NVIDIA Corporation (NVDA) Earnings Call Transcript & Summary

Hi, everyone. Thanks for joining us for our webinar, Improve Retail Business Outcomes With GPU-Accelerated Apache Spark. We want to cover a few housekeeping items before we begin. At the bottom of your screen, you can find various widgets available for you to use. Once open on the screen, they are resizable and movable. If you have any questions during the webcast, you can submit them through the Q&A window. We will try to answer these at the end of the event. A copy of today's slide deck and additional materials are available in the resource list. We encourage you to download any resources or bookmark any links that you may find useful. Here are some tips that can help make this event the best it can be. To maximize the quality of this audio stream, please close any open applications aside from your browser window. Also, a good old-fashioned browser refresh can cure many ills. So if your audio spudders or the slide seem to be lagging, give that a try. You can also try opening this event in a different browser. If you encounter any other technical issues today, please let us know in the Q&A box, and we will help you troubleshoot. Now without further ado, we'll turn the event over to our speakers to begin the presentation.

Welcome to this webinar on Improving Retail Business Outcomes With GPU-Accelerated Apache Spark. My name is Charu, and I'll be presenting along with my colleague, Ward. We'll be going over how NVIDIA can help retailers run their Apache Spark workloads faster and with significant cost savings. There will be a Q&A session after the presentation. So be sure to stick around. Data analytics is used by retailers for many core business functions. For example, companies can analyze customer demographics, purchase history and contact information to segment customers and target marketing efforts, or they can predict future customer demand for products in order to optimize inventory levels, pricing and promotions to meet that demand. All of these use cases take advantage of the vast amounts of data now available to enterprises in order to make business decisions based on data science algorithms. The most popular framework providing data analytics workloads is Apache Spark. This is an open source project that evolved from the earlier origins of big data with Hadoop. The usage of Spark is widespread in modern enterprises. It's used to perform ETL or extract transform load operations for a variety of purposes. Loading raw data into data lakes, loading data in data warehouses, performing analytics and to prepare data for machine learning. For modern retailers, data science is fundamental to their success. However, it is often too slow, overly expensive and fragmented and brittle in its implementation. In years past, the concept of big data was mostly about collecting a bunch of data into a repository and figuring out the right queries to ask the database to find a useful insight. And while that basic workflow continues today, data science has evolved to the point that many, if not most workflows are leveraging data to build, train and iterate on models. These models are used for doing analysis and making predictions or inferences. But these steps take time and require an increasing amount of computational power. And in many data analytics pipelines before we even get to the point of building and training models. It is estimated that up to 80% of the total time is spent just preparing the data to be useful. To date, most data science operations run on massive CPU clusters. Large-scale CPU infrastructure is incredibly expensive to scale. And with the growing size of data, adding CPU infrastructure continues to increase costs. But inefficient computing is only one part of the problem. Unsupported software implementations are equally problematic. While open source software offers great flexibility and agility for data scientists, who are developing analytics, algorithms and machine learning workflows, retailers need assurance that they can get support with a guaranteed SLA as well as professional assistance with implementation and optimization. NVIDIA has created RAPIDS Accelerator for Apache Spark to provide a solution to these pressing issues. This product allows customers to improve their existing data processing workflows and has the following benefits. First, faster execution time. This acceleration means ETL operations can be completed more quickly. It also allows new use cases that might not be possible today. For example, you can turn a best job into a real-time job or you can run an important operation multiple times for Business Day. Faster data preparation tasks enable information to quickly move to the next stage of the AI pipeline. This allows models to be trained faster while freeing up data scientists and engineers to focus on the most critical activities. It also lowers costs, allowing me to do more with less. When jobs complete faster, it saves organization's time as well as operational costs. Finally, you get the backing of full enterprise support. NVIDIA will able to work with you on fixing bugs, solving problems and helping you get the most out of your investment. NVIDIA RAPIDS Accelerator is supported for Amazon EMR, Google Cloud Dataproc and Databricks with support for other platforms planned for the future. Let's take a look at how it works. RAPIDS itself is a suite of open source libraries incubated at NVIDIA, which enables GPU acceleration of data science algorithms. Building on RAPIDS, NVIDIA created RAPIDS Accelerator for Apache Spark. The software works as a plug-in to popular Apache Spark platforms. It automatically intercepts and accelerates operations that can be sped up with RAPIDS software and GPUs, while allowing other operations to continue running on the CPU. No code changes are required. Currently, RAPIDS Accelerator works on Spark, SQL and data frame operations, and it works in Spark stand-alone as well as YARN and Kubernetes clusters. Some key advances in Spark 3.0 have contributed to delivering transparent GPU acceleration. First, Spark 3.0 provides columnar processing support in the Catalyst query optimizer which is what the RAPIDS Accelerator plugs into. When the query plan is executed, those operators can then be run on GPUs within the Spark cluster. Next, Spark 3.0 recognizes GPUs as first-class resources along with CPU and system memory. This allows Spark 3 to place GPU accelerated workloads directly on to servers containing the necessary GPU resources as they are needed to accelerate and complete a job. Here are some examples of the cost savings that are possible with RAPIDS Accelerator. These results are based on benchmark tests comparing accelerated with not accelerated job runs. This test is based on the TPC-DS benchmark and contains over 100 representative queries. You can see that depending on the Spark platform, GPU acceleration can provide up to a 4x cost savings. Even though GPU cloud instances cost more, the faster execution time results in the large overall cost savings. The benchmark testing also shows the degree of speed up that can be achieved with RAPIDS Accelerator. These graphs are based on the same data from the previous slide and represent the overall best CPU-only results compared with the best GPU results on a price performance basis. You can see that RAPIDS Accelerator provides speed ups between 2x and 5x on this particular benchmark. Some customers we've worked with have been able to see much greater speed up as you learn in a few minutes. While benchmark results are great for general comparisons, you will naturally want to know what kind of time and cost savings you should expect for your own workloads. This is exactly why NVIDIA created the Accelerated Spark Analysis Tool. This tool analyzes the logs files for existing Spark 2 or Spark 3 jobs and predicts both the speed up and cost savings with the GPU acceleration. The tool makes recommendations for which jobs should be accelerated, as well as which jobs might not see enough benefit to make it worth your while to use RAPIDS Accelerator. The tool also provides guidance on how to configure GPU cluster to get the most performance and can recommend further optimizations based on an analysis of the initial GPU accelerated run. And now I'll turn it over to my colleague, Ward, who will go over some interesting case studies of companies who we help to transform their business by accelerating their Apache Spark workloads.

Charu, thanks for that great overview on the benefits and the value of leveraging NVIDIA's RAPIDS Accelerator for Apache Spark solution. Right now, my team and the engineering teams at NVIDIA are working closely with a number of customers. We're working to develop a strategy to upgrade their platforms, and we're working with them to identify workloads that would benefit from the acceleration. So let me tell you a little bit about their goals, the journey and the benefits that they're seeing. I'll start by reviewing the case of a large retailer, I think you'll see that they're facing many of the same challenges that you're probably facing. As background, this customer has tens of thousands of Spark workloads and spends in the hundreds of millions of dollars a year on computing costs. 95% of their workloads are run in their primary corporate Spark environment which leverages GCP Dataproc as their platform. They came to us with several objectives. First, they wanted to accelerate their key workloads to meet their SLAs. And secondly, they were looking to reduce their overall Spark costs. We currently have 2 separate projects that are working on with this customer. The first project was more short term as there was a group in merchandising with their own Spark cluster that was interested in accelerating several key workloads. One of their key workloads, which is depicted on this slide was for assortment ranking on their website. It's critical to regularly update the assortments to better align with customer trends and item popularity. Their goal was for this pipeline to run hourly, but the current solution was taking over an hour. To perform this work, they have multiple machine learning pipelines and ETL steps that needed to be accelerated. We started working with the customer in late November. We have a pretty standard process of engaging with customers on Spark opportunities. We start with our qualification tool to identify the benefits of using the RAPIDS Accelerator for Apache Spark. We typically see most data and compute-intensive workloads will benefit the most. Remember, the beauty of the NVIDIA solution is that you don't have to change the Spark application. In the case of our customer, they were still back on Spark 2.4.8. We leveraged the logs from the Spark 2.4.8 environment to identify which applications we could speed up. The first step was to set up a Spark 3.0 environment. Once they set it up, they tested the application under Spark 3.0, and they had no issues with their workload. Next, we installed the NVIDIA JAR files and made the minor changes to the Spark environment variables to load NVIDIA's optimizer. Again, the beauty of the solution was the customer application needed no modification. They ran initial testing and verification to see the expected performance improvement savings and validated their output. The testing went well enough that this team identified an additional 4 workloads that they validated and set up to leverage RAPIDS Accelerator for Apache Spark. After a little over a month of testing, they deployed a GPU cluster into production and the workloads were removed. As you can see from the slide, the overall outcome for this one particular workload was pretty significant. First, we brought the execution time down to under an hour, and we were able to reduce their overall computing spend by 70%, which equated to about $150,000 a year. In doing our initial work and discussing goals, we ran additional testing to identify optimal configurations for the GPU cluster, and we noticed something that was really interesting. We learned that we could either optimize for cost savings or for performance. If you look at the graphs on the slide, the green bar represents their original CPU-based Spark cluster performance of the application. I recommend that best practices are to add 2 T4 GPUs to each Spark worker node. In this case, we had 40 nodes in the CPU base Spark cluster, so we recommended starting with 40 nodes each containing 2 T4 GPUs. Because this was a cloud's Spark environment, this was very easy as all we had to do was change the instance type of the worker nodes. By moving to 2 T4s, you typically see a 3 to 7x speed up with a 40% cost savings. This provides added performance while providing a savings. In the case of this particular workload, you can see from the graph that we saw a 2x speed up and a 40% cost savings. This is reflected in the purple middle bar that you'll see on the graph above. However, if you're merely looking for cost savings, we found that reducing the number of worker nodes can provide same performance with a significant cost savings. If you take a look at the red bars above, you'll see that by using only 10 worker nodes with 2 T4 GPUs. Instead of the original 40 CPU nodes, we were able to reduce the overall cost by 71% while still decreasing the overall time by 13%. The customer was very excited with these findings, as this gave them 2 really simple strategies to follow to either speed up workloads or significantly reduce their costs. Now what was exciting for us was that the results from this first project were so good and they actually led to the second project. For the second project, we reached out and started working with the primary Spark platform team that was responsible for tens of thousands of applications. We followed the same methodology that we have used with the first group. And at this point, we've already -- we've gathered the Spark logs from tens of thousands of their workloads. We run them through the qualification tool, and we've identified workloads that would benefit from RAPIDS Accelerator for Apache Spark. And what was interesting was that in the case of this primary Spark cluster, these workloads came from many different teams throughout the company. So we identified 4 key teams and they ran initial testing on migrating their workloads to GPUs. For all of these workloads, their primary goal was actually cost savings. And based on our learnings, we have configured smaller clusters. At this point, they've validated several dozen workloads, and I'm excited to say that we started moving these workloads into production on GPUs last week. Our short-term plan is to continue to migrate batches of ideal workloads over to the GPU-accelerated Spark cluster. And longer term, the customer is looking to automate their job submission system to intelligently leverage RAPIDS Accelerator for Apache Spark for appropriate jobs. Our expectation is that there are thousands of jobs that will benefit from the accelerated platform resulting in tens of millions of dollars in savings and ensuring that time-critical inquiries meet their SLAs. For a second customer, I'll tell you about Taboola. Taboola is a large service provider that NVIDIA has been working closely with. As background, when you surf around the Internet, and you see context-relevant advertisements on a web page, chances are that, that content is being served up by Taboola. Taboola uses complex Apache Spark data pipelines and models to produce best bidding recommendations. Their goal is to provide the best possible customer experience and assist retailers with increasing their engagement with new customers. And for the environment, Taboola runs their solution on an on-prem version of Apache Spark in a large Kubernetes cluster. So let's talk about our objectives and challenges. The challenge for Taboola was to optimize both data center capacity and costs. The Apache Spark pipelines that they run are compute hungry and have to be scaled often to grow the cluster capacity. Taboola is facing a number of growing challenges. First, as the company's they service continue to grow, their compute demands continue to grow. And with an ever-increasing number of customers shopping on the Internet, again, their compute demands continue to grow. Also, they see seasonal demand creating significant traffic swings that they have to manage. And finally, as they continue to innovate to provide the best possible customer experience, the number of models and their complexity continue to grow. And as you can guess, continue to drive more compute demand. To give you a feeling of the scale of their problem, Taboola analyzes data with over 1,500 distinct columns and analyzes over 1 terabyte of raw data per hour. So to sum it up, Taboola was seeing significant increases in their compute demand that were difficult and expensive to keep up with. So let's talk about our process and our engagement with Taboola. We'll talk about the Taboola journey. Taboola's started on the assumption they could match the performance of many CPUs through GPU acceleration. Their goal was for a GPU to match the performance of 3 CPUs. While they couldn't use the qualification tool to make predictions on the expected performance, they decided to set up a test cluster and use real-world testing to demonstrate the performance of RAPIDS Accelerator for Apache Spark. As an aside, the results, the qualification tool have been very accurate. And other than unforeseen system bottlenecks, the predictions have been very good. For Taboola testing, they took 1.5 terabytes of production data from CyberMoney, and they picked a set of standard queries that represented some of their most complex SQL operations and they set up a benchmark, so they could compare their standard CPU performance to GPU performance. Some of the operations, Taboola tested out included aggregations, sorts, lateral view explodes, distributed buys, window functions and UDFs. Taboola set up several different GPUs from NVIDIA so they could compare the overall price performance. And as for results, Taboola ran a large number of initial tests, and they found the A30 ended up as the most optimum GPU for them for a price and performance perspective. Their initial performance test showed that 3 to 5x speed up which exceeded their initial goal. But in reviewing their performance data, they realized they had run into a number of system bottlenecks. Their analysis revealed that they encountered nonoptimized configuration parameters, disk performance issues and network bottlenecks. And after addressing these issues, Taboola witnessed some amazing results that far exceeded the GPU to CPU factor of 3 that they were targeting. They actually achieved a factor of 20x speed up across multiple workloads. You can see the final performance results of their testing on this slide. For some of the workloads, they also learned that only 2 A30 GPUs could sustain the workload of one where they had 200 CPU cores before as. A result of the Taboola testing, they are now using NVIDIA A30 GPUs and RAPIDS Accelerator for Apache Spark through [indiscernible] scale and cost issues associated with their large data center CPU clusters. Shortly, we'll be releasing a blog that walks you through all the learnings and performance tuning so that you can benefit. As you can see, NVIDIA's rapid accelerator for Apache Spark provides great benefits for traditional CPU-based Spark customers. It provides application speed up, cost savings, less infrastructure and the ability to easily scale. And with that, let me pass it back to Charu who will tell you about how to deploy the solution in production with enterprise-class support, something you typically aren't able to have with much of today's open source solutions.

Thanks, Ward. Those were some fascinating stories of how customers are seeing real value from GPU acceleration of their business-critical data analytics workloads. Now I'll go over NVIDIA's enterprise support and services for RAPIDS Accelerator. Rapid Accelerator for Apache Spark is available as part of NVIDIA AI enterprise 3.1. In the full life cycle of AI, which includes data prep, training, optimization and production deployment, RAPIDS Accelerator sits in the data prep phase. As you have seen, however, Apache Spark is used not only for machine learning data prep, but also for other data analytics workloads. So the scope of RAPIDS Accelerator is greater than just AI. When you buy a license for NVIDIA Rapid accelerator, there are quite a few services that you're entitled to. First of all, customers get enterprise grade support. This includes SLAs for response with access to NVIDIA technical support experts. There is also a long-term support for stable product software branches, which allows you to use software and production without worrying about having to continually upgrade. Customers also get priority when it comes to notification for product issues such as security and bugs. If you choose, you can also pay for a higher tier of support which includes 24/7 support and an even shorter response time. It also includes the services of a technical account manager, who works on your behalf to ensure you're getting the most out of your investment. Secondly, licensed customers didn't take advantage of professional services to assist with migrating their Apache Spark workloads to GPUs, including expert guidance on optimization and best practices. Finally, a license gives you access to NVIDIA education services, which includes both self-paced and instructor-led classes on a wide variety of data science topics. For retailers who are ready to move beyond testing and POC and start using RAPIDS Accelerator in production, the NVIDIA AI enterprise license provides key services that enterprises need. There are a couple of ways that customers can license RAPIDS Accelerator. First, customers can imply existing entitlements of NVIDIA AI enterprise. The end user license agreement allows you to use one license per GPU. Second, customers can work with NVIDIA for a private offer arrangement. The private offer is a custom quote that's negotiated with NVIDIA for a longer-term commitment at a discounted rate. NVIDIA AI enterprise in the marketplaces of cloud service providers gives enterprises who have committed cloud spend agreements to be able to burn down their existing credits. We'll close this webinar with some recommendations on how to get started. The best way to get started with RAPIDS Accelerator is simply to try it out in a proof of concept. As we've seen, you can use the plug-in on existing Spark 3 jobs without any code changes. And the Accelerated Spark Analysis Tool can guide you towards the best initial candidate jobs to test. The tool will also recommend how to configure your Spark cluster for best performance that can also help to the cluster, so you see the most benefit. You can sign up with the URL on the screen and an NVIDIA solution architect can work with you throughout the whole proof-of-concept process. Thank you very much for watching our presentation. We hope you learn about how RAPIDS Accelerator for Apache Spark can potentially improve your own business. And we invite you to sign up for a trial.

All right. I think we're ready to take some questions from the audience, feel free to type them into the Q&A. So one question I see is which GPUs do you recommend? And how does the performance change for various GPU models? So generally speaking, you get the best performance gains on GPUs with more memory. So that would be like the V100 or A100 or that class. That's because the larger memory can sustain more concurrent tasks. However, in terms of price performance, we see the best price performance when you use more inference class GPUs such as the T4, the new L4 or A10 or even A30. So if you're looking for price performance, you would go with one of those, if you're looking for the absolute best performance, look for a GPU with large memory.

Great. I'll take a question, Charu, since you took the first one. So I see a couple of questions came in about what platforms is this supported on? And what's great about it is that we've targeted all of the key Spark platforms that are out there. So whether you're using EMR from Amazon or you using Google Dataproc, a lot of the work that I've done has been with Dataproc. Or you're using Databricks. We have customers using Databricks from AWS or also Microsoft. So the version of Spark that you're probably running today we've already tested with and are working with customers around. And the other thing is like the Taboola case, right? They're just running straight Apache Spark on their own on-prem. So they're not using a commercial version of Spark. So the -- what's nice about this is that you have the flexibility to leverage this speed up in any environment that you might be working with. Another interesting question kind of rolls into that one also is -- can I run this on Spark 2? And the answer is no, all the integration that we've done is with some features that we're in, Spark 3.1 and later. And the other thing I'll tell you about is that there's some great performance benefits that came along with Spark 3. So even if there are workloads that maybe aren't as applicable for running on top of GPUs, your workloads are going to see speed up. And so our recommendation is start your migration and you're testing on to Spark 3 or 3.1 in our case to support our accelerator. But start that work today, so you can get to that platform, if you're not already there because all your workloads will see speed ups. Again, the beauty of the work we've done is that for these large complex queries that you're running, we speed them up, but we do a great job doing that. And we obviously significantly improve your overall spend. So it's a win there, too. But there's some really basic, simple, small data jobs that run relatively quickly. That it might not be worth leveraging the GPU and the beauty of the qualification tool is if you use that tool, let's identify your -- and as a starting point, your top 10% of your workloads that are really going to bring value back to you as a company and start there. And then you can make decisions moving forward on what makes sense, what are the next workloads you should tackle. What are some of the other great questions we've got.

Okay. I see one question here. A couple of related ones. What efforts needed to convert from CPU to GPU? Like do we need developer time? Or do you have to do any code changes? And kind of as we mentioned before, the beauty of the solution is that you don't have to implement any code changes. It's done transparently for you. Once you deploy the plug-in, you just need to change a few configuration lines when you are configuring your cluster and then it'll automatically take advantage of the GPU. So you don't have to bother your developers any of this. You can just do it and it becomes transparent. And so in terms of the configuration, the tool that we've been talking about this Spark Analysis Tool, it will help you. It will initially recommend some configuration parameters to use. And then after you've run the initial run with GPUs, it can go back, analyze that run and recommend some further tuning. So that's really the beauty of this. You can kind of implement this without having to do anything that takes away from your developers' time?