NVIDIA Corporation (NVDA) Earnings Call Transcript & Summary

Hi, everyone. Thanks for joining our webinar today on accelerating AV development with DGX Cloud and NVIDIA AI enterprise. Before we begin, I wanted to cover a few housekeeping items. You'll notice on the top-right corner, a button called more information. If you click on this, it will be a drop-down for a few links. Some of those links already have a checkmark next to it, which indicate that they're already open. All of these windows are movable and resizable. You'll also notice a feedback survey link. We would appreciate if you took a moment to complete the survey at the end of the webinar. You'll notice on the left-hand side, there's a related content window. This has links to various webpages as well as our automotive newsletter and a PDF of the slides that are being presented today. Finally, there's a Q&A widget, where you can ask questions at any point during the webinar. We'll do our best to answer all questions that come in. You will also get a link to a replay of this webinar once it's concluded. We hope you enjoy the webinar, and if you have any questions, please don't hesitate to ask, be it the chat window.

Accelerating autonomous vehicle development with NVIDIA platforms. Hello, everyone. My name is Dean Harris with NVIDIA. I'm the Global Business Development for Enterprise and our Automotive business. I'm very excited to be here today to talk about how companies are accelerating autonomous vehicle development with NVIDIA platforms. We have a packed session today, so I'll go ahead and get started with our agenda. First thing we'll cover, is taking a look at the AV industry and software market. We will then talk about some public examples of customers already scaling up for AV, then we'll jump into talking about the AV development workflow and challenges that are present from both a software workflow and hardware or data center perspective. We will then speak about how NVIDIA cloud platforms are helping to accelerate AV development. We will then round out with some concluding thoughts and move into the Q&A session. Please feel free to submit any questions in the chat during the session, and we'll also take live questions at the end. We know that autonomous driving capabilities is complex and time-consuming to develop and test. In a recent report, S&P Global stated that vehicles contain more than 100 million lines of code and are becoming increasingly complex. As this chart shows here from McKinsey, the increasing software complexity and broader deployment of vehicles are contributing to the strong overall growth in the software market. They state 3 key factors to the strong overall growth of the software market, including the increase in software complexity in the domains, most influenced by ACES trends. Second is customization effort for integrating functions in a growing number of platforms and growing labor costs for software developers. They also stated that by 2030, AD software, excluding ADAS, is expected to represent more than 1/4 of the market value and grow at an average yearly rate of over 20% until 2030. So as we can see, there's clearly little doubt about the growth potential and complexity of the autonomous driving market. Software complexity is just one challenge. Another challenge in the AV industry is the ability to scale. There are numerous public statements around the growing scale that is seen or needed to safely deliver autonomous driving. Tesla Motors for example, recently spoke about how they are leveraging NVIDIA GPUs to power their AI supercomputer. The firm also built a compute cluster fitted with 5,760 NVIDIA A100 GPUs back in June 2012, and Tesla's latest investment in 10,000 of NVIDIA H100 GPUs, dwarfs the power of that supercomputer. Aurora is another example. They have been public about how many simulated miles they had driven. And by the time of this public statement, they had driven 6 billion miles, with a target of 9 billion miles by the end of that year in 2021. China is also a quickly emerging market with many AV companies working on the technology. To name some include Li Auto, XPENG, NIO and numerous other companies. We will continue to see more and more EV companies talk about their scale as advancements are made in their work. AV infrastructure at scale; the left side is showing a trend in adoption or scale of adoption over time by some of the largest adopters of our hardware for AV development. I think what's important to show here is that as recently as 2017, these customers all had less than 128 DGX or equivalent 8-way systems. Most had 0, in fact. And you'll see some have now ramped up to over 1,000 DGX equivalents or 8,000 NVIDIA GPUs. So I think for customers who are not yet at that level of scale, this can seem intimidating. But it's even more of a reason why it's important for us to convey the value and benefits that come along with working with NVIDIA expertise and taking advantage of NVIDIA AI platforms wherever possible. The right side is a snapshot in time. So today, the scale of adoption of DGX equivalents. So again, interesting for customers to think about where they sit on this graph and how much ground they have to make up to keep pace with industry leaders. We spent over 10 years developing our own end-to-end AV platform. And as a result, have tens of thousands of hours of engineering expertise to leverage. We understand the challenges firsthand and therefore, are best equipped to provide expertise and solutions in these areas. In a recent announcement from our CEO Jensen on stage with Foxconn, he spoke about how we are developing an AI factory for the vehicle. This vehicle will go through live experiences and collect more data. The data will then be fed back to the AI factory, which would in turn improve the software and then update the entire AV fleet. This entire end-to-end system, the AI factory to vehicle and back around, is what we have called AI factory. More to come on this in the future. We know developing AV is extremely challenging. Developing autonomous vehicles comprises of 3 main steps; first is data collection. Data needs to statistically represent ODDs and scenarios that occur in the real world. The more vehicles that drive in the real world, the greater the cost of acquiring useful data. In addition, capturing data for edge case scenarios can be difficult and dangerous to encounter or reproduce in the real world. Data needs to be diverse, covering weather, lightning, traffic, road conditions, road layout, objects, et cetera. As you can imagine, it's impossible to control for diversity in real-world collected data. For example, how do you recreate a natural event such as a snowstorm with certain conditions and outcomes that apply to the needed driving scenario? Such data types that are necessary for development, such as depth, instance, labels, surface normal, velocities, optical flow, these cannot be collected with precision in the real world. The second is AI training and development. Deep neural network development and refinement requires labeling and curation of thousands of hours of data. This is an incredibly labor-intensive process. Curated sections of sensor data must be labeled with auto labeling techniques and human supervision to add bounding boxes, semantic mask and object classes required for DNN training. And in many cases, some data can be impossible to label, for example, blurry or occluded objects. In addition, preproduction design and testing of sensors, vehicle controllers, planning and control modules, perception algorithms in the real world is costly and time consuming. And third, we have testing and validation, verification and validation in the real world makes up a significant amount of development cost and time. Every layer of the AV stack must be thoroughly tested and validated on the possible scenarios that the vehicle could encounter in the real world. Issues need to be mined to find the unknown, a costly, time-consuming and uncertain process. To prevent regression, AV developers need to test changes in the software and sensor configurations. They need to run through scenarios repeatedly, which is difficult to achieve in the real world and also time consuming and costly. The AV CI/CD pipeline, the massive computational challenge; so how do we know that these DNNs know how to recognize all of these objects and react accordingly in the real world. The answer is the data center. Before they even hit the test track, autonomous vehicles and the AI that powers them, drive miles and miles in the data center. Networks are trained on massive amounts of driving data, until they perceive the driving environment flawlessly. They are then tested and validated on even more data and simulation, before they're deployed in the real world. And it doesn't end here. Networks are continuously trained on new data and functionality in the data center and then upgraded in the vehicle. In the top left-hand corner, a 50-car fleet, driving 6 hours per day can generate 1.6 petabytes each day of data. This data needs to be transported, encoded and stored. This data needs to be curated to identify the important data that needs to be labeled and used for training of the model, and this is usually around 10%. This can be an extremely labor-intensive requirement, with thousands of labelers working to ensure quality with the data frames. Many current AV development companies are using 20-plus models, and we'll talk more about this in the coming slides about new approaches with single world models. Our replay customers are testing against thousands of hours of sensor data and repeat this daily. And as I spoke earlier about a customer example, simulation can help provide a massive amount of driven miles, and can help find the most critical scenarios to test before deployment. As I spoke earlier, approaches to AV development may differ. We have many customers using multiple DNNs, as shown in the left-hand side, in some cases, anywhere from 20 to 50 DNNs, all supporting perception, mapping, planning and even interior sensing applications. We are seeing some AV customers starting to look at a new approach called AV 2.0. This includes a similar software stack, but this time, the majority of the modules are integrated into us, called a one large world end-to-end model. There are some new characteristics associated with this new approach. For example, you need video clips, not just images, to feed through this new world model. We have seen anywhere from 1 to 10 million video clips for this particular approach. The architecture of these world models are transformer-based models or generative AI models. In this case, you are basically predicting the next frame of the video. This will change the entire paradigm of the software stack and its capabilities moving forward. NVIDIA is also moving towards this new approach and in a phased plan towards AV 2.0. More details to come on this in the future. The AV development workflow; AV software usually consists of 4 main areas; perception, prediction, planning and control. Perception, is understanding what is around us at time, T. There are different models to detect objects and providing an understanding of the surrounding environment. Prediction is what happens at T+1. Knowing the perception and prediction, this can help drive actions or specifically what should be done based on what we know from the details. We know AV software stacks have a lot of deep learning models, and many are accelerated with GPUs. There is a huge dependency on AV development and testing workflow. This slide is a representation of the majority of most AV companies, except for a few customers who are taking a different approach, as we mentioned in the previous slide. Starting in the center, there is data ingestion. A lot of data but yielding only a small percentage that is important to the model we are interested in. Most of this data is staved off or discarded. Labeling is required on this 10%, but the data keeps growing and you simply cannot keep scaling up labelers, so often companies use already trained DNNs for auto-labeling or AI-assisted labeling. Often, human labelers can also be used to help with confirming accuracy. In addition to this, many companies are using synthetic data generation to assist with corner cases. Expect 20 million labels to help with training 30 to 40 models for a larger part of the perception and prediction software stack. And for online, offline test validation or replaying data, you can expect 10,000 to 20,000 hours of replay data and targeting hundreds of hours of simulation. Each of these areas pose unique challenges, which require the best-in-the-market networking, hardware and software platforms to help accelerate these development workflows and processes, and inevitably accelerating software delivery and efficiency. Each of these areas we have noted at the bottom of the image, with some of the NVIDIA speed up benchmarks that have been achieved by using the NVIDIA AI enterprise software stack, which we'll talk more about in the coming slides. Now we're going to spend some time talking about the different areas of autonomous vehicle development workflow. We're going to talk about some of the challenges we see in the industry and then some of the benefits seen, leveraging NVIDIA expertise and NVIDIA software. First step in any autonomous vehicle workflow is the data collection or preparation. There's no question that data is king, and not only about quantity of data, but the quality. There are numerous challenges with data procurement, including how to create and collect data for those rare critical edge cases, domain generalization, dynamic scenes and multimodal sensors and sensor fusion. For the first challenge with rare edge cases, automotives need an economical way to create or collect data for rare or dangerous use cases. Think about an avalanche or flooding scenarios. These events are very difficult to recreate and are critical to ensure safety. Companies use simulation or synthetic data generation to create the high-quality data needed for these scenarios, to test against and to help ensure safety. With advances made in the area, AVs continue to be challenged with these rare edge cases and until sufficient data and training happens around the particular edge cases, AVs will continue to struggle with dealing with those scenarios. Companies also adapt strategies around domain generalization, to adapt and reuse data from one domain and fine-tune with data from the new domain. For example, using common data from one city to another and supplementing it in various ways for the data that is different. Another common challenge with data is the area of dynamic scenes. These are extremely complex versus static data and require extensive and diverse data coupled with advanced modeling techniques. Sensor fusion or multimodal, integrating and blending different sensor types such as LIDARs, radars, cameras and more, continue to bring benefit but remain a complex challenge to accurately fuse the intricate data together for reliability and trustworthiness. As we have mentioned with some of the challenges, data procurement and processing can be an expensive and timely exercise for AV companies. Many advancements in this area continue to progress and provide both efficiencies and performance and costs, helping advance AVs to become more reliable, efficient and safe. NVIDIA is working with hundreds of automotive companies across the industry. With NVIDIA's AI platform of software, hardware and enterprise support, we help bring a comprehensive end-to-end capability that helps streamline the customer's AV development workflow. Our platform is designed to create intelligent and low-code solutions, and with our AV expertise help accelerate development workflows and processes, inevitably accelerating software delivery. Specific to the data processing workflow, we've done considerable work with our customers who are leveraging our pre-trained models from NDC to help annotate images. Leveraging NVIDIA encode and decode for image processing, using our software SDK called TAU, for fine-tuning models and even leveraging our Triton Inference Server for serving DL models. As part of leveraging our expertise through our NVIDIA AI enterprise offering, we have seen customers achieve as much as a 50% reduction in manual labeling efforts and resulting in a 30% speed up in labeling throughput. Training deep learning networks is a critical, costly and time-consuming part of the AV workflow. We have seen as many as 10 to 20 deep neural networks for L2, L3 driving and as much as 30% to 40%. And in this case, Tesla Motors, which have publicly expressed using as many as 48 deep learning neural networks, to help propel their AV efforts. As I mentioned at the beginning of the webinar, there are even new approaches evolving in how training is performed with any number of models versus a single world model. We have also spoken about the compute scale required for AV training with as many as 80, up to 320 8-way DGX nodes when testing 10 to 20 deep neural networks for L2, L3 driving, as much as 640 to 1,600-plus nodes for the training of 25 to 50 deep neural networks, and even higher scale for increased model complexity and level of driving. We have numerous customers who have publicly mentioned the massive scale they are using for the training of their AV stack. We spoke earlier about the challenges with data procurement processing, both access to the right amount of compute, coupled to the right type and amount of data that is critical for the training phases. It is critical to understand this phase and continue to look for areas of efficiency and optimizations, all of which can help reduce costs and accelerating your time to market. Accessing the NVIDIA AI platform, we have seen customers take advantage of our SDKs. An example is TAU, which allows you to train, adapt, optimize your model in hours versus months. As part of NVIDIA AI enterprise, we will continue to optimize our GPUs against machine learning frameworks and taking that unknown from your developers, allowing them to spend their time on model development. But lastly, we have seen numerous customers seeing a 3x or greater speed up when moving from Ampere to Hopper GPUs. As mentioned in the earlier section, Synthetic Data Generation, or SDG plays a key role in AV development. SDG can help in various areas, including developing the diverse and rich data needed for handling rare and dangerous use cases. There are many areas that humans can't label. SDG can be used to create the ground truth data, such as occlusion, depth, velocity, adverse weather conditions, collecting all the data you need and for the rare and unknown use cases is simply cost prohibitive. SDG can help save costs from the collecting and labeling real data and providing it faster, all while reducing errors, helping increase the accuracy and addressing many of the areas around labeling our data restrictions and regulations. Depending on the country, there might be regional restrictions or regulations in place around data. This, therefore, requires AV companies to collect or recreate the data that is needed to train the AVs in a specific country. In all of these cases, it opens up the potential for errors and challenges in achieving the accuracy needed, and in a timely manner for the data in question. Moving on to validation, replay and simulation; some of the common challenges here include the concept of open versus closed loop validation. Open loop refers to fixed sensors or fixed platforms, with no feedback mechanism and therefore, can only add value to a certain point. In addition to this, there's an ongoing cost for resources, operations and labeling that is needed to support these validation efforts. Some of the benefits seen with NVIDIA software, include pre and post processing of images using our DALI SDK. This is an NVIDIA GPU accelerated library for data loading and preprocessing to help accelerate deep learning applications. Second there is Triton, which is our optimized inference server, which is a critical part for AV replay. We have seen customers achieve a 6x or greater replay speed up on inference tasks for AV data mining and AV replay. We have one public customer example with Neo, explaining their speed up on 100,000-plus inference tests per day for AV data mining and AV replay. Our DRIVE Sim platform enables physically accurate, closed-loop software-in-the-loop and hardware-in-the-loop simulation. Data center challenges around autonomous vehicle development; on the slides earlier, we have seen the massive industry scale already underway at many AV companies and others soon to come. As the market continues to evolve and mature, there will be more and more pressure on these companies to come up with ways to accelerate deliveries and look for areas of incremental efficiencies. As part of this, access to performance GPUs when scaling up, will be critical to the AV industry to keep pace and deliver on milestones. For platform challenges, the increasing complexity with AI software, accelerated hardware and the required performance networking continues to evolve in the industry. Working with NVIDIA can help overcome these complexities using our NVIDIA AI platform, which includes accelerated and optimized software, hardware and networking. For data center space and power, there is an increasing energy usage in data centers. With the rise of AI, these massive models are delivering new use cases, which are demanding more and more compute. Data centers need to become more efficient and accelerated compute with GPUs drives efficiencies. GPUs can handle compute-intensive functions that use less energy and partitioning or multi-instance capability allows boost and utilization, allowing significant reductions in OpEx and CapEx. We have spoken about the AV development workflow and many of the challenges that AV companies are facing. We've also spoken about some of the benefits that some of our customers are leveraging through our NVIDIA AI enterprise software stack. I would like to now speak about how our NVIDIA cloud is helping automotive customers accelerate their AV development pipeline with NVIDIA DGX Cloud and Omniverse cloud. Introducing DGX Cloud, which we are calling an AI factory in the cloud, we hear time and time again of how enterprises are continuing to look for ways to optimize and accelerate their AV efforts. Performance, software, access to the latest GPU technology, enterprise-grade support, storage and egress and access to an NVIDIA AI expert are what makes DGX Cloud unique. NVIDIA AI enterprise software suite is our enterprise-grade software that powers the NVIDIA AI platform. It includes AI workflows, frameworks and pretrained models, which help organizations get started more quickly with achieving AI outcomes. It includes AI and data science development and deployment tools such as TAU, TensorRT, Triton Inference Server plus many more. It also includes GPU and network operators for cloud native management, container orchestration and infrastructure optimization software for virtualized environment or for scalability. Base Command platform is how users access and interact with resources in DGX Cloud. It abstracts the hardware resources on DGX Cloud to constructs that are identifiable by AI practitioners. DGX systems are abstracted as compute instances. Storage files are abstracted as data sets and workspaces. Each DGX Cloud instance includes 8 A100s or H100 GPUs and is based on the proven DGX architecture, delivering industry-leading performance. With DGX Cloud, you get access to the best of NVIDIA AI. We can help your team get on stock and help optimize your AV workflow. NVIDIA AI experts are available in every step of the AV workflow. Omniverse is a software platform and suite of preloaded reference applications and services. Omniverse Enterprise leverages the NVIDIA RTX technology, and can scale up and down from workstation to the data center, depending on the customer environment and use case. Omniverse Enterprise is used to develop applications and Omniverse cloud is then used to scale and collaborate. It can be used for many areas within automotive, including concept and styling, design and engineering, software and electronics, smart factory, retail experiences and autonomous driving. In the context of autonomous driving, Omniverse can be used to create synthetic data with DRIVE replicator. As we spoke earlier in the webinar, this step is crucial in the AV workflow to help provide the much-needed data for different corner case scenarios and to help reduce data collection costs from the ground truth sources. OV can also be leveraged for AV simulation using the DRIVE Sim application. NVIDIA AI Enterprise, an end-to-end full stack solution, which provides reliability and security for production AI. It consists of 4 important layers; from the bottom, you see infrastructure optimization and cloud native management, where orchestration layers are essential to optimize your infrastructure to be AI-ready. AI and data science development and deployment tools includes the best-in-class AI software that is needed for development and deployment. The entire software stack can be flexibly deployed across accelerated cloud, data center, edge and embedded infrastructure, wherever you choose to run your AI workloads. Applications can run anywhere that NVIDIA infrastructure is available. So now I'd like to recap how NVIDIA AI enterprise has been applied to the AV CI/CD workload. In the earlier slides, we spoke about some of the challenges in how NVIDIA offers software capabilities that streamline the AV pipeline, including data prep, training at scale, deploy at scale with synthetic data generation, replay and simulation. We've been able to work closely with our automotive customers who choose NVIDIA AI enterprise to see the listed performance and speed ups outlined here. The more you buy, the more you save through streamlining and optimizing your processes. Our technical experts can work with you to select the right NVIDIA SDK for your needs and pool from thousands of man hours of GPU optimization experience. Concluding thoughts; so this brings our session to the end. Before we move to the live Q&A session, I'd like to close out with some concluding thoughts to help shape what next steps might look like for those interested to speak further and see how NVIDIA expertise can help you with your AV journey. First, understanding your current and future state is critical. As part of this, we can help understand your key challenges and areas that need to be addressed. We often engage with our customers through a scaling workshop to get a good understanding of where you're at and where you would like to be at some point in the future based on your milestones. Customers come out of these deep dive sessions with a better understanding of their challenges, what is needed to progress and agree on a path forward for the required technical resources and compute to help optimize, accelerate and ideally help ensure you meet your AV software deadlines. The importance is understanding where to invest and how to plan appropriately towards successful delivery of AV for your organization. NVIDIA is here to help you in this journey. And one more note before moving on to our live Q&A session. We would like to make everyone aware about our upcoming GTC conference that will be back and in live person in San Jose in March of 2024. This is the world's premier AI conference with over 600 sessions, 200 exhibits, networking opportunities plus much more, in covering many automotive topics. With special relevance to our session today, the NVIDIA DRIVE Developer Day at GTC is a series of NVIDIA led sessions focusing on AV development. Register now for early bird pricing. We appreciate everyone's time here today, and this concludes our session. We will now move over to the live Q&A portion. Thank you.

Hello, everybody, Dean Harris here. So I hope the webinar has been useful so far. We're now into the live portion of our session for Q&A. So just looking through the list there, please go ahead and submit any other questions that you might have, and we'll certainly do the best we can. We do have a question here from the audience. What platform does DGX run on? So DGX Cloud, again, is our NVIDIA cloud offering, and it is offered on leading cloud suppliers. So today, it's OCI, Azure will be very closely announced and then GCP and working on others. Another question here on DGX Cloud, who do we speak with or how do we purchase DGX Cloud? So like most of NVIDIA's products, we have an NVIDIA Partner Network, NPN that we would work with to transact an NVIDIA solution. So if you have an NVIDIA partner that you're working with today, you want to ask them about how I purchase DGX Cloud, because you would have existing contracts and agreements written up already, and they could help you with that. If you're looking for a recommendation, please reach out to myself or your NVIDIA team for the form, and we'd be glad to help point you in the right direction. Another question here is, when will H100 and H200 be available in cloud? So we are working with our partners to get this rolled out. You will see some announcements coming out soon, hopefully. But I would -- from a timing perspective, I would look towards early 2024 Q1, early, and H200 shortly thereafter. So it's just around the corner. Seems like DGX Cloud has a lot of interest, another question here. It looks like signing up for DGX Cloud is a lot more complicated and involved than signing up for AWS, Azure or GCP, for example? So again, back to how you would consume DGX Cloud. Again, it is a cloud offering, so similar to signing up for a consumption model or an agreement with a cloud partner of yours that's similar for DGX Cloud, there would be a commitment term, a contract written up. And again, you would work this through your NVIDIA partner that you're working with today. And again, if not, we can certainly help recommend somebody who can help provide the agreement, a contract pricing, et cetera, that you would need. But it should not be more complicated certainly. I mean it's similar to a cloud offering, and again, like any term or contract you would need just need to have that written off and agreed upon. Another question here is when to use DGX Cloud over Omniverse cloud. So DGX Cloud, again, is our training multi-node training platform, obviously targeted for AV training and other use cases. And then Omniverse cloud is specific for the 3D application workflows, factory design, marketing, digital twins, et cetera. So they are different solutions, but they can coexist, for example, in supporting any particular automotive workflow or environment. I think we'll share the slides. And again, if anybody needs to reach out to me personally or the NVIDIA team, we have the contact form or you can reach out to me, I'll pop my contact info in the chat here shortly.

All right. Well, thank you, Dean, and thanks to all our attendees for joining our webinar. As Dean mentioned, the slides are available. Right now, if you look at your related content section, you should see a download the slides or view the slides button. There will also be a follow-up e-mail with a link to the recording, as well as a link to the slides. We hope you found this informative and we will be hosting more webinars in the future. So we hope you can attend them in the future as well. Thanks again. Have a great day, morning, evening, depending on where you're located, and hope to see you at a future webinar.