NVIDIA Corporation (NVDA) Earnings Call Transcript & Summary

Hello, and welcome to today's webinar on Networking for AI. My name is David Iles, and I'm a Senior Director in NVIDIA, responsible for AI networking solutions and ecosystem. And in today's webinar, we're going to talk about really three things. First, we're going to talk about AI impacts data center design and how running AI on a network in a data center that wasn't designed for is going to give you suboptimal results. Then we're going to get into our two networking solutions that we have from NVIDIA. First, we've got the NVIDIA Spectrum-X, which is the world's first Ethernet platform designed for AI. And then we're going to talk about the NVIDIA Quantum-2 InfiniBand platform, which is -- has a lot of the features of Spectrum-X and then a few more. So hold on tight, and we're going to deep dive and provide some new benchmark and the like. So first off, let's talk a little bit about AI and frameworks and the infrastructure to pull it off. You will see here is a full stack solution from NVIDIA where we famously make great GPUs. But the GPUs that we have, have been constantly improved and optimized for AI. And you'll find that we also make CPUs, like Grace CPU. That also is optimized for AI. And then you see we've got DPUs, which are kind of optimized for a couple of different areas. But we have a SuperNIC that's optimized for this east-west traffic for GPUs, for AI fabrics. And then we've got DPUs, which are optimized for kind of your north-south. And then we've got switches that are also now optimized for AI. And so all of these comes together with the software and the platforms like DGX and HGX systems to kind of optimize form factors for, again, for AI training and inference. And if you look here, you can see some of these acceleration libraries. And so this is something that's constantly evolving and is optimized again for AI. And we're going to talk a bit about how that impacts your data center design, how these acceleration libraries -- and it's important to know a little bit about because it affects how traffic flows across the data center. Okay. So with that, let's get into what different types of data centers are and how they are impacted and need to be designed correctly for getting the highest levels of AI performance. So we'll start with the AI factories at the bottom. And this is -- what is an AI factory? An AI factory is a data center that is designed to do one thing really well. And that is AI training for very large AI models. And everything chosen in the AI factory is built and chosen for that specific use case. It's -- so the CPUs, the GPUs and the network. And so -- and the network for AI factories tend to be -- and what we think in our point of view is that the NVIDIA NVLink in combination with our InfiniBand is really the gold standard for AI fabrics. And the idea is that these things come together in AI factories to produce the highest level of AI training performance per watt and per dollar. It's the most efficient way of doing AI in these large scale models. And so this is what we have been doing for quite a long time. We build them specifically for each use case, and they are pieces of art. They're amazing. However, we've had a bit of a kind of like an iPhone moment for AI, where with -- along with ChatGPT, people have figured out that Generative AI is useful to make everyone's lives a bit more productive, professional lives more efficient and businesses to be able to do so much more. And so everyone wants Generative AI for their applications, all the enterprises out there. And so what we find is that there is a strong desire to build smaller clusters. And the main consumption for that is cloud, private cloud, public cloud. But you'll see that cloud is very different than an AI factory. So you might be surprised to see, but an AI factory frequently will have one tenant. So if OpenAI is -- if you look at their factory, well, they've got one tenant and it's OpenAI, right? So -- and they've got like one job running, and it's GPT training. And that's very different than the cloud environment. So the cloud environment, well, it's optimized for multi-tenancy, in fact, that's maybe the most important characteristic for a lot of the cloud networks, is that you can have multiple tenants, thousands and thousands of them, all running at the same time, protected from each other. And you look at the type of workloads, they're all running at the same time as well. You've got thousands and thousands of them. So kind of the polar opposite of the AI factory. And so it shouldn't be a surprise that when someone takes a GPU server, and they pluck it out of an AI factory and they plop it into a generic cloud infrastructure, well, those GPUs and that -- those servers that performed very well in the AI factory running about half speed in that traditional cloud environment with traditional legacy Ethernet. And we'll get into why that is. But we see there is a need for Generative AI cloud that do multi-tenancy, but also perform at optimal speeds. And so just like AI factory and the cloud are really defined by the networks that tie that data center together, we started looking at an AI cloud, what that network looks like. So if we look at a network for any kind of an AI cluster that's doing training, you're going to see a strong recommendation from us. And really the best way to go is you have a separate fabric for the GPUs to communicate with each other. And we'll call that the AI fabric or east-west traffic. And it's really -- it's for that communication from GPU to GPU that doesn't conflict with the general purpose traditional Ethernet. And that's important to have its own special highway because it's going to be built differently. It's like -- it's a racetrack versus general purpose road on the traditional north-south network. And if you were to kind of look at under a microscope, how these things are very different, it starts with the protocols. If you look at the general purpose cloud network, you're going to have a lot of TCP. And by a lot, you're going to have hundreds of thousands of flows, and they're all relatively small flows. You look at an AI fabric, it's all going to be UDP, right? It's all RoCE. It's RDMA traffic. And the flows themselves are going to be relatively fewer flows, but each of those flows are going to be monster. They're like elephant or super elephant flows. These are whales. And this turns into a problem when you want to run AI on traditional Ethernet. It's not very good at load balancing those monster elephant flows that you get with an installation. So that's problem number one. The other thing is if you look at the jitter tolerance, what does that mean? Well, we're talking about latencies and a traditional cloud network, where it's connecting servers to clients that might be on the other side of the planet. And so the latency doesn't matter that much on the left-hand side here, this general purpose cloud, but it matters a lot for an AI fabric. And not just the average latency, which does matter, right? Average latency matters. However, a lot of times, average latency as a figure can hide and cover a world of sins in that those outliers, like 1 in 100 flows suddenly has terrible latency. Well, that is a long tail that happens occasionally and it just kills AI performance. And so some of you might be wondering, well, why? What comes down to those applications that we allude to at the top of this, where the applications on traditional cloud, they're very loosely coupled. They might talk to each other, but they're not highly dependent on each other. Unlike AI, where AI is very much like traditional kind of high-performance computing, where they're tightly coupled and you have a team of GPUs, then they are all working in tight coordination and they're doing things like collective operations, which if you're from the HPC world, you know that, that's something where each one is waiting for the others to finish. And anything that slows down your network is going to slow down your GPUs, and you don't want that, right? That's -- idle GPU time is wasted GPU time. So we created Spectrum-X, and it's really -- its purpose is to extend kind of those generative AI capabilities that you would found in an AI fabric network and bring them to a mainstream hyperscale cloud. So this is something that is an end-to-end solution with our SuperNICs and our Spectrum-4 switch, where they come together and they work in a very tight coordination to make AI work better. And in fact, it's optimized around making NCCL work better. And so that NCCL, if you're unfamiliar with NCCL, it stands for NVIDIA Collective Communications Library. And the way to think about it is any time you have an AI workload that spans across multiple GPUs, NCCL is what tells those GPUs how they inter-operate, which part of the job they do and how they communicate, what path they use to communicate. And so you can imagine that has a lot of bearing on the network, and you want the network to be optimized for how the GPUs are communicating. The other thing you'll find is that NCCL is changing, right? It changes with every new hardware platform that comes out. We have a new NCCL release. With every new AI training model, large language model, we get NCCL optimized for those environments. And so it's important for continuous innovations on the libraries, and it's important for the network to keep up with those innovations and to adapt to them. And so we're going to -- the next two items about adaptive routing and congestion control, we're going to talk about in just a bit. We're going to get in quite a bit of detail, and show why, again, AI affects your data center network design. And so we have this thing called Spectrum-X. And the value proposition is pretty straightforward, right? There's three main points. And it starts with performance, right? So you want the network to run faster in an AI environment, or another way of putting it is you don't want the network slowing down your GPUs. You want the highest performance network that can make your GPUs perform better. So when we talk about performance, we're talking about the network enabling the GPUs to perform at the speed of light, at the kind of peak performance. Number two is the fastest time to AI. So another way of looking at this is now that we've had that iPhone moment for AI, and everybody -- all these enterprises want a solution for AI in their data center, well, they don't have time to do science experiments and they don't have hundreds of engineers to kind of play with different models. When they want to get AI running, they want to get it now and they don't want to wait a year. So the fastest time is to pick a proven design that is based on real-world experiences and best practices. And that's what we put together with Spectrum-X is a way of building turnkey AI clusters that are automated and they're ready to go with great cookbooks and reference architectures and the like. So that's the number two, fastest time to AI. And number three is around really those two things, and then turning them into a dollar value. So it's the best return on investment because your network -- the right network will pay for itself in a matter of months by making the AI training times improve, by making the GPUs more efficient, by making the storage and the whole AI cluster come together and perform the way you want. And so it pays for itself very quickly. And again, we're going to drill down on that in just a second as well. So what I'd like to do is talk about two things that Spectrum-X does that's different than your legacy Ethernet environment. And specifically for AI, like why -- what we're doing for AI. And so what I'm going to do, I'm going to grab my little laser here, and we'll talk first about adaptive routing for RDMA. And I point that out because there's -- you probably heard there's other adaptive routing solutions out there, but they tend to be, we'll call them, general purpose adaptive routing. They work great. They're fine for TCP. They're kind of there for campus networks and regular data center networks, and that's fine. But we've built here is an adaptive routing that specifically for RoCE, for RDMA traffic, for lossless environments. And in fact, we don't even want to use this feature on campus or regular non-AI environments. You want this for the lossy -- sorry, the lossless environments, and that's what this adaptive routing is built for. And this solves a bit of that problem with those large elephant flows that traditional Ethernet suffers with. And so the problem for traditional Ethernet is there's a kind of a rule that says, every packet in a given flow has to take the same path through the network. And that rule is there so that when the packets arrive at the far end on the receiving side, they don't end up out of order because, usually, the way it works is a packet's out of order, it gets dropped and then you have to retransmit. And so what we've done with the NVIDIA Spectrum-X is we have this adaptive routing that works a bit different. It allows for us to break that rule, that traditional rule that every packet has to take the same path. And instead, what we do is the packets, as they're being sent out by the BlueField SuperNICs, they identify which flows we can do kind of aggressive load balancing, this adaptive routing. And we have -- in the Spectrum-4 switch, we have a fine grain adaptive routing in hardware. And what it's able to do is load balance packet-by-packet even in a given flow and send those packets down the links that are not congested. So we can avoid congestion and we can get even spreading our traffic across the network. And you can see that in step three here that you see some of the green packets are spread across different spine switches and some of the purple packets are across different spine switches. And that would be a no-no with traditional Ethernet, right? You couldn't do step three. You wouldn't see that. You'd see all the green on one spine and all of the purple on another spine. And so this allows us to then increase the bandwidth. So if you can see at the bottom, you have traditional Ethernet, where you've got all of these elephant flows, and over time, what you end up seeing is only getting about 60% effective bandwidth when you run AI on a traditional leaf spine network for Ethernet. So only 60%. And we increased that 1.6x in terms of just throughput and get up to about 95% effective bandwidth, nearly perfect load balancing of the packets. However, in order to do this, we need the BlueField SuperNICs to do something special at step 4 here because what you can see is that those packets are now arriving out of order, right? They're no longer in the right order when they get to the SuperNICs on the right in step 4. And so in step 4, going to step 5, as the SuperNICs are sending these packets into memory, as they're placing them in memory, they're putting them in the correct order. And this is the kind of the magic that allows us to do that out-of-order load balancing and getting really effective bandwidth and much lower latency because, again, we have less queuing going on here as well. So this is something that, again, our adaptive routing, it's built specifically for RoCE, for RDMA traffic. So now we're avoiding congestion and we're avoiding some of those latencies associated with it. But I think it's also important, there's another kind of congestion. And so this is the second kind of key thing that we do with a Spectrum-X that's different than traditional or legacy Ethernet, and this has to do with congestion control that's programmable. And so what we see here, if you look on the upper side, this is kind of what happens with legacy Ethernet. When I have AI running in an environment, there's a kind of congestion called [ NCAST ] or many to one. And this is pretty common in AI. There's an operation called AllReduce. There's a NCCL AllReduce. And what happens is you have multiple servers sending multiple streams to one receiver. And so in this case, I've got two 400-gig streams going into the network, and they all converge over here and they have to try to get out a single 400 Gb link. And that's never going to work, right? You're never going to get more than 400 Gb of throughput out of a 400 Gb link. So 800 Gb going in, 400 Gb coming out or trying to go out. And what you end up with, with traditional Ethernet, even with RoCE, even with RoCE version 2, you end up with this flow getting slowed down not to 400 Gb, but under 200 Gb, more like 140 Gb. But -- and that alone will be bad. However, there's another thing that happens here around performance isolation or what some people call the noisy neighbor problem. And this is workload B. So workload B, maybe it's been operating just perfectly for months. It's doing its AI training times like a clock. They know exactly how long these things take to run. And then all of sudden workload A starts up, and we have noise, right? So noise is an important concept here. Noise from workload A is now impacting workload B. And so workload B also suddenly, instead of getting 400 Gb of throughput going to workload B, it also gets throttled back to around 140 Gb per second. So you're getting far less throughput here, and you're having this noisy neighbor problem. So what do we do to fix this? Well, we have some very clever congestion control that's in hardware. And so what we do is we use the hardware-based flow metering inside of the BlueField SuperNICs to control the injection rate, how fast packets are going in so that we don't send more packets going in at a higher rate than can come out. And how the BlueFields know how many packets sent in is they send probes. So they send probes through the network and the Spectrum-4 switches accelerate those probes, they allow them to bypass congested paths, and they provide good contextual telemetry information so that the BlueField-3s on the left-hand side will know when there's congestion and react to it at sub-microsecond kind of hardware scale reaction times. And they're able to then adjust that injection rate. And the net benefit is, a, both workload A and workload B, well, they both get 400 Gb or nearly 400 Gb of throughput in this scenario. But also, you no longer have noise from the perspective of workload B, noise causing this noisy neighbor problem. They don't have that problem anymore. They're able to run perfectly well. And I'll just also point out that we have tuned this congestion control feature to work in conjunction with the adaptive routing feature and also to work with NCCL. However, you may find that you've got your own NPI workload or some other workload. And so we have made this programmable so that you can tune it and change it if you want, although I think best practice is if you've got an end-to-end NVIDIA solution, it's going to work out of the box. Okay. So let's prove out some of these features in a real environment. And so in order to prove out Spectrum-X and all of the things that it can do, we are building a Generative AI cloud supercomputer in Israel. It will be the most powerful supercomputer in Israel. We call it Israel-1. And it will have 256 HGX systems, which means it will have 2,000 H100 GPUs. It will have 2,500 BlueField-3s and a bunch of Spectrum-4 switches to build a 3-tier network, which I'll show you in a second. But the goal is to be able to have peak AI performance of around 8 exaflops. So why this matters to you is this is a public proving ground to show, here's how we build it, here's how then we generate reference architectures and we benchmark. And I'm going to get into some benchmarks to prove out, here's the value of Spectrum-X with our GPUs. So one of the proof points is around digital twins. So NVIDIA, we're huge proponents of digital twins. And we use them as we have been building Israel-1. And so we have kind of two different kinds of digital twins. One is the Omniverse where it's a physical digital twin of the data center where we map out airflow and power and how the cables are laid out. And that's kind of the physical environment, and we do that with Omniverse. But we also have a logical digital twin that looks into the network logic. And so what we do is we create a digital twin, and we simulate all of the switches and all the cables and all of the servers with each one of their NICs or DPUs or SuperNICs are all simulated so that you can verify full data pack connectivity. You can verify your configurations are correct. You can test fail-over scenarios. You can test your zero-touch provisioning. You can make sure your automation's working. And it's built to scale. So if you wanted to play with digital twin technology, it's available on our NVIDIA Air. And so anybody can go to air.nvidia.com and access this or access Air and create their own digital twin. But this is a great environment for multiple people to test and use and verify that things are working so that when hardware does show up, you can be up and running very quickly. So this is part of delivering on that idea of time to AI, right? The accelerated time to AI and digital twins help you get there by having everything working before hardware even arrives. And then, of course, you can also test changes before you push them into the real world. And so let's move forward a little bit. So Spectrum-X, there's a reference architecture that we're building for each kind of main platform use case. So if you remember that first slide where we talk about these different platforms, DGX, HGX, OVX. Well, each one of those things gets their own reference architectures, cookbooks, if you will, so that you can very quickly build your own AI infrastructure that matches best practices and known working configs and known performance levels, which we're going to get into. And so the way we build all of our reference architectures is on this concept of scalable units, which I'm going to get into in just a second. In fact, let's show a picture of scalable units. So if you look at how we build all of our -- whether it's our SuperPODs or BasePODs, with Spectrum-X, it's no different. The reference architectures, they're built using these scaled units or SUs when you see that. And so in SU for us here, it's 256 GPUs, and that's kind of a stamp. So that if someone said, Well, I want to build -- I want 500 GPUs. Well, they're easy. Well, that's two scalable units or you can have three scalable units all the way up to 32 scalable units that are all just follow our standard reference architecture. And this concept, it doesn't just work at up to 8,000 GPUs that you see here. If we go to the next slide, you can see we can scale into the tens of thousands of GPUs. And so we have built the Israel-1 so that it's following this model that you see here, where we've got three tiers of networks, even though we don't really need three tiers of networking for the scale of Israel-1, but it's built to prove out the benefits of the Spectrum-X infrastructure. And so I want you to look at this picture and recognize how I've got a scalable unit #1 on the left and another scalable unit. Because as we start talking about benchmarking, a lot of times, what you will -- imagine you've got an AI cloud. If you have an AI cloud, what you have is customers that come in, and they want to use those GPU resources for a time, right? They will use -- and so you will allocate GPUs. And so it might be that if they're lucky, they get all the GPUs in the same scalable unit, and then all the networking for that is kind of really local to there. And some -- there are some cloud guys out there that spend a lot of time really trying to get that localized GPUs. But what happens over time is as you get more users coming in and leaving, those workloads get spread out. And that's where noise becomes a bigger issue, where you've got multiple jobs running at the same time. And you need to protect each of those workloads from each other. You need that performance isolation. So hopefully, take a picture of this in your minds as we talk about benchmarking here in a second. And one more note before we get to benchmarking. If you look at the application stack, you'll find that we're going to show you some benchmarks that kind of each tier of this model, the stack. And as you go kind of further up the stack, well, it becomes more valuable to the person that uses that particular model. So for example, if I have a GPT benchmark that I show you, well, that's great for anybody running GPT, but it's not useful for Llama or NeMo exactly, right? They're going to want their own model benchmark. And we're fast at work, creating lots of benchmarks on that. However, you can see that all of those models run on AI primitives, right? They all run on NCCL. So if I have a NCCL benchmark, well, that's very useful really for all those models, even if it doesn't like measure in terms of AI training time. And NCCL, of course, runs on top of RDMA. And so as we go through these benchmarks, we're going to start at RDMA and then we're going to move up the stack. So let's show the value of Spectrum-X. So we start with RDMA. And so again, I want -- now I want you to call back that picture that I said, where we've got all those scalable units. And so in this case, I've got 80 nodes, right? So I got all these nodes, and they're across a couple of different scalable units. And so they have a benchmark running. And the question then is, well, how much bandwidth can 1 node, 1 queue pair? So one GPU talking to one other GPUs. So we've got a -- and when we say QP, that's a queue pair. And what you see here is that there's a theoretical maximum amount of throughput that you could get when you take out the overhead of protocols and that, and it's about 388 Gb per second. And what we see with Spectrum-X is we get nearly that number. We're at 380 Gb per second for this node-to-node, single-path bandwidth here. And what we see with legacy Ethernet, we get about 82 Gb per second. And I'll just -- this is throughput. And if I do the exact same thing, I'm going to go to the next slide, this is the amount of latency that you'd see on legacy Ethernet versus Spectrum-X. Again, this is in that three-tier network model where I've got traffic going from one scalable unit to another. And I just look at one GPU talking to another GPU, and go, well, how long is the latency between those? And you can see with Spectrum-X, we're about 12.5 microseconds, all in. And then you see with legacy Ethernet, we're at 57 microseconds. And this is the same hardware actually. But we're not introducing some slow legacy Ethernet switches, right? These are really the lowest latency switches on the market with this kind of capacity. However, on the right, you don't have those Spectrum-X congestion avoidance features. You don't have those adaptive routing features. And so it can only do as good as legacy Ethernet can do, even if it's the lowest latency Ethernet switches on the market. And then on the left, you see once we turn on those advanced features, we're able to drive down latency while we increase throughput. So let's move up the stack a little bit. So now if you picture that triangle, that pyramid, well, now we're not talking about the RDMA level. We're talking about NCCL. And so NCCL has built into it a bunch of tools so you can verify the performance of your AI cluster. That's kind of nice, right? Well -- so in this case, we've got an all-reduced isolation test. And this is, again, a standard test built in the NCCL and in one of those -- the Israel-1 type of configuration, where we've got different kind of scalable units. And in this case, I've got 8 nodes, and they're talking to each other. So we have a workload of 8 nodes. And so again, this is a very common scenario in an AI cloud where you've got a customer. They've got a use case. They need -- and they're willing to pay for 8 of your servers for some amount of time. And at the same time, you have a bunch of other workloads running at the same time, and those -- that's what we're going to call noise, right, the noisy neighbor problem. So this is noise here. And the question is, can you protect this application from all that -- those noisy neighbors from the noise? And what we see here with Spectrum-X, again, we're able to achieve very nearly, the theoretical maximum. And what we see with legacy Ethernet is two things. One is we see this -- and I'm just going to grab my pointer here again. Legacy Ethernet, on average, if you run this 100 times and you average it out, you're going to get about 143 Gb per second between these nodes. And -- however, you might occasionally see a maximum of 170 Gb per second, which means that sometimes it's a lot lower, right? This is the average. You can imagine the low, it's going to be low. And this is kind of jitter. This is how it changes from run to run. And this becomes important as we start looking at not just a NCCL all-reduced test where we're protecting from noise, it also is impacted by how jobs are placed. So as I mentioned before, job placement can matter on some cloud infrastructure. And if you were kind of look here at the top, we've got this case where we've got the optimal job placement where all of the nodes are talking, and they have the best path and they don't have to worry about like a lot of noise and stuff. And what we see here is that even in optimal placement, your kind of legacy Ethernet, you end up with this jitter that goes from, again, kind of the maximum performance they could get to the kind of average. It's going to be somewhere here. So it's going to be vary, right? That's this jitter. And that's consistent whether you have optimal placement, you have average placement of the jobs or even the worst-case placement, where every time -- if you look at job one, it has the work spread out all the way across the data center and has to go all the way across the network, and you see this jitter. Now let's compare that to Spectrum-X. And with Spectrum-X, what you see here is that the -- we get kind of peak performance. It doesn't matter where the jobs are placed. If they're optimally and they're really close and minimal hops away from each other or they're in the worst case, we get around the same performance. And the other thing I'll point out is there's not a lot of jitter, right? There's no variation in runs. So you run it 100 times, you get the same performance 100 times in a row. And so this is where we start looking at the top of the stack. So large language model performance. If you have around 2,000 GPUs, we're going to get 1.4x higher large language model performance, meaning the AI training times are going to run 1.4x faster. And before we get to kind of the final top of the stack, let's also just mention that there's another kind of problem that we've noticed with legacy Ethernet when you're running AI jobs on it. And it has to do with when you have a link failure. And link failures are very common in a data center. You can imagine you've got tens of thousands of links and -- in a large data center. And one of them is going to go wrong every so often. And what we see is with traditional or kind of legacy Ethernet, an outsized drop in performance, meaning -- let's imagine you've got 10 links and you lose one of those links. Well, you might be natural to assume the performance is going to drop 10%, right, 1 out of 10. But what we see is a much larger drop in percent. And it has to do with how these NCCL flows get balanced. And what we have is kind of a -- with Spectrum-X, we have a way of kind of intelligently rebalancing the NCCL flows to avoid the failed paths. And this is something we'll refer to as global adaptive routing, and it's something that's important for dealing with these small outages that happen regularly in most data centers. Okay. Back to benchmark. So we're going to talk about FSDP Llama. And again, this is where we've got 16 nodes, right? They're all talking to each other. And then I've got 38 other nodes, they're doing something else, right? There's some other customer. But if I'm a customer of the AI cloud and I've paid for 16 nodes doing Llama, well, I want them to perform very well. And so what we see here is that in this situation, we've got a noisy neighbor problem that shows up with legacy Ethernet. We have a performance isolation problem. And so what this is showing is an iteration. So how long does it -- you're doing a training workload. And each iterations, you might have thousands of iterations that you have to run through, right? So thousands and thousands of iterations. But each iteration on the legacy Ethernet is going to take 26.2 seconds in this environment versus Spectrum-X, which achieves really the theoretical maximum. So 15 seconds for the 16 nodes to run this Llama, and that's 15 seconds per iteration. And you can imagine this is why things start to matter. Looking at it a little differently. So there's a tool with GPT-3, where we have a large language model that's spread across three scalable units, right? So we've got 128 GPUs, but they're kind of spread across these different scalable units. And this is meant to show in only 128 GPUs, what kind of slowdown that you're going to see, right? So you're going to see a massive kind of slowdown when there's noise. So when there's noise, boom, things start slowing down for legacy Ethernet versus Spectrum-X, where we're able to kind of maintain the same performance regardless of whether there's noise or not. And so this is my last benchmark. And in this case, we're going to leave the Israel-1, and we're going to look at results from public cloud infrastructure. This is existing infrastructure where we know the performance that people should get out of their GPUs. So if you have 128 GPUs, we can see that on this traditional Ethernet infrastructure, they're getting 18% less performance than they should with 128 GPUs. However, as the scale increases to 2,000 GPUs, the drop in performance gets much larger, which, if you'll remember, the network makes a bigger difference as the size of the cluster increases. And so if we were to kind of put this in dollars and cents and say, well, what's the ROI? What's the total cost of ownership? And I won't get into all the math here. But the key takeaway is Spectrum-X is a -- it's a premium product. It delivers high value, and it pays for itself very, very quickly. And the point I like on this chart is that if you look at that 1.4x, performance per GPU advantage, so your GPUs are going to perform better. So another way of looking at it is you can get the network for free if you want it, and you still wouldn't use it, right? Your AI data scientists will be very unhappy that you save money on a network, but you now have to buy 40% more GPUs to maintain the same level of AI performance. So the network really matters when it comes to AI performance. And it's not, again, so much how the -- how much throughput you get out of one particular link or not, it's really about how fast that AI cluster runs. What are your AI training times, and that's with the network [Audio Gap] NVIDIA has a large number of partners that are offering Spectrum-X as part of their AI solutions. This includes Lenovo, HP, Dell, Supermicro and really anybody that offers NVIDIA-certified platforms. Now I'd like to talk about the third agenda item for today's webinar, and that is the NVIDIA Quantum InfiniBand platform. This is something that actually a lot of the features on Spectrum-X come from. So while InfiniBand is a 400-Gb per second platform for the switch, the deep use and the ConnectX HCAs, you'll find that there are some similarities, right? Both platforms can do 400 Gb per second. They both have adaptive routing, and they both have congestion control with that performance isolation feature. In fact, the Quantum platform has had the congestion control and the adaptive routing for longer than Spectrum-X did and it's been proven out in some of the largest supercomputers in the world. And so we got to learn and take some of those learnings from the Quantum platform and apply them to the Spectrum-X platform. However, there are some things that the Quantum InfiniBand platform can do that Spectrum-X could not do. InfiniBand offers the lowest latency network for AI solutions, and that is a clear differentiator. But where InfiniBand really stands out with the Quantum platform is the data reductions that you see on the right-hand side of this chart. SHARP is an acronym and it stands for Scalable Hierarchical Aggregation and Reduction Protocol. SHARP was originally designed to improve the performance of MPI collective operations for high-performance computing and supercomputers. And it worked by offloading collective operations from the CPU to the switch network. This eliminated the need to send data multiple times between endpoints. SHARP has been enhanced to now also offload machine learning collective operations to work with AI environments and not just from CPUs, but it also accelerates GPUs. This innovative approach decreases the amount of data traversing the network as aggregation nodes are reached and dramatically reduces the machine learning operation times. Implementing collective communication algorithms in the network also has additional benefits such as freeing up the valuable CPU and GPU resources for computation rather than using them to process communication. SHARP has a high impact on MPI performance. And you could see that with this benchmark, where we have two benchmarks, one showing the benefit of latency in a NCCL all reduce environment. And on the right, you can see the benefit to bandwidth as the buffer size increases in 128 node DGX cluster. And if we wanted to show the benefit of SHARP at the application level, if you remember the top of that pyramid, you can see even on a very small 8-node cluster, SHARP improved the performance at the application level by 15%. And then SHARP also works in multi-tenant environments. So even though we position Spectrum-X for multi-tenant environments, we have customers that are building large cloud environments with Quantum-2. And so it's important for SHARP to be multi-tenant aware and capable. So in summary, we have two network solutions for AI: we have Spectrum-X; Ethernet built for AI; and then we have Quantum-2, which is InfiniBand built, again, for AI. And both of these offer simplified turnkey deployments. So these are designs that are optimized based on real-world deployment. These are leading edge optimizations, meaning we continue to tune as NCCL changes, as new GPUs come out, as new AI models come out. You can count on having your best level of GPU performance and AI training performance by having the right network. And on the right, you see all of these things are adopted and available from the major server OEMs out there. And if you want to learn more about AI networking, what we have is a couple of additional webinars. The first one on top, it's much more about the DPUs and what -- how they work with NVIDIA AI systems and kind of what the functions are. And the other one, you can see power networks for AI clouds. This is more of a workshop. So it's going to be far more technical than what we've been talking about so far. And with that I'd like to thank you for your time, and I hope you enjoyed the webinar. And what we're going to do is open up for a few questions here. So thank you very much.