Advanced Micro Devices, Inc. (AMD) Earnings Call Transcript & Summary

Good morning. How is everyone this morning? All right. Big, big welcome to Advancing AI 2024. It is so great to be here in San Francisco with all of you. So many press, analysts, customers, partners, and lots of developers today. And welcome to everyone who's joining us online from around the world. It's been an incredibly busy year for AMD with lots of launches across new products in our PC and our Embedded portfolio. But today is a special day. Today is all about data center and AI. We have a lot of exciting news and products and so let's go ahead and get started. Now at AMD, we believe high-performance computing is the fundamental building block for the modern world. And we are really committed to pushing the envelope to help use technology to solve the world's most important challenges. Whether you're talking about the cloud or health care or industrial or automotive or comms or PCs and gaming, AMD products today are used by billions of people every day. And for sure, AI is the most exciting application of high-performance computing and drives the need for significantly more compute as we go forward. So let's talk a little bit about AI. Actually, I think we're going to talk a lot about AI today, if that's all right with you guys. Over the next decade, AI will enable so many new experiences that will make computing an even more essential part of our lives. If you think about it, AI can help save lives by accelerating medical discoveries. It can revolutionize research. It can create smarter and more efficient cities. It can enable much more resilient supply chains. And it can really enhance productivity across virtually every single industry. And our goal at AMD is to make AMD the end-to-end AI leader, and to do that, we have 4 big themes. First, it's about delivering the best high-performance energy-efficient compute engines for AI training and inference, and that's including CPUs, GPUs, and NPUs. And you're going to hear me say today, there is no one size fits all when it comes to computing. The second is really to create an open, proven and developer-friendly software platform, and that's why we're so excited to have so many developers joining us here today. And it's really about enabling leading AI frameworks and libraries and models so that people can use the technology and really co-innovate together. The third piece of AMD's strategy, and you're going to see a lot of our partners and customers, whether on stage today or throughout the show, it's about co-innovation. Like there's no one company that has every answer. You actually need the entire industry to come together. And so for us, this partnership is about including the entire ecosystem, including the cloud, OEM, software, new AI companies. And our goal is to drive an open industry standard AI ecosystem so that everyone can add their innovation on top. And fourth, we also want to provide all the pieces needed for our customers to deliver their total solutions, and that includes not only at the chip level, but really at the rack, cluster and data center level. So when you put all that together, we are really committed to driving open innovation at scale. And on the silicon side, and you guys know that we spend a lot of time on the hardware, this means driving the bleeding edge of performance in CPUs, GPUs, and high-performance networking but all to new industry standards. On the software side, we're going to talk a lot about software today. It's about enabling the industry's highest-performance open source AI software stack. And with our acquisition of ZT Systems, we're going to bring all of those elements together to really offer a complete road map of AI solutions. So when you look at what business leaders are talking about, when I talk to business leaders today, everyone is focused on: number one, how do I use AI as fast as possible; but number two, how do we maximize the impact and ROI for their AI initiatives? Now when we think about AI, it really is about choosing the right compute for the right application. And looking across the portfolio, we see a lot of different pieces. Starting first on the CPU side, we have lots of opportunity to really think about, today's AI is really about CPU capability. And you see that in data analytics and a lot of those types of applications. So for instance, in enterprise, predictive analytics are actually used very often. But you also see in generative AI applications, you actually rely on the GPU. So we're starting to see a lot of conversation about agentic AI and these new workloads. And these are the idea that LLMs can actually be tuned to automate very difficult tasks and actually reason and help us make decisions by using natural language. So when you look at agentic AI, you actually require significantly more general purpose compute as well as AI compute. So you see CPUs handling things in the data pipelines, and then you see GPUs for training, fine-tuning and inference. So when you look at all of these compute needs, we certainly have the best portfolio in the industry to address end-to-end AI. So a little bit about our portfolio. We've built our leadership data center compute portfolio over multiple generations, starting with our EPYC CPUs. Since launching in 2017, EPYC has become the CPU of choice for the modern data center. The largest cloud providers offer more than 950 EPYC instances and have deployed EPYC widely throughout their infrastructure on their most important services, things like Office 365, Facebook, Salesforce, SAP, Zoom, Netflix, and many more. And on the enterprise side, numerous large customers have also deployed EPYC on-prem to power their most important workloads. Today, you're going to hear from all of our server -- our largest server OEMs and they provide over 350 EPYC platforms. If you put all that together, we're very proud to say we exited the second quarter at a record 34% revenue share. On the AI side, we launched MI300 less than a year ago to very strong demand, as the large infrastructure providers like Microsoft and Meta deployed MI300 to power their most important AI applications. Instinct platforms are now available from every major OEM, and numerous cloud providers have also launched public instances, making it easier than ever to use AMD Instinct. Now customer response has been very, very positive, and you're going to hear from some of those leaders today, leaders like Cohere, Essential, Luma, Fireworks, Databricks and many others have been able to bring their AI workloads to MI300 very quickly, and you will hear at leadership performance and TCO. So we've got a lot of new news today, so that includes CPUs, GPUs, DPUs, NICs and also enterprise Copilot+ PCs. So let's start first with the core of the data center computing, the CPU. I've spent a lot of time with CIOs recently and what everyone is thinking about is how do I modernize the data center? They want a CPU with leadership performance, efficiency, and TCO. With Turin, that is exactly what we've delivered. Today, I'm super excited to launch our 5th Gen EPYC portfolio. It all starts with our latest Zen 5 core. We designed Zen 5 to be the best in server workloads. And that means delivering an average of 17% higher IPC than Zen 4 and adding support for full AVX-512. Turin is fantastic. It features up to 150 billion transistors across 17 chiplets, scales up to 192 cores and 384 threads. And one of the things that's very special about 5th Gen EPYC is we actually thought about it from the architectural standpoint in terms of how do we build the industry's broadest portfolio of CPUs that both covers all of the new cloud workloads as well as all of the important enterprise workloads. And things like building 5th Gen EPYC CPU at 5 gigahertz, that was because there was a new workload. We were seeing the need when you think about industry-leading performance for AI head nodes, that frequency becomes really important. And that's an example of where we broadened the portfolio with Turin. Now how do we do that? Turin actually uses our industry-leading chiplet technology. This is an area where we have innovated ahead of everybody else in the industry. And that allows us to optimize for both enterprise scale-up as well as cloud-native scale-out workloads. What we do here is we use a consistent ISA and socket, which is really helpful for developers as well as our customers. And we maintain feature parity, including support for next-gen memory and I/O. We've also extended EPYC's confidential compute capabilities with the addition of trusted I/O that enables Turin to communicate securely with GPUs, NICs and storage. All right. I'm going to show you some pretty chips here. Here we have Turin. Now this version of Turin is actually 128 cores and it's optimized for scale-up workloads. It has 16 4-nanometer chiplets. So if you look outside of the ring, you see the 16 4-nanometer chiplets and a 6-nanometer I/O die in the center. And we've optimized this for the highest performance per core because it's extremely important in enterprise workloads. When you see that software is often licensed on a per core basis, you want the highest possible performance per core. Okay. Now this is also Turin. This is the 192-core version of Turin, and this guy is optimized for scale-out workloads. And here, we used 12 3-nanometer compute chiplets and it's the same 6-nanometer I/O die in the center. And this version of Turin is really optimized for cloud so applications that benefit from maximum compute per socket, this is what we do. Thank you very much. So with these 2 configurations, our 5th Gen EPYC portfolio is the broadest portfolio in the industry. It scales down to 8 cores at extremely high performance per core to up to 192 cores with extremely high performance per socket. And you can go across a wide range of TDPs, and what that does is it just enables customers to choose the best operating point for their specific needs. Now Turin is great but I also want to remind everyone, it actually builds on our strong track record of execution. We've built a 5th Gen EPYC CPU that runs at 5 gigahertz, delivering industry-leading performance for AI head nodes in many applications, but you also see that we've gotten up to highest core count 11x performance. So let's now take a look at Turin performance. We're going to compare many of the things in the next few slides to the competition's top of stack, Emerald Rapids. When you look at the competition's top of stack, a dual socket 4th Gen EPYC server is already 1.7x faster on SPECrate 2017. And with 5th Gen EPYC, the performance is fantastic. We extend that lead with 2.7x more performance. Now we know it's a very competitive space, and we fully expect as our competition launches their next-generation CPUs and they ship in volume, that Turin will continue to be the leader. For the enterprise space, there are many commercial software stacks that are licensed per core. And CIOs want to optimize the cost by running solutions on the fewest possible cores. When running these workloads on-prem, again, 4th Gen EPYC is already the performance leader. And with 5th Gen, we deliver 1.6x more performance per core than the competition. That's 60% more performance with no additional licensing cost. Now let's move to relational databases. Now these are also critical workloads for things like transaction processing and analytics, and MySQL is widely deployed in the enterprise and the cloud. This is another area where EPYC already offers leadership. And now with Turin, that performance capability increases to delivering 3.9x more than the competition. In video transcoding, you can also see again that 5th Gen EPYC significantly extends our lead, and we are now 4x faster than the competition. Supercomputing is also another one of those really important workloads, and it has been a place where EPYC has continued to lead. We're already the world's fastest CPU for complex modeling and simulation software. And with Turin, we extend that lead because of the Zen 5 IPC increases as well as our full implementation of AVX-512. And so we now deliver 3.9x more performance than the competition. And what that means is that for researchers who are really doing the most difficult simulations in the world, they get to their answers much faster if they're using AMD. Now enterprises are also running many, many more of their applications, their AI applications on their CPU. And this is another area where Turin delivers leadership, 3x faster performance on traditional machine learning and 3.8x better performance on TCP-XAI, which is actually an aggregate benchmark that represents end-to-end enterprise AI workloads. So when you put all this together, I want to give you the business reason why people are so excited about Turin. According to IDC, nearly 75% of enterprise customers refresh their server infrastructure every 3 to 5 years. Now if you look at a typical data center today that an enterprise may have, it might be running something like 1,000 Cascade Lake servers. That was the top of the stack in the industry from 4 years ago. 5th Gen EPYC can do that same amount of work of 1,000 servers in just 131 Turin servers. Now just think about what that means. It's like a huge benefit for CIOs. You can replace 7 legacy servers with 1 single EPYC server, and that significantly reduces the power that you need in your data center, it lowers TCO by more than 60%. And when you add on top of that the enterprise software licensing costs, that means an enterprise can break even on their investments in as little as 6 to 12 months. So that's good for CIOs and it's also really good for CFOs who want to optimize their CapEx spend. And on top of that, it also creates space for all of the additional compute that you need, whether you're talking about AI capacity or just adding more general-purpose compute capacity. So that's the benefit of the new technology. Now I really love talking about our technology but I love more having our partners talk about our technology. So to understand how EPYC delivers the most -- in the most demanding environments, let's welcome our first guest, a very close AMD partner who runs some of the world's largest and most advanced data centers. Please welcome Amin Vahdat from Google Cloud. Hello, Amin.

It's a pleasure, Lisa.

It is so great to have you here. I'm so excited. Google was really one of the first to adopt AMD at scale, and we have learned so much from our partnership with you. So can you tell us a little bit about our work together and all that you're doing in this gen AI era?

It's my real pleasure, Lisa. Thank you for inviting me to join you. Our partnership with AMD goes back a long way. We've been using EPYC CPUs to power our general-purpose instances in addition to our high-performance and confidential computing solutions. Just last year, we launched C3D, Google Cloud's 4th Gen EPYC-based offering for general-purpose workloads like web servers, databases and analytics. In this new era, C3D is a compelling option for many AI workloads. And frankly, the demand for ML compute power is insatiable. I see the rise of generative AI as truly the biggest transformation since the beginning of the Internet. We're on the cusp of a change that will redefine industries, amplify human potential, and create unprecedented opportunities. To me, that's what makes partnerships like ours so incredibly important. We need to constantly evolve our hardware and software architectures to respond to the demand we are seeing from our customers, whether it's raw performance, cost, convenience, or energy efficiency. Lisa?

I really look at this, when I think about the work that you're doing, Amin, as pretty amazing. And I know, look, beyond AI, we're also doing a lot of work on your internal workloads as well as some of the third-party workloads on EPYC. So can you just share a little bit more about what you're seeing and what customers are seeing?

Absolutely. We've been using EPYC CPUs for multiple generations to serve our Cloud customers and our internal properties at Google. That adoption has been driven in large part by the gen-over-gen performance and efficiency gains we've delivered together. For example, Snap used EPYC-based virtual machines on Google Cloud to reduce their AI inferencing costs by 40% and improved their performance by 15%. Those are some big numbers. Thirteen months after the introduction of C3D, KeyBanc, one of the largest banks in the U.S., is seeing cost efficiencies modernizing to C3D. Striveworks and Neural Magic are running inference workloads on CPUs, saving money without sacrificing speed. And that's just to name a few. Thanks to our collaboration, C3D has been one of our most successful VM instance launches to date with up to 45% faster performance than previous generations.

I love hearing those numbers, Amin. It's wonderful to see how great C3D is doing. Now Google is totally leading in AI innovation, and I know you're doing so much. Can you talk a little bit more about sort of your vision for what's happening in AI and the role EPYC plays?

It's going to take innovation on every single front. Releasing new Gemini models, software, CPUs, GPUs, and TPUs. But it's bigger than that. We're rethinking our system and infrastructure designs from the ground up. That's where our AI Hypercomputer comes in. It's a supercomputing architecture that's designed to combine performance-optimized hardware, open software, leading ML frameworks and flexible consumption models to maximize the return on AI investments. EPYC CPUs are an important part of that stack, offering a cost-effective and seamless option for AI workloads. Sustainability is also a key part of Google's strategy during this transformation. And so I really appreciate AMD's focus on delivering excellent performance per watt, an increasingly critical metric for all of us.

That's wonderful. Look, we totally agree, Amin, and it's great to see all the innovation that you're bringing to the market. Now we're super excited about launching Turin today, and it's another area where, frankly, our teams have been partnering closely, and you've given us just fantastic feedback. Can you tell us a little bit about your plans?

Thank you, Lisa. Turin's a beautiful chip and we're really looking forward to the continued collaboration. It was a pleasure joining you today, and congratulations on all the success. We are looking forward to partnering with you to deliver Turin-based VMs early next year.

That's fantastic. Wonderful. Thank you so much, Amin. Super exciting. Now let's turn to data center GPUs and our next-generation Instinct accelerators. Last December, we estimated the data center AI accelerator market would grow from $45 billion in 2023 to more than $400 billion in 2027. And at the time, I remember people asking me, "That seems like a really big number. Is that true?" Since then, AI demand has actually continued to take off and actually exceed expectations. It's clear that the rate of investment is continuing to grow everywhere, driven by more powerful models, larger models, new use cases, and actually just wider adoption of AI use cases. So now as we look out over the next 4 years, we now expect the data center AI accelerator TAM will grow at more than 60% annually to $500 billion in 2028. For AMD, this represents just a huge growth opportunity. We launched our MI300 family last December, and we had leadership performance and very strong customer demand. And I'm very happy to say that the ramp has gone just extremely well. In the last 10 months, we've been laser-focused on ensuring our customers get up and running as fast as possible with maximum performance right out of the box. And to do that, we've had to significantly drive improvements and continuous improvements across our ROCm software stack. We've integrated new features. We've optimized new libraries. We've enabled new frameworks, and we've significantly expanded the third-party ecosystem that supports ROCm. As a result, if you look today at MI300x performance, we have more than doubled our inferencing performance and significantly improved our training performance on the most popular models. Today, over 1 million models run seamlessly out of the box on Instinct, and that's more than 3x the number when we launched in December. And we recently also completed the acquisition of Silo AI, which adds a world-class team with tremendous experience training and optimizing LLMs, and also delivering customer-specific AI solutions, again, to help our cloud and enterprise customers get to market as fast as possible. The improvements we've made with ROCm are enabling our customers to see great performance with MI300. Now let me just show you a little bit about what customers are seeing. We've now worked with many customers across a wide range of workloads. And our experience has shown that we run out of the box actually very well, so most things run quite well. But with just a little bit of tuning, MI300X consistently outperforms the competition, which is H100 in inferencing. So for example, using Llama 3.1 405B, which is one of the most newest and demanding models out there, MI300 outperforms H100 with the latest optimizations by up to 30% across a wide variety of use cases. And we've seen this across a lot of different customer workloads using many different models, including Mistral, Stable Diffusion and many others. So this allows our customers to really leverage the performance advantage to build their own leadership AI solutions, and you're going to hear some of those key use cases a little bit later on. Now we're always pushing the limits on performance so let me show you what's next. Thank you. Today, I'm very excited to launch MI325X, our next-generation Instinct accelerator with leadership generative AI performance. MI325 again leads the industry with 256 gigabytes of ultrafast HBM3E memory and 6 terabytes per second of bandwidth. When you look at MI325, we offer 1.8x more memory, 1.3x more memory bandwidth, 1.3x more AI performance in both FP16 and FP8 compared to the competition. And when you look at that across some of the key models, we're delivering between 20% and 40% better inference performance and latency on things like Llama, Mistral, and Mixtral. And importantly, one of the things that we wanted to do was really keep a common infrastructure. So MI325 leverages the same industry standard OCP-compliant platform design that we used on MI300. And what it does is it makes it very easy for our customers and partners to bring solutions to market. Now when you look at the overall platform with 8 GPUs, we deliver significantly more AI compute and memory as well. The 8-GPU version features 2 terabytes of HBM3E memory and 48 terabytes per second of aggregate memory bandwidth, enabling our customers to run more models as well as larger models on a single MI325 server. When you put all of that capability together, what you see is that MI325 platform delivers up to 40% more inferencing performance than the H200 on Llama 3.1. And a lot of people also are doing training. And when you look at training performance, we've made very significant progress on optimizing our software stack for training on a growing number of customers. And what you see with MI325, we have excellent training performance that's very competitive compared to the competition. So as you can see, we're very excited about MI325. The customer and partner industry has -- interest has been fantastic. We are engaged across all of the leading OEMs and ODMs. We're on track to ship production later this quarter with widespread system availability from Dell, HPE, Lenovo, Super Micro and many other providers starting in Q1. All right. Now let me invite my next guest to the stage. Oracle Cloud is one of our most strategic cloud partners. They've deployed AMD everywhere, including CPUs, GPUs, DPUs across their infrastructure. And to talk more about our work together, please welcome Senior Vice President, Oracle Cloud Infrastructure, Karan Batta. Karan, so wonderful to see you again. Thank you for being such a great partner. And actually, you were also at our December event, so thank you. You talked a lot about how OCI is adopting EPYC across your platforms and services. Tell us a little bit about what's been happening.

Yes. Again, thank you. Thank you for inviting us again. It's very exciting to be here since last December. AMD and Oracle have been working together for a very long time. Since the inception of OCI in 2016, AMD EPYC CPUs are deployed across 162 data centers across the globe that covers our public cloud regions, our gov regions, our secret regions, even our dedicated regions in alloy as well. And we've had tremendous success on our compute platform offering bare-metal instances, virtual machines on Genoa-based E5 instances. And then also, we also use, at the base layer of our platform, we also use Pensando DPUs so that we can offload that logic so we can give customers the ability to get great performance instances.

Look, we love the work that we do together and it's not just about the technology, but it's also about what we're doing with customers. I know that you're very active with Turin. Can you talk just a little bit about some of that?

Absolutely. One of our largest customers today, cloud-native customers is Uber. They are using E5 Genoa instances today to actually get a lot of performance, efficiency. And they've moved almost all of their trip-serving infrastructure on top of AMD running an OCI compute. So that's been incredible.

That sounds pretty good.

We also have Red Bull Powertrains that's developing the next generation of F1 engines for the upcoming seasons. And then additionally, on top of that, we have our database franchise, which is now powered by AMD CPUs. And customers like PayPal and Banco do Brasil are using Exadata powered by AMD to achieve great things for their database portfolio. So that's been incredible.

We are super proud of the work we're doing together on Exadata. I think that's just an example of how the partnership has grown over time. So look, the momentum with customers is wonderful. We love the work that we're doing on compute. There's just a little bit of something called AI right now. So let's switch gears to talk a little bit about AI. You just recently launched your MI300 instances publicly. Can you talk a little bit about that?

Yes. It's been a great collaboration between the 2 teams. We recently made generally available MI300X. We've had incredible reception internally, externally. And we're working with customers like Databricks and Fireworks and Luma AI to run incredible inferencing workloads on top of the AMD GPUs. Additionally, on top, we have seen incredible levels of performance for inference for running things like Llama 3.1 405B. And so we're seeing great efficiency and performance on top of AMD GPUs. And we're incredibly excited about the road map that you've announced, and we're excited to work together on the future of that road map.

Yes. Look, it's really, really cool to see what customers are doing. You talked a little bit about the road map. You talked about the importance of partnership. So what's next on the horizon?

I mean, first and foremost, we're very excited about Turin. We're going to be working together to launch E6 instances on OCI compute later next year, so very excited about that. So that's our compute family. We'll continue to collaborate on the GPU, scale up the capacity for MI300X for our customers across the globe, across all types of regions. And then again, we will continue to collaborate on the DPU architecture as well with you guys.

That's fantastic. Karan, thank you so much and thank you for the partnership.

Congratulations. Thank you.

That's great to hear. Now let me bring another partner to the stage now to talk about our partnership from maybe a different angle, from more of a user angle. Please join me in welcoming Naveen Rao from Databricks to the stage. How are you, Naveen?

Great.

Thank you so much for spending some time with us today. It's been a pleasure working with you and your team. I'd love for you to share just a little bit about Databricks and what you guys do.

Yes, absolutely. We're pioneering what we call a data intelligence platform, which combines the best elements of data lakes and data warehouses. The platform enables organizations to unify their data analytics and AI workloads on a single platform. A crucial aspect of our mission is to democratize data and AI. This democratization is driving innovation across sectors, enabling companies to make data-driven decisions and create AI-powered solutions. Our team at Mosaic, which is now part of Databricks, has been pushing the boundaries of what's possible with AI. And we're not just developing models, we're actually creating entire ecosystems that make AI more accessible, efficient, and powerful for businesses across various industries.

Look, I think you guys are doing amazing work. We completely agree that our goal is to make AI as broadly accessible as possible, and that's why partnerships are so important. You've also been very active in using our technology. Can you share a little bit about MI300 and what you've been seeing?

Yes, we've been on this journey for a little while with you, and our collaboration has been exceptional. We've achieved remarkable results, particularly with large language models. The large memory capacity and incredible compute capabilities of MI300X have been key to achieve over 50% increase in performance on some of our critical workloads.

That's a pretty good number, right?

We'll take it. And that includes things like Llama and other proprietary models that we're working on. The MI300 GPUs are proving to be a powerhouse in the AI computation and efficiency area. And we're excited about the possibilities that opens up for our customers going forward.

Yes. Well, thank you, Naveen. Look, it's great to hear about the performance on MI300. Now you've also been very active in giving us feedback on the ROCm stack, and we so appreciate it. ROCm plays such an important role in helping people use MI300 and our MI road map. You have a lot of AI expertise as well. Can you talk a little bit about ROCm and what you've seen?

Absolutely, yes. We've been working with ROCm since the Mosaic ML days, which was even late 2022, I believe, on Instinct MI250 and have published those results on the ease of transition from other platforms using ROCm and even scaling across many GPU. So we observed very closely that ROCm's capabilities have expanded significantly in the last year, and now it supports a wide range of features and functions for AI workloads, and the performance improvements have been substantial. Many of our models and workflows that were originally developed for other environments can now run seamlessly on AMD hardware with no modification. Working with AMD, the AMD team has been an absolute pleasure. Together, we're optimizing at multiple levels of the software stack and for AMD GPUs, which translates to better performance and efficiency for our customers.

I liked what you said, no modification. Did you say that?

I did.

You can say that again.

No modification.

Look, Naveen, we are actually thrilled with what our teams have been able to accomplish. I mean, we love working with teams like yours because it's like a very, very fast iteration and innovation cycle. What's next on the horizon?

Yes. I'm incredibly excited about the future actually. We've done a lot with MI300, but that's really just the beginning. We're looking forward to the continued optimization efforts, not just for the 300X but also for the new 325X and the upcoming MI350 series. And we're excited by the compute and memory uplifts we're seeing with these products, especially in new things like FP4, FP6 data types with the MI350. So on the Databricks side, we're working on new models and techniques that will take full advantage of these hardware advancements to further improve training and inference efficiency and making advanced AI capabilities more accessible to more organizations. The combination of AMD's cutting-edge hardware and software innovation is helping to democratize AI and make it more powerful, efficient and accessible. So we're not just pushing the boundaries of what's possible with AI, we're working to ensure that these advancements are practically and responsibly applied to solve real-world problems.

That's fantastic. Look, Naveen, thank you so much again for joining us today. Thank you for the partnership. And we look forward to seeing all the great things you and your team are going to be doing.

Thank you.

Thank you. So that gives you a little bit of a flavor of how users are seeing our Instinct road map and our ROCm software stack. Now let's turn to the road map. As we announced in June, we have accelerated and expanded our road map to deliver an annual cadence of Instinct GPUs. Today, I'm very excited to give you a preview of our next-generation MI350 series. The MI350 series introduces our new CDNA 4 architecture. It features up to 288 gigabytes of HBM3E memory and adds support for new FP4 and FP6 data types. And again, what we're thinking about is how can we get this technology to market the fastest? It actually also drops into the same infrastructure as MI300 and MI325 and brings the biggest generational leap in AI performance in our history when it launches in the second half of 2025. Looking at the performance, CDNA 4 delivers over 7x more AI compute, and as we said, increases both memory capacity and memory bandwidth. And we've actually designed it for higher efficiency, reducing things like networking overhead so that we can increase overall system performance. In total, CDNA 4 will deliver a significant 35x generational increase in AI performance compared to CDNA 3. We continue our memory capacity and bandwidth leadership. We deliver more AI flops across multiple data types. When you look at the MI350 series, the first product in that series is called MI355X, and it delivers 80% more FP16 and FP8 performance and 9.2 petaflops of compute for FP6 and FP4. And when you look at the overall road map, we are absolutely committed to continue pushing the envelope. And we are already deep in development on our MI400 series, which is based on our CDNA Next architecture that is planned for 2026. Now Microsoft has been one of our deepest and most strategic partners across our business, and it's played an incredibly important role in shaping our road map. I recently sat down with Chairman and CEO, Satya Nadella, to talk about our collaboration. Let's take a look. [Presentation]

Satya, thank you so much for being here and thank you for being part of our Advancing AI event today.

No. Thank you so much, Lisa. It's a real honor and pleasure to be with you.

So Satya, AI is transforming our industry, and Microsoft has truly been leading the way. Can you just tell us a little bit about where are we in the AI cycle? What are you most excited about? Where are you seeing the adoption?

First of all, it's always exciting, Lisa, for both of us and all of the folks. I mean, there's a new platform being born, right? Because in some sense, I'd like to say this is probably a golden age again for systems, right, because of all the innovation that you are doing, the system software innovation and, of course, the application innovation in AI. And behind it, these things that all of us now call the scaling laws. It's very much like Moore's Law. Now you have these scaling laws that are really creating, I would say, abundance of compute power. And in fact, it's interesting to think about it, right? It's the combination of silicon innovation, systems software innovation, algorithmic innovation and even sort of good ways to sort of synthesize data are leading to perhaps 100x improvements for every 10x increase in compute power. So there's clearly something afoot. It's a super exciting thing and it's not long -- I mean, I've never seen, Lisa, this rate of diffusion ever before, right, which is having lived through both PC client server, web Internet, mobile cloud, I would say, one, it builds on all of those previous things. And so therefore, the rate of diffusion of this throughout the world is pretty exciting to see..

Yes. No, I think you're absolutely right, Satya. It's been incredible, the amount of innovation that's happening in the industry. And frankly, it's been incredible the amount of industry -- the amount that Microsoft has brought to the industry in terms of getting AI innovation out there. I know that we're personally using many of your AI tools at AMD. Now we are so excited about the partnership that we have together. We've been long-standing partners across all aspects of our business and your business. Can you talk a little bit about the partnership and especially our work in data center and AI infrastructure has been really accelerating?

No, absolutely. I mean, to your point about the long-standing partnership, there is not a part of Microsoft that we're not partnering with you, right? When I look back and think about it, right, we are inventing a complete new PC category with you yet again. We historically always worked with you with -- comes to our gaming consoles. We started working -- in fact, you and I first started working together when both of us were not CEOs, when we started really doing the cloud work. And so we made progress. And then in the last 4 years, in fact, it feels like it's been 4 years since we even started on really adopting your AI innovation for our AI cloud as I think of it. And I think the interesting thing there is not only the silicon pieces that you brought but even the software work that you've done, how -- and because at some level, it's that close feedback loop between emerging workloads, right? In this case, these emerging workloads, which are these training and inference workloads aren't like anything we have seen in the past, right? These are synchronous data parallel workloads that require a very different way to think about the software stack and the silicon stack. And the jointly optimizing it here, we now have in our fleet, MI300. It's sort of -- we did all the work to even benchmark the latest GPT models. We are seeing some fantastic results. We have customers coming, now have choice in the fleet to be able to sort of really optimize for different considerations people have, latency, COGS, performance. So it's fantastic to see the progress the 2 teams have made. And I know it's been a lot of hard work. And that's what it takes, which is to be able to sort of see the new workload and optimize every layer of the stack.

Yes, absolutely, Satya. First of all, I have to say, we are so proud of the work that we've done together on MI300, getting it into Azure. It was absolutely hard work, as you said, but a huge thank you to your engineering teams, hardware, and software. I know that our teams couldn't be closer in how we've really brought that together. So let's talk a little bit about the future. I mean, the thing about AI is, I always say we're just at the beginning of what we can imagine with AI. And it requires an incredible data center infrastructure, a vision for that and really, optimization on all levels. I think one of the things that's most unique about our partnership is we've talked about bringing the best of each other to really form that vertically-integrated stack. So can you talk a little bit about the road map? We're excited. At this event, we're actually talking about our accelerated road map with MI350 coming next year, MI400 series coming in 2026. Much of that has been work that we've done together. So yes, can you talk a little bit about that?

First of all, we're very, very excited about your road map. Because at the end of the day, if I sort of get back to the core, what we have to deliver is performance per dollar per watt, because I think that's the constraint, right, which is if you really want to create abundance, right, where the cost per million tokens keeps coming down so that people can really go use what is essentially a commodity input to create higher value output, right? Because ultimately, we'll all be tested by one thing and one thing alone, which is their world GDP growth being inflected up because of all this innovation. And in order to make that happen, we have to just be mindful of the one metric that matters, right, which is this performance per dollar per watt. And in that context, I think there is so many parameters, right, which we think about what you are all doing. What's the accelerator look like for all of this? What's its memory access bandwidth? How should we think about the network, right? So that's a hardware and system software problem. So that's something that collaborating together to create, I think, the next set of breakthroughs, which create -- for every 10x, we actually get 100x benefit. That, I think, is the goal. And I mean, very excited to see how the teams are coming together, whether it's OpenAI, Microsoft, AMD, all working together and saying, how can we accelerate the benefits such that this can diffuse even faster than what we have? So we are looking forward to your road map in 350 and then the next generation after that. And the good news here is the overall change has already started and we build on it. And the fact that now all of our workloads will get continuously optimized around some of your innovation, that's the feedback loop that we've been waiting for.

Thank you, Satya. We are so proud of the deep partnership we have built with Microsoft. And as you heard from Satya, we see even larger opportunities ahead to jointly optimize our hardware and software road maps. Now Meta is another very strategic partner who we are collaborating with across CPUs, GPUs, and the broad AR ecosystem. They've developed EPYC and Instinct -- they've used EPYC and Instinct broadly across their compute infrastructure and share our view that open standards are extremely important. To hear more about that, please welcome Meta VP of Infrastructure and Engineering, Kevin Salvadori, to the stage. Hello, hello.

Hi, Lisa. How are you?

Kevin, thank you so much for joining us today. Thank you for the incredible partnership we've built. We are actually so honored to be part of Meta's infrastructure. Can you tell us a little bit about our partnership and how that's evolved?

Sure. Well, we first started partnering together back in 2019, but things really took off in 2020 when we started to design in the Milan CPU into our server fleet to support our planet-level infrastructure perhaps. So supporting Instagram, Messenger, Facebook, WhatsApp, that's when it really kicked off. Subsequently, our collaboration of advanced compute infrastructure has enabled us to scale our AI deployments, really meeting what's the seemingly insatiable demand for AI services. And Genoa and Turin have been essential for us to optimize our workloads, and we're really excited to now be pairing AMD's compute with AMD's MI accelerators that really help us innovate at scale. So we really see our partnership with AMD as essential for us to scale AI going forward.

No, it's absolutely fantastic, Kevin. Thank you. I think one of the things that I've been super excited about is with each generation of technology, Meta has expanded your deployments. So can you talk a little bit about sort of what drove those decisions and where we are today?

Sure. Sure, I can. You're right. With every EPYC generation, we've continued to expand our deployments. And when you serve over 3 billion people every day, which is what we do, performance, reliability, and TCO matter. And you know we're a demanding customer but...

Just a little demanding.

Just a little demanding. But simply put, you and your team at AMD have continued to deliver for us. So last year, we announced Meta's at-scale refresh with Bergamo, driven by a 2.5x performance uplift, higher rack density and energy efficiency. And that all drove a better TCO for us. And I'm happy to announce to everybody something you already know, that we've deployed over 1.5 million EPYC CPUs in Meta's global services.

I like the sound of that. That's something we call at-scale deployment.

That's serious scale.

Look, we are also like so excited about our AI work together. One of the things I've been incredibly impressed by is just how fast you've adopted and ramped MI300 for your production workloads. Can you tell us more about how you're using MI300?

I can. So as you know, we like to move fast at Meta, and the deep collaboration between our teams from top to bottom, combined with really a rigorous optimization of our workloads has enabled us to get MI300 qualified and deployed into production very, very quickly. And the collective team works to go through whatever challenge came up along the way. It's just been amazing to see how the teams worked really well together. And MI300X in production has been really instrumental in helping us scale our AI infrastructure, particularly powering inference with very high efficiency. And as you know, we're super excited about Llama and its growth, particularly in July when we launched Llama 405B, the first frontier-level open source AI model with 405 billion parameters. And all Meta live traffic has been served using MI300X exclusively due to its large memory capacity and TCO advantage. I mean, it's been a great partnership. And based on that success, we're continuing to find new areas where Instinct can offer competitive TCO for us. So we're already working on several training workloads. And what we love is culturally, we're really aligned around -- from a software perspective, around PyTorch, Triton, and our Llama models, which has been really key for our engineers to land the products and services we want in production quickly, and it's just been great to see.

I really have to say, Kevin, when I think about Meta, I mean, we do so much on the day-to-day trying to ensure that the infrastructure is good. But one of the things I like to say is you guys are really good at providing feedback, and I think we're pretty good at maybe listening to some of that feedback. But look, we're talking about road map today. Meta's had substantial input to our Instinct road map, and I think that's so necessary when you're talking about all of the innovation on hardware and software. Can you share a little bit about that work?

Sure, sure. Well, the problems we're trying to solve as we scale and these new AI experience, they're really difficult problems to solve. And it only makes sense for us to work together on what those problems are and kind of align on what you can build into future products. And what we love is we're doing that across the full stack, from silicon to systems and hardware to software, to applications, from top to bottom. And we've really appreciated the deep engagement of your team, and you guys do listen and we love that. And what that means is we're pretty excited. The Instinct road map is going to address more and more use cases and really continue to enhance performance and efficiency as we go forward and scale. And we're already collaborating together on MI350 and the MI400 Series platforms. And we think that's ultimately going to be to AMD building better products. And for Meta, it helps us continue to deliver industry-leading AI experiences for the world. So we're really excited about that.

Kevin, thank you so much for your partnership. Thank you to your teams for all the hard work that we're doing together. And we look forward to doing a lot more together in the future.

Yes. Thank you, Lisa.

Thank you, thank you. All right. Wonderful. Look, I hope you've heard a little bit from our customers and partners as to how we really like to bring co-innovation together. Because, yes, it's about our road map, but it's also about how we work together to really optimize across the stack. So as important as hardware is and we know that software is absolutely critical to enable performance AI solutions for our customers, so to talk more about the progress, and we've made just fantastic progress over the last year on ROCm in our broader software ecosystem. Please welcome SVP of AI, Vamsi Boppana to the stage.

Thank you, Lisa, and good morning, everyone. As you just heard, AMD platforms are powering some of the most important AI workloads on the planet. So today, I'm excited to tell you about the tremendous progress we are making with ROCm, our AI software stack that's making all of this possible. Two years ago, when we laid out our pervasive AI strategy, we made open software a core pillar underpinning that strategy. We said we would partner deeply with the community and create an open ecosystem that is able to provide a credible alternative for delivering AI innovation at scale. And today, we are there. AI innovators from the largest corporations to exciting startups are delivering their most demanding workloads on our platforms. ROCm is a complete set of libraries, run time, compilers and tools needed to develop and deploy AI workloads. We architected ROCm to be modular and open sourced it to enable rapid contribution by AI communities. It is designed to connect easily to ecosystem components, frameworks like PyTorch, model hubs like Hugging Face. And over the last year, we've expanded functionality in ROCm at all layers of the stack at the lower layers, from coverage, for platforms, operating systems, to higher layers of the stack where we have expanded support for new frameworks like JAKs. We've implemented powerful new features, algorithms and optimizations to deliver the best performance for generative AI workloads. I am so proud of what our teams have accomplished this year. ROCm really delivers for AI developers. We've also been partnering very closely with the open source community. Our deep partnership with PyTorch continues with over 200,000 tests that run nightly in an automated fashion. Our CI/CD pipelines ensure that when developers anywhere in the world commit core to PyTorch, it gets automatically checked, that it works well with AMD platforms. That's what has enabled us to ship with day 0 support for PyTorch. And we have expanded support for key frameworks with significant work on JAKs this year, ensuring robust functionality and support for MaxText. Work on Megatron-LM has also been super crucial for us for our expanding training engagements. Now vLLM has rapidly emerged as the open source inference library of choice in our industry. We are delighted with our close collaboration with the UC Berkeley team and the open source community behind vLLM, that's been crucial for delivering the best inference solutions for our customers. Hardware-agnostic languages and compilers like Triton are strategically important for our industry. Triton offers a higher level of programming abstraction with increased productivity and still delivers excellent performance. Last year, we announced that Triton supports AMD GPUs, and we delivered on that promise. And we've continued our close collaboration with the Triton team to ensure there's expanded functional coverage and excellent performance coming out of Triton for AMD GPUs. We've continued to add coverage for emerging frameworks and technologies, and I'm delighted to share today that SGLang, which is an emerging inference serving framework now offers AMD GPU support. In fact, I am delighted that the creators of all of these key open source technologies, Triton, vLLM, SGLang and many more are all here speaking at our developer event. And all this great work is resulting in great support for AI workloads and models on AMD platforms. Hugging Face is the largest and most important model hub in our industry. We announced our collaboration with them in June last year with the goal that any model that's on Hugging Face should run on AMD. And today, I'm delighted to say that over 1 million Hugging Face models now run on AMD. This has been made possible by our close collaboration over the last year, an effort that ensures that all their model architectures are validated on a nightly basis. And it's not just about the number of models. We've done extensive work to ensure that the most important models are supported on day 0. For example, when Llama 3.1 model came out, those ran on day 0 on AMD. And perhaps even more importantly, several of our partners like Fireworks offered services immediately thereafter on AMD platforms. And as you just heard from Lisa, we delivered outstanding performance across a diverse set of workloads. We have been relentlessly focused on performance from the latest public models to the flagship proprietary models. With each ROCm release, we have delivered significant performance gains. Our latest release, ROCm 6.2 delivers 2.4x the performance for key inference workloads compared to our 6.0 release from last year. These gains have been made possible by a number of enhancements, improved attention algorithms, graph optimizations, compute libraries, framework optimizations and many, many more things. Similarly, ROCm 6.2 delivers over 1.8x improvement in training performance. And again, these gains have been made possible by improved attention algorithms like FlashAttention-3 that is supported, improved compute communication libraries, panelization strategies and framework optimizations. It is these huge performance gains that have been key to driving competitiveness and momentum for our Instinct GPUs. But look, model optimization is not the only requirement for AI. AI production often requires data processing, RAG, pipeline development and many, many more things. There's often significant effort that customers need to put in to realize the value of AI. To solve this last mile of customer AI needs, we acquired Silo AI earlier this year. Silo AI was Europe's largest private research lab and built a stellar reputation, helping customers implement over 200 production AI solutions in the past few years. With 300 AI experts, including 125 AI PhDs with deep, deep deployment experience, we are thrilled to be able to offer our customers now the ability to implement end-to-end AI solutions. This exceptional team has also been behind the development of some of the most important European open source language models. And I'm thrilled to share that those LLMs have been exclusively trained on AMD platforms. Now I've shared a lot about our software progress, but perhaps the best indicator of our progress is what AI leaders who are using our software and our GPUs are saying. So it gives me great pleasure to invite on stage 4 remarkable AI leaders whose work is at the cutting edge of AI, to stage. It's an honor to have them here to share their perspective on the future of AI. So please join me in welcoming these outstanding innovators. Dani Yogatama, CEO of Reka AI; Dima Dzhulgakov, CTO of Fireworks AI; Ashish Vaswani, CEO of Essential AI; Amit Jain, CEO of Luma AI. So great to have you all here with us today. Thank you.

Dani, let me start with you. You are an AI trailblazer, having worked on groundbreaking projects like DeepSpeech and AlphaStar. And you've actually seen the potential of multimodal AI before many others. Tell us a little bit about what you're up to at Reka and some of the exciting work that we have been doing together.

Yes, sounds great. Yes. Thanks for the intro, Vamsi. At Reka,we provide multimodal AI that can be deployed anywhere. Our models understand text, images, video and audio, addressing the needs of both consumers and enterprises for developing powerful authentic applications in the cloud, on premises, and on devices. We are really, really excited. Our models are optimized to run on AMD platforms from high-performance cloud GPUs to AI PCs.

That's awesome, Dani. Thanks. It's an exciting vision. Now Dima, you are one of the original leads that built PyTorch. You are also a co-creator of ONNX. You're very well known in the AI ecosystem. So tell us a little bit about Fireworks AI and how your open source contributions are shaping your work there.

Yes. Thanks, Vamsi. So yes, at Fireworks, we offer a platform for productionized and generative AI with key focus on inference speeds and cost efficiency. So we help companies ranging from startups such as Cursor to enterprises like Uber and DoorDash to basically productionize the latest and greatest open source models across text, image, video, and other modalities. And actually, I'm particularly excited to be here today because open source is the center of ROCm stack. And as PyTorch maintainer, I actually worked with AMD over the past 5 or 6 years to make ROCm the first-class citizen and first class back-end for PyTorch and optimize multiple layers of the stack for that. And actually, this foundation helped set Fireworks when we started [ 14 ] inference to ROCm. And of course, this year, we worked very closely with AMD to achieve the best in industry performance and leverage all the low level capabilities of MI300X.

That's great, Dima. It's been great partnering with you in the open source over the years and now at Fireworks. Let's go to Ashish next. Ashish, you're well known to all of us. You co-invented Transformers that led to the generative AI revolution here. Tell us a little bit about how Essential is planning to change the world of AI models and some of the work we have been doing together on MI300X.

Absolutely. And Vamsi, at Essential, we believe that in the future, there should be no obstacles to knowledge work. If you're an analyst or you're a scientist, if you want to do manual work in a computer for complex research, we want to build products that you can go to that will help you get unstuck. If you're under time pressure, if you're curious, to help you make progress, get answers and do your work much better, right? And today, we're relentlessly focused on building a tool for financial research. We're building a financial analyst that anybody can actually use to do their research and finance and where -- it's day 0 for knowledge work, and we're actually incredibly excited to build together with AMD with the MI300X and also the advancements that you've made on ROCm and also the entire training stack with JAX. We're a full-stack company so pretraining to post-training to model alignment, we're incredibly excited to work together with you on this.

That's awesome, Ashish. It's been great to partner with you on your training vision. Let's go to Amit. Amit, Luma turns a simple text prompt into something beautiful, right, stunning videos and 3D models. Tell us a little bit about Luma and the work that we've been doing together.

Awesome. Thanks for having me here. At Luma, we are tackling one of the hardest problems in generative AI, which is how do you model the world and create really high-quality, consistent, controllable intelligent videos? And we do that just from very simple text We have this platform called Dream Machine, and we have seen a great deal of traction from various industries like advertising, education, entertainment. But also we believe this has huge potential to change how people communicate and share their ideas in the world. Video is the medium through which most information travels in the world today, and we think we can actually bring a whole level of democratization to that. It just doesn't exist right now. And yes, we are very excited to push the boundaries of this very critical but also difficult AI research and product milestones with like the powerful GPUs that AMD has.

It's great. It's been wonderful partnering with you. So Dani, let's just go to you. It's been great to watch your models come to life on MI300X. Can you talk a little bit about your efforts? How was it getting on to our stack and what are you seeing?

Yes, thanks. We ported our models to the MI300X and even AI workstations in just a few days, including the full application stack. What's even more exciting for us is we were able to meet our performance goals within about 2 weeks. ROCm's open source platform really allow us to move fast, enabling seamless integration, adaptation from MI GPUs to AI workstations. It's been great.

Awesome. It's been great to watch you guys bring all these models to life so quickly. It's been absolute pleasure. Dima, can you talk a little bit about your experience? You serve pretty large models. How has it been on MI300x and how you're seeing performance, TCO?

Yes. So actually, I mean, frankly speaking, when we started integrating our inference stack with ROCm, we were like pleasantly surprised how easy it was. And since this success, actually we're able to offer Fireworks inference API running on AMD platforms hosted by Oracle Cloud on the day when Llama 3.1 was released.

Yes, that was amazing, yes.

And actually, I mean, a lot of sense to outstanding capabilities of AMD GPUs and particularly, their leadership in HBM capacity and. When you serve large language models, that comes really handy. For example, if you are trying to serve Llama 405B in original position with full context window, it allows you to still stay on a single server and simplify complexity quite a bit. And of course, prior to this Fireworks inference engine AMD GPUs can really demonstrate competitive performance compared to other GPUs and other accelerators. And overall, like this basically allows to do really efficient deployment of large-scale AI models in real-world production with higher speeds and lower costs.

That's awesome. This is exactly what our engineers dreamed off when they built MI300X. Ashish, it's been super exciting to watch your team train on MI300X. Can you share a little bit about how the scaling is going and your experience with ROCm?

Actually, I want to start by thanking the entire engineering team at AMD. It's been phenomenal. The advancement in such a short time has been absolutely incredible, right. So very excited. Yes, I wanted to start by saying thanks and it's been wonderful to partner. And we're seeing sort of per device best-in-class performance. The linear scaling characteristics are extremely exciting to scale to our large training workloads. And of course, the advancements all through the ROCm, the training stack and the blessing of being able to continue to use JAX has been painless, that experience has been painless. And just seeing how much has been done in such a short time, I think we're going to do something incredible together.

That's awesome. Ashish, when you say linear scaling, it's like music to our ears.

To ours as well and the team, some of which are here. Yes.