International Business Machines Corporation (IBM) Earnings Call Transcript & Summary

Ladies and gentlemen, the program is about to begin. At this time, it is my pleasure to turn the program over to your host, Wamsi Mohan. Please go ahead.

Yes. Thank you so much. Good morning, everyone. I'm delighted to welcome you to day 2 of BofA's Virtual AI Conference. Today, we have with us IBM. We're joined by Matt Hicks, who's the President and CEO of Red Hat. As a reminder, Red Hat was acquired by IBM back in July of 2019. It's a leader in enterprise open source software solutions and hybrid cloud for building and managing IT solutions. So Matt, thank you so much for taking the time to be here. We really appreciate it.

No. Thanks for having me, Wamsi.

Great to see you again. Well, let's start maybe big picture. Can you just, at the highest level, talk about IBM's AI strategy and how Red Hat fits in?

Yes, absolutely. So I'll start with Red Hat. I think it's the easiest piece there. We have 2 offerings. And first is RHEL AI. And our goal with RHEL AI is to bring together a model, the ability to put your data into a model and the ability to run it. It's sort of the smallest unit of making a GPU work for you in that space. And then if you're able to be successful, if I've trained the model and I can work it, OpenShift AI can expand that to a cluster. OpenShift AI, we sort of say like it brings the scale, like I cannot be successful with one model, I can be successful with 10 or 100 or a lot of volume. From that point, there are 2 really exciting paths for me. One is sort of depth to the infrastructure where we take this work. We drive this all the way down to Z and the efficiencies in Z. It's at the heart of a lot of businesses on it. So the Telum or Spyre processors, we want to make sure Z can participate in that really efficient running of the models that you build. But at the same point, if you've built 10 or 100 models, we know the next journey for enterprises is I need to bring these closer and closer to my business processes. I need to bring more data to train fit-for-purpose models. And that expansion brings you into the watsonx portfolio, like watsonx data to help you bring your enterprise data in or say, watsonx Orchestrate to actually take these models and integrate them in your business processes. So that's sort of the software infrastructure stack. The last component of this, which is really powerful is IBM Consulting because if you say I like this strategy, I like the software, but I don't have the skills right now to move as fast as I want. IBM Consulting, they know this stack. They have expertise. They have demonstrated. They can do this with like a US Open use case. So they're the accelerant in this, if you like this path and technology to help you get there faster. So that sort of is the AI. It is -- our goals with it is being open, being flexible, but being able to apply it to your business problems in enterprise as well.

Okay. Great. No, that's super helpful. So maybe just following up on that, right? Can you just talk about IBM's decision to open source its code base and the granite models that you have?

Yes. So being in Red Hat, I'm a bit biased to this because we're -- our development model, we use open source for this. But the reason we use open source is for innovation, and that extends to IBM's decision to open source their work around AI, both their large language models, Granite, as well as the techniques to be able to train them, which we call InstructLab. When we talk about innovation, for those that aren't familiar with open source, you can think of it as if you're able to bring expertise across the entire planet to contribute into something, we have seen over the past in operating systems and technologies like Kubernetes, you get the open source model right, the innovation potential outstrips any other model. If you're a company, no matter how big you are, you're limited by the talent you can hire. When you get open source right, you have no limits on it. You can integrate with university talent, with other competitors. But the way you get open source to work, you also have to be comfortable with the flexibility, which is why we picked the Apache license for that. Not everyone is going to want to build on your stack and give the software away. So allowing them to build proprietary solutions as well as contribute back things they want to maintain is usually a really key component. So that's what we looked at in both Granite and the InstructLab was open sourcing this to drive the best innovation model we could and then allowing flexibility so that you don't have to be open source yourself to participate with it and you have the flexibility in commercial use as well.

Right, right. Yes. No, I mean it's obviously worked extremely well for Red Hat over a long period of time. How does IBM plan to monetize this AI strategy then?

Yes. We talked about a few different layers. The monetization strategy is slightly different in each. So if we talk about the unit, the smallest thing, RHEL AI, RHEL AI is going to attach to GPUs that are shipped. If you look at RHEL in the early days, RHEL's job as an operating system was to give you the most bang for your buck out of the CPU that you bought. In the early days, it was Intel and AMD to run any applications and run them efficiently. When you look at RHEL AI, it's very simple. It's the same strategy, but on GPU processors. It brings that independence of -- it runs on NVIDIA, it runs on AMD, it runs on Gaudi with it. So you have that unit and our goal is install that layer, and it will be the most efficient way to get your models running and training on GPUs. You move up one layer to OpenShift AI, that's not GPU tied, that is server tied because now you're taking a bunch of those GPUs, you're making them act as one thing. Same model, which is if you are clustering, we want OpenShift AI to be the best bang for your buck in terms of that server attached model to giving you efficiency. When you go up past that, you are now into different models because it's business integration with it. So you have watsonx on other models in software. They're not directly GPU tied, but they can tie to business outcomes, user models on that. You obviously have infrastructure on Z. So that is sort of that stack there from GPU attach to cluster attach to business outcome attached across the software stack.

And when you think about the contribution from that, how do you see that kind of maybe stack rank over time because, I mean, obviously, the infrastructure needs to get built, but then I can see how applications kind of supersede that over time. So just curious how you think about it.

It's an interesting question. I don't think we know the answer at this point. But where we focus those when we look at RHEL AI, it is the -- how ubiquitous can we make this layer. This is why we've driven an OEM model. We've announced a partnership with Dell and with Lenovo to provide that reach. So you sort of have a common basis similar to RHEL in our experience with server attach. When you get to OpenShift as a platform company, there will always be other options, but we like the affinity of OpenShift AI and the apps that will consume those models in OpenShift. So that sort of moves up the tier. Now to your point there, once you have that, the infrastructure is built. And at some point, in the olden days, I would say this was like the middleware spend versus the infrastructure spend. I think we're going to see the same trajectory play out where the business integration and application will be incredibly valued. This is sort of where the watsonx plays on it. And that will sort of be the next-generation middleware in this space. And we'll continue to see infrastructure grow. We'll watch that attached to it. But all of these phases are pretty exciting. It's getting hybrid infrastructure out there, making customers effective with it and then driving this into their business processes and outcomes with a new form of middleware.

Yes. No, that's a great analogy. Well, maybe switching gears a little bit, right? What's on everyone's minds is really trying to understand what are people really using Gen AI for? And so it would be helpful to see what are you hearing from customers as it comes to adoption of Gen AI?

Yes. So disclaimers being, we focus on the enterprise space. So the trends, I will say, will not reflect what maybe we see consumers do, but what we see in enterprise. So the first trend, I think I am a technologist, but I think most people understand that this technology works because they see it on the consumer side. They see what frontier models can do. But then we also hear this trend of there's not a lot in production. We bucket those into sort of 3 challenges. There's a cost, a complexity and then flexibility challenge. So the cost is, I made this work on a frontier model, but I have tremendous volume to bring to this use case, and I can't pay a tremendous amount per query on it. It's Internet optimization, business process optimization. So cost is a barrier. Complexity in the -- the frontier models, they're trained on Internet data. When you get into enterprise business processes, almost by definition, it's not in the public domain. It's private to you as a company. Your ability to teach these models, your business, your knowledge is another blocker. It is very hard to train frontier, really large models. And the third is flexibility where not all use cases can be tethered to the cloud. And so if you can make it work in the cloud, but now you have to make it work in a car or a factory or a retail outlet, it's a limiter to that. And we're pretty passionate about we think small models, especially with optimization can solve the cost challenge. The InstructLab work we do is -- I describe it as we're trying to bring like put your knowledge in a model to mere mortals with it. We use fancy terms like synthetic data generation, but it's your content, can you train a small model, which we know works at this point. And then flexibility is GPU flexibility, other hardware to allow you to have hybrid options. That's where we're excited. So where I see enterprises stuck is they can make the technology work, but they usually hit one or more of these barriers. That's where we try to intercept them to see if a smaller model, unique training in this area and a hybrid approach can get them past that sort of like trough of disillusionment from the technology side.

Would you say 2025 becomes the year of enterprise AI?

I really think the -- at this point, we have good infrastructure options, both public cloud and private. We know the technology works. And to be honest, we know the technology works in smaller models. I think 2025 is that first year of real production wins for companies. I think that's critical in any technology adoption journey is the first wins in product are usually the hardest and then you see growth build off that. So I'm excited about next year because I think all the landscape is laid to be able to show customers this is possible even at that first model, first train, first win use case. So yes, I think 2025 will be an exciting year there.

Yes. No, looking forward to that, too. What would you say are some of the industries or companies that have been early adopters with Gen AI? Do you see any particular trends over there?

This is an interesting one because I think there are -- if you look at some industries, a big biotech as an example, they've been doing what we tend to call predictive AI or machine learning for a long time. So they're coming into this space with a depth of data structuring already done. They're going to be looking at like convolutional neural networks versus foundation models with precision on it. So there are a few industries that have already been doing this on the last rev of technology and just approach it differently. It's positive, but they have a depth beyond what most enterprises do. I would say the -- if you put biotech and [ industries ] like that to the side, most enterprises, the trend is, can I take my somewhat boring business processes. They're written in standard operating procedures or they're in people's heads. And can I automate an outcome with an AI model to help my teams get through more volume, get through it more accurately, reduce toil. But that is a very hard challenge on it because the Internet optimizations, they don't have the necessary value. When you get to business processes, they have a neat mix of large language models can read long documents well, but you need that training and precision to be able to automate it. That's where I think most companies are picking the business process, understanding if they have the documentation, understanding if they can tell it's right or wrong on the answer without a human looking at every single one of them and implementing that. That's where the bulk of customers that I work with, they're still in that early stage journey with it.

Right. So in terms of what would you say then are the most common use cases that are being explored? And how does that change as companies get a little more mature down this path around implementing AI?

Yes. I think there's a common set of use cases around support optimization. It varies by industry, but -- and we do this ourselves as well under our head. I have to interact with my customers, and I want to get them the solution to their problem as fast as possible. AI models are a great way to augment traditional support tiers in this -- from the typical chatbot, but also in what we call the needle in a haystack searches where these things can process a tremendous amount of information and find good matches for it. So support is a common use case. Then we get to the G&A world. HR processes are another area where there's tremendous benefit from being standardized, rules are complex around the globe, using these to help point to better process, better associate experience and allows HR to be more efficient with it. And then the third bucket, I would say, sort of falls in operations for it, where from procurement to looking at discounting rules, you have processes that are really specific to your company, and we have been on an automation journey for a decade plus on this. In business automation and RPA, this is just the next evolution of can we do these things better from pretty unstructured documentation. So those 3, I see come up all the time with it. There are others, but those are probably some of the most common.

What would you say in each of these cases, Matt, is kind of the ROI, right? I mean part of like what at least from an investment community perspective, folks worry about is there's a ton of investment being made, but what is exactly the ROI of this and how sustainable is it? And I'm sure you know pretty deep in the weeds how that is. But what can you share with us about maybe each of these different use cases that you just spoke about, like what kind of ROI people should expect? And does that actually improve over time?

Yes. I think, one, it will differ whether companies are growing or whether they're just optimizing in a flat market. But in general, the trend that we look at is the work we do, the cost of that query has to be lower than the outcome that I get from it. Because if you say in support, I'm going to answer a lot of customers' questions better, but I'm going to pay millions and millions of dollars in doing it. It might be more effective if you stay in your traditional support structure. It serves customers well today. It's more effective cost. This is why we're such big believers in the first premise of ours, which is small models and then also being able to run them in the most applicable thing to us because I think the best way to get a return on investment is to keep the unit costs extremely small in these areas. If you can be efficient at the questions you ask and push those down from dollars to pennies on it, then your return is going to be in areas like iTrack and Red Hat, which is basically toil optimization. So how many customers ask a question on our website and didn't put in a ticket and talk to a human. That's a very known cost for us, and I'm able to model the GPU cost of giving them better matches versus the cost of a person to go through that themselves. But I think at the core in any of these use cases, it is pushing the unit cost as small as you can get it. And in our world, that is a combination of starting with the smallest model possible. When you're working with small models, you have to train them on your knowledge or they won't be good -- they won't be precise enough and then basically being able to run those on the infrastructure that gives you the best advantage. It might be cloud, it might be private, it might be at the edge. That gives you the optionality on it. And I think that's where a lot of customers have been stuck to as they know it can work, but they don't know that the ROI is there. We're trying to bridge that to say, we can take the same premise where you know it works, and we can get the unit cost small enough that your return will be close to guarantee using this. So that's our goal.

Yes. Okay. That makes a lot of sense. You recently announced the intention to acquire Neural Magic. Can you just talk about the strategic rationale there and how it fits within your broader AI strategy?

Yes, absolutely. So there are a couple of key tenets of Neural Magic. So first, they're very comfortable. They work in an open source development model as well. So it's an easy strategic fit for us. We know the team. We know where they work on it. And they have a tremendous influence over impactful communities like the LLM. But if you think about this of where it fits in, I'll use the RHEL AI bucket for this. The model, we focus on Granite right now and our work with IBM Research. InstructLab was invented in IBM Research. We sort of refine and carry that out. And then we talk about you have to run these models with an efficiency that is your best unit cost possibility. That's where Neural Magic fits for us. We're able to do this today, but Neural Magic brings tremendous depth from being able to use technologies like sparsification to allow models to run on CPU architectures instead of just GPU. They're able to shrink models so that you can start with a 70 billion parameter model and shrink it to a 7 billion parameter model while losing very little. And then when we talk about just raw inference, that's asking the model a question, they can get the most out of an NVIDIA GPU or an AMD GPU or Gaudi 3 GPU. So that combination, that's where they fit in that stack. And RHEL AI is the central piece to OpenShift AI and watsonx on that for running. So really excited to have them on board. That's an exciting fit to the portfolio.

No, that's great. If we just go back a second to think about how you view the broader competitive landscape and maybe just like when you talk to customers, what is the most common, maybe like stack that you're seeing from an infrastructure layer up to application layers? And where do you think IBM has the most opportunity?

The stack, what I would say is the most common starting point, the only real commonality is a lot of customers start with frontier models. And when I say frontier models, think GPT-4 or Gemini or maybe Claude in this. These are the trillion parameter plus models on it. The stack above that of how they try to integrate it, one, it's pretty nascent. It's fragmented because they usually get stalled down that path for the cost of complexity of training or flexibility of -- I know the technology works, but I can't run it. To be honest, I think the best IBM opportunity is can we bridge customers from the -- I know this works. It actually works great in some domains. If I can be tethered to the cloud, if I have lower volumes, I can make it work. But being able to complement that when you're stuck for what options do you have, which small models do you pick, who can train them? You made the comment earlier of like, well, we need the infrastructure to be built. This is the missing piece of that infrastructure. I think if we're able to build that, so you have a fit for frontier models in some cases, but then you are also unlocked in your use of fit-for-purpose models commercially. Then I think the middleware journey on this and the power that watsonx brings to being able to integrate these with business processes, bringing data in, is a really strong advantage for IBM because they've built a great suite there. We know this works in companies like the NatWest example are US Open that have really adopted this at scale. But it's about building out enough infrastructure so that we can bring that advantage to bear. So I think that will be the focus over the next couple of years to go through that.

Maybe another adjacent question is just around semiconductors, right? Like IBM obviously has a long history there. And obviously, like you've optimized for Z, a lot of capabilities for running AI-based workloads. So I'm kind of curious as to how you think about the longer-term view of whether you would be using more third-party kind of chips? Would you continue down this road of having more custom silicon? How do you think about that? And does it matter?

Yes. No, I think it does. When we talk about optimizing unit prices and costs, infrastructure is a huge component of that. You see custom processors being built basically by all the clouds at this point, and I actually really get excited. You mentioned IBM has a long history. If you look at Albany and the semiconductor partnerships they've done dating back to like Samsung with memory to the options they have of NorthPole collaboration and AI processing going forward, especially when you get to edge in use cases where you can't run off sort of traditional hardware combinations you might run IT off. It is a great tool in IBM's toolbox to be able to bring that they've known how to do this for a long time. The nice affinity with Red Hat is because we already work closely with IBM on Z, we're going to have enablement for Telum and for Spyre and for NorthPole. And if you look at Red Hat's strategy of taking our operating system and pushing it to the edge, whether that is OpenShift running on the International Space Station or whether it's our in-vehicle operating system suite, there's a natural synergy there for customers that do invest in the semi side, to build their own automated infrastructure, knowing that they will have a software stack that can run on that, I think, is a really powerful combination. So that's earlier in the market, but I do think that's a relevant strategic asset to have, especially as things evolve.

Yes. Okay. That makes sense. And then as you think about just within the Red Hat framework, right, if you think about where the infrastructure is kind of maybe not just infrastructure, but where you're seeing workloads within public cloud versus on-premise? How would you say, has that -- I mean, the last kind of 15 years have been a big march towards public cloud, but I think that's decelerating fairly strongly now, at least in growth rate terms. I mean, obviously, we've gone from $0 to $400 billion of spend on public cloud in 15 years. So when we think about like how you are seeing workload movements because you have that visibility in some ways, how would you characterize at the high level, like what is actually happening with the use of multi-cloud architectures? And what kind of dynamics could you share over there?

Yes. To your point, there's been a hard march to public cloud. And I usually tell people, if I went back 5 years, 7 years, most customers would say, I'm not focused on hybrid because I'm going to get everything to one cloud. That's my goal with it. And that goal wasn't realized by a lot of customers, whether it was they continue to have on-prem infrastructure, whether it was other clouds develop new things that they had to participate in, so now they were multi-cloud or they're big, and they acquired companies that had other footprints. I would say that trend plus sovereign cloud, geopolitics, there are pressures to where you have to be comfortable with hybrid at this point. You can't only have one answer if you're an enterprise or a global enterprise today. This is, I would say, directly attributed to why we've seen such strength in platforms like RHEL and OpenShift. Because the minute you have 3 clouds or 2 clouds and on-prem or a sovereign cloud and 2 public clouds, you need some commonality across them to get the best use. So that has been a very strong trend. To be honest, I think AI only amplifies this because you have the problem of training where a lot of the enterprise data still sits on-prem. And you have the problem of deployment where if my in-destination is a car or a factory or a retail branch, Edge really becomes part of my core strategy at that point. So that -- I think you're right on this trend. We have seen the acceptance of hybrid really strong over the last couple of years. I think the work we've done with Ansible and the upcoming acquisition of Hashi gives us a real management strength in that space. But then AI is probably even more hybrid than the core traditional applications, because by design, you're training it on data that is yours and tends to be behind the public cloud move curve and you are deploying it as close to your customers as possible for that unit cost efficiency. And that brings us into some really exciting strategic conversations of how hybrid becomes an advantage for you, not just a reality that was thrust upon you because you couldn't get to a single cloud with it.

Yes. No, that's great. I know we're out of time, so I'm going to try to sneak in 2 questions as one really quickly, if I could. One is just what's your longer-term view on training versus inference in the sense that, I mean, as these enterprises train based on their proprietary data, how much of workload or volume do you think ultimately or time spent would be on training versus inference in the enterprise world? And then maybe just to wrap it up, like what are you most excited about IBM's positioning here in AI?

No problem. So training versus inference, I do think in the next couple of years, we'll see the inference load eclipse training on it. At the same time, to be honest, we try to push training down to -- there are multiple types of training. There's build a new large language model, which is -- requires a supercomputing data center. When we look at RHEL AI, we train on a 4-way or 8-way single machine. So we try to keep training costs fundamentally lower than the mean aspect of it's going to take us $1 billion to train. So we are pushing that down. But to your point of -- 2025 is the year for production workloads. By design, you want to be doing more inference. You want to be asking the model more questions than you are feeding it with data. So that's something we watch there in terms of making that shift. And what I'm most excited about for IBM's positioning is when you have any technology disruption, I think the most challenging thing is can you make it work for enterprises. And there are 1,000 things that have made really exciting technology to me, not be able to make it forward in the industry, whether it's from cost perspectives, application of it, the ability to structure it, trust it. And IBM has the entire portfolio from a software stack that is flexible and gives you the starting points, optionality on infrastructure, all the way to the skills that can help you get there faster. And so I think that's a really -- it's an exciting position to be in to help bring customers to a point of success with the technology, not to stall out on one of those constraints and get frustrated. But -- but yes, we'll see what we can do in the coming years there.

Amazing. Well, thank you so much, Matt. It was a pleasure to have you here and join us at our AI Conference. We look forward to catching up with you soon again. And thanks, everyone, for joining us today. If you have any questions, please shoot us an e-mail. We'll try our best to get it answered. Thank you again. Thanks, Matt.

Thanks, you all.

Happy holidays.