UiPath, Inc. (PATH) Earnings Call Transcript & Summary

Thank you, and good morning. It's a real privilege to be here. This is my very first FORWARD. And I understand that we've got more people here than at any FORWARD in our history, over 3,000 of you. Thank you for being here. Today, I want to tell you about what we've been cooking back in the product and engineering team to try to help bring together the magic of AI work. As the Chief Product Officer, one of the favorite things that I get to do, the best part of my job is to bring together amazing people like all of you, an amazing technology, to bring it and create some of the best products that we have ever seen together. Today, I will walk you through some of that amazing technology. I will show you some of the best solutions we have. And together, I invite you to come have a dialogue with us. Find me, find the people on our team and tell us what's working for you. Tell us what's not working for you so that we can create solutions together that help solve your hardest problems. We've assembled folks from all over the globe here today to come and have those conversations with you so that we can co-create a future. I wanted to start off with some stories. Over the last year, I've had the privilege of talking with many of you about how you're putting the UiPath platform to work for you. And earlier this year, I heard an event about an amazing story. In 2022, British Airways encountered a spike in their customer claims. And their traditional manual processing took 25 minutes per claim, and a lot of tedious work to have to go process these claims efficiently. Together with Tquila Automation, a partner of ours, they've created a brand-new process, reengineered it from the ground up to focus on improving that customer experience while still maintaining the highest compliance standards. That implementation led to some remarkable results, including a drastic reduction in the claim processing time from 20 weeks down to just 6 days. And in 2023 alone, they have processed 55,000 claims and saved about GBP 0.5 billion. Some amazing results. And this successful venture, not only save them a bunch of money and made their processes more efficient, but it also improved customer loyalty. And they saw measurable impact on the Net Promoter Score for their customers and how they think about British Airlines. Next up is one that is very close and personal to me. I have family who are about to be deployed to the Middle East. And it's really important to me to make sure that our service members come home. I'm actually getting a little choked up just thinking about what he's going to be going through. At the public sector event, I had the privilege to talk to the CIO of the Air Force, and she told me how they had deployed over 700 automations across all 90 of their bases. And it is important that they have the best tools possible available to them. And they removed over 33,000 manual steps so that their folks could focus on what matters most, which is bringing home our troops safely and protecting our country in the world. Thank you. And last, but certainly not least, I want to talk a little bit about Lazard. Lazard is a long-time customer of UiPath and a true innovator in automation. They're also nominated for one of our AI10 Customer Awards. But to share a little bit more about what they're doing in AI plus automation, I thought it would be awesome to invite the Global Head of Transformation on stage to share with you a little bit about her perspective on it. So I'd like to welcome to the stage, Mansi Kapadia, to tell you a little bit more. Mansi?

Hi, Graham. So nice to see you.

Nice to see you too. Mansi, thanks so much for being here. For those who may not know about Lazard and your role, would you mind telling them a little bit more?

Absolutely. I'm so excited to be here, so thank you again. I lead innovation and transformation initiatives across Lazard, where I introduce disruptive technologies into the firm and scale them globally across all business units. Prior to Lazard, I spent nearly 2 decades in management consulting, where I helped advise clients and CEOs on how to optimize their technology product and cloud investments and also develop technology roadmaps to enable business objectives. Let me give you a little more about Lazard. So Lazard is a very highly rapid financial services institution, focused on investment banking and asset management. It has a very rich history, which dates more than 175 years. And some great names of finance have been part of Lazard and have turned it into this M&A powerhouse. Our current CEO, Peter Orszag, comes from the White House, where he was the Director of Congressional Budget. And we've been part of a lot of landmark deals, including saving New York City from its bankruptcy at some point. We did the IPO of Alibaba as well. And on the asset management side, we manage more than 200 billion in assets. And I'm really proud to say, I worked for Lazard. It's a global firm. We're based in 27 countries, especially London, New York and Paris. And the C-suite is so forward thinking that they really embrace innovation. And they really believe that automation and AI can transform the way we do business.

That's amazing and very inspiring. Mansi, can you tell us a little bit more about some of the key processes that your automation program is focused on?

Absolutely. Look, for any for-profit business, the bottom line is how to generate more revenue. right? And how to do so with high productivity and get high profit margins. And for banking, it's the same. How do we generate more deals with fewer resources. And I think, so it's more about higher output to input ratio. Now the backbone of any investment banking firm, we have bankers that spend a lot of time behind the scenes, doing data compilation, financial modeling, creating all these reports. And my job is, really, to figure out what is this future -- digital banker of the future going to look like, where they're spending less time behind the screen and more time in front of clients. Because every minute behind the computer screen is a minute that you are not spending, driving business growth, right? So we looked, we did a bottoms-up analysis, try to find out what are the main processes that these bankers are spending time on. And we picked something called the PIB, right? The PIB is called a public information book. We picked this process because we thought if we automate it, you would get the most bang for your buck, because it's something that you can scale the automation on. Now this PIB essentially is a 500-page report, with a lot of data from various sources, that the bankers spend about 2 to 3 hours every week putting together. It has things like 10-Ks, 10-Qs, financial information, brokerage reports, quarterly earnings and so forth. And I think it was very important to pick that right, first use case because we were able to show right off the bat, value, to our C-suite. And that's really what gave us that, stop, buy it.

Amazing. So when you and I first met, back in the spring, I think you were a little ahead of the curve, in fact, and you were talking about generative AI and how that could even help with this process. Could you talk to us maybe a little bit about the evolution of that solution?

Yes, absolutely. So at the core of our solutions, we have UiPath orchestrating generative AI and our core Lazard IP right? And we did a transformative phase approach where we -- Phase 1 was about an unattended bot with a simple Excel form that the user just puts in a company ticker and a date range and sends -- and clicks the send button and gets the output via e-mail. Fast forward to Phase III, now we have an attended bot with more complex forms. We also migrated to UiPath apps. So we kind of used all the core components of the UiPath platform. So orchestrator, attended apps, unattended bots migrated to apps, all of that. And then this really benefited us in terms of adoption because now people are able to configure their outputs and make them more bespoke because every client needs a different type of pitch book. And then it's a lot of ease of use as well since we migrated to apps. Now I said all of that, it's still PIB, which is still a 500-page report that people need to read before going into client meetings. So we added layers of generative AI and ChatGPT and we allowed our users to summarize the output. So now they can digest the information more quickly, and we've condensed a 500-page report into a 1-page report. We've also worked a lot with LlamaIndex and web plug-ins to get the most up-to-date information. And we've given the end user the ability to extract information, extract critical information, so that they can make business decisions more quickly. So that's how we've incorporated generative AI. And our bankers are really, really excited because now they have to do less mundane work, and now they can focus on more higher-value activities.

Fantastic. Could you briefly tell us maybe some of the highlights of the results? I know you're going to be speaking on the Vision stage later today, so people can get the deeper, but maybe you can tell us a little more.

Absolutely. So look, I think generative AI has been compared with the PC revolution or the iPhone moment, but I think its comparison with electricity resonates the most. You don't really think about when to wash your clothes because the electricity is just there. And I think it's the same with UiPath and automation. They have so many different uses, and we have embedded these technologies in so many of our core processes that it's become a commodity, right? So I mean the obvious benefits are time savings. You've created capacity for our bankers to do -- drive more business, spend more time with clients. We've elevated the type of work they do. So there's definitely a lot more job satisfaction. We've -- we're giving them a lot more with the time that they're spending. And then, of course, this also helps us with talent attraction and talent retention. Said that, in terms of hard numbers, the PIB alone, it's estimated that we're going to save at least 100,000 hours per year. And then now -- remember, the PIB is only one small part of this pitch book process. If we look at the full pitch book process and we get every banker to adopt it, we are looking at savings of at least 500,000 per year. Now fast forward into the future, I think the goal is to have every analyst to indeed be that digital banker of the future and leverage gen AI into their day-to-day efforts. Right now, we've looked at different use cases, and I'm sure all of you have ChatGPT. Some of the proven ones are around summarization and sentiment analysis and training. I think the next step is looking at content generation. And I think we're also looking at, in the future, things like idea generation. But all in all, I want to say that Lazard has really embraced emerging cutting-edge tech like this to fast forward into the future. And I'm really proud to say that I work with -- at a firm where the leadership has such a long vision.

That's fantastic. Mansi, thank you so much for joining us and telling us about what the future looks like at Lazard.

Thank you. Thank you, Graham. Thank you.

Hopefully, you're as inspired as I am by Lazard and their awesome story. Thank you, Mansi, for sharing that. We've been on a journey, and I want to tell you how our customers have been able to achieve some of these results. We've been on this journey to become a full-fledged comprehensive business automation platform, not just a collection of tools but something that helps you discover, automate and operate at scale. I also want to tell you about how AI is playing a more important role. AI has always been a critical part of the business automation platform from UiPath, and we are placing more emphasis on it than ever. And many people ask me, well, why is that exactly? That's because AI plus automation is magic. And frankly, AI without automation is kind of like having a brain without a body. You need the critical context that automation can provide, and you need to be able to take action from the decisions that AI can allow you to make. When you're in a call center trying to respond to a customer's request, you need to know who that customer is. Are they entitled to a higher level of support? Have they been speaking to people in your organization already? Do they have open orders? Do they have open issues? Those are critical pieces of information to respond appropriately and in time. And you want to be able to take the appropriate actions. You want to be able to go fulfill the request of that customer and, when it's necessary, to ask humans, put them in the loop, to make sure that the right decisions get made. What we've heard from customers is that AI is really powerful, but it's also causing concern. Sometimes you don't want to have an AI hallucinating an extra 0 on a check that you want to cut. You don't want it hallucinating about an employee that you want to be able to hire, and that's where it's critical that you marry an automation platform with the magic of AI. And at UiPath, we are embracing the generative AI. It's a massive accelerant for us, opening brand-new scenarios I will tell you about, and we are marrying that with specialized AI. When you need a faster, more accurate response, specialized AI, for which we have 70-plus built-in models, may be the right solution for you. And in our platform that is open, flexible and responsible, it's critical that you use the right tool for the right problem at the right time. So let's dig a little bit deeper into the gen AI side of things. A couple of months ago, at our TOGETHER event in London, I told you that we just announced our connectors for OpenAI, both directly and through Azure, as well as Amazon SageMaker and the Google Vertex AI. So if you made a bet on these providers, they are key partners of us, and they are sponsors here at the event. But we have not stopped. I'm excited to announce a brand-new set of connectors. So now we are going to stay ahead of the curve for you so that you can make use of the best latest and greatest generative AI, whether it be Llama, whether it be Claude, whether it be the entropic partnerships that we've now developed. These set of connectors will help you stay on top of what's going on in generative AI, and I'm also proud to announce that the connector builder is now going to GA next month, so you can build and train your own and plug them in. Now let me talk about specialized AI. Last year, we talked about the acquisition of Re:infer. We have now built that as a native part of our application. So that is now reborn as communications mining in the automation cloud. And combining specialized AI and generative AI, we have become and been recognized by analysts as a leader in intelligent document processing because of the completeness of that platform and because of the magic of specialized plus generative AI coming together. And more importantly, that's had massive results for customers like Expion who transformed their business around processing claims. They're a low-cost provider -- they're a cost management, sorry, provider for the health care industry, and they are using robots and AI together to download, process and review those claims in record time, saving a ton of manual work and time so that people can get those claims that they need faster and have better health outcomes. And that resulted in a 600% increase in the number of claims that get processed every day. So what's next? We are not stopping there. For specialized AI, I want to show you what the future looks like and how we're going to decrease the time to value and make it easy for anyone, whether or not you have any machine learning or AI expertise, to create your own specialized AI and put it into action. What I'd like to do is to show you a little video about how these solutions are going to come together and introduce you to [ Anna ] and [ Jacob ] who are first-time homeowners who are going to be dealing with a warranty request on a broken refrigerator. Let's roll the video. [Presentation]

Pretty exciting, isn't it? So what you just saw, and what I'm proud to announce, is a set of capabilities, active learning for document understanding, generative annotation so that you don't have to spend so much time labeling and generative classification and extraction to build into your workflows for intelligent document processing. Based on our internal benchmarks, we've been able to save over 80% of the time it takes to build and deploy specialized models. So what am I going to talk about next is how to make sure that you can trust that your data is being used responsibly. One of the biggest concerns that every customer asks me is how do I make sure that my data doesn't end up in the wrong hands. These generative AI features are very powerful, but people want to make sure that we are going to -- that they are able to trust and they have transparency and control over how their data is used. With the new UiPath AI Trust Layer, your data flows from your enterprise through the UiPath automations and the data is encrypted, both in transit and at rest, to make sure that before it reaches the AI Trust Layer, you can be sure we're treating it the right way. It lets you establish governance policies. It lets you monitor and audit all of the activity that's happening, and it allows you to mitigate the sensitive data to filter out the private and sensitive information so that it never gets to the LLM from the third-party in the first place. These capabilities all go through the LLM gateway, which then allows you with built-in policies that will prevent the retention of the data and training by any third-party model, so it never gets reused by anyone else. These capabilities are going to all come together for our generative AI experiences in brand-new ways. What I've been talking to you about so far is about how generative and specialized AI affects the existing set of automations you've been thinking about. But what I want to do now is actually help you, cocreate with you and understand what the future is going to look like, what are the new scenarios that generative AI and specialized AI will unlock. And we believe that with you, our customers, we've been on this journey together to have world-class computer vision, to have the best specialized document understanding models, it's only with your guidance that we've been able to do that. So I have a challenge for you as we go into the future, and I have some questions that I'd like to answer here together with you. Those questions are about what comes next. What if every employee in your organization could be a developer? What if creating tests was just as easy as going down the hallway and picking up that Red Bull? What if every user could be a power user? And what if there something that was, yes, a little better than a copilot but for automation? Well, we believe there's an answer to these questions. I'm proud to introduce you to the newest member of the UiPath family, UiPath Autopilot, or as I like to call it Auto. UiPath Autopilot is an amazing set of capabilities and it's going to help improve the lives of developers and testers and analysts. Everyone from the intern to the CEO will be able to make use of Autopilot and the capabilities that it brings to the UiPath automation platform and do things that never before were possible. And to show you some of this, let's start with the developers and what we're doing in UiPath Studio for workflows and apps and expressions. So I'd welcome to the stage, Noopur Inani, to show you something we used to call Project Wingman, Noopur.

Thanks for that warm welcome. I am so excited to show Autopilot in action today. Now for the purposes of this demo, I'm going to be playing a persona. I'll be the Partnerships Director at Desk Experience. So as the Partnerships Director, I'm frequently onboarding new vendors into our different systems. They may fill out this information on a paper form, the vendor information form. And then I enter that data into Salesforce and NetSuite before I eventually tell my team on Slack. It takes me about 4 to 5 minutes. Now in that same amount of time, Autopilot is going to help me generate an automation for the end-to-end process. We'll start with digitizing the form. Now Autopilot for apps supports the generation of apps based off of screen shots of legacy systems, text prompts and PDFs, like this one that we just saw, using UiPath Document Understanding. Autopilot is also going to help me by generating some entities in data service to help me store those submissions and the data from them. Now we can see that the form has generated, and we can see the submission for the data entity and data service. But to truly transform this business process, I want to connect this to an automation. So I'm going to ask Autopilot to help me generate an automation. I might ask it to generate a description with OpenAI, create that new account in Salesforce and notify my team and inform me on Slack.

So what's happening behind the scenes here, Noopur?

So here, Autopilot for Studio is using a combination of our connections to Salesforce and Slack from integration service and activities from Studio to generate the automation workflow. Once it's generated, I'm going to have the opportunity to review the workflow and even change the prompt if I'd like. Now Studio Web is already making automating for developers very easy. But Autopilot is truly making it faster and easier than ever.

That's amazing. There it is.

So now that I've got this workflow, let's move it into Studio here. As I move it into Studio, I'm going to see my activities populate, pre-configured with those exact connections we talked about. But I forgot one piece of the process. I usually onboard the vendor into NetSuite before I send that message in Slack. So let's use UI automation to do this. I'll go ahead and add that use browser activity for UI automation. And then generally, the next step is to indicate exactly where I want activities for that UI automation, whether it's selectors or whatnot. So I'll provide the tab as well. And this is particularly helpful, right, because I take quite a bit of time usually to identify those activities and selectors to interact with. I'll try again here. And then I'll actually provide Autopilot with that prompt. So it's as simple as telling it exactly what I do, which is add a new vendor. Once I click generate activities, it's going to go over and start to look at the different pieces of this automation to really identify where to interact.

Wait, are you doing this? Or is it doing it by itself?

Well, Autopilot is doing it for me. So not only is Autopilot identifying the activities and selectors for me, but it's using UiPath best practices to do this. Now this is particularly important for me because I want to make sure that these selectors are robust enough that they can withstand any of the different changes in that application.

I remember that taking me a long time when I first did it.

Exactly. So we can see that Autopilot found those activities for me. And we can see that it has going to basically take those activities, all those selectors, put them back into Studio for me. Now there's one last piece to this and kind of a finishing touch for this flow. Usually, what I like to do is really make sure that all of the web addresses that my vendors input are secure, just so that we don't have any issues going into NetSuite as we onboard this information. To make sure that the automation can account for this, I'm going to create an argument, I'll call it web address. And then usually, I would find myself on Google or Stack Overflow to figure out what the expression looks like. But I'm actually going to ask Autopilot to generate it for me. So it's going to be as simple as, "make sure that, that web address argument has HTTPS."

I remember taking so much time with my big O'Reilly book sitting next to me doing this.

Exactly. Especially for those of us that are new with automation and potentially new with programming, creating expressions can be really overwhelming. So to be able to do this with just a click and a type is really helpful. Now in the time that it took me to onboard one manual vendor, I was able to create an automation for the end-to-end process and really free myself from ever having this data entry manually again. It's really AI at work.

That's awesome. Thank you so much, Noopur.

Thank you.

All right, Ingo. I think we know what's up next. Show us the new stuff in test.

Thank you so much, Graham. Ladies and gentlemen, I'm not just playing as a software tester right now, I am a software tester. And Graham, we know it, and every other tester knows it, the time needed for testing is always infinitely larger than the time available. On top of that, too many repetitive manual testing tasks typically slow us down, allowing those [ critical paths ] to slip through and hurting the software we release. So let's now see how Autopilot helps to fix that issue.

Let's do it.

Now to do that, let me go to one of my testing projects here in Test Manager. And as you can see, this project is all about a web application. And for this web application over here, we see that we already defined, yes, several requires for this application. Now going to one of these requirements, we see that this requirement over here is expressed as a user story, contains information about the user flow, the application logic, and it also contains information about the acceptance criteria as defined by the product manager. Now to generate enough tests for this particular requirement, all you need to do is click on this button over here, generate test. And what Autopilot then does in the background, it analyzes the logic of that requirement, analyzing the relation between all this acceptance criteria. And once done, Autopilot will notify you like this to then, yes, show you all the critical, the top test cases Autopilot has found for that particular requirement. So these are displayed over here. Now you won't be 100% satisfied with what Autopilot suggests all the time, so that means you can also add your instructions over here to, yes, make Autopilot, generate the test cases that are specifically tailored to your needs. Now once you're done with that, you just select the test cases you want to create, click on create over here, and then those test cases you just generated are being linked to the requirement. Now going to one of these manual test cases we just generated, we see that we don't just generate the name of the test case, we also generate step-by-step manual instructions that allow you to immediately execute that manual test case. But the story doesn't end there. You know it, Graham, we, in UiPath, we live and breathe automation. So the next step is pretty clear, we want to automate those manual test cases. So to do that, let me switch to Studio. And this project here in Studio is linked to the project you have just seen in Test Manager. Now whenever we do that, we immediately show you all the manual test cases we can find in this project in Studio, so you can think of it as a to-do list for your test automation developer. Now to automate one of these manual test cases, let's, first of all, load one of these manual test cases here in Studio. And when we do that, we convert all the manual test steps we just generated in Test Manager here to code comments, as you can see. Now what you just see are our brand-new COVID automation capabilities that allow developers to write any type of automation [ in the shop ] in Studio. Now with the object repository containing here all the elements of the application you are testing, your buttons, your links, your tables, you basically have everything you need in order to turn this pile of text into an actual automation. How do you do that? Well, you just select the text comments over here and click on generate code.

Wow. So what's happening here, Ingo, behind the scenes?

So what Autopilot does now behind the scenes is it analyzes all the manual test steps. It builds a cohesive story from all these manual test steps. It cross-checks with the object repository to figure out are there technical elements that can be used for the automation. And of course, Autopilot constantly impacts also with our driver framework to build the actual automation. And Graham, the magic just happened, so to say, right? Now the best part of it, you can also customize the automation according to your needs. So this is just a suggestion Autopilot does. So now let's do the final [ proof ] and let's also see if we can execute this automated test case. So Studio now compiles the project, makes this automated test case ready for execution. And here you go, the automated test case just kicked off. Now ladies and gentlemen, that's the short story of how you can not only generate manual test cases from requirements but also automate those manual test cases in just a matter of minutes with Autopilot. That's all I have. Thank you so much.

Amazing. Thank you. So you just saw some of the magic of Autopilot for developers, for testers. We are also announcing Autopilot will be available for our full discovery suite, including communications mining and process mining, so that with pure natural language, you can describe the data that you want to get, the insights that you get, and we prepare it for you immediately. So across the entire organization, Autopilot is going to lend a helping hand. It's your partner for AI at work to help developers, help testers, help analysts do their jobs better in ways they could never do before. But like I said before, we are actually cocreating a lot of these experiences, and some of our customers are actually starting to use them today. And so to tell you a little bit about their experience with these technologies already and where they'd like to see it go, I'd like to welcome one of our most innovative customers, Dentsu, to the stage to tell us About that. Please welcome, Flo Ye. Now Flo, for those who may not know Dentsu or your role, would you mind telling them a little bit about yourself?

Absolutely. My name is Flo Ye, I'm the Director of Automation Solutions at Dentsu. Obviously, our work is most known in the digital media advertising world with over 60,000 employees globally. I manage the global automation team. So we have around 2,000 people with different persona in our global automation community. Last year, really great to be back on the stage with Noopur again. We share our success story around automation operating model. So this year, with that framework putting in place, we're able to deploy many gen AI, including Azure OpenAI. And most notably, we also deployed another dentsuGPT, similar to ChatGPT, but built on our secure network with also additional data internally for Dentsu.

Amazing. So we just highlighted a bunch of new capabilities. And I know that you're very excited about what generative AI can do. Can you tell us a little more about how you're putting it to use in your...

Absolutely. So one thing that we really value, it's our citizen developer community. We already have more than 200 CitDevs certified to become CitDev at Dentsu. But I'm not going to lie, it's been really difficult to keep growing the program in the past year because it is a lot of initial commitment to do a 13-hour training that's on your academy. You might -- it doesn't seem a lot, right, 13 hours. But for people who actually have a day job, what we call it in finance and maybe they are doing a new ad campaign as a creative, that's a lot to do in addition to what they already have to do on their day-to-day. And then I think just seeing the Autopilot, it's going to be an amazing tool for them to overcome that barrier -- entry barrier and actually minimize the initial learning curve. I'd love to give an example of I think there's a notion of how AI are taking jobs with all the new buzz coming up. But I always love to say that, just like the cameras, back in the 1800s, I'm sure it's a big production to just take one photo. But with the evolve of the technology, we see these amazing images and videos on Nat Geo or on Planet Earth, right? We don't take away the talent from the photographer behind the lenses just because they have fancier equipment. Now same thing I see with our CitDevs and Autopilot, AI isn't taking job away from us, just transforming how we do our job. So we're really excited at Dentsu to make sure that we utilize this technology, take away the tedious part of their job and task and actually having our team members to focus on human-centric work and work that require their creativity.

I think we completely agree. It's a great vision that you have for your employees, very uplifting. Maybe it's magic one time. So you've been a long-time customer of UiPath, Tell us where you'd like to see all of this go?

Absolutely. I think that gen AI -- and I've seen some preview of what UiPath is going to bring the magic. Gen AI is amazing now to give us useful information, right? My e-mail's writing are way better using all these gen AI technology. But what I think can take it to the next step, it's really not only to provide me the information maybe, for example, tell me that I pay for a taxi to come here, so based on the receipt that I received in my e-mail, but actually performing the action of submitting that to Concur, for example, our internal expense portal. So I think UiPath is in a very sweet spot to be the united front to bring action across different platforms and ecosystems.

And that's really exciting. That's a great idea. So I would invite you and everyone here to come tomorrow to see Daniel's session, and you might get a surprise or two.

Absolutely. I'm really excited.

Thank you so much for sharing your story, Flo. It's really inspiring, and thank you for cocreating Autopilot.

Thank you so much.

Thinking about the future of work and you might get a glimpse of Autopilot in action for everyone. So what did you hear today? I'm really very excited to announce the new gen AI connectors, the faster time to value with our specialized AI for Document Understanding and communications mining, the new AI Trust Layer so that you can be sure that your data is being used properly and the brand-new generative AI experiences that only UiPath can bring to you with specialized AI, generative AI and automation together. I hope you're as excited as we are. Thank you so much for being here. [Break]

Good morning. Welcome back to day 2 of FORWARD VI.

It's been amazing already.

So I hope most of you had the opportunity to come to the automation celebration last night, which was what AI cannot do. We had some incredible food, but the acts were awesome. I want you to know that there were 3 sisters, their group's called Sorelle, which means sisters. And they were the runner-up in the U.S. The Voice last year. And we had this French beatbox group called Berywam, which everyone loves, and it was great. And the bartenders are making sort of drinks and juggling bottles. It was -- everything was entertaining, even the food.

So our theme should be AI that doesn't work as well?

Yes.

Yes. I mean, I think for me, well, my feet hurt, which I think is because of all the walking. I imagine you all feel the same. But I went to a lot of sessions yesterday and walked around. And I think we wanted to really bring the customers forward here, and I hope you all saw some amazing customer stories. I'll give you a few of what I saw. Well, first of all, my favorite one was Sam Balaji on stage yesterday with Rob Enslin from -- Sam from Deloitte -- when he said he committed to 10% of the Deloitte workforce becoming trained on the UiPath platform. Do you know how many employees Deloitte has? Over 450,000. That's amazing. And that's really cool. That shows, I think, what our technology is all about. And certainly, there are many other partners here who are equally committed, and so I don't want to not highlight them, but it was really fantastic what Sam said. But then Cathay Pacific from Hong Kong and British Airways from London, they were talking about AI work, these amazing airlines. VMware and Paychex, I saw them talking a bit about their views of what -- our top 4 features beyond RPA. And then one of my favorites not because I happened to bump into them early because they were practicing and they were really working hard and they know who they are here, but I had to go see them, and that was Nicole Otero who runs Navy -- the Department of Navy's Automation as a Service COE. And Rebecca Young, coincidentally, also from Deloitte is an ISV. And they gave like deep advice on how they're taking automation across the whole of the Navy and transforming the government. So look, there's many amazing examples, Mary, but those were really highlights for me.

So today, we have more breakouts starting at 3:30 and also the industry summit this afternoon, so make sure you don't miss those. The keynote theater this morning will have -- we have the Digital Economist from Stanford, Erik Brynjolfsson. And Bobby will wrap that one up with Erik asking -- Erik wants to have questions. So we want you to go to Expertsville after The AI 10 Awards this morning. That's what's going to wrap this morning's session. But speaking of questions, please bring your questions to Expertsville. We have assembled the world's best UiPath experts there, and we want you to try to stump them with your questions. So please do that. And then the day will wrap up today with the unwind happy hour in Expertsville at 5:30. We'll have a drawing for the fast-forward game that you can play, check out your mobile app. But we should get this party started, yes.

Well, speaking of the best experts, we would not have gone from RPA as a tool to automation and platform to AI at work without these 3 amazing amigos, the trio. And we're going to bring one of them out. He's our Head of AI Strategy. He'll kick this off, Dr. Ed Challis.

Wow, it's amazing to be here at the beginning of day 2 of this incredible event and what I really believe is actually kind of the beginning of a new era, the era of AI. And I think all of us, in 5 and 10 years, we'll look back and say, "2022, 2023, those were the years when this kind of things really started to change." And so I wanted to take this opportunity to kind of facilitate a conversation about where we're at in AI, how we got here, where we're going and kind of what we need to do collectively to make this future a positive one. So to have this conversation with me today, I'm joined by 2 major colleagues. Firstly, Professor David Barber.

It's great to be here. So I've been working with AI for about 30 years now, incredibly. Well, actually, [ also ] I'm an AI researcher, I head the UCL AI center in London. But I also work here at UiPath. So we have an amazing AI research team here at UiPath. We are working on some of the most amazing impressive technologies, I think, that we're going to be seeing in the next months and years here in the products that we're making as well. So I'm super excited about AI. It's come a long, long way in the last few years, and we're going to talk about it more today.

Amazing. And I'd also like to welcome Luke Palamara to the stage, our VP of AI Product Management. Luke?

Hi, everyone. I'm Luke Palamara. I lead AI Product Management here at UiPath. I've worked my whole career shaping enterprise products with AI, including a decade at IBM Watson. But here at UiPath, I really see my role as creating this fusion between AI and automation. Ed, I think the AI landscape has never been more thrilling to navigate than right now.

And I'm Ed Challis, I head up AI strategy, and I kind of think of my role as somewhere between you both. So my career has been a mix of AI research and building kind of new AI products through the startup I founded. And I kind of sit between your team that's developing the new models, these new cutting-edge kind of capabilities and the product managers and engineers that make it real in the platform for our customers and our partners. So I kind of wanted to start this session with the professor, with you, David. I'm going to give you the difficult task of trying to explain like where are we in AI, how do we get here, how would you kind of summarize that.

Yes. Thank you. So it might good to go back a little bit in time and think what was the initial goal of the field, right? So actually, AI, you may not be aware of this, but it's not really a new thing, right? It's been around for at least 50 years. And actually, our ambition has always been to make these systems that can do things like humans can do, right, this cognitive tasks. And it's actually, in some sense, a very inspiring message, right? That's been our hope and ambition for a long time. In some ways, it's actually very well aligned with the UiPath mission, right, accelerating human achievement. That's a really similar kind of goal. So there's no coincidence that, in some ways, you're seeing this convergence of these ideas, UiPath and AI. And I think actually, it might be good to think about the ways that we've been trying to do this. So there are different ways to try to achieve this goal of AI. And actually, some of them are very logic-based, right, sort of playing games like chess, et cetera. But the other way to do it is to try to replicate the way the human brain works. So let's just call it like an artificial neural network. So in our brains, we've got these billions of neurons and they're all interacting and connected in complicated ways. The idea was, could we build a system like that? And the idea was, well, it might make sense because, if you can build a system similar to the way a human brain is actually constructed, maybe it can solve similar problems like humans could do, right? So we would really like to do that kind of processing that brains can do. But actually, that's also a very complicated thing. You've got so many of these neurons and how to connect them is a very complicated problem. And the amount of compute you need to actually do that, the amount of data you need to determine those connections is actually also very, very difficult. So whilst the field has been around a long time, it's only relatively recently that we've seen the accelerated progress in the sense that actually these systems are becoming very, very useful. And if you look a little bit at the figure here, this is an interesting graph, I think, where you can see that, from 1998, on the left, we started to document how well we were doing against certain -- against humans, the average human level. So on the vertical axis, you can see there's 100% human performance at the top. And the top -- the most left one is things like handwriting recognition, right? So you can see that actually, over time, steadily, we get this improvement to the point where maybe around 10 years ago, actually, the systems are now outperforming the average human. So these are the neural network, the artificial neurol network systems. And then we sort of looked at other specific problems as well, like, say, speech recognition, image recognition. So for example, I give you an image. Is there a cat in this image? Or what is in this image, right? So the machine has to recognize that. And you can see that, again, things went well. Things were going quite progressively good until around 2012, 2013, things really started to accelerate very dramatically. And that point was the introduction of the GPUs into the whole game, right? So that's really a major thing because the amount of compute power that they had is huge compared to what we had previously. And it's actually quite difficult to comprehend this. We are used to every year, our companies just get a little bit faster, 1.5x, 2x faster than the previous year. But this compounds. And even though the artificial neural networks at the time were very similar to what was actually proposed 20 years before, in 2012, they were thousands of times faster to the point that basically, it would have taken 100,000 years to train that system 20 years ago, that state-of-the-art system back then. Now actually, it would take over many millions of years actually to train, say, a large language model, obviously, with the hardware from, say, 25 years ago. It's actually that level of acceleration. We're not really conscious of it, but it's a major, major thing. So you can see that we've done really well with each of these individual tasks and they're getting better such that, actually now, it's pretty much -- if you've got enough data and you've got enough compute, the computer will always beat the average human. It's really at that stage. Every single one of these tasks has been knocked off, right? But in some ways, that's only part of the story. These are perceptual tasks. The machine is perceiving a world, it's recognizing objects, it's recognizing speech, et cetera. But that's the kind of -- that's only 1/2 of the story, right, in terms of intelligence. And what we'd like to have is something else. We'd like it to generate things, right? So humans actually are also very good at creating things. So in some ways, we'd like to have that ability as well. So we started now with very specific tasks, again, like generating a completion of the sentence or, say, generating maybe a translation from one language to another or generating some code. And these were all quite good, quite reasonable. And you can see again on the graph, towards the right, that the -- because of the compute and the data, the amount of time it took to exceeding the performance actually was going down. But then around 2019, 2020, we've had the amazing idea, a crazy idea even, so what happens if we now try to take maybe a huge chunk of the Internet and try to just make the model generate the next, say, word and, say, sentence, a random sentence, taken from the Internet, given the previous words in that sentence. And also, it could be a language or it could be code, it could be anything, right? It could be all kinds of stuff. And the amazing thing is that actually, surprisingly, these systems, if they got enough data, they learn all kinds of incredible facilities. So not just to generate because of code or to complete a sentence but actually to do things like logical reasoning, at least to some degree, to understand much more about our rich cultural heritage because a huge amount of the linguistic information of humanity is actually being ingested by these systems. And actually, that's really a very, very powerful thing. So it's really, I think, one of the most inspiring things that happened within the last, say, 10 years, 15 years in AI. But this is really, really cool and they're very creative. So the good thing is they're kind of like built like the human brain, right? So they're good at cognitive tasks. The bad thing is the [ build has ] human brain, in the sense that they're also fallible. It's hard to interpret and they can actually make mistakes like humans can as well.

Well, just to summarize 70 years of research, I guess, it's kind of amazing. But definitely, we've gone from this transition of AI just being a kind of perceptual kind of classification engine into being something that's much more creative, generate text and code and super fast and all underpinned by this incredible growth of compute. But from a great kind of fundamental perspective, in the world of business, Luke, how would you summarize where we're at with this technology and specifically kind of generative AI?

Yes. So as things stand right now, Ed, in the world of at least enterprise AI, things are kind of bifurcated. On one hand, we have these specialized models that are highly tailored and specific and they're trained on these narrow domain-specific data sets. And specialized models have these traits like being really fast, they're predictably good, they're relatively inexpensive to run and they have this detailed understanding of data that's really important for businesses, like being able to understand things like invoices, bills, receipts and being able to extract information from those. But the challenge comes in that these documents can be nuanced in different kinds of ways from company to company and from use case to use case, things like the language in them, the visual layouts and the documents. And so specialized models can be trained to understand these nuances to get the highest level of accuracy. And actually, some of the hidden cost of specialized models is this training, if you don't have the right or proper tooling and methods to really make them specialized enough for your use case. Now on the other hand, we have these large language models, which you were sort of alluding to, David, which are based on these massive corpuses of data, and they power generative AI today. And generative AI, of course, promises vast capabilities. But today, it's largely relegated to improving personal productivity. So for example, it's great at writing tasks like summarizing information. They can analyze data quite well. But it may or may not be correct. And it is limited when it comes to very domain-specific type of tasks that are so important to enterprises. But really the future that we're all working towards and the future that we all want is AI actually doing work. It needs to go beyond just this personal productivity. Otherwise, its potential becomes somewhat capped. And so AI has to do work. This is the biggest difference between personal productivity and really enterprise productivity. So for example, if your company could hire just an aid that could advise, but not actually do work, like how valuable would that really be? So we need AI that doesn't just advise but actually undertakes tasks. And that's really where the frontier lies. It lies in this transition from personal aids to really enterprise catalysts for AI.

Well, yes, I think that's really clear and kind of helpful way of breaking it down, this kind of personal productivity tool versus this kind of enterprise productivity tool. And I like that expression, kind of words are cheap, we need to actually do stuff. So I guess kind of could you try and summarize like what are the pieces we need to put in place to get this technology into a place where it's actually doing stuff for us? Like, how can we make that real?

Yes. I think there's really 3 critical ingredients to make AI really productive in the enterprise: their context, action and trust. So first, we could tackle context, right? So AI is really only as good as the information that you give it. Just on this stage yesterday, we had Walter Isaacson who's an amazing author. I'm actually a really big fan. I've read many of his books. But would you go and hire Walter Isaacson and, on day 1, put him in front of your contact center or on your help desk and have him answer e-mails to your customers. Well, he's an amazing writer, has incredible writing capabilities, but he wouldn't have the information about how to respond to customers. They're based on the historical conversations that your company has had with them or interactions or know what type of level of service that this particular customer needs to be provided based on their status or know how to actually take the request that they're asking, like upgrading an insurance policy or canceling a policy and knowing how to actually get that work done. So generative AI, similarly, has this need for context, where to enter data, what to look for. And in the UiPath platform, this really comes together in our integration service that creates this bridge from the context that's locked away in the data sources and being able to pull that data in and feed it to the AI so it has the context, so it can get things done. And then next, and this one is really critical, is action, right? So what good is knowledge if you can't act it. Benjamin Franklin who is actually another subject of Walter Isaacson in another book, Benjamin Franklin said, "Well done is better than well said." And I think that's really true for AI as well. AI needs to do more than just generate text and images. It needs to move data flawlessly from system to system. It needs to be able to respond to customers based on, again, the historical interactions that you've had with that customer in the past. And then it needs to be able to know how to place an order when inventory is low. So this is what UiPath really is all about at the end of the day. It's about putting AI to work. It's about bringing this rich capabilities that allow AI to take action and not just be an adviser. And then I think the third, but certainly not the least, thing here is around trust. So leaders really want to embrace generative AI, and really all forms of AI, but they're wary of risks. And I think this is like really well reflected in a poll that happened last year that was run by Pew Research, in that 38% of people in the general public in the United States were more concerned than excited about the use of AI. And then the same poll was run again this year, and it went from 38% to 52%. So for enterprise leaders to adopt AI, they need to trust it. And there's obviously a lot of issues around trust right now, and there's a lot of ways to tackle that, and we can talk about how we unpack the trust issue.

Yes. I mean that context in action piece is very clear, right? But trust is the foundation. If we can't trust these systems, if we can't get buy in, if we can't validate them, ultimately, we can't use them. And so I kind of want to just like focus on that trust problem. And it's something I think about all the time. It's something we all think about at UiPath all the time. We really see kind of 3 core challenges there, 3 core kind of questions. The first is information security. So we might all have seen these stories where someone's query into a GPT model got leaked to another user that's absolutely kind of critical. We've also -- everyone's kind of aware of these issues around, just as you were talking about -- I like the idea of Walter Isaacson to run your payroll -- imparting specialized knowledge and how we can make these tools, have the kind of the lingo and the knowledge of our processes and exceptions. And then obviously, there's that hallucination question that comes up in almost every conversation here. So we're working super hard to address all of these, and that's why I think it's so great to see one of Graham's announcements yesterday around AI Trust Layer. This allows all UiPath users to get that kind of governance control, visibility on all of those large language model calls, complete visibility filtering, and all of that control that you're CISOs and your CIO will on around that. But if we kind of move on to some of those other questions, I wanted to kind of think about imparting specialized knowledge on these systems. Really kind of one of the -- there's two ways there, right? One is to give a really great accurate prompt, which is the context piece. And UiPath is clearly super strong at getting that information into the LLM. But the second core tool there is fine-tuning these models, making -- giving them kind of specialized information. Fine-tuning models can be expensive, but active learning is a really amazing tool to make those super fast and efficient. I was wondering if you could come to a moment, David, to explain the whole concept around active learning.

Yes. Sure. Thanks, Ed. So maybe it's good to think about traditionally how are these machine learning models and the AI systems train, right? So they typically train on the basis of label data. So if you think about, for example, making systems that can recognize a cat, then you have a lot of images, right? And a human would say, well, that image contains a cat, that image doesn't contain a cat, et cetera, but all this time manually labeling images. Then you give it these pairs of images and labels to a machine learning algorithm and, after some time, it would come up with an algorithm that hopefully is good at predicting whether cats are in an image or not. Fine. But that's a lot of human labeling effort, and that actually has become probably the biggest bottleneck in actually making these systems work. But in active learning, we do it a little bit differently, so a smarter way to make use of your human labeling effort, your employees and your company's time. So the system actually -- once it sees a 1 or 2 labeled cats and non-cat, actually, it's already started to get some sense of what's a cat, right? So maybe when you show it a new image, it might say, "Well, I'm very confident this is actually a cat. I don't need a human to tell me this anymore," right? But this thing here, it might be a dog, I've got no idea what it is, I've never seen anything like that before, and I ask the human to step in and tell it if that's a cat or not. So in this way, actually, the machine then -- every time it gets a label from the human, it again updates, right, and then it increases its knowledge, in this case, about cats. And that means that the overall amount of labeling effort required for the system to converge to a very good cat classifier is actually much lower than it would have been if you just label 10,000 images of cats and non-cats to start with. Maybe you can get away with only 1,000 or even less. And so in a similar way, in this figure here, you can do the same kind of thing for documents. And if you want to train the system, which you can do accurately, say, document classification or generation, if the machine is very confident about what it can do in that case, it just outputs the same. If it's not confident, it passes it off to humans and says can you actually help me out here, can you give me a label for this thing. The human gives it a label for that. Machine then recalculates, recreate its own parameters, it learns more and then the process repeats. And actually then each time, it starts to improve very quickly.

Yes, it's almost like -- I view active learning almost as a kind of a conversation between 2 colleagues. You only ask questions about the things you're uncertain. You don't ask the same question again and again and again, which is why so much -- kind of using AI systems is often so laborious when it doesn't have that active learning component. And then I think one of the hugest components, tools in our tool chest, so to speak, to bring safety and governance and control to the use of this technology is obviously human-in-the-loop supervision, human-in-the-loop review. And I wonder if you could kind of talk about that for a bit.

Yes, I really think that the human-in-the-loop capability is one of the most important capabilities when it comes to making enterprise AI really productive in businesses. So one of the misconceptions about human-in-the-loop is a lot of people think it's just about correcting errors from the output of the AI, inserting a human to review it, correct errors. That's obviously a critical part of it. But it's really also creating trust, coming back to the trust topic. It's about bringing oversight and accountability within AI processes. It's also about reassuring stakeholders that AI decisions can be validated, they can be checked and they can be overwritten, if necessary. So in the UiPath platform, it's really action center is where this all comes together. It's sort of this hub that enables humans to sign off and approve AI decisions.

Amazing. Well, I think we're almost at time. So over to the professor, where do you -- I guess, what are your predictions? Or how do you see the next few years evolving in a very...

Well, I mean, there's some obvious things, right? Things are going to get faster, cheaper. This is kind of a no-brainer that there's multi-modality. These systems will be good, not just in, say, reading and generating image and text but the all kinds of modalities, speech, video, et cetera. But if you go back to the way these systems are built on the human like brain-like structures, actually, they're good at things that humans are goo at, but they're not so good at things like calculating or logic, et cetera, right? So imagine that you could marry that sort of human cognitive-like capability with the system and also it can calculate, it can remember, it has access to a huge amount of information and never make a logical error. That's an incredible step forward for humanity. And I think that's actually going to be the next grand goal for AI, to marry the concept with the calculating, the reasoning capabilities of the machine itself. So I'm very excited about that. I think it's going to be a huge step forward for mankind.

Yes, I couldn't agree more on. I think it really echoes everything like Walter Isaacson was saying yesterday. It's almost the barriers to using technology are becoming less and less, and that's really part of our mission. And it's almost -- we have the components to make this safe, to make this responsible, to make it governable. And the kind of the question -- the responsibility is on us to build the -- we're not passive participants in this, to build like the processes, the tools, the systems we want our businesses to be based on. So with that, I want to just say thank you to everyone for having us. It's been amazing to have this conversation with you all today.