UiPath, Inc. (PATH) Earnings Call Transcript & Summary

Welcome everybody to UiPath's AI Summit for 2023. We're really excited today to have a number of different sessions. First, we'll have this keynote session here, and we'll have a number of breakout sessions across five different industries and three exciting product sessions as well. Don't worry if you can't attend all of them, we will send out the recording afterwards as well. And then finally, we will reconvene on a really exciting session about what's next for AI and automation. You definitely won't want to miss this last session. We'll be making some exciting announcements and also we'll be highlighting some new capabilities with product demonstrations. I can guarantee you will see some things here that you haven't yet seen from UiPath. My name is Luke Palamara, and I'll be your host for this session today. I lead the AI Products Group at UiPath. I'm very excited and honored to have a couple of very special guests that will join me today in the keynote session. A little bit later, we'll have David Barber join us. David is the Professor and Director of UCL's Center of AI and also a fellow at the Alan Turing Institute. He'll discuss advances in Generative AI and how it can be implemented in enterprise setting. But first, we'll have Boris Evelson from Forrester to talk about the current and future state of AI, including emerging tech and other trends. Boris, I'm so excited to have you with us today. Welcome to the session. And over to you.

Excellent. Luke, thank you so much, and thank you, UiPath, for having -- giving me and the Forrester chance to present. Really looking forward to the session. And, Professor Barber, David, really excited about what you have to share with us. But we, at Forrester predicted last year. We've been watching AI obviously, for as long as most of you on the conference with us today. But it was last year when AI, we believe, reached this critical mass and we predicted that it was going to become an indispensable trusted enterprise coworker. We started talking about AI-led digital workers. We started talking about AI in the loop applications. And indeed, today, we have confirmed those predictions. I mean our clients have confirmed these predictions. Our readers in a broad market, 74% of the respondents to our annual survey. Now these are global data and analytics leaders. These are global digital leaders. These are global IT and automation leaders. So almost 3 quarters of them predict that -- not predict, actually report that AI is providing positive impact on their organizations. The benefits of AI are almost innumerable. But today, we will concentrate on two top benefits. As you can see, automation of internal and external processes are the total benefits out of the top 10 that our clients are readers have reported to us. There are -- while there are multiple use cases for AI, but once again, improving business automation, improving business workflows, workflow orchestration and intelligent automation, as we call them, is one of the main use cases #2, out of top 10. And last but not least, in terms of the numbers that our readers report to us. So there are multiple techniques that are being used for AI. AI, it's not a single monolithic thing, right, that sometimes a media and reporters present to us in newscasts. It's based on multiple different techniques. And today, specifically, we will do somewhat of a deeper dive into two techniques for that AI is based on its knowledge base or rules or based AI. Sometimes some vendors call these capabilities symbolic -- reasoning or symbolic AI and machine learning based AI, all culminating again, there are multiple -- a myriad of use cases. But today, we're going to do a deeper dive into the use cases of AI for NLP, for natural language processing. Now there are multiple definitions of what the natural language processing is. You see on the screen a lot of fancy words in computer science and the way computer and humans interact. But at the end of the day, it's text to data. You've got a lot of unstructured text and we've got a lot of unstructured text in semi-structure documents. You can't really act or automate processes based on this. You first need to extract structured data from documents. And that to me is one of the main components of NLP. And what I just talked about really covers -- mostly covers the deferred part. An NLP market segmentation that we had forced to see that, that's the analytics part, right? Text to data, sometimes synonymously use the term that I'm sure some of you are familiar with is text mining, right? So text mining on the text analytics getting structured data from unstructured text, that's the first part. The second part is conversation. Intelligent chat bots, digital assistants, et cetera, et cetera. And a very exciting latest development that I know Professor Barber is going to cover in great detail is natural language generation. That's the term we've been using for a few years, but the latest and very interesting, exciting topic of generative AI. NLP is here and now it is not something that you need to be just watching or experimenting or prototyping. Earlier this year, Forrester proposed 10 top emerging technologies that everyone should be working with. And as you can see, NLP is smack in the middle, in the center of technologies, emerging technologies that you should be implementing. So not just watching, not just experimenting, but implementing today. If you don't do that today, you're going to be following behind competition. Now another interesting market segmentation. As you can see, NLP, just like AI, has many, many different components, and that's why sometimes when you get three people in the room, and they start talking about NLP, they may talking about 5 or 10 different things. So It is very important to understand the market segmentation, all of the different capabilities. So specifically within text mining and text analytics part of NLP, we see a very distinct market for people-oriented text analytics. This is unstructured text that people generate. All of you generate that when you write e-mails, when you post something on social media, and there is kind of a cousin to that market that we call document-oriented text. So it's still something that people generate, but now we're talking about not completely formed text, but semi-structured documents that have sections, that have forms and numbers and tables, et cetera, et cetera. Very, very different capabilities in terms of different requirements for volumes and complexity of documents and latency and what we're going to talk about in a couple of seconds, natural language understanding, understanding more complex concepts and just topics or numbers. Some of the typical use cases for each of the market subsegments, I'm sure you're familiar with all of these customer experience and contact send the conversation intelligence of the two most popular use cases for the -- for analyzing completely unstructured text, right? When you talk to a call center agent and that conversation is transcribed to text, that's completely unstructured, right? And then when we talk about semi-structure documents, documentary oriented text the most event use case here is ITEP or intelligent document extraction and processing, all of the other use cases such as data protection, information governance, some domain-specific applications like a legal industry is e-discovery or in real estate or any other industry contract analytics, but they're all based on this one main use case of intelligent document extraction and processing. And that's exactly what we are. Now I kind of hopefully narrowed everyone down to a very specific topic, very specific capability that we research at Forrester that UiPath actually supports. That's the main use case that we are talking about today. Again, just a quick sidestep. I already teed up the concept that AI is not a monolithic set of technologies. It is variety of technologies. And at the very least, we suggest that everyone thinks about AI in two parts. It could be knowledge-based AI, right? Even before the advancements in machine learning and the compute power that became available on us to run machine learning algorithms, complex machine learning algorithms, we've been coding, I used to do that 30 years ago. We were coding AI-type application, right? So machine learning doesn't have the monopoly on AI, but indeed ML-based AI applications where you don't code the rules, but rather you just give the machine the inputs -- required inputs and desired outputs and the machine figures out on its own, what's the best algorithm to transform those inputs into desired outlooks, right? So that's how we see another segmentation of the market. So what's very, very important and that's really the crux of my presentation, all of you really, really, really need to understand what happens under the covers of natural language processing and specifically intelligent document extraction and processing. Because what happens when things break, what happens when that wonderful black box that vendors are selling new stops working. So you really need to look under the hood. And what happens under the hood are a lot of very interesting and quite complex things. So you need to have the platform connect to all of the data sources that exist in your organization. Not only do you need to connect to those data sources, but you need to be able to extract and integrate and reconcile and normalize data from these multiple data sources. Text doesn't always come in the form of text. I already teed up what we call speech analytics, right, or transcribing a speech -- recorded conversation of contact center to text. A lot of documents have images on them. They may have logos. They may have bar codes. They may have cursive writing, so transforming images to text is a required capability. And that's one data ingestion integration for multiple channels, omnichannel, data integration is what we call it, starts happening. And that's the time when you would need to enrich the data that you're integrating with your domain specific lexicon, right? Every business, every industry vertical is very specific lingo and lexicon. And you've got product hierarchy economies of products and your business areas, legal structures, et cetera, et cetera. What we call ontology so you need to enrich the data with all that. And only then and only then at this point, we get to natural language processing because this is where we extract some simple concepts from a text such as numbers and entities and topics. Then we get to more complex, what we call a process natural language understanding more complex concepts such as a sentiment, emotion, intent, efforts, risk levels, et cetera, et cetera. And as I just mentioned a minute ago, all of these techniques can be either knowledge-based or machine learning based. And at that point, we got to the part that I was talking about earlier. Now we have structured data, right? Now, we have topics and entities and sentiment. And now -- and you can start doing all sorts of exciting things with that. So first of all, you can start analyzing the data. And most importantly, you can start automating your processes, right? And this is the crux of what UiPath does and what they're going to present. But very important to remember that this is not the end of the story. And that today, there is really no such thing as lights-out operation. There's always a human in the middle. There's always someone who needs to be verifying the accuracy of the system and retraining machine learning models and enhancing knowledge-based AI model, so there is always a human in the middle, human in the loop. You need to plan for that in terms of your budgeting and your strategy. So what is it that I'm trying to impress on all of you. So number one, do no treat NLP and intelligent document extraction processing as a black box. You need to look under the hood to understand what's happening. Understand the different AI techniques that are being used for each application. Because if there is a knowledge-based AI technique, well, who is going to be maintaining those rules? Is that you or the vendor, you've got to plan for that. If it's machine learning-based technique, well, who is going to be running the entire ML OPs cycle. So who is going to be verifying the accuracy drift, potential accuracy drift of the model and who's going to be doing retraining of the model, right? So today, and I'm going to jump -- my apologies, I'm going to jump to #4. Today, we still think that hybrid AI, a platform that's based on the combination of both machine learning and knowledge-based AI is the best approach for most of the use cases. Number three, do not get to plan and budget for human in a low price. This is not a full -- yet not a fully automated capability. And go back to that slide where I showed you all of the components and then -- and therefore, my #5 and my last recommendation, please, please, please buy these solutions. Do not build them. So now I'm going to transition to the next part, Professor Barber, I know, is going to have some exciting information to share with us. We at Forrester absolutely do believe that generative AI promises boundless potential, but to be careful. I'm sure Professor Barber will talk about this. There are still lots and lots of challenges and issues. We do not think that generative AI is ready to completely replace anything. So we recommend to our clients that they take baby steps and they use generative AI to enhance their current applications rather than completely ripping and replacing something. But I know Professor Barber can run circles around me on this. So I'm going to turn this over to him.

Great. Thanks, Boris. Wonderful to be here. Hi, everybody. So I'm David Barber. And not just at the AI-centered but I'm also a distinguished engineer here at UiPath. So today, I wanted to tell you a little bit about generative AI, how the heck does it work. It's a very exciting technology, but it's also maybe a little bit mysterious so I want to sort of lift the lid a little bit on how it works. And also where I think we're going in the future or so particularly that UiPath with this stuff. All right. So what is genitive AI? Well, essentially, these are our models. There are couple of models that can generate new objects. So there may be, for example, generate new text or images or sound or video, all kinds of different modalities, and maybe even a combination of those things as well. So this is a very -- it's a very old area, but the capabilities have really increased dramatically in the last few years. So you might think that actually, if I ask some machines that generate something. Well, you could just look it up, if you go on an Internet and can look up what I'm searching for asking to be generated. And to give you a sense that, that's not what's going on here. We've come up with this phrase a Chrome plated duck with the Golden Beak arguing with an angry Turtle in a forest. Now probably never in the history of humanity as somebody thought of this exact sentence, right? It's just totally -- you can't find this out there on Internet. And if you put this into a machine and ask you to generate an image of this sentence, this is where it comes up with. This one is from Google. But there's a whole bunch of related techniques out there. And it's pretty good. So he's got this -- even the reflections pretty accurately. It's not just simply looking stuff on the Internet or in some sort of database. So you can take this even further. You can say, well, let's use this to generate, for example, paintings like so you say, well, we generate an oil painting of some scene, make a description of it. And here is one. This has actually entered into a fine art competition. And guess what? it won the first prize. Of course, the entrant didn't say that actually it was a computer generated thing until he won first price, but it just shows you how far these kind of systems have come, they're really very powerful. Okay. So what else can you do with this stuff? So in a more presenting area, you might say, well, how can I use -- so text generated some generation systems to do things that are useful. So for example, here's one from open AI, ChatGPT, candy writer function takes a string input to capitalizes the first letter of a word. So that's the input, and the machine then responds with -- function. It generates some Python code. Even explains to you why it's generate code this way, and you can then run it hopefully, you that was very well. So these are really amazing things. They're really impressive. You can use these kind of capabilities in all different ways, for example, create job interview questions, you can respond to customer e-mail, all kinds of things. You can understand what people are writing in terms of the computer code. But in some sense, this is just the start, right? These don't necessarily bring new products that you could use sort of end-to-end fashion within an enterprise. So really the question is how to take advantage of these kinds of capabilities. So before we do that, you might also think how on the earth this will work. So remarkably, at least the text generation, these and what was the train simply to predict the next likely word. So if I gave you, for example, to training incentives is like the cat sat on the mat, the cat sat on the table. And then I asked you, okay, given the context, the phrase, the cat -- what is likely to be the next word? If you can look at your data state and say, well, a door go, is that range shelf, chair, and lamp, these are getting more likely. So actually, what the machine does is it tries to compress. And given the context in this case of the previous five words, what's the probability of generating the next word. And what is done, you can actually use that to take an input like the cat sat on something and then generate sychastically, one of the likely so it might take generates the cat sat on the mat. Now the major thing about this is that if you have a long enough context of the history of tokens or words, actually, these models start to get incredible capabilities. It turns out to be useful in order to able to predict the next word, it's useful to understand the structure of language itself, whether that's human language or computer language. So in this sense, you could then take a phrase like the cat sat on something, and it will predict the output would be, for example, mat. But if you keep doing -- you say the context on the mat. In this case, this is just a few tokens or words, and you'll predict the next word is you had and you get recursing this process and the keratin and generating longer and longer sentences. And in a similar way, you could do this for computer code, all kinds of things. Now in reality, the length of these things is quite long. There are several thousand words. These are very hard to train. Just to give you a sense, this is not easy. These models take months, if not almost a year to actually train. This is an open source project, going online. Last year, and it's on Twitter page. This is a very, very, very complicated project. And it's not a sort of a light heart task to engage with this. So that's fine. That's all fun. But in the enterprise, I think it's much more interesting to take these individual capabilities and put them all together to orchestrate actually how you might call these individual functions. And I think that's really where companies like UiPath are going to be major leaders. So you might have, for example, things like a database or a CRM or a web search capability and how you can call these will actually be orchestrated by the language from itself. So to give you an example of this, here's an input, which is a customer query from somebody who's complaining about some parts that are now being delivered. Then actually, what happens in this case that AI can read this model, it can extract the data, it check if this customer is it by -- customer in terms of a customer, you can check devalue, the database, the state of the delivery. And then we ship from these language models, things like ChatGPT to generate customer response, like the thing here from the left. And actually, this is really a great example of a time saving, labor saving process. You can even then alert the account owner that has done this, ask for approval from a human before actually sending the e-mail. And maybe you could update on database to say that it's actually responded to the query. So this is really an amazing way to save time and effort on people to get them freed up to do more interesting things. And we can actually also make sure UiPath that we do this in a way which is actually really going to be safe and secure for our clients. That's all fun, right? But how do you do this in a way which is actually scalable for an enterprise where you need to train these models on proper in your enterprise, how to do that rapidly, quickly, safely is something that UiPath would help you with. They've got to be guardrails. You've got to make sure that these systems don't make fantasy generations. They don't say bad things, bad to clients or call the wrong kinds of functional. Certainly, you need to have good humans down the loop to make sure that these models are actually doing the right kinds of things and to learn the from and when he says, well, actually, that's not correct, how do they want to improve themselves very quickly. So in summary, basically, AI is really exploding. The number of things which are coming out each week is just mind-blowing. This is going to have a major, major change in the workplace. And I think we hear UiPath is really in the driver seat to take advantage of these changes. But we do need to make sure that these systems are not going to go off the rails. They don't do crazy things. So enterprises need to make sure that these models are compliant. They're consistent. We understand as well as they can do when they are likely to work. And just to let you know, of course, these are super exciting times for us also at UiPath. We are very busy working on bringing these lease technologies to the real workplace. So thank you very much. I hope you enjoyed that. And I'm going to pass you back to Luke.

Thank you so much, David. That was an awesome dive into how generative AI works. And I think you made some excellent points to think about as the folks on the call here think about it as they deploy these AI technologies in the enterprise. Particularly, both you and Boris touched on the importance of keeping humans in the loop with AI to increase adoption and accuracy. And I think that's a really a point that really resonates well with how we think about the UiPath business automation platform here. And now, so you're all probably already familiar with the UiPath business automation platform, and it's comprehensive suite of enterprise automation capabilities. And while we call out AI and NLP in this graphic, what it doesn't show is the level of depth that AI has really already woven throughout our platform. So let's take a minute and so just a few other ways that AI is deeply integrated in powering some of the UiPath's most powerful products. So naturally, we should start with one of the key products in the discovery portion of our platform, Task Mining, where we utilize our proprietary computer vision and OCR models to understand all screens and interfaces that employees interact with it to track common processes. Task Mining applies task extraction models to help determine which tasks are best to automate. With communications mining, you not only discover and understand your business communications in detail, including intent and sentiment, through the use of our large language model, any user Technical or not, can utilize active learning built into the product to train, update and deploy new model versions. And most of you are probably aware, that document understanding is a game changer in the document processing space. And it utilizes a variety of custom models we've developed for everything from invoices to passports to insurance and banking forms. And in combination with AI Center, you can fully customize models or bring your own in to ensure that there's no document process that you can't automate. And then lastly, we have Clifford AI. This is the newest on the list. It's currently in preview, but it's also one that is most likely to be used by everyone in the audience today daily. It brings together everything from UiPath with their vision capabilities, to GPT and other ML models to enhance our everyday productivity for moving data between documents and applications, to filling out forms, to extracting key information from images, and we're going to be sharing more and demoing the latest build during the final session today. So ladies and gentlemen, you won't want to miss this one. And while we're sure that we built AI to all the rate places for our products, we also know that you want to build automations that use partner and third-party AI functionality. The UiPath platform is open. And on the right, you can see a wide range of powerful connectors and activities that can all be easily added in your workflows, allowing you to operationalize AI like no other platform really can. And we also provide options so you can bring your own models and deploy them in the UiPath platform, as well as call virtually any third-party model, no matter where it resides using custom studio activities as well as the ability to build your own connector with our connector builder. And so you probably all, at this point, keyed in on to the first two on the list of the right, the connectors for open AI and Azure Open AI. So generative AI has been on the top of mining for us too. And we are super excited that is available now in preview. And in the final session today, again, you're going to get to see a demo of these and details on how you can start easily using GPT directly within your automation using these new connectors. So it's going to be an exciting stuff. And so what's next? So we have a jam-packed agenda lined up, starting with our industry sessions. These will be directly followed up by our product deep dive sessions. And then last on the agenda and what I'm most excited to share is our final session called building the future of AI and automation from the lab to the road map. And I mentioned this session a few times. This is where we're going to be providing additional information on the open AI connector clipboard AI, our product road map and the winners of the AI Community awards. We also have some surprises in store so you don't want to miss this one. But if you do miss it or any of the other sessions, remember that all of them will be available on demand. So with that, thanks for joining us. Enjoy the rest of the 2023 UiPath AI Summit. Have a great day.

Hello, everyone, and welcome to our session today. Thanks for joining. I'm excited to get to our conversation around AI-driven automation in financial services. But first, let's meet our esteem panel. First up, we have Shameiz Hemani, he's the CEO of Greenlight Consulting. I've invited you to UiPath Platinum partner. We've been working very closely with since 2018. Welcome Shameiz.

Thanks, Bill. Pleasure to be here. Hi, everybody.

And next up, we have Nitin Purwar, who leads many of our industry engagements with financial services clients. He also spent a career in banking the prior to UiPath. So welcome, Nitin.

Thanks, Bill. Look forward to this one.

And to round it out. My name is Bill [indiscernible] and I'll be your host today. So gents, let's jump in, shall we? Nitin, let's start with you. Why do we need AI-driven automation in financial services? And what are we trying to solve for?

Right. I think when we look at the financial services landscape, I think it's pretty vast, right? So again, don't want to generalize what we do. I'll pick up maybe a few pointers here. So there are two broad kind of angles for us when it comes to application of AI. So one is the documents, right? Obviously, banking and financial services as an industry is very, very document heavy still. And the variety of documents we receive vary from structured to semi-structured to unstructured documents, right? And I think financial services organizations spent a lot of time in dealing with these documents and processing these documents. And especially the nature of documents start moving from structured to semi-structured to unstructured, the amount of time it takes to handle these documents obviously, increases exponentially, right? And this is where I think we have started looking at capabilities of introducing AI where you can actually play in models. So today, we have capabilities around our understanding machine learning models, which allow you to train your own models for extracting data from documents like the passport and identity-proof documents in certain markets. Then I think -- but we also realized that financial services is so huge that the amount or the types of documents you handle today can never be catered by our model. So we also provide you that capability to train your own models. So that's where I think a lot of training, self-training kind of capabilities come into picture. So that's on the document side. Now the other side of unstructured data comes in the form of the various channels of communication. So especially, we are seeing a big explosion in terms of e-mail communications and virtual chat-based communications which are happening between the customers and the financial services organizations. And I think that's another big area where we are trying to solve this problem of handling the unstructured communications coming through the banks, it could be in the form of e-mails or it could be in the form of SAT or SMS, right? So that's where, again, we have some models available, which allows our customer to understand those incoming from our customers, right? And then the last pillar essentially is for application of AI, especially in a world and we are sitting at a time where a lot of people are talking about things like ChatGPT and what we can do with capabilities together. I think we start broadening the overall scope of use cases for us when we start combining automation with some of these next-gen kind of capabilities which are in the market like ChatGPT. And then also, I think we are also doing a lot of things around analysis, like predictions and classifications. So we do have a lot of models available to do that. So I think the idea is to essentially find some of those biggest problem areas and try to solve for those areas using some of our capabilities in the platform. And I think we'll talk a little bit in more details around specific examples as we move forward in this one, but I think that's how we are positioning ourselves around the AI space for financial services.

Thanks, Nitin. It sounds like a pretty complex problem. So Shameiz, you've worked with a number of clients very closely over the years. And as Nitin just shared, the capabilities that are offered to those clients is continues to expand, right? So we're really moving beyond just core RPA. How have you seen that broader expansion that capabilities, the toolkit expand and start to really transform customer journeys. How are they deploying all of those integral parts?

Yes, it's a great question. I think we talked about business cases and ROIs, but one of the things that I think you have to step back a bit just to understand where we are today from where we used to be. If you think -- I think my first mortgage customers, whether in 2017, 2018. And if I remember, that conversation is really tactile. He's really focused on a specific problem they were having with servicing, right? You've got a patient property taxes, there's an escrow account, there's tons of transactions. Let's just fix that. And really, at that time, that's what the tool is really good at. You can go in and you can do an automation and off you went. It's been about 5 years, 4 years, the tools actually become extremely sort of expansive and good. And so most people that are watching this today, all of you that are watching it, your titles have probably changed, right? We've got Chief Automation officers. We've got VPs of automation. And people are asking us to do more. And how we can do that is we can actually just zoom out. And because of the capabilities of the platform, not only am I looking at sort of the servicing module for property taxes now, I can step out and look at the whole mortgage journey, both from a customer perspective, from a business perspective and from an employee perspective and impact that whole process, a real process automation end-to-end process automation. And just to give you an example today. The reasons we didn't do end-to-end automations before was, it's really hard to handle all these handoffs. You'd have to spend months sitting with subject matter experts to understand what's going on. With the continuous discovery tools now, we can accelerate that and watch things go across business lines, really easily, so we can capture what processes are and understand them really quickly. As Nitin was talking about with the document understanding tools and the communication mining tools. We can actually look at sentiment and understand what's going on in brain for the robot. And then, as always, take that RPA and be the hands and go out and finish the work. So all of a sudden, we can go back to our stakeholders and our businesses and our customers and impact the entire journey. I've got a customer right now who is looking at the entire mortgage area, and they're doing it because they want the customer experience to go better. They feel they're pretty efficient internally, but they don't feel that customers are seeing that today. So they're using the app to facilitate the workflow. We're using action center to do given the look for using DU to understand all this documentation that goes back and forth. And we're speeding up the process. And making it sort of more seamless, both for the organization, for the employee and the customer. Does that make sense?

Yes. I think that resonates a lot, right. I think a lot of organizations we're working with today are focusing more on who their end customer is, right, and how they can improve their lives in their day-to-day and their interactions, right? Less about the internal focus and more. What it means when they look outward. So Shameiz, let's stay with you. You mentioned the mortgage journey in some of those customer stories. Can you share some more details? What are some of the benefits you've seen? And kind of how does it all come together?

Yes, that would be great. Like I think we have the slide about the mortgage sharing up right now. So that journey is kind of important, and each step has a lot of importance. Let's -- I'll talk about a customer that we have that does a lot of non-conventional mortgages, they have broker-led, broker channel. And in the nonconventional world, in the broker world, it's not really 10 basis points difference in the percentage you give a customer that's going to drive whether they select your bank or not, right? It's speed of giving them that pre-approval. So historically, if you're trying to increase your top line, you get more transactions or you can get faster, which is when we look at ROI historically, you're thinking about, okay, well, what's the transaction volumes, how much am I saving? But here we are talking about speed to the customer so that they'll select you, right? So if you think about the other areas that are important. It's how fast do we tell the customer, yes or no. And then how right are we when we say, yes, on the pre-approval. And if you say no on the approval because we kind of did a little more due diligence, we spent more time, we looked a little deeper, we really like, "oh, we're not." That becomes a sentiment issue with that customer, they're not going to use us again as their bank. And so making that process clean helps a ton on the top line. So I'll spend a little bit of time on intake lender review and pre-approval, especially in the context of you, if that's okay. So let's look at how this usually works. And most organizations, they're not -- they don't have a seamless sort of approach through one system. So a broker will spend a packet. And that broker packet will have a bunch of things in it, right? Customer identification, their existing mortgage, tax forms, employment verification, a whole bunch of upside. Step one is just taking that packet and classifying it. Hey, this is a driver's license. This is a passport. This is a tax form. This is what their current mortgage looks like. This is their current address. And so document understanding is actually great at this, right? Historically, we couldn't do this really well 4 years ago. Today, classification is quite easy to set up and get going. It learns very quickly, and it learns along the way. So when your driver licenses, maybe something changed in your state, you don't really have to go back and retrain the model, it will learn as it goes as it gets use. So let's talk about it. Okay. We classify it now. We've quickly taken this document and we've cut it up and we've put it all in its associated system. So whether we put it in like file-folder system or in the loan management system, okay, we've classified it. The next -- in the old world, once someone did that, they would route it. They would say, okay, we got to this to an underwriter that specialize in this person is a sole proprietor, right? And there's -- there's five people in the organization that deal with sole proprietors, we got to just commit to something. If I looked at many of my customers, that could actually be the long pole in the process. Getting set of documents to the right person is pretty difficult. But now with DU, not only are we doing the classification of each of those documents. When we read them, we can start making assertions. Oh, I'm looking at this employment letter. This person is a sole proprietor. I know the sole teams, these five people. I can go to those five people's calendars, see who's the freest and give it to that person, so we know we can reduce the lead time. So again, we're taking that information, we've gathered from the machine learning model that we've pulled out of the documents, and we're routing faster. The other thing we can do is actually do a bunch more due diligence faster. So if you think about the worst thing that can happen to a customer, you get a pre-approval, you're all good, you go buy a house, you go back and they're like, no, we're not going to get a mortgage. And all of a sudden, you were alternate right now. And historically, the reason for that is because we don't do as much due diligence on pre-approval as we do on the final sort of adjudication, right? And what the robots can now do is take some of that and read it, work for you. We can validate IDs. We can make sure the driver's license is legitimate to make sure that picture looks right. We can make sure there's a signature on the document where it's supposed to be. We can validate that mortgage that they have is the right address and validate if that addresses the house but up for sale that we expect it to be. So all those things terawatt now do, and no human has come into picture to validate. What that does in terms of next the robot actually will create the risk score card. So all of a sudden, we're taking all that information. We're taking all of the individual pieces of that, that are important. We've routed it, but we've also could take that and give a risk score card. And risk score cards, there are sophisticated ones, but most ones are to just excel. Somebody has created an excel spreadsheet with a bunch of formulas that was going to give you a red, yellow, green depending on those or maybe or no. And what the robots can now do is prefill all that, validate it, standardize it across your organization and then go use action center to come in and say, "Hey, we've actually given this person the green. Is there anything you need to see underwriter, usually, they're like now and -- the approval. So something that could have taken 24 to 48 hours to get a pre-approval rollout, sometimes longer. Now, it's around 5 minutes, right, which is extremely impactful, both to the customer and to the organization. But now if you think about all that time the underwriter has, they're setting it on the outlooks, right? They're looking at those yellow ones and they're growing, okay. Should I underwrite this, the rollout couldn't make the decision because we've got a few things to figure out. And all of a sudden, you can be more right on the amount of times you say yes. And you might be able to say yes to more than you used to because you said, no, because you didn't have time to get to it, and there was in a yellow state. So those factors sort of drive the process really quickly and push out. The other thing that's great about this process is sometimes the brokers forgot something. And that lead time between you realizing they forgot and what the arm -- kind of drives a lot of sort of back and forth on lead times. The robot can do that for you because document understanding has now pulled out and said, "Hey, I still need this one piece. I don't have any proof of identification," and it will go back to the broker directly, instantaneously almost and then come back with that piece of information. For the customer, it just looks like you know what you're doing and you know how to do it quickly. And so that area, I think, we've had a ton of process, and then we can walk through some of the other stuff a bit later, but -- that's where we've used to the most impact.

That's really interesting. And it's a critical journey for any lending institution, right? It's a highly emotional time. And certainly, there's a lot of impact to improving that customer experience. So double benefit there when we can get customers and banking customers can provide loans to their end users, their end clients quicker, right?

One more thing, Bill, that I forgot if you don't mind, if I could add. Let's talk about the employee experience, we forget about that all the time, don't we. Underwriters are expensive. They know what they're doing, and they're fairly good at their jobs. You know what they don't like doing is the same thing over and over again. And what you're doing with this process is they're actually giving them all the stuff that's put in heart, right? The -- this is a cookie cutter. It's not stuff they can do to sleep. The ones that we talked about that were yellow really take experience and understanding and sort of brings out the best in that underwriting team. And so all of a sudden, you've gone from being pretty bored at your job to going back to how it used to be when you ask to do a lot more work. I don't want to lose that in sort of the benefits of this approach. Sorry to...

No, I appreciate that. And you're right. We often focus solely on the customer. There's obviously people supporting. There's -- we all ourselves are subject to that experience every day, right, and trying to improve our workday in life and how it all relates. It's a good point. So, Nitin, I know you've engaged with a lot of banking clients in recent years. What kind of impact have you seen customer make with AI? Shameiz provided a good insight from a document perspective. I know that -- we've done a lot of work from an e-mail perspective as well.

Right. Yes. I think -- and again, let me pick up from Shameiz's point around employee experience and then I'll come to the customer experience part as well, right? So today, when I look at the amount of e-mails received by banks, right? And obviously, COVID kind of accelerated the trend of using e-mail as a channel or virtual channel. So I think the people who are sitting in some of the customer servicing departments, I think they're spending a huge amount of time in reading through these unstructured e-mails, right? And I think that's where I think we thought that, look, can we actually look at some of those e-mails and have machine learning models, which kind of interpret those e-mails for our customers and then also do certain actions post interpretation of those e-mails, right? An e-mail obviously is a very generic term. So there are other things like looking at service requests or complaints and disputes, all those sort of things, which are received by a financial services organization. I think skimming through all of that data, all of those request obviously, is a very, very time consumer activity, from an employee standpoint, right? Now let's look at it from a customer standpoint. So I mean, typically, what happens in a financial services organization once an e-mail is received, and say that e-mail is related to, say, cards, right? You have specific department who's taking care of request related cards, then you have to route those e-mail that particular department. And then that department will look at the e-mail and then respond to that, right? So it peaks almost around more than say few hours. In some cases, it can take a few hours to say a couple of days as well in order for a customer -- for an organization to respond to that e-mail, right? However, a lot of cases, customers cannot wait, right? There are issues which are really, really urgent. And this is where, I think, models like these where you allow the machine learning models to interpret the e-mails, expand data from those e-mails. And then, if possible, also respond to those e-mails as well. It makes a lot of sense because obviously, it improves your customer experience. We brought down that time of response to customers by almost 60% to 80% or in many cases, even 95% where we were able to introduce auto response framework in a complete way, right? So that's one, how you can improve your customer experience as well. The third thing is it also allows you -- we have models today embedded, which allow you to also understand the sentiment of some of these incoming requests queries from customers. That also shows you what is the perception or where the customer is facing a problem, right? So we were solving this for one of the banks where the customer, the head of customer servicing asked us, look, can I actually get a detailed report of what is the sentiment of the e-mails or the requests that are coming from my customers right? And it could be -- there are certain products which are frustrating customers, there are certain products where customers are actually facing a lot of challenges and there's an immediate need to go back and look what the problem is. And that's where I think we have -- within the models, you will see that there's also a sentiment analysis part engaged, right? Now let me pick up one. We've done this for several banks. And again, across consumer, commercial wholesale and your capital market. There are a lot of instances where we have seen this kind of communication being applied across various types of e-mail requests or complaints and disputes. Let me pick up a very specific example here, which we did for one of the consumer banks, where we applied this for 3 product categories, right? So the product categories were, I think, cards, liabilities and loans. So these were the three product categories across which we applied this model. So the model first did a -- so it did two level of classification. One, first classification across these three categories. And then there were 32 other subcategories, there are three -- which the model kind of classified. So that's one. So you receive an e-mail, you categorize that e-mail and then you understand the intent of the e-mail. And the second is extracting relevant field level, fields from those e-mails as well. And once you have the fields identified, then you can -- and that is where you combine your RPA with machine learning models. So then what action you have to take based on the incoming request or based on the fields that you expected, and that's where the actioning part comes into picture. And this is where I think we actually created an auto response framework for around 30% of the e-mails, which were falling within those buckets, where very contextual response goes back to the customer, in some cases, like you might have to go back and look at your core banking data to find out what the challenge is. And then respond with a very contextual kind of response to the customer and kind of resolve that query. So it obviously -- again, putting some numbers here. One, I think we gave almost 95% accuracy in terms of the multilevel classification we did. And then obviously, we brought down the time. So typically, in this kind of a scenario, handling each e-mail might take 4 to 5 minutes. And that's when a person is actually assigned to that e-mail. I think obviously, we brought that time down completely because in cases where we created auto response and with journeys, I think, obviously, the time literally comes down to by almost 98% to 99%. So I think it's a great example how you can actually leverage machine learning models, and this is where I think we have invested a lot in our communication mining capabilities as well to make some of these channels, e-mail and servicing channels very, very seamless for our customers.

Thanks, Nitin. Definitely some eye-popping stats on that slide there, right? ROI is just really, really impressive when you think about what this particular use case is driving for the bank. It's very significant. So guys, let's talk about what's next, right? You're having a lot of conversations with clients today. There's a lot in the market, a lot of interest around ChatGPT, customer experience, a whole host of topics centered around AI, right? What are some of the most interesting conversations that you're having with clients recently? Maybe Shameiz, let's start with you.

Yes. I got to meet one because we're thinking about a recession, this might be apps. You can think about renewals in the mortgages, okay? Let's stay with mortgages, right? That's something nobody caps about, right? If you're a bank, and you have someone that's got a mortgage with you, the chances that they're going to renew with you are really high, right? And so when you think about the stack of things you care about, renewals kind of fall to the bottom. Now, we're coming into a recession or the market, the housing market might be slowing down. And so acquisition of new mortgages becomes a little more -- And all of a sudden, your eyeballs go to make sure you retain the customers you have. So we have a pretty cool. You working right now with the customer on the renewal side of things. So this customer is getting about 15,000 renewals a quarter, right? And we usually talk about, hey, let's figure out these problems that have all this high value in high scope. Renewals is one of these things that if you're going to lose about 5%. If you could save a percent, it's a huge number. But you don't know who to focus on. So we're working on a machine learning model through AI center that looks at historical renewals across organization and picks which accounts if things are going to be problematic and maybe won't renew. And then it takes that and hand it to a customer service rep to go accident and go talk to somebody. So before you actually just wouldn't know who to call, you kind of try to call everybody and guess what, you call very few people and you were guessing. And now we can take that small team and focus them on things we think are going to be the core issues, so that they can have a better shot at getting a renewal on the ones that they should be read about, which I thought was really neat, right, and really recession centric because keeping the accounts have clearly important.

Yes, fascinating, right? I mean it's really the introduction of different technologies focused on revenue, right? We spend a lot of time with talking about automation and what it can do from a cost savings perspective, but retaining, capturing revenue fascinating. Nitin, how about you? I know you were in some recent client conversations. What was the topic and kind of what was the sense on those clients?

Yes. And again, I know some have mentioned about downturn good. But when we look at these test situation, right, obviously, clients look at more personalized services, right? So one, I think, obviously, today, in most of our conversations, I think we are talking about outcomes. How can we make offers. So how can we create more personalized in nature, right? So I'll give you one example, where -- today, we are working with the bank where we created models for classifying expenses of the customers, right, across different categories, right? So say if I spend more on travel, for example, I would need offers which are more related to travel. So how can you make your offerings more contact to all the customers. So we are actually having a lot of conversations with customers where we are looking at data lakes and how we can actually pay motion-learning models using that data. In case of IPA, obviously, we have some good machine learning models, which allow us to train new models like this expense classification living expense classification kind of things. And then obviously, our next logical action is going to be classified, launch any contextual person. So that's number one. Then obviously, the next big thing in the market is ChatGPT. So today, no conversation goes without having a discussion around ChatGPT. And again, it's very relevant because GPT obviously opens a host of use cases around summarization, and I was having this discussion with one of the leading wealth management firms where they said, "Look, can I actually combine, say, ChatGPT with automation to create pitch books?" Which are very contextual, right? You leverage automation to connect with ChatGPT, click -- enter the relevant details and then pick up very contextual data and then obviously use the power of automation to bundle that up nice-looking presentation, which can then be used by our wealth advisers as well, right? So I think we are finding a lot of interest on how you can combine automation with some of these generative AI tools coming in the market. And I know that a lot of customers have already announced those partnerships, UiPath -- it is also a lot of work around integrating with ChatGPT and launching a host of use cases that can be solved combining the two technologies.

Yes. I think you're spot on ChatGPT, right, top of mind for pretty much everybody in the market today. But I appreciate both of those stories. I think they're very exciting and -- and exciting as we move forward, right, and to see what's next. So thank you. So that does bring us to the end of our session. I want to thank you all for spending some time with us today. And of course, thank you, Shameiz and Nitin for sharing your insights. We'd love to continue the conversation. So please be sure to check out the links in the Resources section for more information and enjoy the upcoming sessions. Thank you, everybody.

Hello all. Welcome to UiPath's AI Summit. And in this insurance breakout session, we will be interviewing Andrea Simpson, IT manager of Robotic Process Automation at USI; and Thach, who is Director of Digital Innovation at Hub International. And the topic that we will be discussing today is transforming the insurance operations. While optimizing cost, how AI-powered automation can transform the operations in the insurance industry.

Welcome, Andrea, and welcome Thach. Just as a quick background, in 2018 and before 2018, the automation was looked at as taking RPA as a hammer and chasing the nail. Mostly in terms of bottom-up automation. And then it moved into insurers started thinking about doing a end-to-end automate. But still, they are all going to be purely repetitive, 100% at patches automations. And then the situation came where there were opportunities where the cost levers have been merged into customer experience levers. And the best way was enabling the human workforce -- and that is how the executives started looking at automation. The key reason being in 2022 and 2023, there was -- even 2022 was called as a year of great resignation, where the workforce change happened, and that actually increased the operational cost because a lot of people were moving laterally, but for a better salary. And also, the remaining workforce was looking at adding value to the enterprise and thereby getting a better deal on their salary. So this was happening. And many executives are thinking in a comprehensive way of automation that the automation of [indiscernible] can provide maybe a 8% to 12% efficiency, can I expand the data to 15% to 20% of efficiency by giving the human in the loop and handing over the automation to humans for a better decision-making and more accurate decision-making thereby delivering customer experience. So given this background, I wanted to ask you some questions to provide a pragmatic insight to our audience. And the first question, Andrea, I can field it to you is, can you please walk through your automation journey from the time you launch it as a RPA program? And then now you are already embarking on your journey on expanding that with AI machine learning skills. And you can include some of the key challenges, learnings and what are the state of the program. Can you please provide us some insight here, Andrea?

Sure. Thanks, Sathya. We started our automation journey in 2019, with UiPath. We actually started as a directive from our Board to find places to innovate. So we started looking for ways to use newer technologies to imitate [indiscernible]. I had some experience with robots previously for more simple things like work creeping. But we -- I took on in the past of looking at areas in the business where RPA can really solve some more issues and can be tested. They start with our accounting functions actually because those are generally standard repeatable processes. And from the time that we launched until we decided to actually bring AI into that with about 3 years, we started with some more simple things, and now we've started incorporating AI into that so we can get a better usage of the RPA bringing AI into it as well. Currently, we have around 15 robots in production with several more rolling soon. And our senior leaders and Board have been impressed with the adoption and success of our RPA program and really want to continue to see us push that into more AI spaces to solve some of our current issues. One of our biggest lessons learned and kind of key takeaways in that it's on sale. There's still many different areas where you try a robots or RPA doesn't go as of instability as systems that you're integrating with or lockouts from a vendor just network fleet why. So we've really taken away from that, that we try to quickly determine is this going to work? Is it not? If it is to going, if not, say, okay, so free to sail and keep moving. But we really went from just testing out and seeing some areas where we save some time with RPA. A couple of things that we learned was -- we have some staff that is a high turnover just due to the great reputation that you mentioned. So we've tried to solve our -- those areas by RPA and now entering AI. And we continue to move that forward throughout our current -- as bringing AI in.

Okay. Thanks for that insight, Andrea. Very interesting that your Board has decided to actually launch the RPA program, but also gave you some time line to go test and learn, and then provide a very big baseline. You are thinking strategically, right? So you need some time to do that and then provide that the foundation or baseline so that you can go for a moonshot. That's a great insight. And also definitely on the learnings, learning for [indiscernible] where the failure points are, how we work towards it. That's a great one. On Thach, coming to the same question I been asking to you because you have also branched into AI mission learning on one side and also the human in the loop on the other side, right? Like how do you seamlessly transition that the repetitive automation into a much more meaningful way of looking at the automation and also you have an excellent framework of business IT collaboration. So I wanted to ask the same question. So how was your journey and what are the different lessons learned and how you made it successful?

Thanks, Sathya. One of the things that I think our journey is similar to everyone else where we have similar stages, right, where we have adoption move into scale and then we move into expansion, right? And I can talk a little bit about how we determine that we got to each one of those phases from the perspective of adoption. One of the first thing we did was from the business side, similar in terms of the accounting and the finance team, right? Because there are a lot of things that we've done manually and a lot of data, right? And so it fits perfectly with automation opportunity, right? So from that perspective, adoption was very well received and adoption enable us to start the program at where we start giving funding, right? And through that adoption, what we've been able to do is scale out our program, right? Scaling, meaning from everything from the technology perspective, giving sure that we have the licensing that we have, make sure that we have the partners that we deal with, so that way we can deliver as much as needed and as much as in time constraint. And then we also now scale out in terms of the products of UIPath, right? We use everything from AI Center to Automation Hub to Communication Mining now. So we're really scaling out from every perspective needed for automation going forward, right? And then we feel like we're at the expansion stage right now. What I mean by that is we're able to bring on new projects, deliver more solution at the scale that the business needs, right? So we are expanded because we are no longer a resource for the business. We're actually a partner for the business, right? So I think we've grown to a level where automation, which is a tool, but now it's become an enabler for us, right, and we're no longer a resource or a partner with the business so that we can expand on, one, the delivery and two, of the business from a multiple perspective. From the time perspective, from the cost perspective, and from a scope perspective, right? So we're able to help the business delivery with great velocity, okay? Now that, along with being able to realize ROI, right? We do monitoring, we lock, so we can report out those ROI. And then there's scope, right? Because we're enabling the business to leverage automation as part of their strategy and vision will enable them to look beyond what they saw before, right? And so now they're giving us really as a partner as intricate part of their strategy to move forward. And part of that, too, is we've been able to use now the enabler for them. We enable them to reduce complexity, enable them to leverage the data more effectively, right? It's not only volume of data. We enable them to use data enrichment data, right, along with the solution that we provided.

Thank you, Thach. It's very insightful again. I'm not very pragmatic, how do you start a journey during the proving years, you started as a technical project and then move that into your business technology collaborative initiative because that's very important, right? Like because of the art culture also where the innovation is bred into always that business technology collaboration is at a higher level. And the success of delivering those digital initiatives have also been higher, I agree. And Thach, again, asking this question, you mentioned about the business value that you have started delivering. What are the top functional areas and also top 3 use cases that you see that there is a tremendous potential that you were able to unlock the capacity or deliver business itself? And what are the steep KPIs, right? As you see only the cost savings are the capacity release? Or is there anything broader KPI that you were able to deliver?

So first, I will go back to the other. The biggest KPI or rather value realized, value proposition that we've been able to do is enable the business to use a lot of levers that you mentioned, right. Before, without it, without automation, they weren't sure how they would scale in terms of there will always be manual test needs to be done. However, now it's not about clicking of automation. Now it's about decision-making. Now it's about analyzing data. Now it's about aggregating data, and now it's about reducing complexity. So those kind of soft KPI that enable business to trust us more, and therefore, really becomes more of a partner with us and really see us as an opportunity to reduce the gaps that they see and literate before because it's not part of their solution mindset before. Now automation becomes their solution mindset and that enable them to, one, new market expansion without fearing that, hey, you know what, we can't keep up or that because automation really enable business to scale and expand with their own strategy. We don't go in there and change our whole strategy, right? We enhance their strategy, we enable their strategy. So that's 1 of the reasons why we've been pretty successful with the business area because once -- like the old saying, right, "Success reap success." Once we start giving success to engage in with 1 business area a another business there comes in and say, "Hey, you know what, we want to be able to leverage you the same way that the business area has." So within how a lot of the business area or a lot of automation that we've done is really about brokerage management, handling how we handle data from brokers, how we handle document for the brokers because as you know, insurance are a lot out of documentation, right? And other part of the business area is really, we do a lot of it. and a lot of M&A requires data migration. And so we've been able to help the business area to integrate a lot of the M&A acquisition into our system very easily. And that's one of the reasons why I mentioned data as being very important, and we've been able to automate a lot of that for the business. And then obviously, there's the operation side, right, a data update from everything from our CRM to our broker management system to our accounting system, right? So a lot of data entry and also a lot of data updates and correction, and how that enables the business to be very effective in terms of not having to do rework or be able to find gas very quickly.

Awesome. Thank you. And Andrea, again, coming back to you on. What are the use cases that you captured? And how do you look at that as a benefit lever? And then setting these use cases, expanding into the AI as well as delivering next best results. Can you give us some insights there?

Yes, absolutely. So a couple of the ways that we've really found use cases is around our direct dual commissions. That's a huge area for us where we've put a lot of time and effort into not only building our RPA side of it, but also introducing the artificial intelligence side. For our brokerage leaders, you have thousands of different templates from different carriers of commission statements they all look different, even the same periods and do a lot of different ones. We're able to use the AI there and machine learning for it to tell us and post our commission statements for us. So bringing that in has made a huge difference of how we're able to actually process direct-bill commission statements in a timely manner and turn those around with a lot less staff required to do that work. We're looking at probably an annual savings of over $1 million a year just for the trial commission posting once that's final, fully automated the way we have it working now. We're in process of integrating that AI side of that right now. A couple of the other areas where our use cases have proven really successful is just getting our underwriting data into our agency management. So it takes a lot of effort to get application information from a client, if that entered into the agency management system because there's a lot of data that goes along with February application. We've been able to fully automate that process where we can take the application, imperation, extract it, enter it into our EMS, and that gives our producers and marketing teams time back to do what they do best, produce. And market to carriers, here the policies that we're trying to place. So those have been two big time savings areas. And then something as simple as one of our first automation was automated remittance e-mails, just doing something as simple as sending out payment remittance advice to our carriers. That alone saves us like 800 hours a year in man time. Just to do something like that because with every payment, we were adding to full report, and e-mail. So we've found great success in everything from the simple things like automated e-mails of the more complex things like the data entry of applications and then the even more complex direct build commission hosting. So we found a lot of success with that. And with each one of the formations that we've done for these use cases, that has spread into other areas of the business. And now we have a list of people from the business going to pay, we want and automate this for us, automate that for us. And so really, we've started building a lot of big pipeline of here's automation, here's where it makes sense for us to actually automate those and building a center of excellence around that so we can build in a timely manner on the things that make the most sense financially for us to one. Did I answer all your questions, Sathya?

So it really resonates when we discussed about like 2 different areas, right, so Thach mentioned about the acquisition merger and the inorganic and organic growth. And you mentioned about how fast you are growing as a right, the growth. But the growth with to optimize the cost and optimize workforce that supports our growth is resonating well in both the stories actually. Then sort of asking you a little bit leading-edge question, I'll start with the tack here. What are the imperatives, the next year or this year or next year, imperatives that you see that the insurance industry is going through and how this yes capabilities are actually supplementing some of the imperatives or satisfying the imperatives. And most importantly, what are the major concerns or challenges that will be actually hampering the adoption of AI and how you are planning to supplement that with automation so that you can remove those challenges and help to drive the future industry imperatives. So can we get some insights there?

Sure, Sathya. So here's a question after you for our imperative if you are patent for us automate GPT, right? And so what we'll be able to do is grow through a lot of -- without having to do manual work, right, not only manual work but big business area, but now you work in terms of developing automation, right? Because automation itself requires some development right now. And one of the imperative we're trying to get to is to automate smartly intelligently, right? And what does that really mean? Leverage AI machine learning as much as possible to gain insight to gain opportunity and to even code themselves, right? And so as Andrea mentioned, document processing, right? When you have a lot of document out there that are different type -- a lot of variety, right? -- you want to be coded for each one, right? You want to machine learning to be able to recognize a able to extract data effectively, right? So I think that's the -- one of the biggest imperative is how do you automate intelligently without having to deliver manually, right? And I think that's one of the big things. And then from that, what it does is allow us to service the business more, and therefore, they can service the customer more, right? And so one of the things that we're also trying to do is increase engagement with the business in terms of aviation. So what we were able to do is create a chat to allow them to submit idea, right? And that really build up our pipeline because we're -- we've been able to scale and we're at a stage where now we can expand, right? And so expand, we can be reliable because we know we can be -- we can deliver a proven solution, and we know that we can scale them, and we know we can repeat them, right? So all those things allow us to support the business confidence, especially when they want to do new products, there was is agility, right? As you know, insurance can be a commodity, but also could be an opportunity, right? There's a lot of opportunity out there. Your product and services in agile, right? And also, too, to support that, you really need operational resiliency to make sure that it runs smoothly, right? Getting the business is one thing, keeping the businesses sometimes can be harder, right? So I think that's where the operational resistance comes to play. And automation, intelligent automation enable us to be confident that we can be resilient because decision-making, we can do delivery a lot faster, right? And then as the diagram here, new frontier innovation, right? I was kidding about automate GPT. One thing I follow out there where you can actually ask about just to do a U.S. design for you by just giving a sentence. So it's not only conversation, but actually delivering automation for you, right? And so I think that's where I see a future for automation here and especially for you at top. The action is taking in terms of communication mining, a lot more intelligent product coming.

Thank you, Thach. And Andrea, if you can provide some insights there, that will be great?

Absolutely. So one of the things like every other industry, the insurance face more challenges hiring and retaining staff. And for everywhere. You have to be able to hire, retain staff on a consistent basis. And a lot of especially younger employees tend to want to have all of the tools available to them to automate. So I think one area that all together, that we're pushing towards and will be imperative in the near future is having a set up like a lot are user having automation tools right in the hands of the end user. It's like a big push that will come and is already starting to come where we really need a Bot Fever user. And then the other big area where it's always going to be something I think we're improving and moving forward is getting data like Thach said, data is kind of everywhere and everything. We need data to be able to do everything from retaining clients, getting new clients, modeling, rating, all of those things, you need the data and the insurance industry has historically been a little behind the curve in becoming electronic. There a lot of our data, but that we're really moving towards the future now of intelligent automation needing to use that machine learning to retrieve data, so we can use it for all the many things to really get to that new frontier, which is the GPTs of the world and being able to have some of that where we can say, I need a GPT give me a lot where I can retrieve the data from this form, that's standard to the industry and not have coders in the back end going and trying to formulate all of that. So big areas, I think will always be moving further and further into the data mining world and using intelligent automation there as well as really we have to retain talent, and we have to hire that talent. So behavior analysis kind of folds into that to hire those talent and then retain them with giving them a lot for everyone. Well, that's what...

Sorry, sorry, Andrea. Go ahead. Go ahead.

That's it.

Okay. It's excellent time because what we have also seen is that when we talk about data, brokers are in a very pivotal place there. You know the data -- customers' data better than anyone else. And the underwriters are actually in the downstream to get the data and underwrite policy. But before that, the risk management, not just to the broking, risk management is where most of the value is based on that you can add value to the customers and give them the right policies, the right coverages, the right to match the right risk at the right pricing. That is -- so the data is very fundamental. That's a great insight, Andrea. And coming to the last session of our -- last section of our session. And can you provide us some key takeaways or tips for the audience who are starting the AI journey?

Absolutely. So one of the absolute key takeaways from us is get senior-level engagement early and often. It's been critical for us to have that buy in from either senior leaders or our Board throughout the process. Don't be afraid to take time to build up. You don't have to have everything automated day 1. Test and see what really works for you and build up from there. That's really been our key takeaways. And advice is just make sure that as you're automating, you have of BCP plan going on as well, what happens if you're down for some period of time and SLAs with your business partners. If you have those in place, your business partners are going to trust you. And as you automate things they're going to see the value and really start pushing and being early adopters and really taking off with it and selling you to everyone else because they some success with your team. So that's my key takeaways and tips for the value.

Awesome. Thank you, Andrea. And Thach, if you could provide your top tips for the audience to take home.

Yes, I do agree 100% with Andrea there. The things I'd like to add is as automation mature also, right? Our takeaway, our kind of rope to automation can be different for someone who's taking on right now, right, because the product, the industry, the technology has matured so much that someone can leap frog us if we were just told them, "Hey, you know what, do this small automation would click first."They're like now that I can produce that for right click from a software somewhere, right? So also look at where the state of automation is the product out there, the tools out there, right, can do a lot of things that automation was not capable before, especially intelligent automation, use that first rather than manual kind of click automation, right? And the piece is also to is they'll sell automation short, meaning we can't do what I need to do. If they worry about someone having tribal knowledge or, hey, there's so much decision-making? Well, that's what intelligent automation does, right, is then able to make decision for you based on data because even human being, sometimes we don't know everything. But obviously, the bot can handle as much data as you need, and that's where data is critical, right? To have intelligent automation, make sure you have the data out there and also 2 is a cycle, right, where the bot would enrich the data for you, validated for you in order to help you make decision-making, right? So I think that's really going forward. I think that's really important.

Thank you. Thank you, Thach. And thereby, we come to the end of the session. And thank you, Andrea and Thach for your pragmatic insights. This has been an excellent webinar, and we are actually happy to host you and get all your insights. And thank you, audience, for your being here. And for further insights, please connect with our industry leaders. One is Elaine Mannix and me. We together have 60-plus years of experience in the industry and happy to connect you with some of our industry-leading customers and have a conversation on how do we automate it with AI and mission learning and provide typical insights for the employees to make better decisions and drive your growth with optimum cost and efficiency. Thanks a lot, everyone.

Hi. Welcome to the UiPath AI Summit. This particular segment will be devoted to document understanding in health care, so driving the mission impact in health care with AI, automation, document understanding. So I'm Lisa Weber, and here are our speakers today. I'm Lisa Weber. I've been with UiPath for about 5 years now. I am in the provider space. And my background has always been within the hospital operations and trying to understand where automation and technology can play a part in really elevating the care we deliver to our patients. So with that, I'm going to turn it over to Javed and ask him to introduce himself.

Thanks, Lisa. Hi, everyone. My name Javed Ali, and I'm part of UiPath Healthcare Industry practice team, here. I'm also going to complete my 5-year with UiPath. I'm focused into healthcare payers business? I mean, how and what can be utilized there? What are the technologies with UiPath and how UiPath AI center as well as Document Understanding is helping our customers. Thank you.

Great. Thanks, Javed. And we have a distinguished guest today. We have Suresh Kumar. He's from Exponent Health, and I'm so glad that he is doing this. So I'm going to let him introduce himself right now. Suresh?

Hello, everyone. Thank you for inviting me to this panel. I'm Suresh, I'm the Chief Transformation Officer for Exponent Health. But what we do is for out-of-network claims, we price them and also help a lot of providers with obtaining the right contracts, right, fair price for the out-of-network claims. And we provide no surprise delight solutions as well. I've been in health care industry for 25 years. I've been on both sides of the payer and providers. So I understand both sides are the problems. And we leverage technologies very nicely with some of the things that we are doing here and excited to share that with you all.

Great. Thank you. So just to kick it off, gosh, one of our biggest challenges is the paperwork, right? We've so much paperwork in healthcare and provider space and the payer space. It's -- they're saying about 2 hours every clinician spends on managing their paperwork, looking for the paperwork, transporting charts, manually entering the data into electronic systems. So there's just -- I had it from my experience, and so job where does paper play a huge pain point in the payer world.

Yes. So basically, if we look into the payer side of the business, we are starting with onboarding of the member providers. We are having claims intake, claims reviews. It might be a clinical review. We might have some threshold for the claims. So let's say we are looking, okay, we need to validate or we need to review the claims, which are coming up over $100,000 or $200,000. Or we might look into the utilization review where we need to check the if all the details, which are being utilized by the providers as per the medical necessity. I mean these are the areas where a lot of documents are being utilized. So we have seen -- we are having a claim intake or authorization intake analysts, which are sitting in on the systems, they are validating the document. They are looking into those forms, maybe a CMS 1500 form or any type of Preauth or maybe Appeal form where they need to type all those information into system. And because they are typing that, they might be missing some of the information, which is actually taking all those claims into our editing queue, which is actually impacting the auto detection rate for the player. And when we are again looking into the auditing of those claims or authorization reviews of those claims, again, a bunch of pages are there. So medical history, we might have 50 pages, 100 pages, how those information, which is lying somewhere in between maybe at the 50th page or maybe 70th page. However, U.S. RNs are actually drilling down or scrolling down and looking into those information and wasting their time. So these are the challenges which we are facing to healthcare payer one. And also, I just want to add that if we look into the provider side of the RCM business we are having submission of the claims, categorization of the documents, taking those into specific categories. And I mean we are having a different type of correspondence, maybe denial correspondence, where we are getting multiple documents, how we can categorize how we can expect main information out of those. So these are the different challenges, which we have seen. And I would say, Document Understanding from UiPath is actually helping our customers to look into those areas of the automation I mean we know that we can start automation on our simple to automate processes, but these are the areas where now the customers are looking and they are finding big opportunities for automation.

Well, that -- you really solidify the fact that there's a lot of paperwork in the healthcare world of both the payer and provider carries. So document understanding, what is some -- it's not really OCR. It's not computer vision. We kind of describe it as the intersection of document processing, taking a paper or any PDF file or anything. And giving that information use artificial information, AI to extract that information into these discrete pieces and then we use PA to take those discrete pieces of information and put them into another application. So again, extracting data and using AI to interpret this information and the meaning from a wide range of document types. It's easy to create new and also deploy our pretrained models that UiPath already has and document understanding claims forms, the CMS 1500. We've already pretrained our model to do this. So we also have with Communications Mining platform, which starts to understand the such analysis of what it's at and assisting with the sorting, responding and routing of those documents and that information as well. So why is it so beneficial in healthcare? Well, these are the key takeaways is that it really enhances the staff experience and eliminates all the paperwork. So all that paperwork we see really gives a boost gives people time back in their day. When the administrators and the clinicians and all the others that have timed back in the day, what happens, it improves patient care delivery, both the payer and provider aspect. Kicks of that ability to have instant and up-to-date information, again, improving the patient care delivery. And it increases the data accuracy and input. No more errors from taking information off a piece of paper and entering it manually into another system that's reduced to so the errors and the lead work. So now let's dive in. So Javed, you've kind of explained where you've seen a lot of the big areas from the provider side, payer side, we're going to provide just 1 example from the payer provider side. What we see is a lot of our medical assistants. They're on the front line. There are so many facts that come in. What do they do with these facts? They have to get that information from the facts into the EMR. So that's where we're leveraging our document understanding, extracting information from these fax machines and then putting that information directly into an epic or serve in an appropriate field. So again, information is there instantaneously. There's no rework, there's no errors. So excited to get to use, Suresh. So help us understand like your business pain points and why you thought, hey, I got to leverage some document understanding some artificial intelligence to really boost my business.

Yes. In my experience, I've used 2 areas out of all the things that Javed mentioned, as we said, there are many areas where we need help. And certainly, I used it in 2 areas. One was in the provider side where we are taking the remittance advisers that come to the provider from payer to the provider and digested to see how much should be the final patient balance should be. And there is a huge data processing that happens there. And now the key area where we leverage recently is the place where the claims are coming in. A lot of still, there are a lot of claims that are coming in manual, and we are using those manual claims and there was a team of operational staff that usually put that data into our system. Again, there could be manual levers as they are typing in as well as it increases the time in which we can return back the pricing. And that was a problem, like the efficiency, the speed at which you are able to price the claim, send it back so that water gets paid in a timely fashion. And the patient doesn't have to worry about the additional bills that keep coming from the provider. So there is -- we were able to address the speed aspect as well as the capacity aspect. Like every day, we could only do about 75 claims a day per person in that manual process. Now with the overall automated process where the bard logs into the payers claim system, post the image from one of the drives and then actually comes into our system and entered first document understanding digest all the information that is present in the claim. And if there is any validations that it doesn't pass through, it's automatically able to kick it off, saying, hey, there are errors here, so it goes back to the payer so that they can address them. And the ones that are actually done moves into automatically are entered into our system. Pricing is done and written back. All of this happens in a fraction of minutes like the whole process completed now compared to it probably would have taken a day for us to get that. So that was very useful for us. And it just -- one thing I would say is our operational leaders were very keen on bringing efficiency into this process. That's the key driver was as much as I would love to use more and more and more technology. It's also bringing the operational leaders along in leveraging this technology, stitching the new processes understanding what can be improved even if it was not designed that way in the beginning now with the technology, how do we need to rearchitect the process so that they can take better advantage of that. And they were all in it so we were glad to see the benefits of it.

That's fantastic. So how long does this take to get up and running? Was it an extensive period of time before you started realizing value? Or tell us about the roadmap.

It was actually fairly quick given that we are dealing with standardized format, right, like [indiscernible] and the UBO 4s. So we were able to -- again, the document quality was different at multiple times. So we did have to train it. So within a span of couple of months, we were up and running. And this included the complexities of not just document understanding, but also connecting into the client systems, pulling the claim. That was actually the bigger part of the problem was we had to use the UI and DU computer vision as well in order for us to be able to navigate through the client screens and go full the image and then put it to the Document Understanding and send it to the action center, if there is anything that needs to be manually looked into. So I mean, as you mentioned that not only the DU part, but you were able to connect to the customer systems and able to utilize computer reason to extract the information. Now how you were able to identify that, okay, the UI part DU is the best document data extraction capability, which you should use. How are you able to choose?

Two things that went into the tool selection. One is we didn't want a pure play just a document understanding solution alone because then we would have had to build a lot of integration components along with it in order for us to be able to connect it as well as design and workflow around it in a manual process around it. So there are things that I would have had to build a lot more components with UiPath since the lot of adapters that came along with it, like so for connecting it to Citrix and stuff you guys already have those adopters that make the life easy. And also, when we have to do anything with the web API calls you have I can involve the entire process with an API. So that's the next use case that actually we're working on we have mobile app with a number of experience at members to take a proverb picture of a provable. And that goes to UiPath and UiPath automatically digested and then send it back to us so that we can actually analyze and see if a member was balance built or not. So the -- I would say, the ability to easily integrate with our existing processes as well as ability to build all the workflow and the additional components needed is what made us go with the UiPath.

And so -- just one thing to add here. So now you are expecting information. You are pasting that information into the specific claim system. We are also having a capability which might add will in future for you, which is extracting those information and converting directly into EDIs if that would be required. So we are having that capability as well. If you need to convert EDI, so CMS 1500 or [ UB4xEDI37 ], that can also be achieved quickly. Now when you identified and other capabilities to use into the specific use case. You started working. What was the time -- I mean team has taken to complete and to go into production and have some value out of it?

Yes. As I said, like 2 months, it took us to just complete the entire document understanding completed and stitch the workflow and probably a few more weeks of testing. And within 3 weeks, we were up and running, and we dialed up to almost like, I think, 400, 500 claims a day, pretty quick. I mean it was all a matter of like 3 to 4 months. We were up in production, realizing the value, and we were able to allocate the resources who were manually doing this work to other value-added services.

Well, so if I'm not wrong initially, you were able to do only around 70 claims a day. And then after automation, you are able to achieve that threshold up to 400 plus?

Yes, Yes.

Do you have any other -- do you have projects on your road map? Any other areas you're going to start expanding to?

Yes. So there are -- as I said, like I'm fascinated by the lot of tools at the UiPath is a toolbox. There are so many tools in it, right? We're looking at leveraging even the testing manager is another area where we are looking at a lot of regression testing speeds earlier. We had built some in selenium, but maintenance becomes a big development exercise. We're trying to shift that into more of a business-driven testing. So we are moving it -- we are experimenting with now test manager to see if we could automate build the all the regulation test speeds and the analysts can maintain it without having much of a technology development knowledge. That's one area we are exploring. And as I said, even expanding the use into more of a provider build aspect where we can actually look at and see if there is a balance bill for that. If the number has been balanced bill are not so we are leveraging it. And that is a more real-time analysis. So the app is a -- each member downloads this app and as their claims get finalized, they get a notification, and then they just snap the proved a bill that they have received, and it automatically reconciles to their EOD. And the moment they snap it, we are sending it over to UiPath via API calls. So the orchestrate is modeled into an API. So we picked it up from there. And then it actually processes that does the data extraction, the document I understand it goes through the document understanding. And we primarily chose you at that again for this particular use case because there are many provider bills. The formats are that every claim system that you work with has a different format. And they produce different patient statement models for a different type of accounts as well. And just going through and taking each of these as a templates would have taken forever. So we said, why don't we let the engine bottle understand as we do it, we will probably initially start doing a few manually. We -- and then the tool can pick it up as we do one more manual one. automatically learn the new model and then go from there. So that's the reason why we chose that. So that's another one that's active union works.

Great. So another question for you, and this is what we're thinking we're coming up with our sentiment analysis tool, we call it communications mining. So any thought on how you might be able to leverage that in combination with everything else you're doing to accelerate your journey, you really build up your automation program?

Yes. For out of network, we are in the negotiation business as well, right? At the moment, we are looking at a fair price. We -- there are certain reference pricing and the sentiment of the provider that we take into account what we call a toward operation, whether a provider will appeal or not. And that we did them in based on all the past negotiation history. So in the past, it had gone back to that provider and suggested a different price, it accept it, we do not accept it. If we accepted it, how long it took, how many rounds it does there is a ton of document, a ton of data that we have generated in the form of commands in the past. We have not really leveraged that do a good job of actually seeing what time of the day we should look at -- talk to this provider in order for him to say yes to a particular pricing? Or like say what price should we proactively put it in front of the provider. And we are leveraging AI to a certain extent for doing this intelligent allowable that actually is mining some of these data sets that we had available mostly in the structured data. But now there are -- I can see there are areas where we could leverage sentiment analysis and looking at how the conversation between the provider and the negotiator went in either provider accepting that particular negotiation are declining that negotiation. So that gives us more ammunition as the negotiator is starting the conversation with the provider, they may be better at to say, use starting from use these, this is how -- these are the words we used to present your offer to like here is the pain points that typically this provider has. If you hit these 3 pain points, then we will access the pricing that you're providing?

Pacing. So Javed, kind of same question for you. I know you work with a lot of our payers. Where are they kind of on the road map with adopting these types of technologies as well, getting past the document understanding with this sentiment analysis that we're rolling out our communications mining tool?

Yes. So as I mentioned in starting itself. So until now, we discussed about the data extraction from the clean farms, which are structured from now -- when we look into the medical documents, if we are looking into clinical reviews or auditing use cases, where we need or the nurses need to go through a bunch of documents. And then they are looking specific areas, what type of treatment or procedure was done on the patient and how it was done. So they need to validate if these procedures were done as per the policies from CMS as well as -- and I mean, as per the medical necessity. So these -- those are the use cases where the communication mining or the summarization of information of medical or sentiment analysis is going to help big time because through these capabilities, the content would be summarized. So in that way, our apps as well as some capabilities where we can -- the Board can summarize those information and have that information available for them. So now they might have the document available for them as well, but they don't need to scroll up and down and look all those information because that based on these technologies, that information would be available some for them in our short-term areas. So they -- because their time is very valuable. They need to take their decision if this -- the claim is -- I mean, all the details are as per the medical necessity or not. And they are spending on administrative tasks, their time is being spent on administrative task, look into all those details. Now that detail is available quickly for them, then it is going to help them take time. From HD point of view, but from the accuracy point of view because their decision is going to impact the complete I mean life cycle of a claim as well as of the authorization type of fitness.

Great. So it's helpful for me on the provider side. So we tend to like see what you guys are doing first. So where we're starting to use it in the provider world is in baskets. So there's in baskets and there's just a lot of information. So how can we kind of figure out what the real intent is? And should we escalate that? Or is this just the just nearly asking for direction. So kind of getting very granular and very prescriptive about what we can do with this information, not a lot of data. So we're really supporting our physicians and nurses in that way in that let us offload these erroneous invested questions like what are the directions of the hospital that you shouldn't really go to them. So where should that go? So that's what we're really just starting with our journey in the provider world for that. So we're almost coming up against time. So one last question for you guys is what are the key -- what key check ways do you want the audience to know, Suresh, as you half up first here?

So two things. I would say, one is leveraging technology to improve consistency, speed and efficiency is the document understanding and anytime we have manual process, exploring whether this is a tool can truly replace that. Bring the efficiency and consistency is an important thing that I think people should look at for sure. The second thing is there is going to be an element of human and technology interaction. That is what defines the success or failure of a lot of these initiatives. And making sure that everybody like especially the operations leaders who are going to be actually owning these technologies, we, as technologies only put these tools out there and guide on how it should be leveraged. But in the end, the efficiency and actual outcomes are derived by those operational processes. So bringing them along, making them as a part and making them feel like they're interacting with the technology versus the technology is competing with them is a key element that I would suggest everybody to keep in mind so that they can actually see the success of these programs, again, in the entire process, right, making sure that we use our human experts on the right of value-added services is the key we don't need to use the human expertise on things that can be automated.

That's fantastic. Well, I super appreciate you being here, Suresh. You've been a wonderful guest, but I wish your company biggest success. Javed, you're my cohort here. So you always bring the good information. So again, super thanks you guys really want you to get out there and kind of think about where and the provider in the payer space, can you really elevate your staff experience. How can you improve your care delivery and what really increases your data access? And I think you guys have really don't paint a picture of where document understanding and some other type of AI initiatives can really tie that in for us. So again, thank you all.

Thank you.

Thank you.

Hello, everyone. Welcome to AI Summit and the public sector industry record session. We're joined today by 2 industry leaders who recently held a few of our largest federal and DoD agencies, graduate from what used to be traditional rules-based automation to AI and ML-based interlinked automation. Jeff and Viktar, thank you very much for your time today, and we'll invest to share your AI joining with us. We'll start with a brief roundup introductions. I'm Pradeep Paruchuri. I'm the Director of Customer Success for UIPath's public sector practice. Over the past 5 years, I've had the opportunity to work with 100-plus public sector agencies to then realize the automation use. And over this span of 5 years, I've seen the graduation of a lot of our customers from basic RPA to intelligent automation that is infused with AI. And I'm really excited to bring that experience to our conversation today. We will now turn it over to Jeff for a brief introduction.

Hello, everybody. My name is Jeff Haberman. I'm one of the directors here in some consulting, specializing everything from budget analytics to RPA implementations all across our both DoD and civilian clients. So it's I'll be here today to talk more about our experience. Thank you.

Thank you very much, Jeff. And now I'll turn it over to Viktar for a brief introduction.

I am Viktar Zherdetski, I'm CEO of an Anika Systems, and also veteran software development, automation and [indiscernible] implementations. Having driven many automation implementations within federal government. I also worked on many solutions to help agencies to innovate at the same time, address challenges of implementation such as integration, scalability, governance compliance, life cycle management and many others. In my own Anika Systems, I work in alignment with our technical investments with customer priorities and site-driven innovation programs to help our customers with their digital information journey. And it's really great to meet here today. Thank you.

Thank you very much, Viktar. So to set the stage with regards to AI and public sector. We've seen this to be a natural evaluation of many automation programs at both a micro level as well as like at a macro level for the industry. At a micro level, many public sector agencies that started with simple rules-based automation, a few years back and have realized tremendous value in their programs are now moving into more advanced automations that often require AI and ML, that's all just increasingly complex challenges. At a macro level, this is where the industry is moving towards it. The UiPath platform that, in many ways, was a golden standard to help organizations with rules-based repetitive automation test has constantly evolved over the past few years to make AI and ML more of a central. So when we talk about AI and specifically AI and public sector, we talk about its application across 4 broad pillars. The first pillar is visual understanding. And this has, in many ways, been the central theme of automation technology, which is how do we intelligently recognize the metadata about the applications that you're trying to automate so that one, we are able to eliminate any automation blind spots. But more importantly, number two is like how do we promote resiliency of this automation so that they'll always work. The second pillar is Document Understanding. A lot of public center agencies are struggling with paper-based prices like our away. So how can we apply the latest advancements in AI and ML to help us understanding what is this documents so that we can automate the end-to-end process. These documents can be structured, semi-structured, sometimes can images, sometimes handed and documents, et cetera. So it's really important to have like an AI and ML-based platform that can eliminate any of those automation clients. And then the third pillar is conversation understanding, which is now getting into natural language processing and a more conversational AI so that we can intelligently automate at such as e-mail, implement [indiscernible], provide communication channels to cities themselves. And then the fourth pillar is called process understanding, which is how can we apply AI and machine learning models to uncover the complex process details that exist in this massive public sector. Now there is a misconception that public sector is behind on this whole AI cart. Whereas in reality, public sector agencies are actively pursuing multiple use cases. And are now pushing their automation programs to evolvement to using more of this end-to-end UiPath business automatic platform [indiscernible]. So we will talk about example in public sector today. We will start with Jeff. So Jeff, like you recently embarked on this journey to enable a very effective document-based AI ML usage. Can you describe problem statement to us at a high level and like how you realized you needed to move from basic RPA 101 to more of an intelligent automation and with AI and ML?

Yes, absolutely. So similar to what you were talking about earlier with understanding the manual processes of whether it be PDFs, hard copies. That was our kind of our use case with one of our clients, particularly within invoices and purchase orders that were coming in and needing to process those through the financial systems. So as is today, there was a lot of manual intervention in terms of opening an inbox, scrapping those, the details from that actual PDF understanding if it met the business rules in order to go into the financial system. And if not, there was a lot of investigation on the analyst point that needed to be done. So that was our main use case for automating and we were able to -- on the first of our journey, we took the standard features that came out of the box with UiPath. So using Studio, OCR recognition to taking an anchor based approach to grab the data from each one of the PDFs that were coming through various agencies and then processing those through our financial system. What we found with taking the out-of-the-box features was that there was only a 54% success rate, given that we have our family great here, we needed to make improvements in order to be successful for our customer. So embarked on a journey to implement Document Understanding. And then really from being able to build the models and build the template to the Document Understanding comes with, we were able to take that 54% and turn that into a 99.5% success rate by being able to take that unstructured data, put it into a structured data element and then process those individual records appropriately.

Thank you, Jeff. Again, like the comment you made about improving from 55% to 99%, that again, aligns with the theme of how do we use AI and ML to eliminate automation blind spots. So document-based intelligent automation requirements are coming up a lot, like Jeff mentioned. However, there are a lot more AI and ML use cases that are outside of price. And now to you, Viktar, like you recently embarked on this AI prototype for a large federal agency that involve extra speed and assets. Can you describe like the high-level product statement and how you realize you needed to move from basic RPA to AIML-based automation assets.

Absolutely. And I guess I want to start with just already of some of the recent breakthroughs that enabled us to achieve what we've done. And there is certainly the growth of AI-based capabilities that we've seen in recent years, reaching human parity, right? We see a voice generation and understanding character recognition, language understanding and many others. You probably noticed like even your personal assistance, such as Siri, quite recently became much better in understanding voice commands of the patient, and actually speaking in pretty much natural voice. So that's kind of like what started us thinking about the art of possibility in the first place. As for the use case, it was a perfect example for just that. So we had an agency that had a task of converting documents into audio files to enable vision in part recipients to receive the information. And in the whole process, they had to use a third-party vendor to record audio files and then these files were shipped on physical CDs. So just think about all of the logistics necessary and the data exchange. It was a pretty complicated process. And it was taken days, right, to get those recordings back to the agency. So we started first thinking about how we can extract these documents. And as a company, we've worked with many industry-leading on-premise and cloud OCR platforms in the past. And we found that UiPath Document Understanding and the standing framework was regulated in delays. From our experience, Document Understanding from UiPath was able to provide us the highest accuracy among many commercial OCRs. So that's why we selected that as our foundational piece for this automation to handle document extraction as well as the orchestration of the overall process. So once we were able to extract the data from the files, we also leveraged several of cloud-based technologies to enable us to complete the rest of the work. For the text to speech generation, we used Google Voice, Google text-to-speech, and that pretty much held the problem of the automation of all your file creation process. The agency also needed to perform quality control or the audio files with our own staff to ensure that there is no discrepancies in the content, and that was also a very tedious and time-consuming process as their QC team had to listen for hours and hours of audio files, just looking for some of the spoken worlds or portal or the potential issues. To automate parts, we needed to figure out how we can compare the core files back to the original document. So we selected Azure Cognitive Services to solve this step. And now we were able to achieve 2 text files, one extracted from the original and the other extracted from the voice file and now they're really easy to compare and we can identify the discrepancies with the algorithms and also that allowed us to quantify the number of issues and types of issues anatomies and decision points whether the quality is acceptable or do we need to bring human in the loop to do the video chat. And that was already much easier process. Also, now we're seeing some of the cases where we can make just a complete data-driven decision that the file doesn't actually need validation if we do not see any discrepancies. So I think it was really excited in case.

Awesome. And these types of complex projects, they often involve the clearable integration of multiple tools and technical. For example, you mentioned Azure Cognitive Services, right? So in your experience, how easy was integrating all of these multiple tools and technologies with the UiPath?

Yes. I mean we had to integrate a few tools and certainly have to have a diverse team that has experience with them. But the integration itself was not that complicated. I mean we had to write a couple of lines of code in Python to wrap up these calls around the cloud APIs and the choice of language was primely based on our current skills in the team because we were focusing on the speed of results on this effort. From UiPath side, we used UiPath Studio to develop the bot itself and the workflow logic. We also used a couple of the orchestrated features such use to manage the file uploads handling and processing and assets to secure store credentials for external services such as our subscription credentials in some of the API keys. We also leveraged UiPath integration service, which is the component of UiPath platform that really makes an automation of the body applications much easier. So essentially, it helps us to standard advice authorization in indication and helped us to manage on actions that established this connection with the collective service. One other thing, I mean, we're always trying to use modular approach for all of the development. So that allows us to easily switch modules and replace functionality with a different provider. For that particular effort we were planning from the very beginning that in the future, we may want to switch different providers for either voice generation and voice recognition. So we can achieve -- we can always achieve the best results and leverage the best available vendor at a time. And Azure integration was actually pretty simple. UiPath has this activity called Azure [indiscernible] that helped us to connect to an [indiscernible] and control all of the workload with them. So that's pretty much it. As I said, it was not complicated from [indiscernible] side of that.

That's awesome, and on a similar note of complexity, we have all lived through implementing systems in this highly complex, highly secured DOD networks and all the operational challenges that come with it. So a question for you, Jeff, is how hard was it to get like the end-to-end platform installed and configured it in such a controlled environment?

Yes, absolutely. Certainly, a lot of complexity in terms of gathering the right groups of folks together to be able to get on the same page in terms of what the goal is, what the mission is in order to get the application installed. So having the right [indiscernible] administration that have the Linux experience. You have the DBAs that are helping you set up all of the necessary artifacts, getting the right people in the room and really following the cadence of following all the paperwork, policies, procedures that need to be put in place in order to adhere to the highly complex and highly secure environments that we work in. So I'd say that the bigger struggle in terms of outside of having the right group of people involved is just the actual communication that needs to happen. So making sure that when one party is complete with their responsibilities, the other one is following suit and continuing on to continuing to rapidly grow the program and grow the development that we have. So certainly was -- has its obstacles along the way, but overall, having all those things in place is -- it makes it very seamless to implementation.

And that also speaks to the very skill sets needed on a deal, right? So for anyone that's embarking on this journey. Like what would your advice be in terms of what are the skill sets needed, how does the team need to be structured? For example, you mentioned Linux, which is a different skill set than what we needed in the past. But that's just an example, right? Like so what would your recommendation of like advice be for like newbies that are starting in terms of team structures and skill sets?

Yes, absolutely. Obviously, you hit it on the head there, having a dedicated winning [indiscernible] to be able to execute the actual implementation. So you have your dev staging environment. You also have your multi-node environment in production. So having a dedicated resource to take those activities as well as having a dedicated development team on the studio side that can both, do Python as well as the overall -- the development within the document understanding framework, so building the templates out, building the models out, whether it be 1 to 3 people or if you're scaling to a larger product or a team aspect, making sure that the dedicated resources are at least within the DBA sector, you have your database from a Microsoft standpoint as well. And then you also have your orchestrators from a system admin side. So having the Windows experience on that side of the house as well. So those 3 were definitely needed in order to have a successful implementation.

Got it. And then just a follow-up question on that. Is there like an ideal team size? Like would we have like teams as like admins, developers and project managers to start on like a project like how you did like, how long -- how large was the team to do this?

Yes, I would say, at least having a dedicated size of 4: one Linux admin, one orchestrator, system admin, one developer and then one business analyst. So you're able to bring that whole group together from a functional standpoint as well as a technical standpoint. Again, so that everybody roles and responsibilities are taken care of and you're able to both, work in parallel as the technical team is standing up the infrastructure as the business analyst is also conducting the requirements at that point in time. So you're really minimizing the overall time spent with the team in order to want the lights get flipped on from an infrastructure standpoint, you're able to hit the ground running from the studio development and the overall complexity there as well.

Got it. So we spoke about the use cases at high level, so the need for AI and ML-based intelligent automation. We also spoke about like the process for like developing this installing this and configuring this. Now moving over to the value realization side. A question for you, Viktar, is measuring ROI for traditional RPA is fairly straightforward, right? You just like measure it based on how much effort you see. However, when you talk about like AI and ML, there's sort of a lot more value drivers than like a typical RC. For example, in your case, it could be compliance. It could just be that the customer like pedal agency was never able to get to so many audio files, et cetera. So can you talk about like your experience about what the anticipated value that this large pedal agency will ones?

Thanks, Pradeep. So I think, generally, there is same approach for the ROI rate and the value return, because at the end, we automating the processes and trying to see time and budgets and essentially improve the quality. So we're looking for the similar outcomes is just with AI and ML technology, right? There is new opportunities for automation that were not even considered before. So for that particular effort, I mean, there was a prototype, so we didn't really do the ROI analysis. I can only speak theoretically. But nevertheless, the outcomes of the prototype showed huge opportunity for the process improvement. So as I mentioned, the current process for creating those audio files involving use of third-party vendor to produce them and the complicated logistics with actual mailing on our physical CDs. So we have to manage shipping and billing and vendor relationships and many other things. Now with the implementation of the voice generation is cognitive services, right, based on the data extraction. We've seen the improved from multiple-days process with complicated logistics to almost immediate results. In a matter of minutes, we can have those files available. And then that's, of course, also opens how the opportunity that they can be sent by e-mail and delivered with the new digital ways to the end recipients. So that's outside of that prototype. Another improvement we've seen on the QC side. So same thing right now, the current process has a lot of many labors requiring the QC team to listen to hours and hours of these files, looking for these discrepancies. What automation helped us to do is essentially, first of all, to visualize and identify those discrepancies automatically. So now we can see -- team can see right away how many discrepancies are there in the files and where they are. So now they can focus on just those pieces that require human attention instead of just blandly going over and over all of the content. That also helped us to visualize the metrics, right? So we can understand the file needs to be reviewed at all or maybe the quality is good enough, so we can kind of like set a threshold and automate when we involve human in [indiscernible] when we can just make a decision, then based on this analysis, the file is good enough to go directly to the recipients. So same thing. We're talking from hours and hours of manual labor to probably just a few minutes per file that is needed for human interactions. So overall, I think there's -- it's a multiplier, right, on the amount of time and effort that automation can save to the customer.

Thank you very much, Viktar. Jeff, how about you, right, like can you talk about like the value that this pedal agency derived from this AI/ML-based automation?

Yes, absolutely. And in terms of our overall ROI, we took a look at the -- to be able to process one of these documents from inception to implementation into our financial system. It's taking anywhere between 15 and 20 minutes for the analysts to digest all that data and then make the decision on whether or not needs to go in system. We were able to take that from 15 to 3 minutes. So being able to condense that time and these are happening hundreds of transactions a day, a month, depending on the volume of the fiscal year, we're able to fluctuate that ROI up and down and really allow those customers to continue to add benefit in other areas. So it's certainly been a success for our agency, and we look to incorporate and scale what we've done thus far into other processes that we can take, reuse some of the code that we've already built and then apply those in different areas that are applicable.

Thank you very much, Jeff. Now Viktar, back to you to close it out, can you share some best practices and lessons learned for others that are embarking on your new journey?

Absolutely. So that particular use case showed some new ways for us how to use [Indiscernible] services and really help to automate complex processes that we didn't think about it. So the real lesson learned and probably more of the thought reinforcement was that we have to constantly and continuously keep looking for opportunity to automate and thinking out of the box and also follow the technology trends to really understand where the new ways are available to solve the new problems. And part of that is really ensuring that innovative culture and motivated employees across our team are reinforced and encouraged. We're making sure that every thought, every contribution is considered and every member is engaged into this process. Another thing, it's really they need to have an environment for our teams to experiment with technology, so they can have this firsthand understanding what is there possible and being able to quickly build different prototypes to prove their ideas. We have innovation lab in an ecosystem that is available all of our employees and our customers, and it helps them [indiscernible] new technology and validate ideas and that really drives a lot of them to our customers. And yes, finally, also always think about end-to-end automation because that essentially, you will have to be in a way when you have to bring your prototype to production rate. And if you didn't automate all the pieces, that will be extensive lift to do all of that in preparation for the product environment. And from another side, also most of the solutions today are comprehensive, meaning that you have to integrate many pieces together, it's no longer that one technology can solve the entire use case. So always think about reducibility of the modules. So whatever automations you've done, right, you can package them, so it can be reused across multiple different projects or efforts. And that really helps to scale up our automation capabilities.

Thank you very much for that, Viktar. And now, Jeff, turning it to you to close this out, like, what would like what would you advise other agencies that are going to start on a similar journey of like document-based AI process?

Yes. I'll echo what Viktar said is having the environment ready for your teams to go forth and conquer it. We were able to work with our respective IT organizations and spin up those necessary environments for both, be it orchestrator, the AI center and all the dev tests and environments we're able to scale out after we proved out our prototypes built the necessary [indiscernible] that was required and move forward from there. So without having the back end, without having the support from the whole team involved, you can have 1 champion running one way, but not having the full group along with you, which will cause, obviously, delays in terms of implementation and overall success. So I would certainly say build to start small, prove out those use cases, build confidence in what you're doing and being able to show the value and we're doing it here at 1 small scale look what we can do from larger scale if we continue on with this path. So that would be my final recommendation is in order to scale and grow, show that initial value and bring everybody along the way.

And thank you very much for that, Jeff. Thank you so much, Viktar and Jeff, for participating in this discussion. We hope everyone that attended at this session found this helpful. You got to hear firsthand some use case about, for example, in Viktar's case, how do you eliminate automation [indiscernible] part? In Jeff's case, how do you use AI and ML to improve the accuracy of automation? So please reach out to us if you have any other questions, any other ideas about like public sector and like AI in general. We hope you found this session helpful. If you just want to stay back for the product deep dives, that would be great. Thank you very much, everyone.

Welcome back. I hope you enjoyed the keynote space. It was fantastic. There's so much learning from it. So you are joining the manufacturing industry breakout session. And we have some amazing lessons and stories today from the speakers that I'll introduce in a minute. So we are going to essentially talk about how the digital transformation in manufacturing industries happening by harnessing the AI as well as the business automation. Essentially, the goal is to drive the improvement in the business processes, which result into all the great financials and customer benefits that we know. So with that, let me get started. So we have with us Steve Sykes, he is the Director of Operational Excellence, and I am Bhavesh Joshi, I lead the Manufacturing Industries For Americas. And see you -- and I would have some discussions about what they've done and some of the work that Steve is actually champion at DexCom is just remarkable. So without further ado, lets get started into our breakout session for manufacturing industry today. And I wanted to kick it off by some of the trends that I see in some of the areas that are actually changing for the manufacturing industry. So first and foremost, as we really think about the data and manufacturing industry more than a lot of other industries, the better ones are very data focused, and data is essentially the precursor to having a good AI program. So when we think about the amount of data there is awaited every day, it's about 2.5 quintillion. And the assumption is that between now and 2025, we will generate enough data that has already been generated to now. So the proliferation of the data is just at a very fast paced. So how do you actually do something with the data, so that it's meaningful for AI is really a challenge that a lot of companies are trying to overcome. And when you think about the AI market size, it's growing at close to 39% every year. And so that is a very fast pace. It's an area of opportunity that most people are trying to actually get their hand around, the good ones are making sure they are doing the right foundational things, but there they have right data, scientists programs and also that is coupled with the business and then trying to take advantage of that into the AI model that is fourth coming after that. Now at the same time, the research says that 25% to 50% of the time that the very talented and highly skilled data analytics workers are spending in collecting the data, preparing it and more importantly, quality controlling it, which, as you can imagine, those are needed before you can actually utilize it to do different types of AI from that. So that's an area that when you think about the automation, it can help you lot in making sure that those non-value-added activities are actually automated, and you're not having the right resources spending their time. The fourth trend and very -- it's getting highly utilized, and we have some use cases that we are helping with our customers as well is, as we all know that the net zero is the emission target, most of the manufacturing companies are driving for. And people are making incremental progress. Currently, manufacturing industries generates about 25% of all the emissions. So similar to the data collection and cleansing and quality control aspects when you think about the emission controls too, more than the hardware side of things. There's a lot of processes, a lot of human involvement that happens in the data collection for that as well. And that is where automation is actually playing a key part. And then we have some AI models that we have helped some customers with in making sure that automation and more importantly, AI-enabled [indiscernible] are actually geared from those type of AI models that we have for our customers. So these are some of the unique trends overall for AI as well as specific to manufacturing industry. As we go through the session today, we'll actually talk a lot more about how DexCom has had an intelligent automation journey from just the business automation where they started. So let me introduce Steve again. So Steve, as I mentioned, is the Director of Operational Excellence, essentially just making sure that everything related to the operations is continuing to get better every day, and where they started their automation journey and where they are actually going is just remarkable. So Steve, welcome. We are very privileged to have you join us and also share your story.

Thank you, Bhavesh, and great to be here and be able to share our story at DexCom. Just a little background about DexCom we produce what we call CGM or continuous glucose monitoring device. It helps diabetic patients monitor their glucose levels on a regular basis as opposed to having to finger prick and take their blood on a regular basis throughout the day. And the net result is we've been able to change lives dramatically for those who are able to use our product and go on to live lives much fuller and happier knowing that they can they can be monitored and tracked over time. So really happy to be here. And I want to tell you a little bit about our intelligent automation journey. Today, I'd like to talk about briefly the switching platforms. It's an unusual step that you would normally take when you're on a robotic platform, but we did make a change and took advantage of an opportunity and are glad we did. We also made some changes in terms of how we refer to our organization. It wasn't just RPA anymore. We're into intelligent automation, because we wanted to see the long-term spectrum of growth and maturity play out in the way we communicated to the organization. And we will talk a little bit about test mining and our customer service operation and how it generated the idea of attendant [indiscernible] usage and then document understanding an action center where we're actively using this platform and hope to grow the usage of this platform going forward and a little bit about our future. So when we are looking at which platform is better for DexCom to grow and back in the late 2019, we had an opportunity where we had to rebuild 80% of our robots, because the customer advocacy or a group that reviews technical service records was switching their platform where 85% of the transaction activity and robots were working. So the net result is we said this is a real rare opportunity. We probably never get it again. And so we took a look at the platform that we were on. We knew that we were having problems with that existing platform, where we're struggling with the reporting capability and the lack of integrated reporting from the application. The robot environment was stabilized but only after a pretty long cycle of time, approximately 6 months when we launched this to actually get the robot stabilized. And we had just limited functionality. So when we evaluated the platform, we said, okay, this particular vendor's platform version is going to be upgraded. And then we look -- took a look at UiPath platform and did a comparison in terms of how the 2 platforms would comparatively look, and from a cost perspective, there was really not a big difference. But in terms of the history and capability of what UiPath could bring to the table and the integrated nature of all of the platforms from automation hub to attended bots to document understanding and as such, we felt like this was a much better way to go. Plus it also helped in terms of our business services and where we were going to grow and the resourcing that we would need to be able to put developers in. So we chose to make this switch to UiPath and are very happy we made that switch. And part of the thing that we like to do at DexCom is we like to have a really formal way, kind of using an engineering way of selecting the platform and rationalize why back to the leadership, why we want to make this change. Because in the past, some people made the decisions on that platform, and it might be a bit of a tough put for them to agree to make this switch. So this was a way using this, what we call a [ Pulmatrix ] to compare features and capabilities and score them, so that people understood the rationale as to why we were choosing this route. So when we did the comparison against the other platform, from a resourcing standpoint, this was probably the biggest issue. We have 2 business centers, 1 located in Manila and a new 1 that was starting up in Vilnius, Lithuania. And we wanted to have our development and support teams split between these 2 locations. So resourcing from a UiPath versus this other competitor, it was similar in Manila. But when you get to Vilnius in Lithuania, it was a huge disparity and quality of resources and number of resources available. So UiPath scored quite highly on that front. And infrastructure, somewhat similar, pricing pretty much the same. And the features, UiPath just offer a whole lot more capability. There were tools such as -- similar to task capture or task mining, they were coming on board, but the tools at UiPath had already been proven, where this other vendor was actually newly coming on board with a platform of their own. And then the ease of use, just knowing and operating with UiPath in the past, it's ease of use in knowing the reporting was going to improve dramatically, made it a much more positive option to switch to. So the net result is it scored significantly higher and out of 23 total criteria that we aggregated together into these buckets, UiPath had scored significantly higher compared to that new version.

This a few metrics in product development life cycle. So using this into the evaluation is very unique. I'm sure that this maybe points to your data risk culture as a company, and I think that's just great story here.

Thanks, Bhavesh. Yes, we'd like to use some of the Lean Six Sigma tool sets, and this is one of those. And if we have an opportunity to use them to prove our point, especially on the big business case and a spend, we're getting ready to execute. It helps us have our ducks in a row, so that in case we get pushed back. We even went so far as to look at talent comparison by platform, and I mentioned resources was a critical factor that was a big deal for us in the UiPath selection. When we looked at Vilnius, Lithuania, we found that the other platform resource pool out there for hiring was very small at 22 resources, and you can see the different companies on the bottom right-hand side, where they were allocated. And on the left-hand side, you could see the UiPath resources were like 5x more. And then local university, I'm probably not going to pronounce it correctly, [indiscernible] the University of Technology and Vilnius, Gediminas Technical University, they were pumping out about 60 to 65 certified developers each year. And we saw this as a real opportunity to take advantage of. So that drove a lot of the thinking in moving to UiPath. Nowadays, we were using insights with UiPath and we have integrated capability into our Power BI platform. And we get these published reports out on a weekly basis. In fact, this morning, I was just reviewing the progress we've made so far on the -- on our bots, and we're tracking to a 200,000 hour target this year of savings impact. So we want to make sure that as we're tracking to that, we're on pace to reach that goal. And so far, so good. Also, I want to bring to your attention, you'll notice that in the bottom right-hand corner, the pie chart, that blue space that seems to be heavily weighted around the complaint codes. It used to be 85%. It's now down to 71%. We consider that good, because we're starting to spread the utilization of robotic automation out into the rest of the organization. And on the left-hand side, underneath the process and records, you can see on the bottom left, the different robots that we have in play Dexcom. So just to give you an example, the complaint code is the largest producer of hours saved, but the second largest producer is an attendant bot that we're going to get into here in a minute. I'll share with you how that works. We're really proud of that. And then I want to bring to your attention the bot history review. This is our introduction into the QA organization. The bot history review when production -- a bot is produced and comes off the line and is ready to be shipped, our QA team has to go through and confirm that this bot is ready. There's a physical review and there's an administrative review. And when we do this administrative review, it's about out of 1.5 hours total time for the quality engineers to review. 36 minutes of that time is actually spent doing the administration, checking systems, making sure they're validating what's in the bot matches up with what we have systemically. So the net result is when we built this robot called bot history review, it took the entire 36 minutes out of this review process for them. And so they could just spend their time doing the physical review, publishing the results and moving on to the next bot. And if you can imagine, last year, we produced 50 million sensors around the world. And by 2026, we plan to produce 200 million. So to get to that kind of scale, that 36 minutes is going to turn into a lot of benefit over time as an example. So I wanted to bring your attention to the attendant bot. And I'm going to show you the screen here. It's called Oracle Service Cloud on the left-hand side. On the bottom right, you see a little iCom box. The agents when they're on the phone with our customers are very busy spending quite a bit of time guiding them through either how to use the product or helping them provide technical support if they're having issues. So they use Oracle Service Cloud and the many screens that they pipe through and fill out information and try to service that customer. When we used a tool called task mining, we put a recorder on the desktops of, I want to say, 20 agents, and we ran it for about 2 weeks. And that data was ingested into an AI engine and outcomes a lot of very good analysis and illustration on how that process is performing. When we went through and started to figure out what -- where are we spending most of our time. We found that this particular process using this particular screen was causing the most headache. It was more than a minute of time being taken trying to look up the transmitter serial number and take the selection of the subcomponent item and pick that right item and then also go to the application that was being used. But because they had to go out of Oracle Service Cloud into separate databases, they were basically leaving the screen and coming back, copying information from one source system and pasting it into another or trying to remember that information and selecting the right option in the other. So we created this attendant bot, which would do that job for them. So they never have to leave the OSC screen. And so on the right-hand side, the transmitter serial number is duplicated over there. And all you needed to do was type in the number or reference number and then the robot would just look it up and fill in the subcomponent number for you. And the same would be said with the product line below. Now since then, we've been able to figure out a way to take this little box on the bottom right-hand side and hide it behind the OSC screen as well. So the agents don't have to shrink their window. They actually can use the normal window. And they can even use the regular fields and transmit our serial number, because when they enter the data, the attendant bot -- that's the trigger for the attended bot to actually act and it goes ahead and fills out the rest of the screen. So you can see here, if you want to understand the data on the before side, they had up to 4 loop backs that's switching from the application over to a database and back to the application. I don't know, it was about 14 steps. That introduces about 6 points of failure that you could actually make a mistake in that process. And it would take anywhere from 1 minute to 1.5 minutes of timeframe to complete. But after we put the attendant bot in, we got rid of all the switching back and forth from those applications, and it's only 3 steps. And it took them 10 to 15 seconds to get this process step done. And if you can imagine saving 45 seconds to a minute and then take the fact that there's over 500 people doing this task 30 times a day, it becomes a pretty big impact opportunity from using this platform. And the nice thing now is once you pay for the licenses and have them on the desktop, you can add more functionality on top and the only cost is just the development. So we're really excited about this. And -- yes, Bhavesh?

Yes. So since you are obviously a big champion of lean as well. So in a way, you use the task mining as like a quasi value stream mapping exercise, right, to understand where the [indiscernible] and failure modes and all that was to then come up with a new solution, right?

That's correct. In fact, we like to blend our Lean Six Sigma group and expertise with intelligent automation. And when you do that, you get opportunities like this that will surface. So just to give you an idea, once we launched the attendant bot, we did a comparison and these little blue and green bars are actual results from individual operators and the time it was taking for them to execute the transaction. And we were measuring the percentage of improvement that each agent was getting in their handle time. And overall, it was about a 14% improvement. And you can imagine when you're talking about minutes of time, 14% is a pretty decent impact. And we're pretty happy about those results. So where are we going on our automation journey. Back in 2019, 2020, we made the big switch over the UiPath, and we felt like Phase 1 was let's build the capability in this platform. We currently had outsourced development and support, but we needed to also handle our intake -- the demand intake and make sure that things were qualified well before we actually agreed to build. And we wanted to stabilize on that platform. And at that point in time, from an organizational standpoint, we were just following those groups that were stepping up and saying, I'd like to look at using RTA or intelligent automation. And so we had our basic unattended bots. We launched immediately automation hub. It was a no-brainer to get this going. And we got a nice little [indiscernible] of test capture added in when we had those basic unattendant bots launched. In Phase 2, which is currently going on today, we now do all development in-house out of Vilnius, Lithuania location. We just switched our in-house -- our support from an outsourced model to in-house out of Manila, and we've been continually expanding the functionality by having the document understanding with the action center scanning documents and processing orders out of Germany today. And we've been leveraging the task mining...

May I interrupt you to little -- talk a little bit because that's a very unique, it's a nonstructured data per se that you are using the U for. Can you give us some idea about how and what you've done with that?

Yes. If you can imagine a prescription comes in, in Germany in a dot matrix print, multiple copies, formats, it's old technology, but it's still in use today. And it's on a standard prescription pad sheet. So when it comes in the office, we used to do about 300 a week, and this was a very manual process where agents sat in front of a computer and typed in what's on the prescription. Now we use document understanding where we just scan the record, Action Center gives us the ability to make any adjustments or corrections and then, we can process the letter of authorization request to the health authorities or fulfill the order easily using this platform. It's basically an OCR solution, but works with a small machine learning module in the back end. So hopefully, the percentages climb over time and they're able to handle more volume. And just to give you an idea, there was no additional headcount added where we were growing at 50% per year from 300 to 3,000 prescriptions a week. We didn't have to add, because we had document understanding able to manage the intake. And now obviously, we've got the tender bots and our goal is to expand functionally technically and organizationally and then accelerate the pace of change that we're impacting. So now we are moving into Phase III, and we're starting to push the ideas of stretching the imagination of where we can use the technology. I just came back from Manila and while out there and discussing ideas for report generation, for our workforce management group internally, we figured out the best way to do that is to actually have a citizen developer license added to a very adept Excel user, and they could move much quicker to automate their reports and get that out the door. So some out-of-the-box stretch imagination. We're going to continue to do that. We're looking at chat box with NLP, and we're actively going to start process mining this year. And we believe the next future is in advanced AI. So we need to grow in that space. So that's a little bit about DexCom and our automation journey and, Bhavesh, back to you.

Yes. No. Thank you, Steve. It was incredible. I think oftentimes people also talk to me about utilizing the AI. So as you have done already several of the process mining, cast mining and document understanding and communication mining after [indiscernible] exists. But we also -- and we would continue to partner like if there are some fit-for-purpose AI models that we can generate to help with your business process, we are willing to do that, and I do partner with all of our customers on that. So if you want to talk about that or anything else that we cover today, applying AI into business automation and manufacturing industry, please feel to reach out, but I want to again thank you, Steve, for your time on story-sharing journey that you walked us through. Thank you so much. And with that, you can go back to the AI Summit, and the product deep dives are next. So continue to enjoy the sessions, and tell us now how you like them.

Hi, everyone. Welcome to the document understanding best practices session. My name is Steve Tegeler. I've got Daniel Lerner on as well. And we're going to run through some best practices and some success metrics for those just starting out with document processing or you're well under your way with document understanding. So for the next 30 minutes, we really got 2 pieces to this presentation. One, I'll review kind of some foundational understanding going through some concepts, which Daniel will spend time on later talking about how to optimize some of those concepts I review in this first section. So I'll be talking about the most important thing in any document processing solution. We'll review document understanding for those that may not be familiar with it, so you can understand some of the key aspects, and then I'll go a little bit deeper into RPA and AI in terms of what that overall process looks like, along with some of the ML models and the all-important data set. Daniel will spend time talking about the ever-important business rules, along with discussing overall solution performance and model performance and how they tie together. And then we'll end with the very important data labeling and some best practices there. So let's talk about the most important thing. The most important thing in any document processing solution is to prioritize your business goals. And so if I look at maybe my as is document processing, I've got a document source where documents come in from various locations. And then I have to basically get that data out and put it into a system of record. Now what we need to do is we need to figure out what would be valuable about providing automation here. And so we find with all of our customers, it really falls into 3 buckets. The first one being they just want to have a more efficient FTE processing these documents. So we're going to increase the document throughput per FTE. The second is the overall processing time end-to-end, meaning there's some sort of data in the document that's very important, and there's an SLA with turning around that data that sits in there. So a loan package would be a great example of that. And then finally, we want to improve data entry accuracy. And so this is avoid duplication or rework, meaning even if you have a document where it only takes a minute to get the data out, if you don't -- if that data doesn't get in right, there may be some substantial rework involved. So I have to spend 30 minutes going on -- unhooking everything that was done because that one small mishap. So we find some there. So the key here for you is best to prioritize these 3 and based on that, you'll be able to tie some sort of business value. Now in general, it revolves around the time it takes to process the as is time today, along with the volume of documents, and so in this case, I'm showing there's a triage, maybe you have to split up a file into multiple documents, maybe you have to form to other individuals, and then there's the actual processing of the data; verifying and then also extracting. And so when we look at measuring the solution, here's an example, here, where -- so 12 minutes total, 100,000 documents, that's about 20,000 hours. That's our baseline metric we can work off of to understand if there's actually a value or not or how to optimize for further value. And when we look at the measuring the value of an automated solution, there's something called a straight-through processing rate. So those documents go through the system from the very source all the way to the destination, and they never touch a human, but then inevitably, a human is involved in some percentage. So what time are they taking to actually process the data that didn't get processed automatically? And so correlating this time spent to business value is a big key piece of this. So double clicking here on terminology. So we talked about straight-through processing or the straight through processing rate, the inverse of that would be the human in the loop rate or HITL same here. And that if it didn't go straight through, it probably goes to a human to get validated. There is a third bucket. Sometimes we see documents come through that just don't need to be there. It's the wrong type of document, it's a cover sheet or something like that. Those would be discarded. But the culmination of the straight-through processing, documents go straight through processing versus touch a human versus discarded, obviously, 100%. Now when we do our calculations here, there is a human in the loop rate here, that's the key. We want to find out how many times is a human touching it, because we want to find out their average handling time to get the total time, which will in turn correlate to the business value of the solution. Does it make sense, or do I need to do more optimization? Double-clicking on document understanding and an automated solution. Here, we have our customers, our vendors, sending their documents into some document store, so maybe it's an inbox. And then we've got our document understanding solution and then the data destination. So to understanding encompasses everything in that orange box, and I'll go through that here. So maybe the document comes in as a ZIP file, maybe it's multiple documents. We have to basically split those up. We have to figure out what they are and then we need to extract the data. And there's a lot going on here. This is actually where machine learning lives, and we'll spend some time on that in a little bit. The first step after the data is extracted is, boy, we'd love to figure out if it was extracted correctly, and we'd love to do that automatically. And there are some tricks in something we call business rules, where it can provide that automated validation where we can do some logic on the data. We can maybe do some simple math. We can look at date formats, or we can actually match them with a system of record. Daniel is going to spend a lot of time on this later. So I'll move on to the next step here where if the automated validation wasn't done or we found no, we need to send that to a human, because we have a rule in place to do so, we run this human in a loop process where we involve a document specialist. This is typically the people that are doing it today, although it doesn't happen to be. Typically, we are a document specialist that understands that document to take a look. So with that, they will receive not only the data that was extracted, but a picture of the document itself, and they will be basically given this guided validation process. So they're either going to look at the field and say, yes, that looks exactly like it, extracting it correctly, hit submit, or they'll make a change and hit submit. But fundamentally, they are really, really efficient at doing this versus previously doing it manually. And so once submitted, then it goes into the data destination. So if we bring back our calculation before where we had 20,000 hours, here total for 100% manual. With this case, we're now 1 minute per document, even if 100% of the documents had to go to a human, because we're far more efficient at processing now, we still have this massive reduction in total time spent. And a more realistic number could be 50% verified, and we even further get that to 96% reduction in overall time spend. So substantial improvements here from an overall automation perspective and also from making our people more efficient. Now let's take a closer look here at overall, our end-to-end process when you look at machine learning with our typical RPA process. So we've got our -- generically, our data source, our data destination. RPA here, that's what's in our orange box. From an RPA perspective, we're going to do some, call it, data preparation. So in this case, when we look at machine learning, it typically likes to get the data in a certain format, so we're going to prep that data, we can do that automatically with RPA. Now when we need to get that prediction, we actually send that out to one of our machine learning models hosted in AI center. And in the case of document understanding, that's our machine learning extractor. And you can see I highlighted here in the data prep, you can consider that the OCR phase, where we're actually getting the text out of the document, and we're basically putting in a format to whereas the machine learning extractor can pull that data out and do some predictions. Once we do our predictions, we take that prediction along with a confident score. So the machine learning model will produce a confidence score around what it thinks it's success rate was for that prediction. It goes to automated validation. So again, those business rules where we can go out and we can do some external verification. We can do some simple math. If it doesn't check out, we'll put a human in the loop, and they'll make the corrections and then they'll hit submit, and it will go to the data destination. One thing I haven't mentioned previously is as that human is doing those validations, we can collect some of that data for fine tuning that model. So hopefully, this gives a little double click. And again, Daniel is going to touch on some of these concepts later and how to optimize some of them. But I'd like to spend a little bit of time on the out-of-the-box pretrained machine learning models now. And so when you go to deploy a model within AI center and document understanding, you're actually given a list of pretrained models, meaning right out of the box, you can start processing these types of documents. So you get instant results. Now we always recommend retraining those models for optimization. Now I want to specifically highlight. So these were all the pretrained models, but you can use a custom document type. It's actually the same machine learning model we use in the pretrained, but we are basically able to create a custom document type. But it has no data set associated to it. I'll explain that in a minute. So here we go, we want to create a labeling session here where we've got our fields. We've got our typical set of production documents, and we go through the session of basically highlighting the areas on various documents and mapping them to those fields. The output of that is this thing called a data set. And the data set is either used to retrain the pretrain model to make it that much better, or we'll use that training data set to actually train a brand-new custom machine learning model. Finally, before I pass it over to Daniel, I will talk about model performance over time. So as we look at model performance, and we want to get that straight through processing rate as good as possible, it really comes in the form of a few steps. Number one, choosing one of those out-of-the-box models, you're going to have instant success. You're always going to want to retrain the model, whether it's a brand-new custom model, or it is out of the box, and that provides you even better efficiency and performance. We'll have those post processing and business rules that can do that automated validation. So that increase is straight through processing. And finally, that fine-tuned with validated data that taking those corrections that are done in human loop and making them better. Now it's important to note here that every improvement is very specific to your environment for multiple aspects of the variations that can occur. So keep that in mind, you always want to involve your document [indiscernible] or business owner in this. There's a lot of questions and answers that need to be answered by them. And very important, if you've never done this before, you might want an experienced deployment partner to get this as good as it can be right out of the gate. So with that, I want to pass it over to Daniel, who's going to spend some time talking about some of the key ways to optimize a lot of the things I just described. Daniel, over to you.

Great. Thanks, Steve. Hi, everyone. Nice to be here with you all. So we are going to spend a little bit of time going further into some of the topics that Steve laid out here for us. At the end of the session, one of the key things that we'll discuss here are what are some key aspects for how we optimize our solution. And on the UiPath side, what are best practices that we've seen internally working or also working with customers and being able to show those experiences with you all. We're going to be focusing on first business rules, a closer look at what exactly we mean when I mention, business rules. Business rules is the logic that you can add into your automation workflow, your process to automatically determine if the predicted value or a field or classification is extracted correctly. So we're going to take an example of invoice processing. And if we start to discuss just some scenarios where business rules would help us validate information that is extracting the document, we can categorize them into a few groups. One of those groups is just what can we do from a character validation standpoint? For example, being able to see the model or the solution extracted the correct invoice number, but does that meet the expected number of digits or characters that we expect to be found for that field? The invoice number, for example, business process, you might always require it or expect it to be 7 digits, we can instill that business rule in our automation and workflow to allow the robot to execute and carry out that business rule and return to us whether that business rule passed or failed based on the value that was predicted by the solution. Second, you can have business rules perform basic math operations. You can do things like taking the invoice line items here, something goes up, taking the tax amount. And then comparing the sum of those items against the total amount. And that's a way that you can have business rules cross-referenced against the values of fields that have been extracted and determined if the total amount is going to align with what you expected to receive or process with this invoice based on what was found for other fields that are on the documents. Third, we have another example, which is the state validation, being able to make sure that it follows the right date format, if you're always taking in documents from the U.S. or America, you might expect it to be in a certain format versus if you're taking in documents compared to Europe. And then lastly, one of the more popular more common ones is this external source verification, where we may have information that's found in a document, and we also have a system of record, sources of truth where you expect to be able to go ahead and query those system records in order to validate whether the information of the document is matching up with what you had stored in that source of truth. For example, here, we have a PO number that was found on the document on the invoice. And so we can use that PO number to go ahead and query against a database or a system of record that exists where we have purpose order information. And that personal information can go ahead and tell us whether or not for this PO, it is indeed tied to the right vendor. And if it's tied to the right vendor ID or for some reason, there's a mismatch and when you take some action on that. And we'll go more into kind of what happens when you do have these kinds of system mismatches or document information matches and how to take the next steps for how to resolve that. These are some key examples of how we see business rules being used. And again, one of the main benefits is allowing us to automatically or programmatically validate information being processed from the document as opposed to requiring human intervention. That kind of balance between automatic validation and processing and human intervention are 2 key factors with understanding the overall solution performance. And when we talk about best practices for understanding solution performance, we want to go ahead and really start to understand how we can categorize different exceptions, how we want to categorize the documents that are being processed on any human intervention. So we're going to go ahead and do that with an example here. So if we take 10,000 documents, we can go ahead and say, "Hey, we attracted data from those documents. We went ahead and actually applied business rules to those documents. And for half of those documents, we were able to process those without any manual intervention. For a portion of those documents, the remaining half, those required human intervention, humans in the loop. And so a document specialist needed to go in, provide some input into them, and then we were able to process those after that manual intervention step." What you would want to do as a team is better understand the cause for that human intervention. And we can -- we'd like to categorize the reasons for an intervention into 2 main buckets. One is going to be events due to system processing, where we see information not being extracted from the data correctly, due to an OCR issue. So the actual information couldn't be digitized or read from the document correctly, maybe because the quality was poor, whether there's some pretty sloppy handwriting or the model just did not find the actual field or value on the document that we were actually searching for, it didn't find the right value. So those are system processing events. And then the other category that we can kind of group human main information cause events is wrong or incomplete data, whether in the document or the system of record that we have on hand, external system issues, so there might be it's a mismatch in information or a system might have been offline that didn't allow us to actually plug the information downstream. And then lastly, escalations or alerts based on the extracted data. And we'll go more into an example of what each of these mean, but I want you to keep in mind kind of how we separate these 2 categories. One is going to be more tied to the actual OCR engines or the extraction models that we're utilizing or the classification models and the other is going to be things where things are just due to external influence. Things that we want to go ahead and catch as a part of our end-to-end solution and understand the root cause and get a better understanding of the different dependencies outside of just the document understanding components. One example of an external event that we're looking at right now is when you have information that was extracted, but there -- but it does not -- the cross reference with the source of truth, does not have the same information that was found on the documents. So for example, here, the model for this invoice processing scenario was able to find a PO number. We looked at the supplier address as well that was attracted from the document, and there was a mismatch between the address found on the documents and the address store in our system of record. This is a scenario where you would want to go ahead and either send this information back to the vendor because you need to go ahead and report to them, "Hey, you're using the wrong supplier address? Can you go ahead and correct this?" Or "Is this the right address you want us to update our system of records with." This would be a scenario where the -- from a model standpoint, the right information, the document was extracted, but you want to catch this exception, because there's an external event that you want to go ahead and resolve. A second example is when we have another invoice where you had the PO number extracted correctly, but it was not found in your system of record or source of truth. And you don't want to be paying an invoice, but you don't know if there's actually a PO number allocated to it, especially if it's a PO-based scenario or if you have a total amount that exceeds an automatic payment threshold or that exceeds some kind of automatic reconciliation process where you want to make sure you have a pair of eyes on that document because the dollar amount tied to pass in the document is very high. So you want to make sure with full certainty that you're processing that 100% correctly. These are scenarios where we want a document SME to go ahead and investigate. How we go ahead -- sort of knowing now kind of the different categories here are some examples of those external events, we want to discuss how you make the most of those events occurring and balancing those with the automatic straight-through processing that takes place with your document processing initiative as well. How do we transform those external cumulative events or the system processing events into positive outcomes is by first going ahead and categorizing those into their separate buckets. So we can take this example again of 10,000 documents and say, okay, I have 500 of those 5,000 that require manual intervention, 10 of those are due to an OCR issue. The warrants test correctly, 400, we couldn't find a field. A document specialist is going to correct that. But what I as a team -- what we as a team can do is identify trends that we're seeing with these issues and see, hey, maybe I can create a business rule or close processing change that actually corrects or resolves this very common digitization issue in my process or maybe I need to go ahead and add more samples for [indiscernible] subset of formats and layouts so my model can learn more about these invoices of these documents that the system is just not doing super well on or even opening up a bug or an incident with the UiPath team that you work with, right? These are all ways that we try and resolve the system processing issues. And then for the remaining documents that are kind of -- that are impacted by external influences, we take these 400, 500 documents. These might be, again, examples of where you had invoices with mismatches between the system of record in the document. You had the wrong tax rate that was being used in cross reference but failed in your system of record or you had a batch of invoices that was actually -- that were actually marked as duplicates. And so these are all examples where you would want to go ahead and fix that system of record. You want to avoid reworking any documents potentially even paying a document twice. And this is great that you can identify these or being able to identify these is great because you're going to be able to add automation into your entire solution that can automatically resolve these. Hopefully, that's the goal. And being able to automatically resolve these is still going to add efficiency into your end-to-end solution, even though these 400, 500 documents weren't able to be straight-through process without manual -- without some kind of human intervention, right? You're still going to get the actual resolution automated, which is going to increase your overall business value. Lastly, the key thing I want to make note of here is we're talking about how to go ahead and kind of do a root cause analysis and what steps to take to go ahead and take action on these different events that you're categorizing. The key dependency is making sure you track these, that you have [indiscernible] logging in place, that you calculate and analyze those metrics, whether you're using UiPath Insights for some other existing BI tool. And then lastly, I'll go ahead and -- well, we'll take a closer look into model performance. So we looked at solution performance. And now we can go ahead and take a closer look at model performance, which is just kind of a subset of one of the components of understanding your solution performance. When we talk about model performance, we want to -- and this is tied to those system processing events. We want to go ahead and observe those trends, which we mentioned earlier. We want to understand how we're performing on key metrics like field accuracy or how we're performing in terms of accuracy for specific document sources. So that could be, for example, different vendors or different suppliers that we're receiving documents from. The tools we would use to observe those trends are Insights or AI Center. Insights is going to be that dashboarding tool in the UiPath platform. AI Center is going to be where you go ahead and maintain and deploy your models. Once we observe those trends and kind of identify we want to do a further deep dive analysis into, we want to go ahead and start to understand the root cause. And some of the common root causes that we see are due to inconsistent labeling a bias machine learning model or even low confidence scores, where low confidence scores might actually trigger human intervention, even though the value extracted from the -- by the model was indeed correct. And once we identify those using tools like document manager and like your automation workflow and studio where you might have those low confidence score thresholds, the common courses of action that we usually see are working as a team to reevaluate whether you can change those confidence thresholds you have in place in your workflow, reviewing the data labeling that has been performed, make sure it's not inconsistent or it's not causing any potential bias in the model where it's skewed towards a certain subset of vendors or document sources or even retraining the model on a more diverse set of samples that you can make sure that balanced and it really represents what you're seeing in production from a document processing sample standpoint. The tools that you use to kind of go ahead and usually complete those common courses of action, products that we've mentioned before, Document Manager, UiPath Studio and AI Center, Document Managers where your sample reside, your training samples, that's where data labeling takes place, Studios where you build your entire recipe for what steps the robot is going to execute and then AI Center is where you maintain test and deploy those machine learning models. From model performance, I mentioned data labeling. Data labeling is a key component to any document processing solution that uses a machine learning model. I want to more -- I want to dive a little bit further into how labeling can really impact a solution. And so we'll talk about some common pitfalls and some key items to remember. One of the common pitfalls we see is when a model has been built with noncritical fields included, and those noncritical fields are triggering manual intervention. And so I think the key thing to remember there is that you should really be defining your model to extract requirements. You should be linking every single item that your model learns about to a business requirement, and that's going to be done together as a team, together with the development team, the technical team and the business stakeholders who are tied to that document processing initiative. Second, one of the other common pitfalls is when there's bad training from the machine learning model. When you have inconsistent labeling, when one team member labels a document one way and the other labels in another way, that's going to be confusing for the model. You don't have enough samples, the actual model to learn from or when it's just not -- the actual training sample set is not representative of what you're processing in production. And the key thing to remember there to usually resolve these or avoid these is documenting very common edge cases, documenting the methodology with which we're going to go ahead and label documents as a team and then also having the proper training, right, working with SMEs, the subject matter experts who know the documents very well to make sure information is being labeled correctly, to maximize what the model is learning from and to make sure that it's accurate in order to avoid these common issues that are seeing downstream. And the last couple of minutes here to just talk about what we see coming up in the future. We've talked about a lot -- best practices for business rules, solution performance, model performance and data labeling. From a product standpoint, we have a lot of things we're excited about. So one of those is enhancements to processing on structured documents. We're especially -- one that we're excited about is the integration with communication mining and document understanding. We are also -- we're looking forward to adding capabilities that accelerate time to value. So getting you all into production much quicker. This is going to be tied to making our document sharing services available as APIs, going ahead and adding capabilities that are going to help with -- that are just going to outperform real-time model retraining, that are going to help prevent that kind of model bias that often takes place when deployments get rolled out, enhancing user experience. Also very high in our priority list, we're going to go ahead and try to improve the UX to help users build models faster and reduce that learning curve. And then lastly, we have some deployment insights and operations that are top of mind where we're going to be adding enhancements to the insights, document understanding focused dashboard templates as well as some improvements to the validation station, which business users use to validate documents. We're really hoping that all the changes that we have here in these 4 categories is going to make that learning curve a little bit reduced in terms of getting up and running and getting your automations, high adoption processing into production much, much quicker. A lot if information, but I believe we are out of time. So I do appreciate you all joining us today. And please reach out if you do have any questions. We're happy to help. Please also reach out on our -- or check out our public documentation. We have a ton up there, and we're constantly updating it. It's a pleasure chatting with you all.

Hi, everyone, and welcome to this product session to introduce UiPath Communications Mining. Communications Mining is the latest addition to the UiPath family. We're a no code natural language processing tool to help our customers analyze and automate all their messages at speed and at scale. I'm George Barnett. I'm the product lead for Communications Mining here UiPath. We were previously known as Re:infer. Re:infer was a spinout from the world-leading machine learning lab at UCL, and we were acquired by UiPath last year. Now Communications Mining is integrated to automation cloud and available to all enterprise customers with AI units. In this session, I want to cover why Communications Mining exists, the problems that we solve for the enterprise. I'll then introduce and demo product and close with some of the value that our customers are generating using Communications Mining. So Communications Mining exists because in business, every message counts. Businesses send and receive millions of messages each year, and it's across multiple channels, whether it's e-mail, service tickets, chats or calls. Whether these messages are sent or received, they're internal or external, each message contains valuable information about your customers, about your business and about your products. Almost every process in the enterprise requires a conversation, whether it's our customers reaching out because they want something or speaking with one of our colleagues to get work done across every team and across every vertical, conversations or either a problem to be or an opportunity to be gained in sales, in support or in finance, conversation of how work gets done. But to date, automating and extracting insights from these messages is manual, It's costly, and it doesn't scale. The current methods that organizations use to try and solve this problem are falling short. While they have their use and can provide value, they have significant limitations. Some of the methods that we've seen across our customers and elsewhere in the market to tackle this problem include using surveys and sampling, using keywords or hard code of rules or developing web forms or using ticketing systems. Now if we think about sampling and surveys, these are typically limited in scope, and they only look at a small subset of the data. And because of the time and effort in both in collecting and analyzing the results, they very quickly become updated. Keywords and rules-based systems can getus some of the way there, but they're brittle. If the business or process changes then the rules break and they're not resilient to variation, abbreviations or poor [indiscernible] And all of this means that they're challenging and costly to maintain. Lastly, web forms and tickets are great for simple and known processes, but they rely on people for manual data entry, which is prone to human error. And we often see that people just select the first option available or they select other, and this leads to data accuracy issues. Additionally, we're forcing our customers to use a channel of communication that may not be their preference, and this can lead to a poorer customer experience. Businesses also can't just continue to throw people at this problem. Their services are already at a breaking point. In many of our clients and across the enterprise, we see that almost 50% of people's time is spent managing e-mail and the associated tasks and requests that they receive. And almost 80% of customers aren't satisfied with their ability to use data when managing their customer interactions. And this problem just isn't going anywhere. The number of channels is increasing. It's e-mails, it's tickets, phone calls, slack messages or Zoom meetings and the volumes associated in all these channels, well, that's just increasing, too. UIPath Communications Mining solves this problem. We're a no code state-of-the-art NLP platform. We combine unsupervised and active learning to build customer-specific machine learning models to analyze and automate their business communications. Customers can connect their existing sources of communications. So that includes things like e-mails, service tickets, CRM notes. And we have built-in integrations to systems like Microsoft Exchange and Salesforce or we can use UiPath robots to connect and ingest communications from any source. Once the data is uploaded, any user, whether technical or not, can train custom machine learning models to interpret their communications and extract the data and insights they need to discover processors, common requests and opportunities for automation. We can then connect to downstream terms, either using robots or directly via our APIs, to automate the requests and queries we receive end-to-end. In this slide, I just want illustrate the categories of data that we can extract from each communication. So on the left-hand side, you'll see the original message. And on the right side, you can see the structured data we can extract. So you see that there are multiple requests here. There's a request for a policy renewal. There's a request to update an address. With Communications Mining, we're not just classifying the message into a single category. If there are multiple intent, we can recognize and extract all of them. We're also extracting the relevant entities in order to automate to make these processes. So in this example, there's a policy number, there's an address and there's a ZIP code, and we're extracting all of those. Communications Mining can also be used for sentiment analysis. So as well as the intent and the entities, we're also recognizing that this customer is perhaps frustrated by their interaction so far. I just want to show one more example. This time, it's from an IT service desk type request. Again, we're recognizing the request and the relevant entity. So here, it's a system access issue, specifically with Microsoft Office. But we're also recognizing some policy service-related issues and the relevant concepts. So here, it seems that the message is a follow-up to a previous e-mail and is also urgent. I just want to talk a little bit about the shape of the product and give some of the aspects that make it unique from other tools available in the market. So Communications Mining is completely no code. Regardless of technical ability, users can train and deploy NLP models. We have a fully guided training interface to guide users through this process and in build transparent validation statistics so users understand exactly how the model is performing. The models our users train rely on our underlying large language model, which is a state-of-the-art transformer-based model. Our models are fully customizable, allowing users to extract the specific labels and entities that they need. And 2 team could interpret the same conversations that need very different structured data. This is very simple to manage using Communications Mining. And our models are very fast to train both in annotating the training data and actually training the models. We can go from 0 to a well-performing model in just a matter of hours. Finally, before we head over into the demo, using Communications Mining, our customers are now able to understand what all their customers want, track and measure all the requests, queries and demand that they receive in real time and automate every transactional request that they receive. I'm now going to head over into the demo where I'm going to first give a quick overview of the product and walk through the output of a trained model and explore some of the queries and analytics that we can drive and then the types of processes we can automate. Then I'm going to show our new fully guided training interface, which guides users through the model training process from start to finish. And then finally, I want to demonstrate some of the governance and control features that allow enterprises to deploy this type of technology in production at scale when automating processes using communications mining. So I'm starting off in automation cloud. I'm going to navigate over to Communications Mining. We start off in our data's page where we can see all of the data sets that a user has access to. I'm going to start off and we're going to take a look at this underwriter broker support mailbox. So this is some generated data that we built to replicate the type of mailbox you'd see supporting underwriters at any insurer. On the left-hand side, you can see the entities and the hierarchy of labels that a user has declined to interpret this data. And then on the right-hand side, you can see the messages and the intelligence that we're extracting. So in each message, you can see highlighted the specific entities that we're extracting. And beneath the message, you can see the intents that we're recognizing. So you'll notice, as I mentioned before, that there's multiple intents recognized on each message. So wherever there are multiple requests, multiple intents, we can recognize all of them. And hierarchies -- attached on the labels are hierarchy, which gives you great flexibility in the structured data that you can extract. I'm now going to head over to reports. Reports lets us start to analyze, query the data, see what's really driving the volume of messages that this mailbox is receiving. So we can see that this mailbox received just over 100,000 messages in this year. We have charts to help us understand the proportions, the volumes of the messages that we're receiving. So we have this tree map that looks at the portions of different messages. We then have other charts that show whether they're high-volume requests. The tree map allows us to explore the taxonomy, explore the hierarchy and see what's happening at every level. So I'm going to click into policy. We can see what's really driving these policy volumes. Again, I'm going to click into amendment, and I can see that one of the largest requests -- one of the highest volume requests relating to policy amendment is related to address changes. Any of the statistics that we see in Communications Minin,g, we can double-click to see the underlying messages, which allows you to instantly validate that statistic. So now I'm seeing the underlying messages that Communications Mining is recognizing our requests for a customer to update their address. Now when I look at some of these, they look like they could be a good opportunity for automation where we're recognizing the requests. So we're recognizing that it requests to change an address, and we're also extracting the new address, so where the customer wants to change their address to. So I want to go back to reports, and I want to see how of these requests are we receiving in a year. Is this a good candidate for automation? I've gone back to reports. I'm going over to trends. Trends lets us see how the volumes are varying throughout the year. I'm going to apply a label filter to look only at the messages where we're confident it's a address change. So I'm going to filter here to address change. And that's taken our volume of messages from just over 100,000 to around 6,000. So there's potentially 6,000 of these requests we receive each year that we could automate. But when we looked at the messages, we know that we also need entities in order to automate this process. So I'm going to apply also an address line entity filter, and that's going to show us how many of these requests could we actually automate. So we've gone from 6,000 total requests received to probably 3,500 where we have all of the data necessary to drive that automation. It looks like there was a bit of spike in April. So again, I'm going to double-click here just to verify that these messages that we're extracting the intents and the entities correctly and that these will be good candidates for automation. So we're going to see the underlying messages. And yes, so we can see all of the messages received in April where Re:infer is currently predicting that these are requests to change an address, and we're also extracting the relevant entities needed to drive that automation. So if I hover over the labels, I can see the confidence at which we're predicting these, and I can do the same for the entities. So hopefully, that gives you a little bit of an understanding of the type of analytics, queries and the processes that we can automate. Now I want to show you how we got there. So how you would trade a model in order to recognize these labels and entities. So I'm going over to a different data set. This time, it's a -- it's still generated data, but it's now designed to look like an IT help desk. So now you can see there are no entities. There are no labels. There's no intelligence extracted from these messages. It's just the raw messages where we've ingested them from. This -- in this case, it was from exchange. We do have sort of metadata so we can see things like how many messages are in the thread, how many participants are in the thread, this specific messages position in the thread. As I say, there's no intelligence here extracted. So I'm going to go over to our new Train tab. Train is a fully guided training interface to take our users through the model training process. So at each stage of the process, we present relevant statistics to help them understand where they are in the process and how well their model is performing. And then we recommend certain training actions to build up a the model and improve its performance. So because this is a brand-new data, we have to 2 options. We can either upload the taxonomy, which was if we had a predefined set of labels, it would allow us to quickly upload that, start to teach Re:infer -- teach Communications Mining about that or we can start by reviewing clusters, which is where we use our Discover feature. So I'm going to use this Discover feature. Discover reads through the historical messages and groups together messages where the model thinks they share the same intent. And our job as a user is to confirm that they do share an intent and to give that intent a name. So we can see here that these are all referring to send application, and that application is Excel. So I can now create label. So I'm going to create application label as my highest level, and then my sub-label is going to be Excel. And despite double-clicking, I'm going to apply that label to all of the messages that we see here. Now that's going to send a training signal to the model. It's going to look back through all of the historical data and start to try to find more examples of these application requests specifically related to Excel. You can see here that our models already automatically started training in the background. So my job as a user would be to then go through all of the clusters. So at the moment, there's -- we're showing 10 clusters. That changed over time. Discover learns from the training data and always present new not well covered intent to that user. So we seem to have a -- here it's network connectivity issue. I can move on. We have another application issue. This time, it's Word. So it's application. This is a request of password reset. So very quickly, I can start to build a training to teach the model how to interpret this data. This is a vogue error. And through this guided process, we really reduce the time it takes for the nontechnical users to train well-performing models. So I can continue through these clusters. There'll be 10 in this first instance. When I'm done with labeling those, we can close it. It's going to inform the user of the training that they've just completed. It's going to train a model in the background. Our statistics will update, and our recommended actions will update. So while that's just updating, because we don't have so long in this session, I'm just going to move over to a slightly more advanced model. We can see that we're still very early in the process. We've only reviewed amount of data. We don't have many labels, and our model rating is still essentially 0 because model training hasn't probably started yet. So the recommended actions have updated. And you can see that on the recommended actions, we have these factors, which inform our users how the model is doing on a number of aspects. So balance refers to how well the training data represents the whole of the data set. Coverage is how well we're predicting over all of the different messages. So does a taxonomy of labels cover everything that we see in the data set? And then we have recommended actions relating to improving the performance of specific labels. So here, I can select one of the recommended actions. And again, as a user, I'm presented with a training interface where I'm just labeling one message at a time, and I can quickly read these, apply or create new labels to define the concepts and work through these examples. So it looks like we have a hardware issue. So I can quickly work through these examples, training the model and to interpret this data. When I've done in action, I'm always told exactly what I've done, how that will improve the model. And then I can just move on to the next recommended action. These are constantly updating as the model is trained, and we always inform the user of the most valuable next action they can take. So in a matter of hours, a user would work through these recommended actions. And at some point, they would get to a model, which is performing well. It's almost complete. It's a very nearly production-ready. And you can see that the recommended actions have changed. So specifically we balance. We want to make sure that the training data represents the data as a whole. And that's how you know you can trust this model in production. If we have biased the model in our training -- in how we've trained it, then it won't perform well in a real-world scenario. So again, as a user, I would just click through these recommended actions to improve the model performance. Hopefully, that gives you a good understanding of how users go through training model. Once the model has been trained, and we are happy with its performance, we can then go over to our validation page to really understand and get specific metrics on how the model is performing. So let's think about the address change example that we saw earlier and understand whether the performance is working well and what kind of controls and the governance features we can put in place to know that we can put that into production. So I've gone back to that underwriting admin mailbox. I'm now in our validation tab. Now this gives us specific metrics to really understand the performance of the model. So we have this overall model ratings' for. And that's broken down into a number of factors. So these are the factors that I mentioned earlier. It helps us really understand how well the model is performing and how it's going to perform in the real world. And we also provide a more granular performance metrics. So we provide the mean average precision for the model and the precision recall curve. But then we can look at the specific performance of the labels that we're interested in. So I can search for address change label that we were looking at earlier. I can see precision recall curve for this label. Because this is generated, the performance is excellent. In a real-world scenario, you're unlikely to see the performance this great, but you can get excellent performance using Communications Mining. So say, I'm a user. I want to automate this process end-to-end. I want to understand at what threshold -- at what confidence thresholds, the performance I can receive. And a given confidence threshold will give us different precision and recall. And the way to think about precision and recall is precision is essentially the number of false positives you can expect. Recall is the number of false negatives you can expect. And depending on your use case, you will have a different tolerance to false positives and false negatives. So when I'm thinking about automating this process, we need the business as input on the -- on that impact of this automation to understand where we would set that confidence threshold. Once I'm happy with the performance, I understand the false positives, false negatives I can expect to see in production, I can save a version of the model. Why that's important is it gives us determinism. So we know there is something changes in the model. If additional training is done, it's not going to change. It's not going to impact the predictions that we're receiving downstream. So I can save this version of the model. That's now ready. There's an API I'm going to use. We can use this model in production. I want to head over to streams page. This allows us to set API endpoints, cues of communications to be processed downstream, whether by rewards or by a service directly using the API. So I'm going to create a new stream, and this is a stream for our address change request. So we give it a name. I have the referenced version of the model that I want to use. So in this case, it's model version 7. I can then set the confidence threshold of which I want this to trigger. So we want to maximize our precision and recall. So it looks like we want to set this at around 65%. I'm then going to hit save. This trigger is published. This stream is published. And now it can be used downstream by other systems. That was everything I want to cover in the demo. So I'm going to head back into the slides. So hopefully now you have an understanding of how Communications Mining works, the type of data we can extract, the analytics we can drive and the processes we can automate. Using Communications Mining, our customers are able to drive efficiency by automating, changing or eliminating their processes. They're able to transform their customer experience, to understand what their customers really want and adapt their products and services to meet this demand and improve the ROI of their digital transformation project by using a data-driven approach to identify the most impactful change opportunities likely to generate the most value. But what does communication mean when combined with RPA? And specifically, what does it mean for the center of excellence? Well, it allows you to extend the scope of your automations into new processes and business areas heavily reliant on communications. It allows you to scale AI across the business without relying on machine learning or data science expertise, with the governance and control features you need to use this technology at scale, and it allows you to measure the impact of your automations by using real-time analytics and alerting on the changes that you've put in place. This is quite a content-heavy slide, but really, I just want to point out a few of the key use cases for Communications Mining. They broadly fit into 2 categories: analytics and automation. And we have use cases across most verticals, including banking and insurance, manufacturing, e-commerce. Anywhere there are teams and processes that rely on communications, you can use Communications Mining. Whether it's insurance credits or renewals, KYC and onboarding in banking, order amendments and processing and manufacturing or complaints in any vertical, all these business units have processors primarily driven by communications that are right for automating, analyzing and improving. Finally, to cover a specific customer case study. This is from one of our insurance customers. On the left-hand side, you can see their old process with multiple back-and-forth steps, all requiring communications and routing to the right team before completing the customer's request. With Communications Mining, this insurer was able to get the request to the right team first time and where the processes could be automated end-to-end, remove this manual work from the agents to do less. This has massively reduced the amount of manual admin work for their underwriters, meaning they can focus on their core value-add responsibilities. We've reduced that average time for processing from 3 days to less than 3 hours, and we've saved the business around 90,000 hours in the first year. Thank you for listening to this introduction to Communications Mining. If you want to learn more or understand how Communications Mining can be used in your business, please look a demo or speak to your account rep to find out more. And stay tuned for the final session where my colleagues will be discussing building the future of AI and automation, taking it from the lab to the road map. Thanks.

Hello, everyone. Thanks for joining us today to this session about Leveraging the Power of AI Center and building and deploying custom models. My name is Jeremy Tederry. I'm AI Center product manager here at UiPath. And today, I have the chance to have Russel Alfeche with me. He is a RPA technology leader for qBotica, and he will talk to us about the wonderful use case that he built for one of his customer, building the custom models on [indiscernible] detection and putting that into action using AI center and API. But first, let's an overview of what are the capabilities that we have here at UiPath. So as you may already know, we have different AI projects that we built to our customers. First one being a computer vision that allows you to automate base on screen and exactly what you see on the screen rather than taking [indiscernible]. So that was the first for this attribute. Soon after, we start introducing UiPath document understanding that allows you to automate all kind of documents and extract information from these documents and putting up information. A bit later Cosmin Voicu will join us as a way to automatically discover what process you can automate and how to automate them. And a bit earlier last year, UiPath Communications Mining was started and now allows you to automate all your communication, extracting text, finding email [indiscernible] any kind of communication, for sure. And the last one that will be joining us is UiPath Clipboard AI, which is currently in preview and I will encourage you -- all of you to, of course [indiscernible] preview and start using it and discovering it's new capabilities [indiscernible]. Beside all of this project, we also have [indiscernible] the platform leader, which is UiPath's AI Center. It allows you to manage your data, manage your model deployment as we [indiscernible] models. We also have a lot of out-of-the-bus models that you can use in the AI Center. You can retrain them, manage the deployment, manage the models, changed version, the valuation models. And you can, of course, continually retrain your model based on the data set that you can. Since data, of course, you will get another using another part of the UiPath platform, which is an action center. And you can monitor [indiscernible] using UiPath Insights. UiPath was bulit to be open, and this is why we have a lot of external connectors related to AI but not only. And so you can see a list on your screen here. They are not the only one, they are only the one related to AI, but we have a lot more to [indiscernible]. Let's dig a little bit more into what AI Center offer to you. And the first thing that you will do when you start using AI Center is to start using the model. We have different sources for this model. It can be a custom model. We'll be talking extensively about this today, but it can also be one of the dozens of ML models that we provide out of the box. And I will show you obviously, little bit later. So once you have this model, plus -- and the easiest way to start with an already trained model and just to [indiscernible]. So I'm happy [indiscernible] we'll go to the Studio. we'll simply [indiscernible] an activity, as everything [indiscernible]. And because we are a full platform, because everything is connected in [indiscernible], then you can automatically see all the skews that are available on your tenants. And you will be able to consume them basically by just [indiscernible] model. It just needs to know the reason of some model and what kind of input and output [indiscernible]. And of course, the model will scale as you also scale. You can just press a lot of data. Then, of course, if you have one question on the model, you would need to manage a different version of option model for that to go back AI Center train. And after that, we create a new version. And with a few [indiscernible], you can change the version. I'm changing the version just shortly for you. But at the end [indiscernible] the model is automatically updated, and the robots will now just point to this new version on the model. There's nothing for you to do here. [indiscernible]. This, of course means that you have retrained the model, and this is why we have to improve the section here. That will allow you to get back some data, label some of the data if you need to [indiscernible] new model, [indiscernible] assume or it can be just full on the platform. Then you [indiscernible] model travel [indiscernible]. Once you're confident now should [indiscernible]. Training can be manually [indiscernible] or it can be an automatic trigger to restart every week, every month, depending on your data [indiscernible]. Once [indiscernible] done, we'll have [indiscernible]. I was talking about the different or the box model that we have. As you can see, so this is not [indiscernible]. You have all the models that [indiscernible] So as you can see on the screen, you have different type of documents that [indiscernible]. And you also have different languages such. And if not the model future needs that may happen. Again, we have a lot of different use cases that we need to cover, so we cannot [indiscernible]. We have a document understanding model that we can use to start with. And basically, we'll be able to label your data using our document manager tool. And once your [indiscernible], you can just than generic document understanding model and start deploying that continuing it from your wallets. Document understanding is a bit use case, but that's not only one. And as you can see on the bottom line here, we have a lot of models on [indiscernible]. That's only what is available on AI Center, but don't forget that you also Comms Mining a lot of communication can be on the message in this new product. And last but not the least, we start working with image classifier and images in China. So we have a model, which is currently [indiscernible] to use [indiscernible]. And now let's talk more specifically about [indiscernible]. And to start with that, I will just give you some context on when to apply [indiscernible] models and how you can do that. First, when would you apply your own model and why? The [indiscernible] we have a lot of different unit cases. We know that we cannot go everything. We know that you have some data centers working with you. So for some of them, you will be going to need you on more because there is no other [indiscernible]. Once you realize there's no model, you have 2 different to bring your model. It can be -- someone build your data synergies, you can be found on that I would required for you and you need to rationalize model and put it into action. So [indiscernible] a good way to do that because it will make that very easy to just deploy the model using AI center and then put it into action using our robots and our studio to actually start doing [indiscernible]. But another way, and that's what Waters going to show us is to customize an open source model that's already available. So machine learning, community is very open. You have a lot of very, very good model [indiscernible] customer are most of the time open source. So you can just start with a [indiscernible] model and customize it to [indiscernible]. And with that, I think it's [indiscernible]. So I will Russel show us how he took [indiscernible] model and bring that into action to start only one use case, but multiple use cases that he was [indiscernible] AI Center [indiscernible]. The floor is yours, Russel.

Thanks, Jeremy. Let's see how we could combine 2 state-of-the-art technologies, one in optic affection and one in our API and AI model management. And that leads us to this demo where we're going to showcase a couple of use cases using this capability with the latest iteration of the most popular object protection are, which is [indiscernible]. The result is an automation use case enabled with intelligent image and visual tracking capability. And this shows the sense of being able to bring your own customer any open source models to AI Center and helps you easily operationalize in your business through the help of RPA. Let's have a quick overview of these cases. But first, for the uninitiated, let me give you a quick crash source on [ Yolo ] since this is the model that we're going to use for this demo. [ Yolo ] is one of the most popular algorithms in the object protection space. And as the name implies, [indiscernible]. It requires only a single forward propagation in its [indiscernible] network to perform both [indiscernible] and prediction of bounding boxes for the [indiscernible] objects. In simple words, it's super efficient. And it flows to popular as it both considerable accuracy while maintaining a small model size. Now with the release of [ Yolo V8 ], it has become much easier to use, making it an excellent choice for wide-range of public affection, image segmentation and e-mail specification task. Now on to the use case flow. The next section will show you how the high-level end-to-end process looks like. And I'll walk you to the intake parameters, how it's being model for influencing this whole set in AI Center, and finally, how it generates the output gallery that I just showed you at the start. There are different intake channels that inputs can come to cater these various use cases. In Air Force, for example, we can see automatically segmented clips with permanent intervals, which comes from different critical coverage areas such as baggage claims, arrivals and others. For a small or medium-sized business, there can be image or video feeds coming from stock rooms who continuously track inventory. And last but not the least, the community CCTV feeds can be into the model so as to track people coming in and coming out of, let's say, a particular event or rally. Then the next piece is the inference process, being the inputs from each fuel into a center where the models were trained and hosted. Object affection section task all of the multi-option tracking capability for videos by leveraging by track SDK. We have the base model [indiscernible] contacts teens model the trained with luggage conditions product [indiscernible]. Object segmentation is another type of task for the model, which is more fine grade than object protection as it provides more detailed information about the object's shape and boundaries. The last one is another custom model for image recognition using [indiscernible], which is a popular different learning technique that was released a while back for classifying images into different categories. For this use case, this model is used to categorize the inputs and create the description [indiscernible]. It's pretty much the access for [indiscernible] in the image context generation phase now that it has multi-model capability, although it's built obviously different [indiscernible]. Moving on, after completing the model process steps, the inferred image and video are [indiscernible] into -- from [indiscernible] and then upload that to a file repo like Google Drive and push into data service, which then feeds into a [indiscernible] back up project to serve as a human interface in a gallery like [indiscernible], which can be utilized by the different personas involved such as airport security, store managers, [indiscernible]. The airport security can now have an AI-powered [indiscernible] track unattended baggages in airport because it's important for passengers to always have visual contact with their luggage and other belongings. And unattended goods are considered to be a security risk at airport. And therefore, a bag left unattended, even for a short time, will trigger a security alert when it is detected by the airport staff. Another benefit that this can provide to the airport management is maybe for proper baggage handling and handling claims for damaged baggage in airport because airlines are responsible for repairing or reimbursing passenger for damaged baggages or its contents when the damage occurs while the bag is under airline's control during transportation, which is subject to maximum limits on liabilities. While on the other end of the spectrum, whether it's in a small business or communities, store managers can track that product and manage inventory to properly navigate business demands by leveraging AI cost of models with object detection capability. Now going back to demo, let's see how this process run from intake, processing and to output using this sample image that's shown here and let's do that by running this workflow based off of the use case diagram that we've shown. Just 3 things. To summarize the process, first off, let's just input the image or file from the QR folder, runs object detection model, counts and tracks detected objects, run the input captioning model described in for transaction. And then if [indiscernible] thumbnail and uploads them into Google Drive along with the output image or video artifact. And finally, it then updates a data service entity with the preview thumbnails as well as other pertinent details from the model in order to populate the title, description timestamp and model type. Enough in the talk and now let's go ahead and run the process. We'll take a half second to do the step by step and let's show you the output in a few seconds and verify you added record after a uploaded to Google Drive. And then if you look at the gallery, refresh, awesome. That's all for the demo. Hope you enjoyed it. On that note, let's go to the next part of the agenda.

Thanks, Russel. That's an amazing demo. It was very great to see how you can use all the different [indiscernible] of the platform. So not only in AI center, which is our focus today, but also apps and data service. So thank you for this wonderful demo. But I want to understand more and see why did you choose to bring -- continue on your own model capabilities with AI center on this use case?

Sure. First of all, there are lots of versus for out of the box, UiPath and open source path. I just mentioned the [indiscernible] AI Center that you can leverage machine learning problem areas, whether it's BU, language models, image analysis. [indiscernible] to start something -- with something than without. But at some point, you come across this very challenging use case, just like what I showed that requires to build your own model or even get the state-of-the-art model of the open source and be able to run with it, especially if it's a niche use case, let's say, for operating detection so the ability to bring something powerful into the platform makes it even more powerful and capable and lots of awesome capabilities and solve this moreful use cases.

Yes. That's a very [indiscernible] that's exactly why, we have this future available and why we bring that to our customer. But okay, now we have seen the ways, but we want to understand a bit more of this actually. So let's reveal the magic trick to the audience today and see how you actually do the model going through the call.

For sure. And that will be -- what we'll be covering next. Of course, it goes without saying that you already have your use case, whether it's a reignition problem, [indiscernible] problem to classify, I don't know, cats or dogs or whatever. So you know your inputs and the output that we are expecting. Now going into the next part, which is actually building, right, not sure about you, but I have the memory of a goldfish. So I usually start with an acronym. So I -- from this acronym, FORT, F-O-R-T, let me share my screen, let's look at the hole behind how you can organize the methods and functions. Of course, you have the typical dependencies, module imports, et cetera, which is essential for the whole to run. But in terms of organizing or naming your code, it should have, first off, the class called name. And within this name class, you have a method called predict. This is the most important method. And of course, you have the constructor in it. But in -- let's dig deeper in this sample ML model that we showcased earlier, which is based off of [ Yolo V8 ]. First thing you need to keep in mind is the input file. So the input you're going to feed with a model, which can fall under 3 categories, which is either string or [indiscernible] string, a file or a list of files. So in the predict function, if it's a string, then you just pass it directly to the workflow. If it's a list or a list of files, it would be fed by [indiscernible] so the can [indiscernible] and restore to original format. So in this example, our input is a file, right? And from this -- specifically, this is an image or video file. I want the model to be able to detect objects once I filled it into the predict function, and that's where the custom model comes in. Whether it's retrained or custom, [indiscernible]. The model, again, that I use for this example, is [ Yolo V8 ], which is the latest iteration of [ Yolo ] and provides an SDK to execute different tasks such as -- or modes such as predicting, validating and training. So if I can just show that few real quick. So you have different modes to predict, to train and validate. And in order to specify the task, you just need to call on to the particular model itself. So it can be as default as prediction. This is the model -- a name of the [indiscernible]. And if you do segmentation with the [indiscernible], if you want to do classification, you just [indiscernible] CLS. So that's how you specify the task and the model we are automatically downloaded. You don't have to model. So that's how easy use the SDK for [ Yolo ]. Now going back to the center code, right, you have to -- after you do the prediction, whether it's an image or video, of course, you have helper methods to help you predict which type of input that you have. So if you predict a video, again, the input will be converted into byte array and you have to convert it regional format, then you do the prediction, which is simply us calling the particular mode or particular method in the model. And right after that, the last part is to parse the JSON output based on your -- the output that you need. In this case, we're outputting the predicted class count or class frequency and also the byte array of the image that output that you have, which contains the bounding boxes of the detected objects. And of course, you have this nifty helper methods to help you derive those updates. So that is simply how you build a code. Again, you have the main class with the constructor and the predict function. At the minimum, you need to have those function to have your code ready for AI Center, okay? Now going back to the acronym, we got the app, which is file structure, the [indiscernible] file structure. Again, it would be the main [indiscernible] requirements of text. You have the whole, you know how to organize your code. And now we are on R, which is the requirements that ask how you generate this file is simply by -- so let me delete that file -- is simply by installing [indiscernible] Python model that can help you generate text file based on the packages or modules that we use within the code. So we just run [indiscernible] and then the name of the path where the main [indiscernible], and it will auto generate the requirements, the text. So once you're done with this, the optional step is if you need to have a training capability or just -- the only thing that you need to add to the code is the -- 2 additional methods to do train and save the model [indiscernible]. Obviously, this would contain that call to the train mode in the model that we're using, whether it's object protection or other machine learning model, deep learning framework in Python and a function to save the model [indiscernible] from premium. So once you have this -- all of this dependency study, you can go ahead and zip this file. So the name should be exactly the same name as the root folder. So that's very important. So this is part of the validation. So this is part of the validation. And after this, you'll go ahead and go to AI Center and upload your ML package into the project that you created, you upload the zip file, enter the name, then you drag and drop the file. You have zipped and you are starting drag and drop this file and it'll automatically validate the structure, the code behind whatever you did in the code. And once it's ready, you will have the ML package available here, and you're ready to deploy your new ML skill based upon those packages. This is the version and then create. So once it's deployed and available, now you can use it within your studio to run the ML scale. And you can see it using the ML scale activity that's available in studio, and you can touch tap right then and there. And that it, that's how easy it is to deploy your own custom or open source model into the platform to be able to materialize whatever use case or machine learning problem area that you're trying to solve.

Thank you Russel, that looks very easy when you showed it. And in fact, I hope it's that easy for other users to just do the standard trading and bring a lot of new models to us. Okay. So just to summarize what you just show us some best practices when you want to bring your own model. And first, you need to take a model already that works locally. So it can be a model that can be fully or it can be something open source that you just build the interface out of it, but it needs to work local obviously. I will recommend to start simple as Russel show us today, just with the selling function, building interface for serving then generate the requirement file, that a very important step, most of the AOR are actually coming from this file. And I really like the treat that Russel showed us today about using [indiscernible]. It's a very good way to do it and to make sure that it also works like [indiscernible]. Then zip the file, make sure that you have the right name, upload the zip file. And if everything goes well, then your zip file will be validating, the package will be under process. And you can just go to ML scale and deploy it. Hopefully, you did the process, you deploy it, it's now available. And as was Russel shown us, and you can go to [indiscernible] and start working on. If it's not available if you see any issue, you should see something else coming on the platform, usually, you just follow the [indiscernible]. There should be some pipeline and also issue. If you have a little bit of knowledge in my Python, you can just take action and stop that. Once you're comfortable with that and assuming you need retraining, as a [ trading code ], that's -- I'm not sure of a step. You don't need to do it. But as a [ trading code ], again, zip obviously, push it to the platform. And then try to do a small pipeline with a very small data set, so you don't waste your time in case it doesn't work. And once the model is ready with a new version, then you're good to go and you can go into production. And that's a lot. Thank you all for attending the session today. A big thanks to Russel, and I'm very amaze by this workflow. I hope I will hear from all of you about all the new models that you're going to put in AI center. Thank you all. I let you enjoy the rest of the AI Summit.

Hello, everyone. Welcome back. We've reached the grand finality of the 2023 UiPath Summit. Let's get started with our final session building the future of AI plus automation from the lab to the road map. I'm Luke Palamara, Vice President of AI Product Management at UiPath, and I'm delighted to be joined by an outstanding lineup of special guests today. With us, we have Scott Schoenberger, Senior Product Manager of the Integration Service and Cosmin Voicu, Principal Product Manager of Clipboard AI. Together, we're excited to engage in a thought-provoking discussion on the future of AI and automation. But wait, there's more. We have the surprise guests in store for you who I'll introduce as we progress through our session. Stay tuned. You won't want to miss this. All right. So let's take a moment to recap our journey so far at this exceptional UiPath AI Summit. So we began with an insightful keynote from [ Forrester's, Moris Epperson ] and esteemed professor, David Barber, as they export the most recent developments and implications of AI and the enterprise landscape. Following that, I hope you have the opportunity to attend 1 of our tailored industry SME breakout sessions before diving into our in-depth product sessions. We're really thrilled for you to experience our groundbreaking communications mining, enhance and broaden your document understanding automations, and thanks to AI center, create and deploy bespoke models for those 1 of a kind use cases. So as we transition to our next session, I'd like to express immense appreciation and congratulations to our thriving community. Your creativity and dedication are really what makes this event truly special. So now without further ado, let's dive in. Okay. So I am very excited and proud to announce the first ever UiPath AI awards. Over the years, AI has become increasingly accessible, revolutionizing business process automation across various industries and domains. Our UiPath automation platform has emerged as the foundation of innovation and these AI awards stand as a testament to its impact. Today, we guided to honor and celebrate the exceptional contributions of our AI community in the realm of AI-driven automations. So we have 2 prestigious award categories. The first is AI Ambassador of the Year, which is reserved for our distinguished UiPath MVPs; and the best AI use case, which invites submissions from our global UiPath automation community. Okay. So the primary objective of the UiPath AI awards is to acknowledge and celebrate the achievements of our AI product ambassadors. These individuals excel in utilizing AI capabilities and their automations, while actively promoting the value of AI, inspiring and educating their peers on how AI can revolutionize business processes. So exclusively open UiPath MVPs, our esteemed panel and judges carefully selected 2 winners per region, culminating and total 6 remarkable awardees. So please join me in extending a heartfelt congratulations to the winners whose dedication and expertise have truly made a difference in the world of AI and automation. So first up, from the Americas, we have Russel Alfeche from qBotica; and Sharon Palawandram from Ashling Partners. And from Asia Pacific and Japan, we have Lahiru Fernando from Boundaryless Group; and Nisarg Prakash Kadam from WonderBotz. And last but not least, representing Europe, Middle East and Africa, Leon Petrou from Complete RPA Bootcamp; and Jacqui Muller from Dimension Data, Northwest University. So a big congratulations and heartfelt thank you to our winners for embodying the spirit of true AI ambassadors. Each of these 6 outstanding individuals has been recognized for their numerous accomplishments, which include sharing insights on AI and Automation through online and off-line events, podcast and a lot more actually, publishing leadership content such as blogs, white papers and e-books on AI, creating educational videos, demonstrating proficiency in UiPath AI products like our AI center, AI computer vision, Communications Mining, document understanding, Task Mining, sharing AI success stories and making other significant contributions. So the UiPath awards aims to on the achievements of our vibrant community. And so as we establish this as an annual tradition, we encourage each of you to share your AI TRiSM, enlighten others about the importance and the value of intelligent automation and submit your contributions for consideration in the next year's UiPath AI awards competition. So let's together continue to innovate and excel in the world of AI. All right. So the second category of our prestigious AI awards is the best AI use cases. Participants were invited to showcase the most compelling and innovative AI use cases utilizing UiPath AI products like document understanding, AI center, AI computer vision and more. So our esteemed community members and UiPath jury then cast their votes to honor the top 10 most inventive and value-driving use cases. All right. So these remarkable examples of AI applications really to exemplify the power of AI and transforming businesses and industries. So let's join together and celebrate the creative minds behind these extraordinary use cases. So our top 3 winners are as follows. Pradeep Shukla from Peraton; Michael Sebastian from SimplifyNext; and Pradeep Chinnala from WonderBotz. Let's also extend a congratulations to our exceptional runners up whose outstanding contributions present a significant challenge actually for our UiPath jury in selecting the winners. So your achievements and dedication to innovation in AI are really commendable and deserving of this recognition. So thank you for your incredible work. Now let's take a quick dive into the top 3 winner use cases. Okay. So Pradeep's use case features an e-commerce client who face challenges in organizing products on their website using a rule-driven approach that provides heavily on expert opinions. This static method failed to adapt to evolving customer behaviors and preferences. So the really ingenious solution here involve training 2 machine learning models based on historical data, which then forecasted the next stage product views and sales revenue for each item by leveraging consumer buying patterns. So to ensure responsiveness and the shifting trends, these models are routinely out to refresh with updated data. So thanks to this data-driven dynamic solution, the client was able to optimize their website product arrangement and enhance their overall e-commerce strategy. Thank you, and congratulations, Pradeep Okay. So the next 1 is Michael's use case, which addresses the critical issue of detecting foodborne illness outbreaks, which could have severe consequences for public health. So conventional methods depending on patient reports, which can be unreliable and really slow to collect. So to overcome this challenge, Michael's team harnessed the power of social media platforms like Facebook, Google reviews and Yelp to gather data on symptoms and implicated foods. So they then employ the net NLP to extract valuable inflation base platforms and used AI center for sentiment analysis, text classification and name any of your recognition models. So the innovative approach here allows for rapid identification of potential outbreaks, monitoring their spread and offering an effective solution to protect public health. Really interesting use case. Congratulations, Michael. And last, but certainly not least, Pradeep's use case, explore the challenges that farmers face in making well-informed decisions about their farming strategies to maximize yield. So the solution lied in harnessing machine learning algorithms for crop yield prediction. It takes into account a variety of factors, such as slow composition, weather conditions and more by utilizing ML and farmers received tailored recommendations on the most suitable crops to cultivate on their specific farms. It boosted both production and profitability. So this crop recommender system actually suggests the optimal crop or fruit to grow based on soil and climate condition and that leads to substantial improvements in yield and revenue. Okay. Again, I'd like to extend my gratitude to all the participants and encourage each of you to take part in use case content like this 1 on the UiPath Forum. So your innovative ideas and applications of AI really do continue to inspire and drive us forward. Okay. So now for those of you who have juried alongside us in this path to automation, you're well aware that innovation lies at the very heart of our endeavors. Initially, we empowered robust perceive and interact with screens and interfaces just like humans. Next, we equipped our robots with the ability to read and comprehend documents. And then after that, they master the intricacies of processes and tasks. And most recently, they developed a profound understanding of communications. So all of these remarkable advancements have been made through -- made possible via the power of AI-driven automation. With AI-powered automation, you can discover opportunities for process improvements and automation from your back-end systems to desktop tasks, nearly all of your communication channels, enables you to automate more as well. So no matter the type of screen or interface, you know with the UiPath platform, you'll be able to automate it in nearly eligible documents as well. It's massive amounts of unstructured e-mails. So the AI-powered automation also enables supercharged productivity for the pairing of the latest degenerative AI with our powerful automation platform. And I'm really excited today that with the help of the team, we'll get to share more on this in just a little bit. So AI has been constant in our platform since the beginning. But for many, it's still just the beginning. And because the use of AI has changed so much recently, there are many important questions out there from enterprise leaders. So some of those questions that business leaders are asking, such as yourself, are really around exploring and integrating AI into your enterprise. And so the questions are centered around things like, how do you integrate AI across different applications? How you deploy the latest AI models? And also not miss out on the next big thing? How do you prevent AI for making mistakes? And how do you ensure data security? So this is where the AI-powered automation from UiPath comes in. Much of these questions can be answered by taking advantage of the 3 pillars of AI-powered automation from your UiPath. These pillars support our platform, which enables the use of AI that's open, flexible and enterprise ready. So let's talk about what we mean by open, and explore the exciting possibilities of incorporating AI into your enterprise. So at UiPath, we recognize that each business is unique, and in one-size-fits-all approach to AI simply doesn't suffice. And that's why we offer a variety of options tailored to your specific needs. So first, let's discuss our build-in AI. We've deliberately incorporated AI and machine learning into numerous products to address critical business challenges. You may already be using some of these products and perhaps witness them in action during our product deep dive sessions. We'll also be showcasing some of the latest capabilities later in the session. However, we understand that not every business challenge can be resolved with our out-of-the-box offerings. That's why we enable you to bring your own model, develop your own model or customize any of our prebuild models to suit your particular requirements. And lastly, AI is moving really fast with new models and technologies being released in the market daily. Enterprises need AI-powered automation platform to ensures they are using the latest best-in-class models that apply across all applications. UiPath is an open platform that allows customers to base their automations on the latest AI vendors like OpenAI, Google, Amazon and Microsoft and extend them. All right. So now let's discuss the crucial aspect of flexibility, which we approach in really 2 different perspectives. First, we ensure our platform remains open and adaptable, enabling you to build and orchestrate multi-model automations with ease. So for instance, by leveraging the UiPath platform, we can harness the combined power of RPA, Communication Mining, large language model and invoice model from Document Understanding and OpenAI's generative capabilities to fully automate a customer refund process, only requiring human intervention for approval and validation. Many enterprise automation benefit from the use of multiple AI models as part of automating a single process. And UiPath orchestrates multiple AI models, providing automation capabilities that adapt to your ever-changing business needs. Now second, despite the advancement in generative AI technologies, customers will still need to fine-tune models to their own unique data to ensure accurate and ethical and compliant decisions are made by the AI. So UiPath uses an active learning approach to minimize the time that's focused on instilling the specifics of your business and the minimum amount of training is needed for a high-accuracy model. So communications mining is actually a great example of our active learning approach in the UiPath platform. Okay. And finally, of the 3 pillars, let's dive into what it means to have AI-powered automation that is enterprise-ready. So at UiPath, we understand that enterprise-grade AI requires more than just powerful technology. It requires robust security, compliance and governance to ensure your business is protected while using AI. That's why our platform is designed with these in minds so you can confidently deploy and scale your AI across your organization. Second, businesses can't just delegate their decisions to AI. On its own, AI is frankly too unreliable especially for enterprise settings. AI is at its greatest value in the enterprise when it's coordinated with human decision makers to review and check its work. And that's why UiPath have integrated intelligent guardrails that recognize the uncertainties of AI models, flagging issues and requesting human review when necessary. So I hope you've been able to see that by combining the openness and flexibility of our AI powered platform, with our commitment to security, compliance and governance, you can achieve true enterprise-grade AI. All right. So now recall just for a moment what I talked about earlier. The evolution of the UiPath platform started with empowering robots to understand and interact with streams. Then we evolved to documents and processes and tasks, and most recently, understanding communications. While I'm super excited to announce that today really marks another great milestone in our evolution with the harnessing of natural language to power automation. So the most exciting parts of our session today are coming right up. When we're going to announce a number of advancements around natural language power automation. So first up, let's consider that enterprise solutions often have many different pieces. The businesses need end-to-end solutions to deliver value fast. One such solution that our customers have been clamoring for is IT service management. And to that end, I'm thrilled to announce that we've entered into a partnership with Amelia, a leader in trusted AI, to bring together the power of the UiPath business automation platform with Amelia's enterprise-grade conversational AI to create a fully integrated IT digital agent solution. And with that, I'd like to welcome our very special guest, Allan Andersen, Director of Enterprise Solutions at Amelia to tell you more. Allan, thanks for joining us today, and over to you.

Thank you very much, Luke. This is actually really exciting to be part of this. And so let's get in and talk about what this really means for automation inside of the enterprise, like digital agents have been around for a while. We've actually been in this business for almost 10 years in the market with Amelia. And we've seen tremendous effects on this technology internally and as well as externally. But internally, you can now service your employees across IT, other types of services like HR and finance, but especially IT, you can get much, much faster resolution times, much more -- nobody sitting and waiting on hold. All of these things can actually be done and it's already available. So let's take a look at what Amelia actually does and how she works, okay? So at the beginning of this, Amelia communicates. So she communicates through any channels that you will. We're actually doing mostly voice implementations today. But we're also doing multimodal where Amelia may be talking with somebody, an employee, and they need to validate them or gather additional information. She can do that over SMS. You can also enable her through your teams channel, through other -- through websites, even have Amelia actually go in and read tickets that have been generated in ServiceNow or in another help desk application. So she will pick those things up and start communicating with the employee to resolve their issue, okay? The next step in this is trying to understand what the employee wants. Whether or not they've opened up a ticket, whether they're chatting with her or whether they have actually called into the system, okay? Now those things are what we call classification engines in most solutions. We take that a step further to go to much more comprehension around the actual request. And that is not just intent recognition. It's actually trying to understand much more complete, what is the request really about. So you can get very long explanations, where you get a lot of data to Amelia. Or you can even give incomplete sentences, and she will try and investigate and interrogate you to find out what is it exactly that you want. It have multiple things that you want to do, and you can also switch to other topics that you would want to have resolved in the middle of a conversation. Amelia knows how to handle all of those various things, and she talks over 100 languages, so you can have support across the world. Now the next place that she goes is she has understood what you want. And now this becomes a dialogue. This becomes a conversation. And she tries to extract more information from you to really figure out what is it, not only that you want to do, but how is she best going to be serving you. So she can remember past interactions. She can hand -- you can interrupt her. She can -- you can change your mind in the middle and say, "No, I don't want to do that. I want to do something different," Or the information I gave you earlier isn't actually the right information I have changed my mind. Amelia doesn't get summed up by that, okay? She will also understand when somebody gets frustrated. And probably know is to escalate them to a human. Now as part of this dialogue and this conversation, [indiscernible] solutions, automation solutions, integrations into back-end systems, et cetera. And this is where UiPath comes in and does that very, very excellent, right? So as part of the process, not necessarily just at the end of the process, is when Amelia fetches data or runs automations or actions in enterprise systems. And those could be databases, those could be applications. They are obviously IT service management applications. They could even be all sorts of outside applications that you may use to guide the dialogue and the conversation further along, it could involve ChatGPT could involve many GPT models that we're already working with. Now hopefully, the issue actually gets resolved here and the employee is happy, everything got done to perfection. But sometimes it doesn't. Now -- and then you need to escalate. Now when we escalate, we can escalate into a human. The human can be in the loop. They can send it back. This is also where we're working very closely with UiPath around their human-in-the-loop solutions, like Amelia will even be able to order translate certain things or the agent might be able to return the actual conversation back to Amelia after a clarification so that we're not always trying to actually solve everything through Amelia. It might actually require human approvals or even human involvement to actually resolve the issue. But the key thing when you implement this is to make sure that Amelia actually can understand everything or close to everything that people will come into a service desk or whatever to request. And then she knows, "Oh, I need to escalate some of these things to a human because I either can't do it right now or I'm not allowed to do it." And then finally, and this is probably one of the differentiating things for technology is really how it learns. Now it will learn both during the actual creation of the use cases. We're using GPT models for a lot of those things. But we're also learning from all the digital exhaust that actually comes out from running an IT service desk. What were the escalations? How can I improve constantly? So the technology will actually go through all of this digital exhaust and then present improvement opportunities on a continuous basis. So this is kind of the full solution, go into much more detail, but just know, we've been in this market. We've done IT for many, many years. We understand what people need, and UiPath understands how to actually make all those executions in the back-end systems, or as we'll see even on the actual desktops. So what are the types of things that we normally go in and do? Everything from, of course, resetting passwords, adding people to applications or e-mail distribution lists, handling Outlook issues, VPNs, security, et cetera. But even though on the HR side, all the common things that people will go in and have a request about, leave, have questions about do we support various absence rules, what may be the dress code? All of these things can be answered automatically. And then you can even do personal tasks or other things. So there's a lot of things where you can make your internal operations much, much more efficient and your employees much more efficient as well. Now how does this work technically? Well, there is a lot of integration between the 2 technologies. But what I really want people to focus on here is that there's both a service side integration so of course, Amelia can integrate into that, and then automation can be executed centrally on the server. But the key is also the desktop. So Amelia can actually communicate with the desktop automation solution from UiPath and be able to do things on the desktop as the employee is sitting and watching, talking with Amelia and then Amelia will actually orchestra changes through UiPath on the desktop. So lots of opportunities here, very well integrated solution. So with that, I'm thrilled to be here and be able to talk to you about this and good things going forward. Thanks And by the way, this ending point here with all the mountains, don't think of it as this is a mountain to climb. These are actually proven solutions that we have been implementing for almost 10 years now and UiPath has been doing the same.

Well, thanks, Allan. I'm really excited about UiPath and Amelia joining forces for our customers' IT service desk needs. This is an end solution, combining UiPath and Amelia is super powerful. And I think it's going to help us climb those mountains for customers, right? So now, at UiPath, we're striving always to provide innovative solutions to meet our customers' needs. And today, our team will also be sharing with you what we've been working on around generative AI, a very hot topic, and the amazing work that's been done across our product teams from Studio to Document Understanding to our research labs. But up first, I'm really excited, and I know you will be, too, to hear Scott show the latest updates on how UiPath makes the promise of an open platform come alive with our AI and ML connectors. Scott, over to you.

Thanks so much, Luke. Hey, everybody. Scott here. I work on the integration service team. Really excited to introduce a category of the connector catalog that we've been working hard on, specifically in the area of artificial intelligence and machine learning. And with that, I also want to introduce, we're very excited to present our 2 different connectors in the category relating to OpenAI and Azure OpenAI. So first and foremost, I wanted to do a quick review of what integration service actually is. Integration service is UiPath's answered to API automation. So basically, we have an ever-expanding catalog of connectors, 50-plus right now, we plan on adding many, many more. The idea here is that each connector relates to a specific vendor application, making it super easy to drag-and-drop activities on to your studio or Studio Web canvas. You can easily set up as many connections as you like, and we're really excited to introduce connection sharing. So now you can share those connections across folders and tenants, making it super easy to set up and authenticate to these vendor applications in a way that fits through your organization. One thing that I'm going to get a demo in a little bit is our event triggers. So you have the ability to kick off automations based on events that happen on a vendor application. What I'm going to show actually is, for every new sales force lead, that is created, I want to create or to start my automation. So the use cases are kind of limitless when it comes to how to use event triggers. And a common misconception is that you'd have to have a robot running at all times for those event triggers to happen, and that is not true. If you have an automation setup that is started by an event trigger, that will automatically happen when the event happens. Finally, there are a couple of other things. There's simplified automation design. So basically, we have activity packs, as I mentioned, that relate to each individual connector. Within those activity packs, you have, what we call, curated activities that relate to kind of common things that we would expect users to automate within a vendor application. And then, of course, there would be more generic activities, so allowing more advanced users access to kind of any API methods that the vendor offers or supports I think the real differentiator here is the ability to seamlessly work with both UiPath automation and API automation directly in your studio or Studio Web process. So combining the API activity packs with UI activities is really going to be, I think, super helpful. And finally, security and governance. So integration service unlike how you might set up API automation in the past with HTTPS activities, you no longer have to pull in your credentials for a third-party application from your assets and orchestrator, right? You can set up authentication. You can reference a specific connection directly in your studio or studio web process. So much more secure, easy to govern across a large enterprise organization. In terms of the connectors that we currently support in this category, so this is, again, something that we've been investing in for some time now. We have connectors for Google Vision, which includes their optical character recognition activities. They also offer face landmark and label detection. So within those activity packs, you could select those activities. Microsoft Vision offers something similar. Again, it's a different OCR engine. Image description, tags and object detection are also available there. Microsoft Sentiments, which is also part of the cognitive services catalog within Azure. Microsoft Translator. And of course, our classic activity packs for AWS include Textr apps recognition and comprehend. So I would encourage everybody to take a look at the connectors that are currently available and see if these would be of service or future automations. So what we're really excited, I guess, the bell of the ball here is the OpenAI connectors that we've just released into public preview. We have 2 different connectors. There's the OpenAI connector and the Azure OpenAI connector. Both are effectively the same or offer the same functionality. Two different activities are available, tax completion and chat completion. So text completion will give you access to the, I guess, at this point, legacy GPT-3 models. Those include Davinci, Ada, Curie, Babbage, some of the older text completion models, again, that OpenAI kind of originally offered. And then, of course, chat completion. And this is the GPT 3.5 model. This is what powers ChatGPT. As I mentioned, Azure OpenAI is also available. It's the same functionality. There's little added security and, of course, by authenticating into Azure, your are ensuring that your data is going to be segmented. So that none of the data that you submit to the models is going to be used for future training. I believe that OpenAI also released a new data policy along those lines as well. But for folks that want to make sure that this is going to work within their enterprise organization, Azure is a great fit. So a couple of use cases that might be of interest. Sometimes it's difficult when an RPA developer is putting together an automation and has to also have kind of an in-depth understanding of maybe your organizations Jira schema, salesforce schema, you could use text generation to go ahead and interrogate your organization schema and come up with smart queries to pull out only those records that you want to update in mass. Another, I think, super popular use case is going to be, and what I'm going to show in the demo, is generating text for messages like Gmail, Outlook, Teams or Slack. What I'm going to show actually is putting together kind of a simple marketing e-mail based on a new lead that comes in via salesforce. And then another use case, I think, would be really popular would be generating and building social media content. So something happens, obviously, it's really important to keep ahead of the new cycle. You want your organization to submit Twitter posts or LinkedIn post based on what's happening at any given time. You can use these AI models to generate that content and post those things via integration service connectors. Let's see here moving on. So the other connector that we're really excited to announce, we established a partnership with the Amazon SageMaker team. SageMaker, for those of you that aren't familiar, is a custom machine learning model studio. So it allows folks to build, train and deploy machine learning models using proprietary data or they also offer off-the-shelf models via a JumpStart or AWS marketplace. I think that this is going to be a huge difference maker for us because all of the previous connectors, including OpenAI to a certain extent, are using models that have been trained on data that might not be the same as what you have within your organization. And so this enables you to connect to models that have been released to production. We're hoping that this decreases the amount of time that it takes for a data science team to kind of prove out the value of a particular proof of concept. The connector allows, as I said, you to connect to any model that you have deployed to production. So that includes tabular audio, text and image models. And the use cases are kind of limitless here, right, because these are all -- the fact that you're able to connect to a custom model really makes this applicable to a lot of different scenarios. We're working really closely with 2 service integrator partners, Slalom and [ Redesign Consultoria ], and they are actively in the process of using the SageMaker connector directly in their automations for things like fraud detection, churn prediction for sales, credit rating approvals, insurance policy underwriting. These are all things that are actually actively underway right now. So really excited to see how those things evolve. So for the demo, I'm going to go ahead and show an example of using the OpenAI connector in conjunction with some of our other connectors, including Salesforce and Gmail. So basically, what's going to happen is, as soon as a new lead is created in Salesforce, so anybody could be submitting a new lead. This could be a part of a separate automation even. As soon as that lead is created, what's going to happen is, we are going to submit a prompt to OpenAI to generate a marketing message to that individual and use the Gmail connector to go ahead and send that out. So I'm going to go ahead and jump into the demo. Okay. So this process is going to start with a Salesforce event trigger. This trigger is pulling this specific instance of Salesforce for any new leads that are created, and the output is going to go ahead and identify that specific lead that has been created. I'm going to use that new lead, and I'm going to generate a random account manager name with GPT-3.5. So this is the chat completion activity as you'll see over on my left side here. And again, this is the same model that's being used by ChatGPT. What I'm going to ask for from ChatGPT is to generate a random first and last name. I'm going to use that as our fictional account manager from which I'm going to send the e-mail to this new lead. There are a couple of other options here within the generate chat completion activity. You can give the model additional instruction around how you want them to respond to the prompt. There's a bunch of details directly in our documentation that explain this. And a couple of other options here, but I'm not going to spend a whole lot of time on during this demo. So I'm going to get a generated text output, and that is my account manager. So I set that variable there. The second ChatGPT prompt I'm going to submit is asking the model to write a short introductory e-mail that highlights some of the UiPath customers from that new leads industry. It's going to be written from the account manager that I have -- the name that I generated via other ChatGPT activity. And it is also going to ask or send a couple of highlights regarding case studies that would be salient to that new leads industry and send them a quick link. So I'm going to save that. And the very next activity is going to be this Gmail activity. So sending the e-mail to the new lead with a specific subject line, that ChatGPT generated body. Cool. So now I'm going to go ahead and add a new lead here in Salesforce. So I'll do that. Just use my real name. Now let's go ahead and say, Microsoft, and industry is going to be -- make sure we have to set this, and also the e-mail. I am going go ahead and save this. Now normally, if I have this automation already published at Orchestrator, as soon as that new lead was created, this process would be triggered. But because I'm running this locally, this -- I'm just kind of going through what we call the debug, the process. So it's going to go ahead and pull for any -- when I press play, of course, is going to pull for any new records that have been created in the last hour or so. So without further ado, I'm going to go ahead and execute. Okay. Finished in 20 seconds. I'm going to go ahead and reload here. And you can see this is the new leads inbox. Says dear Scott, hope this e-mail finds you doing well. Used our fictional account manager's name. Highlighted a couple of Fortune500 companies that we're working with. Also sent a URL to the case studies and signed off. So that is demo. Awesome. So I would encourage everybody to go ahead and join the previews. These connectors are SageMaker, OpenAI and Azure OpenAI are all available right now in the catalog. Please do submit any feedback that you have via our community forum. I'll also be sending out a survey to our insiders group, and would love to hear what other connectors might be valuable to you within the AI and ML category? And then also would love to hear how we can make these connectors more effective for your use cases. Thank you so much.

Scott, thanks for sharing the latest on the AI and ML connectors. And congratulations to the team on releasing the new connectors so everyone here can start easily using GPT within their automations. So now a bit more on the GPT front, this time with Document Understanding. So I am very excited today to announce that we've made available a new Document Understanding activity in the UiPath Marketplace, and we're calling it AskGPT. So now perhaps you've been hoping for an easy way to unlock the power of generative pre-trading models on your documents as part of an automation. This new activity that's in preview can bring the power of GPT to your unstructured documents to extract the key elements using natural language prompts. So if you just add in the prompt and then GPT will return the results and give you the answers to your questions. Some of the cool features of this is that you can actually have a user validate what GPT is bringing back for you. You can query the document for various different elements like in this case, if we're looking about what's in the document? In this case, it's a lease. You can also do things like extract the rents amount for the year. So you can do some pretty complex queries, actually it will not just return the answer, will also return the evidence with the question as well. So very cool new activity. Definitely recommend going and checking it out. It's available in the UiPath Marketplace under Internal Labs. You can just go out there and do a search for AskGPT, and you'll be able to find it. And I hope you really enjoy using it. The team has been working hard on it, and we want to really look forward to getting your feedback on it. Now there's still much more guys. And then we've already talked about a lot today. We've had some really cool exciting announcements, but there's more. So earlier on today, I mentioned how we are using AI to supercharge productivity. We've also been talking a lot about generative AI. Well, here's where we bring those 2 together, and we're calling it Project Wingman. So what is Project Wingman? With Project Wingman, our goal is to make sure that every employee, developer or not, is capable of easily creating automation, and, in this case, with natural language prompts. So it's just a few words, you and anyone or everyone in your business can start automating away some of their most boring repetitive work. So let's take a look in a demo. So here we are within studio. We're prompted with a great question, what do you want to automate? Well, you can start by selecting 1 of the personalized templates based on your organization's past activities, or we can have a bit more fun and start by simply telling Wingman exactly what you want to automate. And now let's say that at the start of every week, you hold a team meeting. It's early and usually on Monday, so you always have to end up sending out the recordings to those who overslept. Well, let's ask Wingman for help. Wingman will create that automation for you. Now from the workflow configuration, you will know exactly what's missing and what to add in order to make an effective automation. Wingman will guide you through the entire process. So here, we add in our Zoom account and configure the Slack channel so that the automation distributes the recording to the entire team. And look, Wingman is here to help. And once the Slack message is configured, I can also add this to Confluence as well for long-term archiving. And with just a few clicks, my automation is ready to run. All right. That will take care of an easy, but annoying task. But what about more complex automations? Well, Wingman that is there for those, too. Apart from those meetings, your team has also been on a hiring pretty lately. And your hiring manager keeps sending you large Excel spreadsheets with your employee details to enter into Workday. Let's ask Wingman. Just as quickly as we typed out request, we can see the building blocks of our automation laid out for us. Now this is good, but I also don't want to have to go and hear every single time and run this automation manually. So let's switch to automation to 1 that's triggered on schedule using Wingman's suggestions. All right. So Friday at 4:00 p.m. sounds about right. Great. Now like before at the end, I can choose to either publish this, or I can share with the rest of my business for their own use. Now Project Wingman is still a work in progress, but we are excited to share more with you soon. All right. So with Project Wingman, you'll be able to create automations with natural language, making it easier than ever to get started with automation. You'll be able to change those automated processes with natural language so you can make adjustments on the fly without having to dive into the code. And third, it will provide you with suggestions on how to complete the automation saving you time. So we're making automation creation more accessible and intuitive than ever before. And with the UiPath platform, really the possibilities are endless. So I hope you guys are all excited as I am about that. Very cool new project that we're working on and really excited to share with you today. But now other -- one more thing that's really exciting to share with you today. And on that note, I'm going to pass it over to Cosmin Voicu, who will be sharing an update and demo of Clipboard AI.

Hey, everyone. I'm Cosmin. And today, I will give you a quick look at what we're working on with Clipboard AI. We feel it has a tremendous potential to solve the last mile, so to speak, of copy-paste actions. Okay. So let's jump right in. First of all, what is it? Clipboard AI is a stand-alone application, so it doesn't need studio assistant or anything of the sort. So a stand-alone app that helps you copy data between documents, spreadsheets and applications in which ever combination you want. And after the demo, we'll look at the full list of supported document types and all that. Okay. Without further ado, let's see how it works. When you first install the application, you will get this desktop short cut, which if you start, you will get this tool bar with copy and paste buttons, which, of course, you can move anywhere you want to make sure it's your workflow. That being said, let's quickly take a look at a few examples. For the first one, let's say that you want to do a check-in with a specific airline like this one where you're required to manually enter various boring information like first name, last name and so on. Instead of doing it manually, you can just bring up your passport like this one. Go to the toolbar, hit copy window. It will be either automatically detected as a passport or you will have to choose the exact type of document on this list, then hit next and simply go to the destination and hit paste. And this destination is being analyzed to get the exact required information, and that information will be extracted from the document and entered automatically here as you see. Okay. So far so good. That went very smoothly. Now for the next example, let's take a look at one where you might not always run into the happy path. Because what you've seen is basically the shortest version. Let's say you have to enter an invoice into specific ERP system, and it works the same way. You just need to bring up the document, in this case, this invoice, hit copy window, say that this is an invoice, not a passport anymore and hit next. And the same as before, you need to go to the destination and bring up the toolbar and hit paste. And this time, it's maybe a more complicated situation. And you will get to this window where you need to confirm that everything is extracted correctly and everything is mapped correctly. By mapped mean, the information goes where it should. In case it doesn't, you have, of course, the option to change things around, remove things completely, add transformations and stuff like that, which we'll get into later. For now, it seems like everything went fine and you can just paste it here. Okay. So far, so good. Now this application works in a similar way to the normal copy paste, meaning when you've copied something, it doesn't go away after you pasted it. It still remains in the clipboard so you can paste it in other locations as well. For example, you could paste it, let's say, in new spreadsheet. And by the way, from now on, I'm going to use often -- at least use the shortcut because they are faster for me. And how it works, you just basically select where you want to paste the data and hit paste. And your data is automatically extracted and pasted here, well, along with, of course, all the headers and so on. Now if you would like to basically paste another invoice in the same newly created table, it goes the same way. You can just copy this one. It's -- automatically it's copied as an invoice, you can change things if you would like to. And just like an Excel select where you want to paste the data and paste. And there you go. Basically, the information just gets appended to the table. You don't get a new header and so on because the system basically identifies what do you want to do and tries to solve it for you. Next up on the list, I forgot to open the Outlook. Let's see how that works. Let's say, you need to create a new e-mail and you want to paste some information from this invoice into the e-mail. Just like before you just need to click paste and the data will be automatically entered for you. You can maybe remove some stuff that you're not interested or stuff like that. Okay. Next, let's take a look at something slightly different. Sometimes you may want to get a bunch of information that you get from Excel maybe and enter it into a system like a form. And instead of going through each row manually 1 by 1 and so on, you can just copy the whole table and paste it in bulk. This time, I will just copy a few rows just to not waste a lot of time. This will be copied, and the form where we want to enter this information is this one. So how it works is you've copied the table, you come here and hit paste. And now it looks like the system is not super sure of all the mappings and so on, although they do look correct, and you can just paste it here. And by the way, the system learns. Once you've confirmed mapping the settings and mapping and so on, next time, it will not ask about. Okay. So the system basically entered the first row of data. And now an important step is for you to ensure that this data is entered correctly. This is still early, so we want to make sure we don't mess anything in your systems. So please make sure you verify everything is fine. And once you do, you can just save this form, submit it and bring up the new instance that you want to enter, basically the new row in the table. Once you do that, you can just hit paste data, and it will, same as before, get entered in the system. And of course, with no confirmation, no mapping or anything. But if there is a new field that appears something you will get that's been, again, in case the system is not sure what to do. Okay. I think I will stop here because you've probably got the idea, let's save this one as well. So we don't lose any important information. Next step something even cooler, working with unstructured documents. Unstructured documents are basically documents that are mostly plain text, don't have an intern structure that can tell a robot or a machine, what exactly is going there. And until very recently, the only way to work with this is have a human read them and extract information, if that's what they need to do, or reason about them overall. So let's take this example like maybe you get a lot of contracts like this one, in this case, a bill of sale for a car. And you need to extract the buyer, the seller, the vehicle identification number, date, price and so on so you can place in an Excel table like this one. What you would normally do is obviously select everything by hand, copy it and then paste it cell by cell here. And that is if you are in the fortunate situation where you can actually select stuff, otherwise, you will have to type stuff in manually so not a lot of fun. But with Clipboard AI, you can just do a normal copy and then go here and hit paste. Now this information was correctly extracted, but let's take a closer look. Basically, you've seen me quick fix in mapper button, which allows me to make some changes to the paste section that was performed. So now if we take a look at the initial document, you see that it correctly got the seller, the buyer, the VIN, the date of sale, purchase price and warranty, basically, all the columns that were in this source document. But to show you a few extra cool things, you can do a lot of stuff here. For example, this price, while it was correctly extracted, it's not been the most -- in the best format for a spreadsheet. So for example, what you can do here if you want this just as a plain out number is just to write it as a number. For warranty though, again, it correctly identified the paragraph and expected the information that's relevant to the warranty, but you might want to just a simple yes or no answer, and that's what you need to say, and there you go. And it correctly transformed these values basically into what you need. And again, the same could be set for the date. In this case, it's a nice format, but you may want it, let's say, in the -- either the U.S. standard, or even better in the international ISO standard. And for that, you can apply a transformation. Basically, when you click that transformation button, you get a few examples just to start you off. And in this case, we will just use the first example because it's very common. So we'll just say it from, what is the first thing, day, month, year to year, month, day. And there you go, it correctly transformed the data in the format that you needed. This is a super powerful feature. Basically, with this transformation, you can transform currencies, units of measurement like between miles and kilometers. You can translate stuff. You can extract information further. You can interpret information. You can spellcheck. So there's a lot of stuff that you can do. Right. So now you can just paste your data, and it will be in your correct format, in the correct place and there's order in the world. Yes. So this concludes our short demo. I hope you enjoyed it. And one more thing before we go. Clipboard AI is currently in private preview. So we strongly encourage you to try it out. And most importantly, let us know what you think. Or if there are any interesting use cases that you might like to try this on, or maybe see supported, or any ideas you might like added, so just head on to our insider portal and sign in to the Clipboard AI preview, and you'll be on your way. Thank you.

Thanks, Cosmin. And for those, as Cosmin said, for those of you who are as excited as I am about Clipboard AI, you can sign up for the upcoming public preview at uipath.com/clickboardai. That in itself is a really cool announcement because you're going to be able to soon get your hands on what Cosmin just showed us. Very cool stuff. All right. We covered a lot today. So what's next? So we saw some pretty amazing things. The 3 things that I want to call out. First, if you're a developer, we're running automation program, do check out the latest AI connectors including the ones for OpenAI. Second and something where everyone can benefit from sign up to that Clipboard AI public preview. And last, all these sessions will be made available on demand email out to you on Monday. So if you missed one of the product deep dives or want to rewatch this session, be on a lookout for that e-mail. Well, everybody, it's been a great honor and pleasure. Thank you so much for joining us today at this year's AI Summit. Have a great rest of your day.