ServiceNow, Inc. (NOW) Earnings Call Transcript & Summary

Hello, and welcome to today's webinar empowering our workforce with generative AI. My name is Lamar Mills, and I work on a product marketing team here at ServiceNow. And we're thrilled to have you join us to explore how Generative AI is going to really help us to advance our workforces and we're going to just show a little bit of how -- what we -- some of the bank that we're doing here within a ServiceNow platform because we felt that this cutting edge technology is really poised to help revolutionize the way your employees are working and being able to put AI right there at their fingertips. And so without further ado, I'm going to go ahead and move to the next slide, just talk about, well, safe harbor here -- move over, safe harbor statement, forward-looking statements, et cetera. Just want to make sure that we're going to talk about some things that are going to be forward-looking analysis, so let's just keep that in mind. But then we're going to just move to our speakers. And I'm so excited to have this group of talented individuals that are with us today. We'll start with Mr. Rajeev Sethi. He is the Vice President of Emerging Technologies here at ServiceNow. Moving on to Ms. Tricia Cornish, our Senior Advisory Solutions Consultant, who will be showing us the demo today and showing us some of the great innovations that we have around AI. And then also as we make our way toward him, we're going to talk a little bit more about skills intelligence and how AI can manifest itself within this particular world, it has to be Mr. Nick McClure. And that's myself there at the end, the pretty guy on the end there Lamar Mills. Again, I work in product marketing here at ServiceNow. So just to walk you through our agenda. We're going to just start off a little bit about generative AI, right? Answer some questions. I know some individuals might know a little bit more about generative AI than others. So we're just going to give a little intro, give a little understanding, give a little foundation and background to it, move on to work for some of us, right? You have different personas within an organization, whether it's an employee or a manager or an HR agent, anybody. And so we want to just show how AI can manifest itself and impact each one of those personas and then how we're using generative AI at ServiceNow. We'll follow on with the demo. And then we'll move into skill strategy, right? A little bit more forward-looking, talk about skills, which is really the currency and the connective tissue, if you will, around how organizations are looking to be able to help grow and develop their workforce, and then we'll follow on at the end. With that being said, we're going to move on to the next slide here, and this is a poll question, please, and we would love to have your participation here. So the question is, what is your biggest concern about using gen AI in your workforce? Hallucinations, bias, cost or use of proxy. And I'll give you a few seconds just to go ahead and type in your answers, then you should get submit button there at the bottom for you to go ahead and then submit your answer. And again, we're very excited to have you guys here. We are looking forward to being able to share a lot of great details and insights around generative AI. We know it's a huge conversation. If you get on LinkedIn right now, we know that's on everybody's time line as I'm sure everybody is very curious to see some of the great things that we're doing here at ServiceNow with generative AI? All right. So with that being said, we're going to go ahead and move over to the next slide, which will give us our results. And it looks like user privacy and rightfully so, right, user privacy is something that people are very concerned about data and information. So we want to make sure that -- and as we're going through this, but then also as follow-ups to be able to answer some of those questions that you have around use of privacy and make sure that you feel comfortable with that as you start to explore generative AI, okay? So without further ado, I'm going to go ahead and just get going with the presentation. I must stop talking here and hand over my microphone to Mr. Rajeev Sethi, who can take us away. There you go, Rajeev.

Thank you, Lamar. I'm really excited to be here with all of you. Really appreciate it. And as Lamar mentioned, Generative AI is top of the mind of everybody, like it or not, it's all around us. But before we do that, we know in my role, I've been looking at what is happening in the innovation and you go back -- and go to -- because this was an interesting slide and data, which I came across around innovation because that's what I'm leading at ServiceNow within our DD organization. And that house the impact of innovation on the new occupations overall, how that is impacting the employee growth out there. But -- and if you look at it, last 80-plus years, it was before I was born when this data started getting collected. You can see how the innovation is fueling the growth of the occupation. People are worried about that Generative AI is going to be impacting our jobs and how it's going to happen. But if you look at it, machines have -- which you do repetitive process, replace humans on a lot of places to do those jobs out there. But AI is going to have a very different impact on just our lives across the board and which we can already see, AI has been there for the last 15-plus years. We're just now getting a huge lift towards the AI. It got onto the limelight with all the ChatGPT conversation and generative AI, which is definitely going to have an impact. And we are already seeing the conversations and we'll see how this is going to play out, but we know that every time an innovation happens, new jobs come on the market and definitely the existing ones need to transition to accommodate for the new innovation going on. And that's how the growth happens overall in the industry. When you look at AI, AI as I said, it's not been here just recently came on board. It's been there for a while. And it has been constantly evolving. It started with all the basic AI then went into machine learning, then we have been talking about the preplanning and now we're talking about generative AI. But this is not the linear way or in a way that sequential way things are happening in every space, AI is already there, the reinforcement learning. How AI works, that is another field within AI. So AI has multiple fields out there. And all that is constantly evolving, we are just seeing the tip of the iceberg and we are feeling it through Generative AI. And when we talk about generative AI, it is definitely driving productivity but also AI has been trying to help us with in the operational sense. It's been helping us with optimizing our work and the things which we do and obviously looking at long term how it's going to transform our work and in our personal lives, too. But when it comes to Generative AI, there are a lot of different capabilities we all know. We have -- I'm assuming you have already tried some form of generative AI generating some text, generating some documents, you give a small prompt and you can generate a document, give me like a 16 -- like 200 word document or an output on that way. Code generation is a big thing going on in the developer community, in addition to the code generation, how do I fix some code, how do I optimize my code, that is we are already seeing some of those capabilities there. Speech, generating human-like speech, I'm assuming some of you might have already seen some YouTube videos, both speech and video out there. People are creating podcast now. You can -- which will sound natural. So you have a text which will convert into speech for you and it will leverage an existing speech, which has been recorded. And obviously, generating video, I'm sure you have seen some of those videos there on YouTube or other social media sites where you can generate videos and do animations and have something of video. So generative AI is definitely transforming a lot of different industries out there. And where we see it's going to help us in our workforce is absolutely on the customer support side. We can see the generative AI-powered chat bots and all the other instant, giving feedback to our customers in terms of assisting them to solve the problem we definitely are seeing. That is where it can help you. The other places that we are seeing is analyzing data. We see a lot of charts and graphs out there. Is there a way you can translate that into infer what's happening on that charter graph? Or obviously, we have a lot of data can analyze that. So here is another use case which you can apply or look at it from that point of view because I'm sure we do look at charts and graphs, some of you may be looking at it all day long and trying to figure out what's going on in that, and it can help you in that regard. Repetitive process is a great opportunity to look at it from that perspective and make sure that you can apply generative AI and make individuals productive and drive value in that particular sense. The other area, as I talked about it, is around code generation where you can create applications, you can create -- use AI, prepare programming, you can do code re-factoring including documentation. And by the way, these are all some of the use cases where we have started within ServiceNow to experiment, to pilot, to make our users for -- who last about 6 months, they have already started applying this internally within ServiceNow. And one more area where we see generative AI can help in the workforce is creating personalized training. We talked about how some of the capabilities like around speech and text is you can take that and create personalized training for employees to enhance their skills so that you can target, you can target for a particular say an analyst in the organization or you can plan -- like target a developer in the organization who may need some help in terms of improving their coding skills or documentation skills out there or you can target any other persona in the organization and you can create personalized training for them, which we see is going to be a huge productivity gain. The individuals are little bit concerned about how this is going to impact them, but this is where we help you, help their workforce from all the angles, right? If I'm a support agent, how can I help my customers better. My role is an analyst, how can I do my -- work better and faster out there. So there are all these use cases, which you can apply from all the different angles in the workforce out there. Now -- but when it comes to gen AI, it's great. But we know it's not perfect, and we are all learning as we go by, and that's this webinar, hopefully is going to help you to also get some insights about some of our learning. We definitely see that. It's going to increase efficiency. It's going to make people more productive out there, make better decision-making and have decision-making because now you have some -- you can get some analysis done. And then based on that, you can drive some decision. It's available to you all the time. You are not now relying on some other individuals or others where you can generate some of this information. And obviously, it's augmenting your skills out there. So that -- those are some great benefits of generative AI we see today, which will -- going to drive productivity. But then it comes with some challenges obviously. As you all pointed out some of the challenges you are concerned about VC. Everybody is talking about the ethical using AI for how ethical, how responsible you are going to be in terms of using AI. There are a lot of regulations, which have been talked about in different countries, in different industries. So we just need to be mindful of that. How we're going to transition our workforce to leverage this great technology. People need to make sure that they can trust it with and have the transparency in it so they can feel that it is supported. We know AI is all about data and you need good quality data and you need good quantity data. And you know some of these models, which have been out there, which has been built using -- trained with billions of parameters out there. That's the reason the quantity matters in this particular case because that will help us to get better output on that. And lastly, the effort and the expertise, this is not a magic. We know that. Hopefully, some of you have been experimenting. It sounds magic, but you need to put in the effort and you need some expertise to really leverage this technology in your work environment. Now coming to where internally within ServiceNow, we have been applied some of the overall AI and some of the gen AI capabilities is, as you can see in each function, we are looking at it. And as you know, ServiceNow is a software company and digital technology is where we see on the IT side, where we can apply for case summarization, code generation for our developers. On the HR side from different pieces like change request and skills growth, which Nick is going to talk about, finance for order validation. And these are all some of the use cases of AI. But like in sales side, we have created a sales assist, a Q&A capability for our sales organization. Customer support is a huge one. Supporting you all as customers of ServiceNow, when you go to our customer support side today, and you search for something, you should get some of the genius results as generative AI-generated output out there. Case summarization is being used by a support agents. And then again, legal, there are a few use cases, which we have been started deploying and exposing out there. So again, this is not an exhaustive list. This is just a sample is for you to get a sense of it by each function where we have applied AI overall and some of the gen AI use cases. And what results we are seeing already, we are seeing that especially on the agent side, for the support they are already finding the gen AI results, which are coming back, 75% or higher accuracy out there. That's a huge thing because we know that AI is not going to be 100%, and that is the reason we want to make sure there's a human in the loop before that information is shared somewhere. It just speeds up information for individuals who are trying to generate it or summarize out there. They can get things -- they can get up to speed on the faster on a case so that through summarization, they don't have to go through a lot of work modes. We are seeing some of our initial results in that. And then obviously, when they are trying to resolve the case or close a case, they need to put the resolution nodes. Again, summarization is helping them to do it faster by about 15%. So some great, very initial feedback that we have got and the results we have got very encouraging for us. And obviously, this is just a small sample which we are sharing with you, and then we are exploring in all the different areas with use cases. Now when it comes to how it is helping within ServiceNow, as I said, employees and for our customers, we are as through search and other virtual agent conversation experiences out there. We are already seeing the engagement higher. We are seeing case deflections, incident deflections. And then that is definitely helping our employees and customers to be more productive out there. Agent side, again, a huge uplift because now they are getting these summarizations they're able to generate the knowledge articles, they are able to get context of information in their workflow, which is definitely going to drive productivity. And last but not the least within the developer community within ServiceNow, our DD organization, we can see that the text to code, code explain and code commenting where it is going to just boost the developer productivity out there. So there are a lot of different capabilities, a lot of different use cases within ServiceNow. We have deployed already about 12 use cases related to generative AI for all the different personas. And now what I'm going to do is hand it off to Tricia, and Tricia is going to walk you through actually a demo, which will show you how it is -- how generative AI is helping from an employee experience perspective. So over to you, Tricia.

Thank you, Rajeev. Let me share my memo with you today. Let me share my screen. As Rajeev was saying, when we're talking about using generative AI within the platform, we're not just talking about this 1 technology. We're bringing it together to drive higher engagement with the technology and the platform. And so here, in this first example, I'm going to show you how we just improve the employee experience by streamlining search and making a better experience for David. So in my first example, I want in as my change manager to David. And he just has a quick question around continuing reimbursement, reverse it for continuing education. We're looking at our knowledge base within our platform to identify the policies and procedures that are relevant to us. And so not only are we circulating knowledge-based articles where David has to continue to research and determine what the answer is, we can use generative AI to identify that. Yes, in fact, the company does offer reimbursement and some of the salient details around that are the maximum amount within a year for job-related courses. Now as we scroll down our knowledge search results, we can see that this answer was extracted from this article. And so we can identify if the selection was helpful. We can drill in and see is there anything more that I need to know about this policy. And it's like they have just a quick answer to say, yes. In addition to that, in streamlining the process for David, I'm going to initiate just a notification to say, "Hey, the virtual agent chat bot can help facilitate perhaps requesting that reimbursement. And so I am going to request reimbursement, and now we're taking advantage of other capabilities within the platform, which is just retrieving information from our service catalog. Now we're only reducing a couple of clicks. David could have gone back and searched for the tuition reimbursement requests from the catalog and launched the same catalog from here. But in increasing his experience -- he's going to come back and do more with the virtual agent, he's going to search more and more personalized experience. He's going to complete his request for his tuition reimbursement. He's going to add his attachment, click submit. He is done. So that's our first example. In our next example, is a little more in depth. So any barn is...

Tricia, I'm sorry, a couple of people are just saying they're not seeing what you're sharing. I just want to make sure we're sharing the correct window.

I'm sorry. Sorry about that. [Technical Difficulty] Great. I'm going to go ahead and start that one now. So let's just go through this quick example where I'm showing you an example of the portal. David's going to ask for any policies around continuing education and reimbursement. And as you can see, we have a summarized answer that provides the information in -- a response will be affirmative, yes, the company does offer tuition reimbursement policy. But I view the article, I'm also getting the trigger that alerts David to the fact that the virtual agent chat bot can help him compete to request. It's contextual, it's personalized and in this way for streamlining the process. He's going to request the tuition reimbursement for his agile development class. And he is going to complete the process here. And we're just taking advantage of platform capabilities at this point. We're completing a catalog request for his agile development course. At this point, perhaps we can move to the next example, and the next here where we can catch up with Amy and go through a first scenario, which is a little bit more in depth. Can we move to the next review? Or can I?

It'll come up in just a moment.

Thank you so much for your assistance. In this example, Amy is going to ask for information around parental leave and Family Medical Leave Act. So as she is exploring her opportunities, Amy has some additional considerations to continue. And this big life event is more than just a single Q&A. We're actually going to take advantage of this opportunity to kind of diaper not only in answering this question, but through the entire process. So she is also going to get a personalized response. In addition, we're actually going to take advantage of this opportunities to direct her to the microsite. In improving her overall experience and providing her a journey for monitoring her process as she prepares for her adoption event, as she gets ready for taking leave from the company, we want to make sure we're with her every step of the way. So right now, she is completing this process. And this is -- we've guided her to this microsite using a summarized response. She is going to complete the request for her family leave. Now it's not necessarily perhaps what she might consider standard. She is a newer employee, but she is identifying the timing is fluid because the final adoption confirmation is not yet confirmed, so she's identifying some time periods, she is submitting it. And now she can come back to this microsite and complete the information. The challenge that Amy is struggling with in this scenario is not only issue requesting future leave, but she has a current situation that is also in the back of our mind. So she knows she can come back here as she continues through the process preparing for her adoption event, but she has some additional questions that she wants to address. She's going to go to the virtual agent and ask some additional questions. But our knowledge base doesn't quite have answers around multiple leaves of absence. So she's going to address those here to going to ask, does she have to take for unused vacation days. And in the virtual agent, we're going to give her a summarized response. And so just as quickly as you can view this from the portal, she can view this from her -- from the chat bot. She can continue to ask questions. But at this point, we actually want to the prove source of information, so she can review the information that she needs Thereby, we're also concerned about hallucinations and incorrect information. But at this point, we want to go from information to taking advantage of the sources of data within the platform. So we've answered her questions. One of her questions actually is how much vacation time does she have left. This is where we can use our integration to Workday and surface up-to-date information as of the last day period that she has approximately 13 days accrued so far. That's not quite enough time for her. So she would like to speak to an agent. And when she's speaking to an agent, what we want to provide is the context up to this point. Now when we hand off from Amy to Roger, she is going to see a summarized transfer for what's happening. And so Roger having to surface through the entire transcript of Amy's conversation with the chat bot, we're actually going to use generative AI to provide a summarization of what's going on. And so he can answer his question more -- he can answer Amy's questions more quickly than previously, and that's just a short conversation. Sometimes employees have lots of questions. And so having summarization at work within the interaction is very helpful. Let's move to the third video if we can, and talk about Roger's experience. So Roger picked out Amy's interaction. He saw the summarized transcript, and he's able to complete that chat, but that's not the only thing he's working on. Here, we're looking at the agent workspace, and we're drilling into cases. And this is where we can take advantage of summarization not just at the interaction level, but at the case level. So as we can see, he's reviewing a payroll discrepancy case for David. If you summarize, is it instead of having to review all the activity and all the transcripts, a summary is going to help him get up to speed that much quicker. Using Now Assist, he can quickly collaborate and facilitate resolution with David as quickly as necessary. It's just a time-saving opportunity. Perhaps we can move to the fourth video. And in this fourth video, so our HR experience isn't just limited to what we can do in the HR agent workspace. Our colleagues that are in the service desk have an additional step that's required when they resolve incidents. And so here is an example where Beth is working with Mary on resolving the issue of expense management within Concur, that has an additional required step that when incidents are completed, she has to summarize the activity in the resolution. So this is a standard process. This is a big timesaver. Being able to summarize and identify within the resolution nodes exactly what happened, provides us a greater level of detail, which allows us for additional analysis downstream on resolutions for these issues. If we think about generating content. In the fifth video, we're going to go back to Roger. And what I'd like to show you is what 1 of our developers also created and in this video, Roger is also responsible for creating knowledge-based articles. And as you can see in these knowledge-based articles, he's just putting out content. Well, we're at the end of the year, it's September already. And it's time to start thinking about year-end checklist. If we think about populating just the knowledge-based article, there are some fields to fill out. Sometimes, I think all of us can relate. It's a challenge just to get started. Well, we're going to use generative AI to help us generate some content. And in this example, I'm going to show you how we can use a virtual agent conversation with generative AI through Teams. And so this particular virtual agent conversation is only available to Roger. They're not going to allow all of our users to create a topic for a knowledge-based article. But here for Roger, he's going to identify that he wants to start a knowledge-based article for year-end reminders for HR compliance. And so he's going to put it out there. And people who like generative AI do the research. This is how we're going to generate content and use that as the basis for a knowledge article that other employees can take advantage of. Now, we're going to go through this process pretty quickly. And you can see that he's identifying in this virtual agent topic, what he wants the topic to be and then, of course, what he wants this titled to be. We're going to take advantage of this integration given the moment to take a breath and show him the knowledge-based article that was created. We're creating it within the framework of our existing knowledge-based process. It's in draft mode. It does need to be edited, needs to be formatted, something to be reviewed and approved. But the biggest step, the research step, that's the biggest challenge. And we can see that it's a huge timesaver. And in this early release just being able to relieve Roger of the thinking, the energy towards putting it together, we all know the feeling of once you get started, it's a lot easier to edit or move or format and get it done. And so in this final video, what we can do is, I'd like to think about those unsung heroes and those developers. And in this last video, our developer who's made all of this possible, Elsa Norton. She is focused on an employee recognition program. And when she's thinking about all of the tools available to her, she is a company developer. She just doesn't necessarily have all the rules in place at the top of their mind. Here, we can take advantage of text to code. And we can use this upper text where we identify a comment within our scripting window. We use the text to identify that Command and Enter. Based on our comments, text to code will deliver a suggestion and it will identify a perhaps smarter way to do it. As you can see, not only did it identify that it's going to display an air message but identify that the process will end. If I, as the developer, agree with the coding changes, I can tab through and make my edit. Now this is not automated and final as delivered. We still need Elsa to update the variables. They're too genericized. We need Elsa to review the code for accuracy. But if she doesn't have to remember text formatting or all the different client record requirements, that's a huge timesaver. So these videos were just a couple of examples that we wanted to keep top of mind as we think about generative AI in the platform. It's obviously a rockstar capability in platform full of rockstars. And as Lamar described it is the connective tissue between our platform capabilities. Now I'd like to hand back to my colleague, Nick McClure, where we can talk about how skilled intelligence is made better with generative AI. Nick?

Hello. Thank you. Yes. So thank you for the great demos. I'd like to talk about the number -- the big questions we get when we talk about skills intelligence and how it's useful in the workforce and across the company. And I'll start with what does skills intelligence involve? Why do we call it that? What are we doing with it? So skills intelligence, really the whole basis of skills intelligence is that it involves inspecting both on one end, internal company data. And then at the same time, external industry data and the intersection of those 2 allows us to get actionable insights that drive a skills and data-first approach to hiring growth, employee engagement, retention, a lot of areas across the workforce. This intelligence is crucial to test such as identifying skill gaps, talent needs, emerging trends, all of which help companies and decision-makers make those informed decisions to take steps to stay competitive in the industry. So why is it needed? Companies and customers come to us. They tend to have different levels of skills intelligence already, which you'll see here on this slide. And they increase in complexity from left to right. So on the very left, customers may just have a list of skills. And that list of skills may come from using technologies that we have in skills intelligence across their, say, history of job descriptions that we connect to an export of an ATS system. But really, the intelligence here is really just the ability to alphabetize skills and search them from a list. It's not exactly the most helpful. And then in the middle, we move into what's called the skill ontology in which we know the relationships between skills. For example, you shouldn't have to know -- we know that ReactJS is related to JavaScript, but less related to Java. And this goes across thousands of skills and thousands of titles and the industry data that we have. But it's more than the skills, of course. So on the right, we start to connect multiple different types of ontologies together. So we start in the middle with skills. And then we move on the right and we add in -- in this example, let's say, job titles. We connect the business and medical jobs with skills from system administration information management. And then there's more than just titles, of course. Our ontology that we have in the industry includes certifications, industrial areas, location, it includes a whole suite of things that we're continuingly adding to and improving. To -- we provide this intelligence and additional algorithms to make these knowledge graphs, which is a collection of ontology is useful. As a concrete example, if a manager is searching for employees with website development sales, the manager shouldn't have to list dozens of different JavaScript libraries. We know that those relationships exist. All right. So I think it's useful to show how applicable skills intelligence is across a company. And to that end, here are examples of using skills intelligence, at the employee manager and executive level. For the employees, some concrete examples are, can I use my skills on a finance department project? Who can mention me in Learning C++? How should I learn more about Internet of Things in health care? Or how -- just in general, how can my current skills and objectives that are uniquely mine best contribute towards the overall company goals. So these are questions that employees have. For the managers, some very specific examples that I like to use is I have an idea for our website, who can help me build an MVP? How is my team progressing in their skills? What unused skills do team members have that are in high company demand or high industry demand? And then for the company executives, some specific examples could be what new technical skills are my competitors hiring for? Or we need to expand our semiconductor manufacturing skills, do we train or hire externally? What are my company skill gaps within our industry. And so these are all questions that we really, really aim to provide the intelligence at sitting at the intersection of internal data and external data. So hopefully, I've shown you what skills intelligence is about and its potentials and what it can do. I do want to show you why it's necessary and what it can do and how ServiceNow is set up for success. So first, so first, let's consider the problem of hallucination. So we have customers come to us with 3 general problems. So small, fixed, unscalable data. So they have a list of skills. They have maybe some metadata attached to it. And then using skills intelligence, they can say, the problem that they have is that to address these problems is that we have at skill -- ServiceNow, the skills intelligence has existing and expansive industry data to help augment it. And with generative AI, we can help navigate this data at the intersection of external and internal data. As an example, you may say, how many employees have aerospace skills and where are they? And then with our data sets, our ontology and algorithms, we can help you answer that. Customers may also have a kind of a different problem. They may say, "Well, I already have an LXP and ATS in HRS system, so I have all of these large unwieldy data sets. Each one of those may have skills on even ontologies in them. How do I get them to talk to each other? I have 20 different ways to write Microsoft Office. And so what we do is we provide the machine learning algorithms to be duplicate and merge them. So we have algorithms that say, hey, Microsoft Office V2016 is the same thing as Microsoft Office Security Patch 2, et cetera. So we have algorithms right now that we have built that can merge this. And so with generative AI, we can -- as our ontology is continually updating, we can provide those summaries. We find hundred new skills, external skills, then instances can be notified instead of going through 100 skills to approve or say no to or yes to, we can provide data update summaries with generative AI. Outdated and old data sets are an issue we put in -- customers have put in a lot of effort to inventory skills and employees. But it's really hard to keep that updated. The amount of -- it's a full-time job. Skill strategies don't end. So we are currently right now every on a monthly basis, getting millions and millions of data points externally, applying algorithms to extract this data and extract skills and titles and build new relationships and push them into the ServiceNow. So ServiceNow is set up for success in using generative AI and skills intelligence, and I want to show you why. So I think in -- when it comes to generative AI with the workforce, there's a lot of people out there just applying generative AI in general. And there's a lot of things that we need to talk about. There's a few problems when we talk about as an example to show that ServiceNow is set of process in taking on these challenges. So hallucination. So with generative AI, the results are prone to hallucinations, false information. And so with augmented with ServiceNow skills intelligence data, which we have this ontology that is updating continually, we can ensure that the results are relevant and real. So this is an example I did a few months ago. I went to a very popular large language model, and I said, show me the top certification data analytics managers, and it produced certifications that don't exist. So with our known certifications, we can do this with our algorithms with service -- with skills intelligence and say, these are the real ones that exist. Right. So the next problem, a big one that we try very hard to solve and make it easy, is dealing with unstructured data. For example, if a past generative AI, and I -- and I tell it, hey, what are -- and I ask you what are the skills in this resume, whether it's job description or learning content description, et cetera. It may or may not give me a list of skills. And if I'm trying to treat this with an API or an application, I have problems. And so we, on top of it, have built the proper -- using that ontology in the guardrails for a large language model, we can ensure that we're not -- it's not producing duplicates or typos, if someone has that in their resume, et cetera. So we are ensuring that this is in a very structured way. These results are in a very structured format. And this next problem may not seem relevant to a lot of people, but we see this problem a lot. So customers come to us and they say, "I have all of these skill lists, maybe from different projects or different sources and different job architecture data sets. And if you ask a generative AI model, just straight up, hey, here's 2 skills, marketing income or retail department, marketing income or retail packaging company, it might say, okay, and you ask it, please normalize this. It can -- it may or may not. So we -- this is the fundamental thing of skills intelligence if we try to make it very.

Okay, we are going to take 1 quick second. We're just going to see if we can get Nick back here. Please bear with us 1 second. [Technical Difficulty]

All right. Well, while we're doing that, I know we've had a couple of questions come in on the back end. But this might be a good opportunity for us to just answer a couple of questions then when we get Nick back, we'll go ahead and let him get back into the conversation here. Rajeev, are you available? Or are you still with us Rajeev?

Yes, I am.

Awesome, awesome. I'm going to ask you question to some of the questions that came in, I'll just start with one. What would you say are some of the key considerations for organizations that are looking to integrate gen AI into their existing workflows.

I think -- so Tricia did a great job of sharing some of the demos, some of the use cases and which is now in our platform, which we have deployed internally. The key part is our data. So when you're doing summarization, you can only summarize something which has data out there. And if the work notes you're summarizing, if they're written currently, if then are written in a good way, then you can get good summary out of it. Same thing on the KB generation also. You can only generate, it's all about the data. And then if you are looking at a search solution, which Tricia shared about it, which gave us search results out there, if your KBs are written well, then only you can get good content out at. So it's all about the data. That is what we learned very quickly. And again, as I said, AI or the generative AI or any other kind of AI, it's all about data. So I would say it's a #1 consideration.

Awesome. Nick, we have you back.

I am not sure when I lost connection.

Nick, you go ahead. Just pick up with the current slide, we'll just go ahead and just start from there. I believe that -- I think we should be okay.

Yes. Hopefully, we've talked a little bit about what skills intelligence is about? Why is it necessary? And what it can do? I really hope here on this slide, I can talk to you about how ServiceNow is set up for success with dealing with generative AI and dealing with those challenges. So hallucination is a big concern. So what are -- like if you ask just straight up generative AI, what are the top certifications for data analytics manager is. It can give you certifications that don't exist. And this is a real example from a few months ago. With skills intelligence, we know with skills intelligence that the knowledge graph and the skill and certification ontologies, we know what they are, and we can use Generative AI to build -- you can use this ontology build guardrails around the generative AI. Unstructured data is an issue. So many times, algorithms are required to have specific output structures. While there are more and more lately, research is building these guardrails around LLMs. There's still some difficulties with it without feeling like we're duct-taping solutions around LLM. We are actually solving this with ServiceNow, the models that we build to make sure that when you ask like what are the skills in a resume, we're not getting the typos that are in the resume as an example. So -- and another use case at the very end that happens a lot with customers. Customers come to us and say, "Hey, I have all of these sources of job architecture, skills data, et cetera, it's just really hard for us to normalize, whether it's skills or titles or job architectures, roles, et cetera. So we actually are able to take these and provide the actual standardization with our models that we build with degenerative AI. So we actually have this done now. So yes, I'd like to -- I'll show you examples of problems that we're tackling in the future that we want to with skills intelligence. So for example, career path. So the big problem that we see when we survey customers is that with retention, employees, 1 of the top 10. Actually top 3 things that employee says, there's no career growth. And then the employers will turn around and say, yes, there is. It's right here. And so what that means to me is that there's a lack of visibility. So it would be really nice to have these sort of generative AI skill ontology use cases that say, "Hey, if I'm as a manager and my teammate, I have 1 on 1 in 5 minutes, and I go to their employee portal. I can easily see you can say, a current summarization, real time, what they're doing in the past 6 months based on this data, the employees added these 3 skills in this area, et cetera. We can also go to employees. And based on their unique instead of just creating a predefined say, here senior to principal lead. We can actually say, "Hey, based on your learning goals and where you are, where you are meaning like your unique background you may consider these mentors with similar education experiences or workplaces. Yes. So consider these mentors in a company with some of the backgrounds. Learning content is a big thing that we're working on right now. Everyone in this summarization uses it says, Hey -- and it's more than just people right learning content descriptions and they're everywhere, but no one is writing unique path together. So if I select multiple learning content and I had it to my learning path. It would be great to say, hey, in this sequence of learning content, you will be learning because if I add Python and I add HTML web scraping, it says, that's different from Python and back-end work. So no one is doing that and we want to be doing that. Generative uses, so someone has a learning goal. We have a database of learning mentors, et cetera, we want to be able to provide custom recommendations for the employees that's real time, updated and unique to them. Knowledge is really important for us, obviously. So as a person with very little time signs it, not me, but someone like an executive or a higher up man like Rajeev signs in, and he says, I want to look at my whole department, et cetera. and we get these real-time statistics, you've seen these HR dashboards that have all these graphs of retention, hiring budget, it positions open, close, et cetera. Why not just have a real-time summary at the very top. A few bullet points that says what we're looking at. and then say, as we're adding new skills, the skills landscapes are changing so quickly. Wouldn't it be great to have to say like, as an admin to this instance, a new skill pops up. JavaScript libraries have crazy names. Wouldn't it be great to just automatically have a summary and say, hey, look -- and I know we're running out of time, so I'll move on, but "hey look, here's what the skill is." Anyway, I hope I've covered the web by how and finally where we're going. I'm super passionate about skills intelligence and the possibilities it can use for. Reach out to me if you have questions or type them in chat, I love this subject. Thank you, and I'll hand the presentation back to talk about critical skills in the generative AI journey and the generative AI journey at ServiceNow.

Thank you, Nick. And I can say you covered a large here, 2 parts to it. One is just overall how generative AI is -- can help you to provide the intelligence around skills to help your workforce. And then we know that generative AI needs to -- is going to impact our workforce. So our skills intelligence is perfectly positioned to look at that and help the -- into that journey for every employee in your organization out there. So when we -- let me quickly try to wrap it up with a couple of slides out here. So what are the critical sequels for the future people have asked, I've looked at some of the chat questions out there. We want to make sure that as you all, we are trying to make sure everybody is able to understand the AI is becoming more and more important. If you don't need to be an expert in machine learning and deep learning out there, but you need able to like how AI learn, talk about how it uses data to generate the output so you can build the trust, you can understand it, you can ask those questions. You don't need to be expert out there. But for certain people, you may need to learn about the program, how computer -- you need to program -- do the programming skill, Python, prompt engineering, everything, things like that. And then how do you ask the question to create new things like how you ask question to Jack if you need -- it's just that search is very different than generative AI. So you have to ask the question in a very different way to get the responses. I'm sure you all expect -- have experienced that. And lastly, I think I've talked about data at multiple points in this session today out there, you need to really prepare the data, normalize it, so then only you can get good output on that. But how did we get our generative AI journey started, people ask, at ServiceNow. So first of all, you just cannot say, "Hey, I'm going to use generative AI." But we have to focus on what are the problems you are trying to solve. We start with that. Think about bloating that out. We listed out like more than 90 use cases within ServiceNow in like a couple of days, we looked at it where generative AI can help with that. So once we had the list, obviously, we cannot solve everything. We tried something out. There are different live language models, technology, infrastructure, everything out there. How are you going to measure the metrics out there. Then we looked at it, how we're going to build the architecture for the needs -- for current and future needs and a governance model, everybody talk about ethics, governance and things like that. Everything had to be controlled and in managing that. And then driving enablement. We need to make sure we are educating everybody around how to build AI solutions because it's not only 1 team's job but everything out there. So in a summary, it's all about data. It's about -- critical about user experience. You want to make sure that it's part of the workflow. It just doesn't sit on the side that people have to click on it. You need to provide the transparency that the users can trust the output. And there are multiple large language models. We have our own large language models and now [indiscernible]. But we are also using multiple large language models, and you can definitely follow up with me that how we are managing and hybrid solution out there. With that key takeaways, develop those critical skills of the future, get started on your journey on the AI and then continuously learn and end up -- just you have to keep on experimenting and learning. And I'll hand it off to Lamar to wrap it up with Q&A session here, if you have any questions.

All right. Absolutely. Well, first of all, thank you, everyone, for all of our speakers for just bestowing your knowledge upon us with gen AI and just how we're leveraging that with ServiceNow. We just really appreciate your time. And I just want to just let the audience know, please feel free to reach out to us with any questions, comments. We would love to be -- there was a flu of questions so we couldn't get to everything today, but we're definitely available to answer more -- get more questions and more details around anything that you need. Just be sure there's a resource list. Please be sure to get that resource list, download some of the things there that will be helpful and learn a little bit more about gen AI. And also, there's a survey here at the end that we would love to have your feedback with regards to the webinar. So Also, just keep in mind, K24 for next year will be coming up. And I think there is an upcoming -- let's see here we're at the slide -- sorry about that. There is up K24, keep that in mind for next year as we're going towards that. We always are looking to have customers and just be able to personally interact with you guys as we move forward. So we always love to see our customers there and speak with them in person. And also with that being said, guys, thank you so much. Please feel free to check out our on-demand webinars that we have. I'm here at ServiceNow. We love to get you guys. I'm on there, take a look at some of the great content that we have available for you to learn more about ServiceNow and some of the great things that we do here. And with that, thank you so much. We really appreciate it, and we look forward to talking with you further. Okay. Bye-bye.