ServiceNow, Inc. (NOW) Earnings Call Transcript & Summary

Hello, and welcome, everyone. Thank you so much for joining today's webinar, get ready to upgrade to the Now Platform Utah Release. The safe harbor notice is here. Just to remind you that some of this presentation today may contain forward-looking statements and do not make purchasing decisions based on those forward-looking statements. My name is Shamile Aldossary. I support upgrades here at ServiceNow. I'm super lucky to be joined today by Adam Stout, Director of Outbound Product Management at ServiceNow. Want to say hi to everyone Adam, I think away -- we'll do -- it's great to have you here with us today, Adam. We have quite the agenda for you all. We will start by reviewing the ServiceNow upgrade policy and the value of staying current. We'll discuss key goals for your next upgrade, including ensuring a faster and more seamless upgrade with platform capabilities. Adam has a great demo on ATF for us. We have some best practices for creating a streamlined upgrade project plan and setting up your platform for future upgrade success. We'll also discuss leveraging the ServiceNow store, and we'll cap off with resources that you can take away with you today. And of course, we've saved time at the end for Q&A. So along the way, ask your questions, use the fun reactions tool and your tool bar and we'll be sure to get to those at the end. I'd love to start off a webinar with a poll. So if you could please let us know what release version are you currently on? Are you on Rome, San Diego, Tokyo? Maybe some of you have already upgraded to Utah. We just released early availability just last week. We'd love to see your feedback here. We'll give you all just a few minutes here to answer -- a few seconds I should say. [Voting]

All right. I think we've got most of our responses in. So it looks like most of us here today are on San Diego or Tokyo. We've got some folks who already upgraded to Utah through the early availability program. That's awesome. And we also got some folks here on Rome who I'm sure are due for an upgrade soon. So let's quickly review the ServiceNow upgrade policy. Customers need to be on 1 of the 2 most current releases to be on a supported version. So here support means access to world-class technical health, 24 hours a day, 7 days a week. This help also include support with patches and hotfixes. So this means that, as a customer, you'll need to upgrade at least once a year to stay current. So looking at this chart below our customers with N-1 entitlement on San Diego or earlier or customers with N-2 score entitlement on Rome or earlier, you are due for an upgrade to the Utah release next month. At ServiceNow, we encourage all of our customers to upgrade with each release so that they can access the latest in innovations and security enhancements. So that leads me to my next poll question. Are you planning to upgrade to the Utah release? Yes, maybe you upgrade with every release, or yes, you're planning to upgrade because you're due for an upgrade. Maybe you're not so sure, and that's why you've joined us here today or you know you're not due for an upgrade in March or September. [Voting]

Give you some time there to answer. Thanks again for this feedback. This is really helpful, so we can level set with you all today. It looks like almost everyone got a chance to respond. All right. It looks like most of us here today are planning to upgrade. Some of us are unsure, or no, you're not due for an upgrade, but you're just here to learn what's new and get the latest upgrade best practices. If you're unsure about your support entitlement or what release version you're on, you can find this all in the Now Support portal or by reaching out to your solutions consultant. So as I mentioned, customers get the best out of ServiceNow on the upgrade to the latest release. Not only will you receive the latest platform and security enhancements, you'll also be able to access the new products, applications and features that come with the new release and Utah has some really exciting stuff coming out. Customers who upgrade regularly will also be able to use a supported version of the software. So this means in addition to 24/7 support, you'll also get help with specific upgrade patches and hotfixes. And I want to say this is my last poll question, but I might have another one after this. Thank you in advance for your feedback. When are you planning to perform your next upgrade in the next 2 months, the next 2 to 4 months, 4 to 6 months? Maybe you've already -- maybe you're in the process now of upgrading to Utah through the early availability program? Let us know here in the poll. And I love these reactions that folks are using. If you're looking for that, that should be in your toolbox. [Voting]

Perfect. Okay. All right. So it looks like most of us are planning to upgrade in the next 2 to 4 months. Some of us are already upgrading to Utah through the early availability program. Awesome. Thank you for your feedback and I love that. I love all these reactions, keep them coming. So when it comes to your next upgrade, what goals do you have for your organization? We've talked to some of our customers, and this is -- these are the goals that we've been hearing from them. First, you might want to create a streamlined and efficient upgrade plan that can be repeated for future upgrades. You might want to minimize disruptions to your organization during an upgrade and speed the overall upgrade time. Or maybe you want to ensure accurate and efficient testing before, during and after your upgrade. Let us know here in the poll, which goal is most important to you. If you're like me, you want to say all of them, but let us know which one is a higher priority for you with this next upgrade. This might be your trickiest poll question, but I'll just check and it's our last one. Very curious to know what the responses are. Thank you, folks, for your feedback here. All right. You got 10 more seconds . [Voting]

All right. Okay. Very interesting. So it looks like the upgrade goal that is most important to most of our audience today is to minimize disruptions to an organization with a faster, more seamless upgrade. That's fantastic. We'll be covering all of these goals today and providing some best practices and pro tips. So let's keep moving. So let's discuss how you can create a streamlined project plan that can be repeated for future upgrades. So as you approach your ServiceNow upgrades, you would want to apply smart project management concepts, including securing sponsorship, identifying stakeholders, assessing risks, making time management, estimates and allocating resources. You can use a project management application to track and communicate your progress. I will suggest the ServiceNow Project Portfolio Management Suite, which is included with ServiceNow SPM. That's a great choice, but other tools will also get the job done if you have not purchased SPM. We share to follow the documented process on product documentation. There's a link there in your resources for today. That's a great place to start. And we've had our customers tell us that they've underestimated the resources required to do an effective upgrade. But upon further investigation, we found we didn't necessarily need more resources but they needed people in different or additional roles. So take a look at these roles on the right of your screen here, depending on your staffing model, these could be the same person. And I know some of you on our call today might fit into 1 or multiple of these roles. And we've also seen that our customers who use change management process, deliver high-quality results. So if you don't already have one, you can create one using the Change Management Application, which we have, of course, included that resource to learn more about in your resources box for today's webinar. Communication is a key part of any project and upgrading is no exception. Be sure that you are notifying your ServiceNow stakeholders of the upgrade process, schedule and impact in advance, such as change records and e-mail notifications, notify your biggest audience, the end-users, let them know what to expect from the release and what they can look forward to. We have some sample e-mails that are really great in the upgrade success workbook, which again, is linked in the resources for today. Some of our customers get really creative about how they communicate with their end-users, and we love to hear about this. So if you have any great ideas, put them in the Q&A, let us know on our community. We love to hear about these. Some great things that I've heard is video commercials on a TV monitor in the break room or on some sort of company social site. We've seen upgrade parties. I think the Utah party sounds really fun. We've seen signs posted in high-traffic areas. And my personal favorite is I love a Zoom or virtual meeting background to celebrate the new release. So like I said, we'd love to hear about your creative ideas that you get excited about a release. So do let us know in the Q&A or again, on our upgrade and releases community. So once you've laid out your plan, resources, notified your stakeholders, it's time to get moving and grooving. Be sure that you've read through all the release notes for your target version. So in this case, Tokyo, and then compare them against the release notes for the version that you're currently on. Another pro tip is to use a sandbox instance or a demo instance to further reduce your risk. A sandbox instance is a clone of your production instance that is not a part of the deployment stack. So consider designating a sandbox instance to upgrade first. It reduces your risk because it lets you test with the production data without impacting your development cycle. If you don't already have a sandbox instance, you can purchase one as a part of your company's family of instances or you can use the free personal developer instance from ServiceNow to learn about the new release features. Keep in mind that when you use the free developer instance, you can't clone your production instance, but it's still extremely valuable. So we've included links to both the sandbox instance and the free personal developer instance that you can learn more. You're going to hear me say that a lot. We've got a lot of great legs in our resources today, be sure to check those out, which kind of brings me to my next point. If you haven't already heard about the Now Create Upgrade Methodology. I absolutely suggest that you take a look at this. This gives you step-by-step guidance for each step of the upgrade plan. So if you already have a project plan in place, I would just suggest cross-referencing it within upgrade methodology or perhaps starting here when you're building out your upgrade project plan. All right. And now I will pass it on to Adam. Adam, are you there? I'm not sure if we can hear you. Technical difficulties. Thanks for bearing with us, folks. And all right, we might need a minute or so here...

Can you hear me now?

Yes, we can hear you, Adam.

Okay. Sorry about that. So let's continue on. So as we're going through staying up-to-date, one of the key things we want to talk about is using the store and how we manage our upgrades through the store. If you having gone through it, I know we've done a session in the past about really getting into the details here. But one of the things that we'll cover is that you can go into the store, you can search for all the apps that are being updated and whether that is partner apps or ServiceNow apps, whether they're part of the things you're subscribing to or they're free utilities that come out, you can find those and search those in the store and you can subscribe. So I recommend everybody is going through and finally looking -- searching for things that are interesting to you, the areas, if you have CSM or ITSM or HR, and you can subscribe and then you're going to get e-mail when there is an update. And we release major functionality once a quarter into the store. Patches come out every month and partner apps are released and updated weekly. So it's important to stay there to see what you can upgrade. And what that allows us to do is a little bit different than traditional plug-ins that we have with ServiceNow. Our plug-ins are upgraded with family releases and patches. That's what we're all used to in the big upgrades. We upgrade everything or nothing. With our apps, we're able to upgrade -- we're able to cherry pick what we want to have upgraded. So in some cases, we released a new store app. We just released all the store apps in February for Q1. If it's backward compatible to Tokyo or San Diego, you can go in and you can say, "I want to upgrade this now, even though I'm not going to upgrade my production instance to Utah until March or April." I can take those store apps and upgrade them now once they're available. And the inverse is true as well. I might upgrade to Utah and then come back for an app later on, I can -- whether I'm adding it in or upgrading it later, but it gives you more flexibility. In some organizations, we want to do everything at one time and go through a very rigorous testing, and we only want to do that once. And in other cases, we want to spread it out a little bit because I can upgrade this 1 app that's used by this 1 team this month, while I'm doing my full testing for Utah to upgrade in March. Again, you've got control, stay up to date with the subscriptions. You can search and you can look in the store when you want to, but those subscriptions are really helpful to get that e-mail and give you that incentive to go, "Hey, I'm really interested in this, let's go see what's available." And again, the major functionality comes out from ServiceNow once a quarter with a few exceptions and then patches come out each month, and you're able to pick those up when you want to. So we want to make sure you have it in more -- as most control as possible. What you will see with our applications as well, more is moving from the family release to the store so that you can then, again, decide what you want to do and upgrade how you want to. So even things that were plug-ins originally are moving to the store so that updates, you'll be able to see more and more -- everything new we add is going to basically be in the store as long as we can. But again, make sure you're using the store, you're thinking about the store. In the App Manager, you'll see what updates are available, lots of great stuff for you to leverage. All right. So now we're going to talk into a little bit more, how do we do our upgrade and how do we minimize the disruptions that might happen? First thing that we want to talk about is ATF. And with ATF, I hope everybody is already using ATF. There's a lot of content to help you out -- learn how to use ATF and we're going to do a little bit demo at the end -- I'm sorry, in a few moments. With ATF, I always get the question of how do we get started and the way you get started with ATF is by getting started. This allows us to automate our regression tests, automate our testing. So rather than having to rely on a human to go do all of these test steps that we want to go through, I can record those, input those into an automated test and then I can run that -- put that test in suite, and I can run those automatically, saving lots of time. What this is going to allow you to do is identify differences in behavior, before and after upgrade. We are talking about upgrades, we're talking a lot about upgrading to Utah, but this is true whether I'm going to a major family release or doing a patch release, I'm deploying an update set and deploying a new app, I can run all my automated test before and after and whether it's my full suite of everything for a major upgrade, for a family upgrade or it's just my test suite for this 1 app, I'm going to identify any differences in behavior that exists. Those differences, and I do say differences of behavior are not always passes and fails and it does say pass and fail in the test. But it's saying it's different. Then it will take a human to go through, is that a difference I want or it's an unexpected difference I don't want and whether I need to fix the code, it's an issue with the code I'm deploying. It's an issue with some incompatibility with my upgrade or no, it's right now, it was not exactly what I wanted before. It's better. It's different, but it's better. ATF will bring those all to light, so you can spend your time thinking, spending your time fixing, but not spend a bunch of time just clicking on things to find out if something is not working like you'd expect it to be. So why are we using ATF? We talked about this a little bit, and there's a few different areas that are the main use cases for ATF. So the first one is functional testing of business logic. My business rules firing. When I submit my service catalog item, is it routing for the right approvals. When I approve it, is it delivering it appropriately? Testing that business logically what's important. One of the main use cases for ATF, we do really want to focus on that. It's -- you don't have to test the [ ServiceNow Work, we test of Service Networks. ] If a dependent choice list pulls up the right list, if my reference qualifier works, we test those things. But as my business logic work, particularly what you're customizing and where you're configuring, ATF is really going to help you make sure that you're getting the same thing before and after an upgrade. The second is browser compatibility testing. This is something we have to deal with. In your organization, there's probably 1 or 2 browsers that your IT department supports. We want to make sure that they work with those. They work with the plug-ins that we -- with the extensions that our company uses or organization uses. Browser compatibility testing is going to let us run through all of our tests to make sure it works on your browsers and it does. These tests run on your computer with your browsers, with your extensions. We want to make sure all of that works, ATF's going to let you do that very quickly run through all that. And then the bigger area we're going to talk about is regression testing, some new stuff in here. We'll have the demo in regression testing, which is just did it work the same before or after. Just simply we're not saying if it's right or wrong, We're just saying this is what it looked like before, this is what it looked like after. I saw that more than half of us or around half of us were on Tokyo already. That's great. We're going to talk about the ATF test generator in a moment. That requires Tokyo. So it's going to be really useful to go from Tokyo to Utah or Tokyo to the next patch. For those of you on San Diego, once you get to Tokyo or Utah, you'll be able to use the test generator going forward. Couple of keys. These are just my tips that I always like to talk about with ATF. So I want to make a plan. Shamile talked about having the plan, the thorough plan about what we're going to do, really, really important as we talk about ATF and our testing is it's not just run the test, that's not enough of a plan. We want to understand how detailed do we want to be, what do we want to test, what's key to us. I want to understand and have a plan for how it's going to -- how my tests are going to be maintained. We are going to find some differences in behavior. I want to have a plan to know what am I going to do about it. If my catalog item didn't route for the right approvals, what didn't route for the same approvals it did before, who am I going to contact who's going to be able to say, "Yes, that's right now or no that's wrong, let's go fix it." You want to make sure you know who owns your tests, who owns the areas that you're testing and who's going to help resolve those issues quickly. We might -- we find a bunch of errors, but it takes us 3 weeks to figure out, are they good or bad? Did we really save any time? Less clicks, but not enough time. But if we have that plan ahead of time, things can fly through very quickly. We want to make sure we understand what we're testing, what areas we know we're going to test, what we're not going to test. We want to make sure we know what functionality is tested across multiple test cases. Who's going to own that? If I have to have a -- if I have an interaction between problem and incident and escalations, when I escalate an incident automatically [indiscernible] to the problem, so that could be my team that owns problem, my problem managers or my incident managers. Yes, just let's think about it and have a plan for that so that we know what to do and execute when we do find any issues. When you're writing tests, keeping it simple is key. If you have a big, long test, this results in this through -- kind of the same issues we talked about before, if I have huge mammoth test cases and something doesn't go right, I have to spend a lot of time figuring out where did it break. So we want to have really simple, really short test cases that will tell me this is exactly where the problem is. And then test execution. When we're in ATF and there's -- as you go through some of the training, you'll see this, we can run browser-based testing, and we can have server-side testing. It's important to know what you're testing where. If I'm doing browser compatibility testing, I'm going to run all client side. That's what I'm testing. But if I'm testing business logic and business rules and flows and I'm doing more service side, then I want to be more server side. Server side is faster for us. So if we can have to be on the server side, we don't have to worry about any network traffic, network delays anything that goes on. If we can be server -- we want to be server side. And test execution time does matter. If I'm running 1 test and it takes 20 seconds, it doesn't matter. But if I'm running 1,000 tests, there's a difference between them taking 2 seconds and taking 20 seconds. Am I going to run it during lunch or am I going to run it overnight? It's just something to be conscious of about how I'm building my test. And then really key for ATF. We want to make sure we're testing for the outcomes, not just the configuration. Did it get the right approval, was the service delivered? I don't need to test. There is every form field work. That's a lot of time to build those tests, and then it still doesn't really matter, right? Do I get the right outcome? So make sure we understand how we can write a test as simple as possible to test the outcomes that we're testing efficiently and managing our time using as many server steps as we can and using client when we need to and all encompassed by having a plan about knowing what are we going to test and what are we going to do if we find some differences. We run test just quickly with ATF. ATF has built up a series of test steps to go through, and we can pass data from step A to Step B. So I'll create a user. I'll pass that user to the next step who is going to get a role. And then once that user has a role, that user is going to update an incident. All the data can be cascaded, tested by test step, a great feature of ATF is that once we have finished the test, we can roll back everything. We will roll back everything that happened in that test, so I can run the test again. It means that, on my test systems, I'm not -- I don't have a lots of sludge, I'm not worried about old test runs leaving different things behind that might give me false positives or false negatives. Inside of a test, we perform all the actions and we roll them back. That includes creating users. I do recommend for the majority of the time that we create users, so we're sure that we're not using Adam and Adam change departments. And now that test is invalid. He has different roles. But if I create a user or give it the right roles, I can always make sure that I'm really isolated. It's very simple and very isolated, so I know where my problem lies. Not that a user relied on change roles. One note here is that while data can be passed from inside of a test from test step to test step because we do that rollback, data cannot be passed from test-to-test or inside a test suite. As you go through and you build those, this will make more sense. But again, that rollback that happens really, really important, keeps -- allows us to run the same test multiple times in a day without any issues. And if we're creating those users, we can do more in parallel, right? We're creating the artifacts we need. We create a user, we assign the role, we create the incident, we create a different user to see if they can update the incident by creating all of that together inside of my test, we're able to run more things in parallel, get the results faster. And implementing tests, a couple of things that we already came through. Don't be afraid to use custom-step configurations. I use these all the time. We have all the basic components about inserting records or creating tests. But as you are creating your test suites, you might encompass some of this logic into a custom-step configuration, which will let you just say, "I want to do all my setup in 1-step configuration, so I can reuse that test step in multiple tests without having to build the same 6 steps every time." I don't want to recreate 6 steps and recreate 6 steps and recreate 6 steps. As I'm building out my test, I'll create 1 test step, 1 custom-step configuration, which will do all my setup, which makes my test and really easy to configure. We like to create users and impersonate as the first step. It's really common to make sure that we are testing with the right roles, the right people can do the right things, not just admin. Another noting here about the validation and updates. Updates always work. We want to make sure that it actually updated with what we want, all the cascaded business rules, all that business logic is actually what we want, right? Not that I updated a record and it didn't fail but then I updated it and then I actually get all the results that I wanted, while we're running all of the business rules that happen there, right? Again, we're testing that business logic. And there are certain cases where we're going to be querying for data. We're going to use a record query to get a variable that we want to pull in to pull some record sampling. There's a lot of theory that we can go into about how you want to test, but ATF will support lots of those things, lots of different types of tests, so make sure you just understand what testing methodology you're going to do, what are you going to go with, what are we trying to test, how we're trying to test it, all goes back to that plan. We have a solid plan. ATF will help support to make that happen. All right. So let's get to some of the tooling that comes through. So those of you who have upgraded to Tokyo, I hope you've gone and got the test generator and cloud runner. Awesome stuff that came out in November. I believe it needs Tokyo Patch 5 and then -- and Utah will be fine as well. If you're on San Diego, you're not going to be able to use this yet until you upgrade to Tokyo. With test generation and cloud runners, 2 big pieces of functionality that we put together in 1 app for you to install. And you get this in the store, as is noted here. We have AI-powered test -- regression test generation. What this does is we're able to point to a table or ever -- all the tables and say, generate my regression test. We're going to do this in just a second. It's amazing what it is able to do, but it will go in. It's going to look at the data in your tables. So this is a regression test. If you have a new app, we can't generate test because there's no data. But what it's going to do for the things that I have data in, it's going to go, look through data for the last few months. It's going to look at the common permutations, what are the most common things that happen, and then it's going to create tests that make those things happen. If Shamile submitted an incident, it's going to make sure that Shamile's -- impersonate Shamile and submit another incident and get the same state with the same categories and everything that goes through. It's going to look at your data and generate regression tests. It is not saying that they're right or wrong. It is just fingerprinting. This is what happens today. It'll create those tests, then we'll do our upgrade, and then we'll run those same tests again and verify what it did before, is what it does now. And again, as we go back, it's not good or bad. It's just differences. Hopefully, very little is different. And the difference is that we are there are -- it's now doing the right thing. It's doing what we want. It's doing it better. Great. And if not, we can figure out why it's not working the way we expect it to and fix that. But this can save weeks and weeks of time from somebody having to create those tests. And what's great is since I just press a button to create these tests, I can fingerprint my test whenever I want to. I can create a new regression suite whenever I want to. This is in the store. This is part of the platform. This is -- I want everybody on this call to be using test generation and we'll get power under test generation. It's free to use. You can generate new tests. You generate them, you do your upgrade. And 3 months later, I'm going to do another upgrade. We'll just delete your old suite. It's the old fingerprint, not the new one. We're going to generate our test again with more current data and then we'll do our upgrading come through. It takes a little bit of time to generate it, but let the computer do the work, right? We can generate those tests, and we can fingerprint whenever we want to. Next bullet we have in here is Cloud Runner. With Cloud Runner, we have hosted headless testing. What that means is that it can run overnight. In the past with ATF, we would need to open up our computer to have a cloud -- to have a client runner. So we'd have to open it up and then the test would run from my computer. That is really important if we're doing browser-based testing, we're doing browser compatibility testing because I wanted to run on my browser in my network with my extensions. That's what I'm testing. So that's still there. But for the business-logic testing, the functional testing, I can let it run overnight from ServiceNow's hosted headless testing. You could do this before as of Rome, if you set up your own system to do it, but now we're just going to make that easier for you. And again, it's just going to reduce the amount of time developers need to create tests. It's going to reduce the amount of time we need to make sure that our apps are at the quality we want them to be and that they're stable, that they perform the same way before and after an upgrade. And all we're going to do is press a couple of buttons, awesome. And this is my slide that says I'm going to go, do my demo, so let us switch over and go to the demo. Okay. And I'm just going to look for one of my co-presenters to tell me that they can see my screen before we go on. All right. Great. Thank you. I just want to make sure we don't get too far ahead. So right now, we just -- we have an instance. This I think is a Tokyo instance. So we have our instance. I brought up ATF. I brought a QuickStart test. We didn't talk about these a lot, so I don't forget them. QuickStart tests are the first place you really want to start. If I'm testing any out-of-the-box behavior across the platform, we have QuickStart tests. There are 600 or 700 QuickStart tests available for you today, and they're going to test the out-of-the-box behavior, which is great. We can still test that. What's really meant for is for you to take your -- take one of the QuickStart tests, copy it, you actually cannot modify the out-of-the-box QuickStart tests. We will copy it, make adjustments for any things that you've done, whether it's a [indiscernible] field, we modified our business rule, we have a new flow that comes over. This is going to test the out-of-the-box behavior. And as you make your configuration changes and your implementation, you can make your ATF changes as well. More these get released -- more and more QuickStart tests get released every release with your code. So make sure you're starting here with ATF to really understand where do we start, is it doing what we expected to do, you copy it, you modify it. All right. We're going to be doing test generation in Cloud Runner. So the first thing we're going to do, I'm at ATF, I am going to scroll down to select cloud user. When selecting my cloud user, assuming I didn't time out -- perfect. What this is going to do is I'm going to select a user that my Cloud Runner is going to run us. This is for test generation specifically. In test generation, we're going to go, interrogate your data so that we can generate tests based on your data. We have to have a user who can read your data. So generally, this is going to be sys admin or really we recommend creating a user that has admin access to access the data. This, you may end up changing when you're creating different tests, if we have some really protected data that admin can't see. If I have an HR user, I might have to change this so that my HR admin can read if I'm generating my admin or my HR tests. Generally, you're going in -- you're going to create a user, give an admin, I'm just using sys admin for here, it's fine. I have created that by... All right. So if I pick another user, I could submit to come through. So we'll go ahead and do that. It's going to authenticate my user. Hopefully, this will work. Again, it's going to go to our cloud servers and make sure that it can come in. You'll notice I don't put a password in here. All the authentication that's going on between the -- between the Cloud Runners for testing and the test generation is done with password-less certificate-based authentication. On some subproducts, it's not set up initially. So if you have an issue, just put a case in so we can get it fixed. On your -- some demos, this is the work. On your subprod, this should be fine. I am expecting to run this on subprods, not on production. I may want to install Cloud Runner and test generation on production just so it clones down every time. If I don't do that, then I'm going to have to reinstall my subprods. I'm never going to run this in production. I'm just -- I'm not going to run this in production. So if I'm going to come through, I'm going to change this back to system -- to my system admin because I do need -- again, you need to have a user who can access it. It's going to authenticate as we go through. This is pretty much the only set up we need to do. This is the only pre-setup for my implementation for test generation and Cloud Runner. I did go ahead and install it. So again, once we're on Tokyo, we should be able to do that. You'll need to go to the store, log into the store with your organization -- with an account associated with the organization, opt in to ATF test generator and Cloud Runner, that's the name of the application ATF Test Generation and Cloud Runner, opt-in, then you can go to system applications and you can install it. I've installed it. I've configured my cloud user, make sure you do that the first time, and then we're going to go to test generator. And we're going to go through all the details here. So hopefully, we can follow along. All you need to do is press submit and it's going to work. That's all we have to do. There's nothing to do. Now I'm going to do a little bit more, and I'm going to press -- I'm going to put my e-mail in. On the note, e-mails are -- for most of our subproducts, we've turned e-mails off. We all send e-mails to some test account. This bypasses the normal e-mail notification because we do want this e-mail to go out even on a subprod. So what you put here, you will get an e-mail when the test generation is complete. So again, even if your e-mail and your subprod is turned off or going to [indiscernible] or going to some example account, this e-mail will be sent out to make sure you use a valid e-mail address if you put in, not required, but recommended. Press submit, it's going to generate test for everything. I'm going to go into advance, we'll see what options we have in the advance. We have 2 choices. We cleared you specific table so I could do incidents. And in this case, I'm only going to generate test for my incident table, which is great if I am just trying to test something very specific or I go through this a few times or I can just say all tables. We have the maximum test count, so we could generate lots and lots of tests. We default to 9,999 as the max. I haven't seen it generate that much because it is looking for common permutations, but there is an upper limit. And then the -- that's the total for the suite and then the maximum per table is 50. Again, if I have -- I'm generating this for all of my tables and we're looking through everything, if I have 2,000 tables, 50 tests gets to be quite a bit even if they're running in 20, 30 seconds. Maybe that's not what I want. I would recommend leaving all the defaults until you see some issue where you -- all your tables are generating 50 and you're not getting the coverage you want to see, then maybe we'd increase it. But the default should be pretty good for us. I'm going to turn off advanced. And then once we've done that, I just click submit, and it's going to submit the job to go generate my test. We are not going to wait for this to happen. Depending on your data, it's going to do a lot of crunching on your instance to come through. It is just taking up a couple of threads to come through. There's not a huge impact that you'll notice on your instance, but it is doing some work in the background in your data. This might take a couple of hours. If you have a very large instance with lots of tables, and we said you everything, it actually might take a day or 2 to generate this. So make sure you're thinking about when you want to do this about planning an upgrade, about to start my upgrade schedule, maybe I'll kick this off on Friday afternoon and my subprod, I'll do my clone down to my subprod, generate all my tests over the weekend and then Monday morning, we have that suite which we can run. This is going. We're going to -- well, before we go to the test, I will show you 1 message. If it's running already, I'm going to go back to test generation. It will actually give me a warning. So we can only generate 1 set at a time because this is working. It should queue things up while we're going. So if I do submit some more, it will just queue that up to go, but it is giving me a warning of what's going on. If you want to see what's going on, we'll get the message here for more information, and that's going to give me the -- my queue of what's running in -- what's running. So I can see that my test generation, I've run this a few times before, those jobs were completed and now it's currently processing this third one or you'll see it's queued. The same queue works whether I'm doing test generation or I'm doing Cloud Runner or test user. This is when it's going to the hosted servers to make -- to connect back in to make sure that we can read the data, generate the tests, and come through. And also that when we're generating the test, we only keep valid test. So if we have some data permutation, and we can't get it to do that, we won't create that test. So 100% of your tests are going to pass. If we generate 500 tests, if you go to run them, they will all pass. Anything that wouldn't pass generation won't be put into the queue. All right. So we're looking at what's going on. We're generating our test. Let's go look at the test really quickly. And I'm just [indiscernible] going to tell me here. We'll go, look at the test now or my suites, important note here. The suites, whether it was a QuickStart test, or whether it was a test that I created or it's my ATF generated suite, it's here. So the ATF generated set ones are very cryptically named ATF generated suite. I've done this a few times now. And again, realistically, I'd probably delete the one I'm done with them and just generated a new one, having multiple here is a little odd, that it's not something you normally would do because you would just generate your current set, get rid of the old one. There's no reason to keep the old ones. In this case, I generated for a specific table and so that it has that name in it. So we're going to go into my test suite and we're going to see lots and lots of tests created. So in this case, we have 16 tests. This is a demo instance with demo data. There's not a lot of variation. But what we're going to see is it's testing these different tables as different users and doing impersonation whatever needs to come through. So in this case, we are testing incident as Joe employee using the ESS View and making sure we can do everything that we had before. If you see this test, this has an amazing number of steps. This is not a test that a human could create, it's too detailed. I talked about it before, keep it simple. You don't need to test everything and that comes from how much time it takes to build the test. When I don't care how much time it takes to build the test, I'm going to test a lot of things. So this is testing every field. I put in every -- the urgency of the state knowledges -- fill those all out because the computer did it for me, and I test all those things. If you're building your own test, you won't go into this much detail. You're tested for those outcomes. But for a regression test with generation, we do go into this detail. Again, lots and lots to do here. It's going to take care of all this for me. Now to running. We're going to do the demo of Cloud Runner. Make sure you're paying attention here. I'm going to click run test, and then I'm going to say, use Cloud Runner and that's all we have to do. So now instead of running locally, it's going to go queue it up to run it in my test. You can't execute test and generate tests at the same time, and I didn't go and cancel my other test. So this is just going to queue up and be ready to run. But that's all you have to do to enter -- to exercise Cloud Runner is that you'll see the message it's waiting for the resources to do this. All we've to just say, I want to use the Cloud Runner. You choose, do I want it to run on my local computer with my local browser or Cloud Runner. With Cloud Runner, we can schedule it, and you don't have to have that machine running with the cloud. With your test runners open, it will take care of all that for you. Super easy to use. There's no reason not to use this. For Cloud Runner, we include 2 free levels of parallelism at the same time. So it's going to run 2 tests at a time and keep going through with 2. That works for most cases, it will get through the test in a very reasonable amount of time, the things that come through. If you need more capacity, talk to your account team, we do offer more capacity if you want to run through that. So if I need to be able to run 20 tests at a time, I have 10,000 tests, and I want to make sure I can get through all that in an hour, fine. We might need to pursue a little bit more capacity. But right now, I want you to go install test generation, which comes with Cloud Runner, generate some tests, use Cloud Runner. And the test that you create are the same, whether you manually do it, whether you copy QuickStart test, we do test generation. Again, the profile of what's created might be a little bit different than I can tell. But the tests are the same, so they can move in an update set the same. QuickStart test can run in Cloud Runner. Generated test can run in Cloud Runner. Manually created could run in Cloud Runner. They're just ATF tests. We are just generating them and running them. You get both of those, the part of the platform once we get into Tokyo. And with that, we're going to go back to the slides . And I'm hoping we get -- that we're seeing the slides again. If not, somebody will yell at me pretty quickly. So now let's talk -- we talked about testing. Let's talk about future-proofing what you build for upgrade success, and we'll go through this pretty quickly. I click on the right spot, there we go. So the first thing we want to do, we are testing all of our business outcomes. But your life will be easier when you use low-code tools. So that is Decision Builder, Process Automation Designer, Flow Designer, Action Designer, Automation Engine which is Integration Hub and Robotic Process Automation, RPA. When you're using these tools, you have less stuff to test. If you write code and you're writing scripts, you have to test all these scripts that it works. When you're using an out-of-the-box action in a flow, we test that out-of-the-box action, it works. There's no typos in it. And as you know, syntax that might be deprecated, we maintain that. So all you're doing is assembling things that we're fighting out of the box. If you're writing your own custom actions to come through, you can just test that action and be sure that it works all around. When you embrace the local tools, whether you're a citizen developer or pro coder or somewhere in between, this will make your upgrades go smoother. It will be faster. So make sure we're using those. This session is not about these. But if you don't know what each one of these are and what they do, then make sure you come to some of our other enablement sessions, some of our platform academies. These are awesome tools, really easy to use. What we're seeing now is Flow Designer with a Process Diagramming View. Flow Designer got a new view in San Diego that's continuing to be enhanced in Tokyo and in Utah to allow -- to give me a Flow Diagramming View or [indiscernible] View. Awesome way to see what it's doing and understand what's going through. Process Automation Designer, a similar type of concept. And Decision Builder has been evolving over the last few releases. Every release, you get some killer features on how we can isolate out instead of having code with big switch statements or really complex logic coded in there. We can create a decision which allows me to just change my values without changing my code. And if I keep my code over here and I keep my business logic separated out, again, it's going to make those upgrades smoother and simple -- simpler. So this Decision Builder, in Tokyo, we got the ability to import and export Excel to permit my decisions. Awesome feature where I can -- I don't have to take the time of a pro coder. My pro coder makes a flow, says I want to make a decision, and I can have an analyst or process owner modify that decision. And whether or not they're using ServiceNow or they just want to Excel file and modify it, we're going [indiscernible] we're going to empower that analyst to make really impactful changes, but really, really safe because all they're doing is changing some values in Excel file, we'll load that in. And now the logic, the decision that's being made, what level of approval do I need, when do I need approval, what -- how should this get routed, all of that could be really isolated without having to change any code, which is going to minimize all my upgrade business. And upgrade plans. This came out in Tokyo as well. So this might be the first time that we're using as we go to Utah. My upgrade plan before was a piece of paper, where my admin would go and say, well, we apply my upgrade in my subprod and I take my notes about what I want to upgrade, when I want to upgrade it, which apps I want to install, which version I want to have, which skips do I want to keep, which ones are we going to accept. Instead of having to keep that in paper or in a change task with really complicated things to follow, we can record an upgrade plan. So I do my -- I hit my subprod, I upgrade it. I'm able to recapture everything that I did in my upgrade. I promote that as an app [ into my app repo. ] And then when I go to my next instance and upgrade it, I can just apply the plan from my previous subprod. Saves lots of clicks, reduces the chance for human error. Great feature, comes into Tokyo, and it's going to help us as we go through into Utah and beyond. Also patches. It's not just major upgrades, but it's going to be very, very helpful from drug rates. ATF enhancement, we talked about test generation. We spent a lot of time on test generation and Cloud Runner that is there for you. Again, you need to get it from the store once you're in Tokyo. I believe Patch 5 is the [ new version. ] And with that, I wanted to hand it back over to Shamile to help wrap us up.

Thank you, Adam, and we're -- we have only about less than 10 minutes left, so I'll rush through these last few slides and make sure we've got time to answer some of your questions today. I want to make sure that you all know about the Now Platform Utah Release Upgrade Kit, which will be available in March with general availability. This upgrade kit is fabulous, and I don't use that word lightly because it has a full list of all of the new innovations and features that came out in the release. It's got a playlist, a valuable links, demos, blogs, and all the resources you need to simplify your upgrade. So keep a look out for the -- Now Platform Utah Release Upgrade Kit and you can get this by visiting the upgrade easily page on that customer success center.

It's awesome. Awesome. Make sure you go get it. I cannot tell you enough. You mean you go get that's there for Tokyo. If you're not on Tokyo yet, go get it. If you didn't look at it for Tokyo, go look at the Tokyo 1 before Utah replaces it. Awesome asset for you to help upgrade -- sorry.

Yes. So we then hear it's awesome, it's fabulous, go get it. And our conversation today doesn't end here, do join us on the Upgrades and Patching Community Forum, ask questions and get answers. You could also learn about upcoming webinars and meet-ups just like this, get the latest best practices and connect with peers and ServiceNow experts. I highly suggest that you subscribe to the forum to receive notifications about popular posts and upcoming events. And you can customize your notifications to only get notified about what matters most to you. This link is in the resources. Like I mentioned, we have a ton of webinars, meet-ups, I mean virtual opportunities, all about upgrades, be sure to check out live on ServiceNow, our integrated event series to support upgrades, adoption and deployment of ServiceNow products. And then we also have the Platform Academy series, which Adam leads. We have links again to both of those in the resources. We hope to see you at upcoming events soon.