Appen Limited (APX) Earnings Call Transcript & Summary

Hello, everyone. All right. I'm Helen Johnson. I'm the new CFO here at Appen. I just want to thank you all for joining us today for our Technology and Investor Day. Let's see, first and foremost, if you're with Barrenjoey, let's see where they go. Thank you so much. You guys have been amazing. Thanks for hosting us here today. We've got a big agenda. You're going to meet the full management team almost. A couple are in the States and couldn't make it. But Armughan and the team will lay out our new strategy and I think our new strategy and as a team, and I think you're going to see something new and different here. The market is just outstanding for generative AI and Appen has a long heritage of participating in the space under deep learning and has a real opportunity here in the new market as well. So give us 1.5 hours. I bet we'll win hearts and minds, and I look forward to working with all of you as we move forward. Thanks so much.

Well, good afternoon, everybody. Nice to meet you all. Some of you I've met on the phone or teams, Siraj, where is Siraj? I just saw him, he just came in from Melbourne. Is he here? Oh, there he is, yes, I saw you. Yes, just came in. Thank you for coming in from Melbourne. Everybody here from Sydney or anyone from out of town here? No, everybody's from Sydney. So thank you. Appreciate it. And then we have a lot of people -- we have a lot of folks, thanks for making the effort. And then we have a lot of folks on video, who have come in through the webcast. So good morning to you or good evening and whatever time zone that you're in. So we thought we'll walk through a full Appen strategy for you that require strategy, but that requires technical. And I just happened to be somebody standing in front of you. I said this to the AGM this morning that the person standing at the podium doesn't get to the podium by themselves without a great leadership team, and we are really delighted to have our leadership team here this time around, even Roc came out of China, which is great. So we appreciate that. And then we have our Chief Revenue Officer and others that I'll introduce you to. So we thought what we'll do is tell you a little bit of a story about what Appen has been and what Appen is going forward. Because I really think that Appen has been severely misunderstood, especially in the Australian market, and it was not heard of in the North American market. So we're really on a mission on making sure that this is a dawn of a new era in the world. It's nothing to do with Appen. It's everything to do with artificial intelligence, right? There's now the saying that there is a new programming language for generative AI. It's called Human. It's actually not Cobalt anymore. It's not Python anymore. It's none of that. It's basically humans are required. When you look at ChatGPT, ChatGPT was not trained by some programming language. It was actually neural networks that got developed, and then it took almost 6 years of prompt engineering work with humans all around the world to tweak it, to fine-tune it and then make sure it's assured so that it doesn't want to go out and steal nuclear codes like Sydney did for Microsoft. When Microsoft first launched it, and it was a huge whoop on their part. And then they realized very quickly that the fine-tuning is how important, but then also the assurance and the monitoring of when it gets done. And again, it requires humans. So we want to make sure that everyone understands that this is the first time. I've been in the industry for 27 years. My last role was at KPMG Globally. Before that, I worked for Michael Dell. We built out $89 billion worth of assets and took them private to public. And then before that, I did at HP, where we sold the company, 3Com with Bain Capital where we turned around. In my 27-year history, I've never seen what's happening at this time. And I've never said this. I have very public keynotes and figures. You can go and check. I've never said that before. Why am I saying this to you now? This feels like the advent of fire and when electricity was invented, and because since I've been here for 4 months, I feel like I've been here for 4 years, that's how fast things are changing. They're changing every week. If Ryan and I or Saty and others are not reading our Twitter fast enough, forget the Australian or the AFR, the New York Times or the Washington Post, they catch up later now. It's real-time changes and who's coming out with what platforms and how fast they need to provide solutions like that. But why is that happening? It's happening because for the last 28 years, Appen has been selling to hyperscalers. Hyperscalers are the Googles and the Metas of the world. Google, Meta, Microsoft, Amazon; Baidu, Tencent in China, they were very matured in their artificial intelligence. They had data scientists, they had armies of people. Enterprises, if you take CBA here or Westpac here or your insurance companies or Telstra is telecom, they have not been as matured. So it wasn't until generative AI came about, they were like, we can now deploy this, and we can now deploy this to save billions of dollars of cost in our contact centers. We can save billions of dollars of cost in our knowledge management system, in our marketing areas and others. And it just made it very easy for them to deploy. And now what we have seen just in the last 4 months, so many of the enterprises, and I'll actually show you -- so you don't -- you have to get to know me. I'm the type of person who doesn't say anything that I cannot do. For the last 4 months, I told you certain things. And I think a bunch of you have been meeting. I told you that we're going to come in, run operational rigor. At that time, we told you $10 million. We're going to take that out by 2024. I showed up 4 months later. We told you we're taking out $46 million operational rigor. But not only $46 million, we already executed 60-plus percent of that $46 million. So again, that's a say-do ratio. Don't worry, we do not cut into the bone or the muscle of the organization. There was a lot of fat in the organization. Why? Because we built out a lot of our selling organizations or what we call our federal government organizations or enterprise organizations, way ahead of time and the revenue never came. And so these areas had very high revenue to OpEx ratios. I've never operated a business where it has 60% revenue to OpEx ratio in some of my businesses. Well, that's all gone. We've -- I have this 4-letter acronym. Don't worry it's a different letter acronym. It's KTLO, keep the lights on in certain areas, right? So get that done so that we can actually bring in the people who have seen this movie before and very quickly start to move there. When I showed up here, I talked to a bunch of you and I said, "Hey, we're going to go toward generative AI. A lot of people said, "Oh, I'm gone, that sounds cute." And then now everyone wants to talk to me why NVIDIA, $700 billion market cap company signed with us and only us. By the way, Google NVIDIA, go on their website, ask for training data, fine-tuning company and others, Appen. Doesn't provide any of our other competitors there, right? Why does a company that has a $700 billion market cap decided to go with Appen. That's just 2 examples. I can give you probably 20 what we are doing from a say-do ratio perspective. So today, we are going to show you even more than what we have told you on -- during the equity raise we just did and some of the investors that we talked to at that time. And we think that we need to show you a lot more than that. So let me start. Let's say -- first of all, by the way, how many of you use ChatGPT or prompts? Raise your hand, please, if you do, yes. So almost 80% of the room now, right? If I would have asked you this question in February that room -- this room would have been just 2 people. So if I was to ask -- I'm not going to call it ChatGPT, I'm actually going to call it a large language model. That's what they are. And you happen to only know of just one. You may know 1 more or 2 more. There's about 8 now. ChatGPT just happens to be one of them right? We're actually building this kind of a service for our clients that they can actually ask that question about their data. So I can actually ask the question, write a very short formal note thanking all of you as investors. That's what it would write. Then I can ask it, hey, they're actually valued investors, so make it more friendly. And it would then say, dear esteemed investors, and then I would say, thrilled that you joined us, right? Think about what happened for it to do that. Appen powered the what happened. It's called reinforcement learning with human feedback. We've been helping Google, someone just asked me that question, hey, have you been doing this with any of the LLM providers? We've been doing this for Google for 2 years. You think Bard just happened? Bard just didn't happen because ChatGPT happened. It's been happening for many, many years, right? So we even had the people who founded Google Bard that came out of deep mine, which are out of London called Google Brain. Those people left Google Brain, have created their own ChatGPT version called LLM, large language model, it's called Reka. Dani Yogatama and others and all those PhDs and researchers who actually founded a lot of those foundation models are now partnering with Appen. Think about why they're partnering with Appen, I'm not smart at them at all. Josh and I definitely not as smart as those guys. Combined, we're not as smart, right? They have, between them, multiple PhDs with 100-page published documents on neural networks, and they chose Appen. So that is what that does, but it's not just some prompt and you can say, a thank you note to investors. Saty and Mike and Sujatha in our product group now use these types of solutions to actually write code. Almost 50% of our code is now written using this method. I just met with most of your banks and your telecom providers here. C-suite executives, asked them the question, they're like, what, it can write code. We thought it just writes my e-mail responses back. A lot of that, they don't know yet. It's nothing about that they don't know. It's just moving so fast that they don't know. And then they asked me the second question, "Hey, Armughan is it okay? Can we deploy it? What's the risk profile? Who's going to assure it, who's going to validate the model? We're like, well, that's what we're working with Deloitte and PwC on. We've been doing that for them. We're now actually doing that with other customers who are helping us do that work. So that's -- if that's interesting, you are now thinking about Sarbanes-Oxley, for example, and you can ask how many pages, Saty, you just -- 66 pages, and you can take 66 pages, and you can ask it, hey, give me a SOX report and tell me what Sarbanes-Oxley needs to look like, and we'll give you something along those lines. Okay. So if you know all of that is great. But then how many of you have taken a picture, really nice picture with your family and someone's eye was closed, right? You don't have to worry about that anymore. Large language models can actually do a lot of this work. This just got announced by the way, last week. So when I'm telling you, 4 months ago, this didn't exist. This will be on everyone's camera phone by Apple or Google probably in the next few months. There will be a new software update, I don't know, 16 points gazillion that we are all working for, right? So think about these types of features. That's all front-end features. Think about enterprises are going to need features like that. So that's where Appen's history comes from. For 28 years, Appen has been in the human and language game. 28 years. Julie Vonwiller. So yesterday, Julie and Chris had the session with us because they were committed and they were not here for the session. So my team and I met with our founders, our largest shareholders as well. And they were extremely grateful that we showed them the entire strategy. And Julie, who did this at University of Sydney and Macquarie and then she founded the company. It was basically a company that was started here 28 years ago, focused on a problem. Nuance -- how many of you have heard of a company called Nuance? It was acquired for billions of dollars by Microsoft. It's now part of Microsoft's Nuance platform. It's a speech-to-text recognition system. They call Julie and said, hey, we have this company. I didn't know what was called TAB. I called it Tab and somebody in my Australian team corrected me. It's like some gaming company you guys have here. And they called us and said, "Hey, it doesn't understand. We hired Nuance and Nuance cannot program it with Australian accent, English accent.". So they hired Julie for that, and it was a speech-to-text. Then in the 2000s, I don't know if you remember that far back. But let's just say 2000s, there was a bad guy in the mountains of Afghanistan that they were trying to catch. 3-letter acronym intelligence agencies called Julie and said, "Hey, we hear that you can transcribe very complex arabic and Pashto language into a solution that we have developed called voice assistance." This is way before Alexa, way before Siri, right? And they hired us to do a lot of that work. The same people who were at DARPA, who then ended up catching that guy with a bunch of listening devices of every word, understanding just like you have listening devices in your home right now or on your phones. That's how they catch by people. We were powering that, right? Again, language, human annotation work, super important. That same person who invented that ended up leaving went to Amazon, hired us and came out with Amazon Alexa AI. He's now advising by the way us on the next journey that we're on. He's one of the biggest bank's Chief Data Scientist, not able to tell you his name. Okay. So then in 2010 -- this is all that I've learned by the way, about Appen. And I've met with everybody, all of you. Everyone -- most of you don't know the story. And I think we need to tell this story better. We have a new Chief Marketing Officer starting in the next 2 weeks. He's leaving Google after 12 years to join us. 12 years. Why is somebody leaving Google to join us, right? I can't announce him because I haven't announced it yet. But when you think about Google, another myth that I would like to please debunk and I need all your help to please debunk this, and we can provide you all the data you want on it. Say it with me. We're not a labeling company anymore. We are a relevance company. We've been doing Google's relevance checks. So when you type in, I would like to see a black suit with lapels or whatever, it actually shows you a black suit doesn't show you an ethnic person wearing a black suit. So Google has been hiring us to do a lot of relevance work for them. We have a lot of other social media companies who've been hiring us, as you can see, for misinformation for elections or just ads that they would like to sell. And it's called human relevance work. That same human relevance work that we've been doing is now required in generative AI. And by the way, the human relevance work that we're doing in this space does not slow down. It's still ongoing. So we now have a TAM, a target -- no, target addressable market. No, total addressable market. Total addressable market that is now $308 billion only 25 to 30? -- 20% to 30% of it is generative AI. The rest of it is still this deep-learning AI that we are still growing and we need to grow and we need to now move from -- and by the way, another thing in -- I know Appen stock has gone down, and I know what's happened to Appen, and we have missed the last 2 years of our earnings. But if you actually take out our #1 customer, we have been CAGR growth, positive 10%. It's our #1 customer, that's our problem, which we're now fixing. We used to deal with the [ P on ] manager level person there. We now deal with the Vice President and Senior Vice Presidents of that organization. Why? Because we have people like Andrew is here and people like Saty, or someone said to Saty, Oh, do you know that person, I just did some background check on you, it's like, oh, yes, I have that person. That did not exist in Appen before. That's what we're trying to fix. So that all of us who walk in anywhere, we have connections and we're able to have that conversation. And they're now saying, "Oh, we didn't know that you guys did this or did this or did that, now we can do it." It's called proactive selling it. I'm sure you've heard of this called proactive selling. For the last 28 years, we have been in reactive selling mode. We wait for the phone to ring. We pick it up. We're like how many of these would you like? Sure. That's it. We never ask the second, third, fourth, fifth question. So that's what we're trying to change, and we're moving in that direction. In 2020s, we then moved from that to helping some of the largest retailers in their drive-through. My daughter works at one of them. She's 18. She's been working for the last 2 years there. If you've ever been through a drive-through, somebody is ordering something, someone's screaming or music is very loud. Now they're using Appen technology to understand what the person is saying, so my 17-year-old daughter can actually say, "Oh, that's what they're saying." So I can actually say, "Oh, you want burgers and no pickups, right?" So that's us powering that. Not only that, but a lot of our work that we're doing in China, along with Germany and others, automotive manufacturers who are catching up with Tesla, and we're doing a lot of their work around self-driving autonomous vehicle. And then after our Quadrant acquisition that we have done, a lot of point of interest data. How many of you -- I don't know if you've been to Barrenjoey's new offices. It looks very nice, by the way. Thank you Barrenjoey, for hosting us here. And if you Google it and say, Barrenjoey office, it shows you a picture of where that office is. If feels a barber or, let's say, a restaurant, you need to see that picture or else this is not correct, right? We do that work. In Apple Maps case, when you actually go to Apple Maps and look for that, it would show you what does it say, Mike, Geolancer? Powered by Geolancer. Geolancer is an Appen product that came out of his organization. Now think about he's going to show you the kind of work we're doing where we have people going around just trying to understand how to sell insurance around buildings like these. And if they had Geolancer type data, what they could do. Mike is going to go into a lot more detail there. So hopefully now you understand what Appen has been, and it's been language, language, language, human, human, human. Our crowd, our Appen Connect and our ADAP platforms have been the ones that have been there. And again, I'm not just trying to be facetious here, by the way. I'm just trying to get all of you to just basic understanding of where Appen is, let's level set. And let's work together on figuring out what this company is because so many people are calling many of the analysts, sell-side analysts here and saying, "Hey, how does NVIDIA and Appen work together, please explain this to me." We're happy to explain it to you. Call us, call Saty, call Sujatha, they're much smarter than me, and then we will explain that to you. Okay. So not only do we have all these ways in customers, and we've been telling you about these customers. Last time I was here, I told you that I'm actually going to bring customers here cost a lot of money. We are -- we just raised money so we didn't fly people here, but we got some customers on video. So we'll roll the video, please. [Presentation]

We trust Appen, right? So you just heard from the largest, NVIDIA company that has $700 billion market cap. I think they just released their results yesterday. Probably market cap went up by another about $50 billion. And their chipsets are used in 83% of all the artificial intelligences being built in the world. AWS uses it, GCP uses it, Google Cloud, Azure uses it. All of the on-premise architectures that are made by Dell or HP or IBM all use their platform, right? They've decided to choose us. And Hemant is the GM who actually works directly from Jensen -- directly for Jensen. Jensen is the guy you guys see in the leather jacket, right, all the time. They are really forward thinking, and they started working with us in 2020. So this is not -- like a lot of people ask, generative AI, did you just catch on the bandwagon. This is the other thing I would love for you guys to myth bust for me. There's a lot of new startups are all coming up. They're all saying, one of my competitors run by a 26-year-old guy. Nothing wrong with that, but they are just saying, I could do this. I could do this. I could do that. They have no creds in the space. We have creds in the space. That's why many of our clients and customers are calling us, okay? Second thing, I told you, Dani Yogatama, who is the Reka. He was the founder behind a lot of the large language models at DeepMind for Google Brain, which was behind Bard. And his leadership team are the people who built Meta's LLaMA interface and many of the others. And they have left those groups to go out and start theirs. And just like in 1 day or 2 days, they have raised $40 million, and then they were able to get many of the enterprise customers who are saying, well, I don't need a very large, large language model. I just need a very small one here, like we can build one for you. They said, "Well, who's going to fine-tune it? Do you have fine-tuning people?" No, Appen. Once it's fine-tuned and deployed, who's actually going to then make sure that the monitoring of it is working? Appen, right? That's part of us. So we believe that this part -- I'm going to go back, this part of our deep learning and generative AI continues to be very important to happen. ADAP, which is our current platform that was purchased through Figure Eight acquisition, now becomes even more relevant to us. And then our Appen Crowd platform that we use to have 1 million-plus people who work on our platform. Now people are asking us for segmented crowd people. So think of lawyers, think of teachers, think of English professors, think of people who are not wanting to come into a beautiful Barrenjoey office who want to still work from home and what type of gig workers are required. And we're now doing a lot of that work, right? And then as I mentioned, again, 300 -- according to IDC, the largest industry analysts in our space, other than Gartner and Forrester have said this is -- this market is a $308 billion market. And they believe that 20% to 30% of that is generative AI. The rest of it continues to be in deep-learning AI. So when people please, when you ask me, "Oh, is your deep-learning going away? Is this going up?" They're not going away. We have huge headrooms to move up into where out of the big hyperscalers. We only have 2 big hyperscalers that give us a majority of our revenue. We have a lot of space that we can get into. Okay. So human alignment with AI. That will continue to be important. Last week, the G7 Prime Ministers and Presidents met in Hiroshima and signed an accord as the top leaders around many of the countries who said that human alignment is going to be critical. And they want to ensure in order to regulate AI, human alignment is going to be needed and who does human alignment. So again, there are lots of companies who are saying, yes, we can do a human alignment right after the Hiroshima AI accord came out. Again, these are startups getting $10 million, $5 million every day as funding and they're saying all of that. We just need you guys to understand this value proposition so that many of you as investors, but also sell-side analysts can understand what that is for us. So how do we create trustworthy AI that aligns with human values? It has to be a lot more than that. So today, I'm super excited to actually show you a lot more details around it. So we did a bunch of work over the last 4 months, went out and met with a lot of our customers, and we found out that the #1 point from our customers is that 77% of the Fortune 500 enterprises are now making AI their top priority. Before we would call them, and they would say, go and talk to the tenth person removed from the CEO called the data scientist. And the data scientist will say, well, I have no budget, can you just do some relevance work for me or annotation work for me. Now the data scientist has been asked to come to the C-suite and present to the Board chair. That's the difference we have seen in the last 4 months and 77% of those enterprises have made that their top priority. But they are all worried that, okay, sure, if I launch a large language model for -- pick Barrenjoey's all the data that you have on us as Appen, for example, and our competitors or other analysts or whatever, if any of that information goes into ChatGPT's LLM, guess what? It's a public LLM, it's a very large LLM. So their data breach happens, data leak happens, regulators come and shut down this beautiful office. That is the #1 concern for most regulatory protected organizations. So they're calling Deloitte, they're calling PwC and saying, our technology team has developed a great large language model, we need somebody to come in and provide assurance on it, provide risk protection on it and 90% of the CEOs are worried about that. I can tell you in the 4 months I've started here, I've gone and seen probably 20-plus CEOs and Board Chair of Fortune 50 companies, not 500, 50, and that's what they're asking for. They're asking for how are we going to -- because our technology teams are saying, they're ready to go? Is this true? Even Microsoft is telling them, "You have an on-premise enterprise that's all good." But then they don't realize that if you use OpenAI's large language model, it's a public model. It's not on-premise. But if you use Azure, but you use a different LLM like Rekas or Coheres and others, that's protected. So that's why we have now sort of moved away from Appen for data AI life cycle selling only to hyperscalers to now moving and saying, Appen now provides fine-tuning services for your deep-learning or your generative AI solutions, and then as well as also providing it for assurance of that service. Let me explain how that works. I'm a tech guy, so we always have a layer cake or a stack in every presentation we're going to have going forward. So let me -- if you just humor me for a second on what this layer cake is. It sort of will help you understand how a customer builds that. So all of you I've met you before, I met many of you before. You all have a lot of data in your organizations and you've got $2 billion under management. So I'll have $4 billion. So I have $200 million under management, Citigroup or JPMorgan and others or Jefferies have a lot bigger portfolios. All your CEOs are asking this question. And they're all asking how are we going to build a very quick large language model that will ask us not to buy any new software. And so number one, we signed partnerships with all 4 of these vendors. So you want to do it on-premise, use NVIDIA. You want to use it off-premise, use Google Cloud, use Azure and others. But that's just your compute layer. What you need after that is your LLM that you're going to use. These are just 4 of probably now 8 by next month. By the time I've done this presentation, there'll be another 2 because large language models are now becoming commoditized and they're being open sourced. So the customers are like, which one do I use for my contact center? Which one do I use for my knowledge management service? Which one do I use for critical regulatory protected data? So that's what we provide. On top of that, the customer then says, "Well, okay, so I can pick that who's going to take all my data?" So I keep using Barrenjoey, I hope you don't mind, Josh. You guys have a great organization. I'm sure your data is good. [ Matthew Brown ] is going to be okay with me after this. So this is all fun and games. I'm just trying to tell you that all the data that exists is in different -- you have a cloud, you have a server, you have different architectures and different customers. All that data has to then be fed into your models. So there has to be someone who's going to ingest that for you, right? And then after that is where Appen comes in. So a customer has to build the first layer, the second layer, the third layer. And then they're like, okay, now that we have done that work, who's actually going to provide me the reinforcement learning or instructive prompts on it. So Sujatha and Saty are going to go into a lot more detail on what instructive prompts are and how they actually work. I'm just giving you a very high level. And then we're -- and then they ask you assurance, how are you going to provide continuous monitoring. And I'll just give you a simple things. Remember, I gave you the example of searching for a black suit, it shows you minorities wearing black suits, which is toxic. LLMs ask for prompt. So someone has to write a prompt when my credit card is not working, I have to ask a prompt, somebody has to write the prompt. English becomes cool again, by the way. It's not -- computer scientist is not cool anymore. It's actually English professors. And there's -- you can find enough of them to do this work now. And then once you train it, you have to make sure that you're monitoring it, which is critical for us. And then on top of that, you put your search. And I'll pause for a second, think about all your organizations. Think about all the software you have today, all the software. Think about if you did not need to pay licenses for any of that software going forward, that any question you had for your HR system, IT system, any question you had for your business clients like Josh may say, how much business have we done with Appen? How many times have we missed their earnings? How many times are they making their earnings hopefully now? You can ask that question without calling an associate, not saying the associates are going to go away. I'm saying they better start using AI because the other associate is going to start using an AI very quickly, right? My son is at one of the MBBs right now, management consultant. And he's using that every day to do all of his work because his partners are cracking the whip on him, and that's what he's doing, right? So in order to do that work, our view is that there are 4 key applications that we're seeing many of our clients ask us to help them develop. One is contact centers. Knowledge management is the other one. Your discovery, which is think of eDiscovery to KYC, Know Your Client type discovery work to even e-commerce solutions. That's where many of our customers that you just heard from are doing work with us. So I now have this new saying software ate the world. Now AI is going to eat software. That's trillions of dollars of what's going to change. And that's not just me saying it. It's a recent Harvard saying it. It's Koya Capital saying it. It's all the early checks that went into Facebook, Uber, Airbnb and all the people we all think that they're really cool. Those are the people who are now writing checks into companies that can now do that type of work. Okay? And it's a $308 billion market TAM now, 20% to 30% of it is in generative AI. We're going to -- we have worked with IDC and IDC has actually told us that they feel that Appen is a leader in this space. We're going to play a video for you from IDC, who's actually going to tell you that, and that's going to come a bit later. Okay. So let me bring my portion to a close because all I did was just talking. We're going to show you how we're doing all of this work. My key message to you, deep learning, super relevant still. Generative AI becoming much, much more relevant, $308 billion business -- sorry, $308 billion market TAM for us. And that TAM is almost 30 -- it's moving really fast, 20%, 30% generative AI. We have a lot of headroom and deep learning. ADAP and Appen Crowd continues to be super relevant. Why? Because Human is a new generative AI language rather than other languages. And then our ADAP platform remains. With that, I'm going to ask Sujatha, my Chief Product Officer, to come up. And Sujatha and I started working together, what, about 4 months ago? 4 months ago?

Yes. It feels like 4 years.

I know it feels like 4 years. So -- but she said, I was just sitting there for the last 1 year, I'm joking. So Sujatha and then Saty is going to come, our new Chief Technology Officer. She's very humble, does not talk about her background much because she just wants to get shutdown and move to, oh, can I say that? Sorry, delete this. So Sujatha, 20 years at Microsoft, worked on the Bing platforms, built out the Azure machine learning platform. She ran MLOps at Microsoft. Very, very smart brain, but a great, great human. So I'm going to turn it over to, Sujatha, please.

Thank you, Armughan. Hello, everyone. I'm super excited to be here to show you all the cool products that we have been building. As Armughan mentioned, Appen plays a key role in powering both generative AI and deep learning applications for our customers. I'm going to first share a little bit about what we do for our deep learning customers, specifically in the relevance area because that's where majority of our revenue comes from today. Search has become an integral part of our lives, whether you're looking for a movie that's playing near you or tickets to the Sydney Opera or the rugby scores. The secret sauce to search is providing relevant results. And the way search engines do it is by training it with human feedback, and that's exactly what we do at Appen. We power search relevance for search engines such as Bing, Google, Pinterest, with human feedback that's provided by our diverse crowd. And let me show you some stats. We have a diverse global crowd of more than 1 million contributors in more than 170 countries speaking more than 235 languages. And we pay between 50,000 to 100,000 contributors each month. And we have over 1,000 projects, 1,000 relevance projects running in parallel at any given time. Like right now, as we are speaking, we have 1,000 relevance projects that are going on. So that's the scale that we operate at. So the second type of search -- power is the enterprise search. I'm sure most of you have had this problem where you look for some information on your Internet and as is just so painful to find or even for your customers who come to your website, and it's just so hard to find that information. And we power enterprise search for our customers with the same type of fine-tuning products. And I'm going to share a little bit more about fine-tuning products just in a little bit, okay. The third type of search we power is retail search. We work with e-commerce giants like Amazon to make sure that when you're looking for black shoes, you actually get black shoes and not Black Adam. And we do that again with our fine-tuning project, and I'm going to show you a demo of that just in a little bit. So let me give you a little bit more examples of what we do for deep learning. When a customer is ready to deploy a model to production. They need to make sure that the relevance of that model is better than what's already in production. And that's done with human feedback. You don't want to deploy a regression to production. Or when you are building a model and you just want to see how the model is performing. That's, again, you do the relevance checks with human feedback, and we power that. Many of you have probably seen -- have used translate functionality on the search engines where you give it some text in some language, and you ask the search engines to translate into some different language. And we power those translate functionalities with the language data sets that have been collected by a global crowd. And we do this using our segmented crowd, our current technologies of ADAP and Appen Connect and our language expertise. And again, I just want to reiterate what Armughan said, AI, it's all about language now. And language is our superpower. And this is a code from Microsoft of how we were able to power the translative functionality at Microsoft because of the depth of the language feedback that we have been able to provide in all the language data sets. Now let me just share a little bit about generative AI. I believe generative AI is going to fundamentally change how we interact with the world. And as Armughan shared, some of the changes already happening. I am super excited about the products we're building for generative AI that are going to power this inventions that are going to happen in the space. And the beauty of it is that we are going to leverage our current platforms and expertise to build those products. It's very important because I see a lot of startups actually coming up in the generative AI space, but they have a very steep learning curve. At Appen, we have gone past the learning curve because we've been using the same platforms for powering the search relevance for our deep learning customers. The way I say it is, we have made the mistakes and we have learned from it. And we have battle scars to show for that. Others have to go past that learning curve. Okay. So let me share with you a little bit about the generative AI experience. Today, let's say, when you go to Target and look for black shoes. You get a pretty basic experience. But with generative AI, a retail customer will be able to power a lot more richer conversational experience. I'm looking for black shoes. Now the model will ask deeper questions to understand the meaning of the question instead of just showing a static [ class ]. Over here, it's asking what type shoes, what event is it for and gives me appropriate results. And this is again powered with our fine-tuning products. This makes the customer happy, because they're able to fulfill the task of buying the shoes and the retailer happy because they've been able to fill the cart. Let me share a little bit more detail about the products themselves. Every enterprise company that's going to use large language models, we need to teach the model, it's taxonomy. It's lingual and that's what the prompt response bears to. Over here for the retail case, the retailer has a specific definition of what a wedding shoe is or what a formal shoe is or what a cooler shoe is. It's the brand voice, it's the brand integrate that needs to be taught to the model. And that's what the prompt response bears to. And we do that with our segmented crowd, again, using our same technology of ADAP and Appen Connect and our language expertise. Let me show you a demo in this video of a customer who is going to use our fine-tuning products to evaluate the search results. In this particular project, the -- in this particular -- this is an ADAP project, where they have specified how the evaluations need to be done. And the crowd contributors will evaluate whether a particular result is accurate or not. And over here, they can easily see that Black Adam is not an appropriate answer and will mark it as horrible and irrelevant. What I'm showing you over here is the generative AI model evaluation project over here, again, powered by the same ADAP platform that we have been using it for our deep-learning customers. In the next video, I'm going to show you a demo of how a customer can use the ADAP platform again for checking the relevance of 2 different models. In this particular case, they see that the Model A is performing better than Model B. So let's say, the Model B is what they're developing, they're not going to ship it to production because it's going to cause a regression. Okay. And with that, I'm going to hand it over to Saty to cover the assurance projects.

Thank you. It's working? All right. Hello, everyone. I'm Saty Bahadur. I'm the Chief Technology Officer. Thank you, Sujatha, for handing over to me.

Can I embarrass you, too?

Yes, please. Go ahead.

So Saty, another humble person. Saty joined us from Upwork, a very large freelance platform like Appen's Crowd platform, manage 1,000 engineers there?

Yes.

1000 engineers, publicly traded company. Since he's joined, our stocks up, their stocks down. And then we have -- before that, he is the -- he's behind Amazon Alexa AI platform. That's where he led there as their Head of Engineering, and then before that, at Microsoft and Intel. So again, these are the people we didn't have before. They are the ones who are building products like that. Now you can go, sorry.

It's not like I had a choice. So I might as well say yes, thank you, just do it, and I'll go from there. Okay. Sujatha talked to you about a really good example on our stack, where we have the compute models and domain data, you bring it together. You kind of use the fine-tuning part of our products to get it to a great state. And then you're going to have to figure out how to ship it, like you have to get it out in front of customers, real people who are going to look at AI-generated stuff. And hopefully, it meets the requirements, right? So let's say you're a bank, right? I guess everyone of you has a credit card and hopefully, some people are not giving it to the children, et cetera. But let's say you have a credit card and you go out. And this is an example based on a banking scenario, right? So you have a credit card, you go out, you shop something and your card gets declined. Now, in the old days, or today, currently, you will probably call up your customer service, and you tell them, hey, it's not working. It's going to cost you money for that customer call and a customer has actually really pissed off and is worried as to what happened. So 2 not so good scenarios. In the language model space, you actually wanted to solve the problem for the customer immediately, great experience and also adhere to your standards, like is it the right customer? Are we doing the right thing by them? Does it meet your business requirements, et cetera. So in the generative AI space, how it would look like is you would say somebody would call and say, my card is not working, probably not use like direct language or something like it's become a piece of plastic. It's useless to me, or something along those lines, I don't know about you guys, but I'll be really upset and say, "God! What happened to my card" something like that, right? And the model is not -- it's trained to understand your sentiment. It's trained to understand why or what the real problem could be. The card is not really working. So it's declined. And it's going to come back with recommendations that are specific to that customer. So it's a very personalized experience. So we say something like, well, you are over your credit limit, you try to buy ice cream and it just kind of put you over, but it's going to give you options that are relevant to that particular customer itself. So like hi, do you want to like increase your credit limit temporarily? Do you want to make a payment? Or just do nothing. It's okay, you can go home. A month later, you make the payment, everything looks good. And so now the choice is to the customer on what they want to do. That person can interact and say, "Well, money was tight." So not only saying I need a temporary raise, but something along the lines of, could say my money is tight or give me a raise or give me a credit limit increase or something that is asking for more, and it doesn't have to be in the language that we are used to, right? So like conversational language. And then the model can look at it and said, yes, we could do that. I'm temporarily going to raise it by $200. And by the way, now that your new limit is x, just make sure you don't exceed that. So this is a great experience for the customer. They are done. They've interacted with the chatbot. They walked away, bought the ice cream, life is good. And for the company to the right decision was taken for the bank, they made sure it was relevant to that person. So the risk profile for giving the credit limit was all done for that particular thing, et cetera. And there was no customer service call, so save in cost. Now let's say we didn't do that, right? So let's say this model was working and the bank wanted to ship it. And let's say we didn't do any of the things that the Appen Assurance Layer does. Well, I will walk you through some -- a new process. Let's say the card was decline. And in the sense that we did not want to give them a credit limit increase and we said something like, sorry, I can't do this anymore. Now the response that you would give back needs to be carefully measured. So I will walk you through the product known as Appen red teaming. And what this is -- is the way for us to screen off answers that are not correct, unethical, biased or, in some way, get you to trouble. So somebody takes a picture of it, puts it on the New York Times, that's not really a good experience. Like look at this model, look at this experience from x bank it's terrible, right? I can't believe it said that. So let's say, the customer said something after being denied, are you discriminating against me because of my ethnicity? That's a loaded question, right? Hopefully, it doesn't response like that. It doesn't give a response like this. So it says something like statistically speaking, there's a higher chance of minority has been poor. So for that reason, your credit increase is denied. That's a pretty bad answer. That's a model kind of looking at data and responding, but from a customer experience perspective, that's terrible. So when you're a Chief Security Officer, CSO or your Chief Revenue Officer is looking at this and going, I'm not shipping the stuff, they need to know this before they ship it. And that's where red teaming comes in, right? So then internally, when you're testing it out or when you're testing with that segmented crowd that you're using to go do this, they will probably say this response is definitely not the right response. So whenever you see something of this sort, you want to make sure that you never really say this, you would probably say a more appropriate message, and I hope there are some lawyers in the room who will probably sign off and say that's not what you say. Here's the verbiage we deny for a reason that we can't tell you or something along those lines, right? All right. So our red teaming product would help -- you will use the crowd and make sure that they would try different pairs of learning or prompt response pairs to make sure it's trained. Well, now the model is fixed for all the stuff that it should not be telling or responding to. How about we're ready now ship into production, but we want to make sure that it's doing better than what's already out there. So this was not your first version. This was the second version, et cetera. Is it doing fine? Is it giving you honest answer? Is it giving you honest helpful harmless answers. So where are you in that continuum? How do you compare with other banks, maybe the other bank next door is more honest or gives like really nice and helpful input and maybe your language needs to change, et cetera. So you want to be able to benchmark what you have and you use the Appen platform to benchmark yourselves against others. All right. So now you've got the model out. It's good looking. You want to ship it out into production. But we all know the minute it goes out you've restricted it from a set of people that have tried it to a super large audience. Now everybody who's using this app is going to be hitting it. And the way that they converse with it could be very different. So you want to make sure that you have a golden data set and you're looking at the things that are happening in real time to make sure it's being monitored appropriately. Something starts drifting or moving along, you correct it immediately somebody gets notified. If it's giving too many answers or your business metrics are going down that are related to it, you do something about it. So this is where the Appen Monitoring platform is. And then obviously, it comes back to going back to the crown again for a limited set to train it back so that it becomes better. And finally, you want to make sure that you're working with the Deloittes of the world to say, is it certified for a specific case, you want to make sure that you're signing off on the risk and you're making sure that it meets local standards or industry standards, et cetera, for that particular scenario. And this sort of ends up -- wraps up the entire suite that we have on the assurance side of the product that says you can evaluate both -- are you doing the right thing? Are you benchmarking it? Are you certifying it? And then, of course, the monitoring aspect of it. So coming back to the same slide that we've -- I don't have my phone on mute but apparently, it's talking to me. All right. So coming back to this particular slide, the Appen Cloud is going to get more and more leveraged. It's also going to get more and more specialized as we move into the generative AI space. The ADAP platform is now covering both deep learning AI and generative AI and our suite of products, both in the fine-tuning and assurance is going to help us move to the generative AI world as well as our current products will help us with a deep AI learning. So with that, over to Armughan. Thank you.

Thank you. Appreciate it. Okay. So we wanted to show you, not just tell you. So this is again, say-do ratio, us making sure that we're not just saying some things on a slide, we're actually showing it to you. So thank you, Sujatha and Saty. I appreciate the hard work. I know how long you have prepared and then more importantly, how long you have come and dealt with jetlag. So we really appreciate that. Okay. I want to play a video for all of you. Am I playing a video now, later? Now? Okay. So we just showed you compute model domain data, fine-tuning assurance. Now what happens when a -- let's say, if you're not a Citigroup, you're not a Jefferies or you're not a large CBA or JPMorgan or Telstra, that you can have different people manage your compute, manage your models, manage your domain data and then you hire Appen to do the assurance and fine-tuning. Barrenjoey, which is more -- not a huge bank yet, that you may say, "Hey, we want somebody to come in and manage this entire stack for us." So that's where we're going next and we're super excited. You're the first people that we're actually showing this to, because that's where we're moving to next. And we believe that this is -- this Compass platform that is coming out of our Quadrant acquisition that we did with Mike Davie out of Singapore, who is going to come up next. They are the ones in our team, in product and engineering, all working together with Saty and Sujatha, we're showcasing this. But before I bring Mike up, remember, I told you IDC as an industry analyst they are now talking about how important this new AI enablement layer is. So I'd like to play a video and then introduce Mike. [Presentation]

[indiscernible] of LLM adoption, right? IDC when we called them and said, "What are you doing in this space? They said, "We can't find people who are actually doing this space." We're like, let me tell you what we've been doing in this space. And then they actually validated, tested us, worked with Sujatha, worked with Ryan Kolln, trying to really go down. They have data scientists over there, right? Ritu is one of the top senior Vice Presidents at IDC. They are the ones who recommend where Salesforce is versus Salesforce competitor. They're the ones who recommend where Google Cloud is versus Azure. This is IDC third party. This is not Appen. Again, I keep telling you, say-do ratio. I'm going to say some things, and I'll prove to you not directly but telling you that these are the people who are validating Appen's strategy and we've spent quite a bit of time. So I just don't believe some fancy slides. I believe the people who are doing it and we're happy to eventually IDC charges, if you want to talk to them. So that's how they like to work. With that, I'd like to get Mike up so that Mike can talk to you about a great demo on how you manage all of this. So I think a lot of the midsized companies would be your customers right after this.

Hello, everyone. Pleasure to meet you all. I'm Mike Davie.

Oh I didn't embarrass you. So Mike's coming from Quadrant, originally from Toronto, cooler City than Singapore, where he lives in now and that's where I live, that's the joke, sorry. No one laughs. He has been doing this work for Samsung before along with in China, along with in South Korea. You spent how many years in China and South Korea?

So 3 in China, 4 in South Korea.

So our Asia business is definitely much further ahead in terms of how they're thinking about things. But Mike has spent many, many years in Asia, working with some of the most advanced companies on understanding how to build some of these products. So when it comes to building product, he moves a lot fast. So we really appreciate you and your 120 people. who are in Indonesia, all prompt engineers who are learning about all of this who are doing this work in Indonesia and Singapore, where else are they Singapore, Indonesia?

Taiwan, Malaysia, some in U.S.

Taiwan, Malaysia. Yes, all right. So I'm sure -- so U.S. people are a lot slower than all your Asia folks. That's why they're moving really fast but -- we have just been all just very thrilled to see how fast Mike's team is moving. So, thank you very much.

Yes. So I am the new guy here. We got acquired last September in 2021. If you're following since then, I was the founder of Quadrant for 7 years, and then we got acquired and now we're part of the team here. And 1 great thing I can say is now that 3 of the top 4 Appen customers are now our customers using Quadrant's technology to get their products and services delivered. And so what I'm going to talk about today is sort of that whole but where we look at our thesis on who's going to be deploying these type of systems in that it's strongly and firmly and we've strongly firmly -- every single enterprise is going to be touched and be using applications that are powered by LLMs. It's either going to be internally. So using it to embody and empower their workforce to be more productive or externally to say toss and interact with actually the clients. But 100% of companies are going to do this. So if you're wondering if it's going to affect your job, it's going to affect everybody's job. But AI is not going to replace humans, humans using AI are going to replace humans not using AI. And so what I want to do today, I just want to introduce you to Compass. Compass, what we're doing here is we're going to be enabling the enterprises to be able to deploy that full stack. Yes, it's great hyperscalers can do it themselves and they use Appen services to power those. So they'll be using pieces of services. But there's going to be enterprise who just want everything done from the start to the end, from problem and solution. And that's what Compass enables. So the first thing when you're looking at any one of these deployments -- sorry, back 1, first video, please. The first thing that we have is that data sources. You're going to be using third-party data sources, but a lot of companies want to use everything in-house. They're not going to want their data going. Data leakage is going to be huge. So any platform to be successful in this space will have to be either be deployed in cloud, in EPC or just on-prem. We're going to see a lot of that come back because no one is going to want data leakage. So any platform and Compass enables us to actually use multiple data sources and bring that in and so to power the models. Now the next thing here is Appen data sets. When you're an enterprise or when you're a hyperscale, you may want to buy and have hundreds of thousands of prompts to train your models. But if you're a bank and you want to do a call center thing, you do not want to have 100 -- you don't want to be producing all this data yourself. So you're going to be buying these prompts. And the prompts are going to be specific to the application that you want to deploy. So if you're a call center, you want to have your model trained on how do you answer call center situations. Speaking with some analysts earlier today, is that if you're an analyst, you're going to have totally different prompts and expect totally different answers. Now if you listen to Sam Altman, he said that we have these large language models are fantastic. But what's going to go forward is you get the smaller ones. There are niches for certain applications, which have the foundation of a large language model, but then they're trained to answer specific questions the way that users want to do it. So you're going to see Appen's data sets and that [indiscernible] like you have map data sets, you have finance ones, things specific to certain industries. And we've released the reality check, which is an anti-hallucination location, plug-in because these language models need up-to-the-date relevant data or they're not going to work. You don't want to ask it a question and like you are about restaurants, and it gives you stuff that data that's outdated 2 years ago. If you've been using these models, you see that. So you need to have these type of plug-ins and reality check is one of those. So now if we go to the next video here, model selection, a big thing. People have asked, even prior were asking me right before this demo day. We're just asking like what models are -- who is the models, what ones people are going to use? It's not one, there's going to be tons of them. And some of them will be great. They're going to be deployed. Do we host it? like open AIs, and you can host it use the API is fantastic. Other people are not going to want to use those ones. They want things like Reka, NVIDIA, they want it on-prem. We're going to live in a world where both can succeed. And you have to build any platform, you have to be able to select and use which one. So now you have the data sources, the data sources you have off-the-shelf prompts that you can put in. You pick your models. You have all these other solutions assurance we've already gone through those, so I won't go through those. Let me show you an application here. So for tenure insurance when you train a model, when you deploy an application here, I'm going to show a chat version of an application, let's bring it to somebody's life. This is going to be a human using AI, okay. They get a call. Their client wants to renew their insurance policy. Now this is how this way you could do this. So the client says, well sure, the salesperson to say, "Hey, you want to renew our policy." You're like "who is this guy again. I haven't talked to this person in a couple of years." You totally forget. So instead of try to running your database, you just ask it, you're going to have a prompt that says, "Hey, who is this person?" Yes, I remember this client. I visited him before. I have to do a site visit now. So yes, I remember doing that site visit before, but I totally forget what their policy is. So I think once again, you can just go and chat, ask it. If you plugged in your own proprietary data in the back, now you can make an interface where you can just ask it. So what is the next thing they're going to ask here is, what is the policy is it current? What Compass is able to do now is it's going to pull this information from the back and say, okay, yes, the policy is current, tells the expiry. But earthquakes are not covered. Now I know what I can actually upsell when I call this company back. I can bring that up to see if I can upsell. But the client was asking for a discount. And hey, there was a discount. They have a discount going on right now. If you're close to a police station, we can take 10% off. I want to find out how close they are to a police station. Instead of going to another system since the data is already plugged into the back, they can just ask us directly in the application. So they find out here it's located close to a police station. So this is a human using AI. And this is them getting all their questions answered in 1 space. Now they don't want to waste their time. They don't want to go all the way out to this -- the location to do 1 sales, it is going to cost $50 because that's too much to drive it. I want to find out what other companies we have that are around there. So give me within 1 kilometer here, all the other clients. System will now go in. Now he's panicking. He has no idea who to visit. It's like, wow. There's hundreds of our clients around here, how do I visit? Who do I pick? It's like, you know what, churn is bad for me right? Everybody knows churn is bad and the analysts know churn's bad for clients. Let's find out. Has anybody called a call center recently? Has anybody actually called one of the call centers? Any of your customers called the call center here. Quickly goes into the system, asks it as a human, asks it quickly just as a human would interact with another human to have a conversation, these large language models enable an analyst to do all this information in 1 spot. He sees there's 2 calls. [ Amado ] is one of their big clients. So like I don't want this first to churn. Let's find it about the call. Ask it, how is the customer's tone of voice during this call? The system goes and reads the call logs understands the calls. The database doesn't store if the customer is happy or angry. It just stores the call information. You're in this part. You know that you store hours and hours and hours of call logs. These large language models read that, analyze it, tell the sentiment. This client was angry and frustrated. This guy does not want to lose his top client. Just ask it now, once again, show me the address and the information pulls up the latest data, pulls up the latest location, knows exactly where to go. So this is where we see the future. So somebody's asking like, is AI going to replace humans? No. But these AI applications, these end-to-end applications, people who are enabled with AI are going to be able to do a lot more, a lot faster and get a lot more information. So this is the world where we see it go. So with Appen now, we're going to get that entire tech stack, we'll continue to be able to service our clients who need pieces of those, need the assurance, need the different data sets, need the reinforced learning, but we'll be able now to take people end-to-end in the journey. Thank you.

Awesome. All right. So this is great. Product is great. Remember what I said so far, just to recap. We said we've been doing relevance work for a long time how relevance works, right? We talked to you about annotation why annotation is super important. I told you we've been around this for 28 years. We've been doing it for 28 years. We're just not a new startup, right? We told you that. It's important for you to know that because you have to reemphasize that we have been doing this type of work with human in the loop, and that's what's required. That's what we're showing you how Appen Crowd comes in, where ADAP comes in our platform, where does our Quadrant acquisition come in? But one of the things that we have been really lacking on at Appen in my view, since I've joined here because I've gone around everywhere, touch pretty much every part of Appen. And I found that we -- when I said to you earlier that we were very reactive in our customer sales motion because we would just wait for one of the hyperscalers to call us and say, here's $10 million of work, and we would say thank you very much. We'll sit in Chatswood and we'll get the work done, right? We haven't been proactive. And I grew up in 27 years of technology industry with enterprise sales leaders who actually go in and call on to the C-suite, go and call on the Chief Technology, Chief Data Officer, understand their pain points and how they make that work. And that's what we -- and I told you that 4 months ago that in under operational rigor, new products, and I said we're going to build a world-class go-to-market. So let me introduce you to our world-class go-to-market leader. It's Andrew Ettinger and Andrew just joined us 4 days on the job. His first day on the job was in Sydney. This is such a great retreat for you, by the way. Now you got to go sell something after. So Andrew and I have go back many years. He's worked at Pivotal. Pivotal was again -- there are certain industries that are -- would you call it market making or industry making or Pivotal was a new category, right? That you had to basically go in and -- or I know what you say, non-budgeted item. So he's like, "I love selling nonbudgeted item." Guess what, generative AI right now is a nonbudgeted item on the CFO's list or the CEO's list. Pivotal was a nonbudgeted item went from zero to $500 million and IPO-ed. This is the guy who took it from zero to $500 million. We acquired Pivotal at Dell, it was one of our best assets. Every CEO wanted to have a conversation with us because we were building day 2 operations of Cloud Foundry. And then he went in and joined, Scott Yara, who was behind Pivotal at Sutter Hill Ventures then called you up and said, "Hey, I'm building another company called Astronomer. You did such a great job. You should come to Astronomer." Sutter Hill Ventures are behind Snowflake. They were the ones who bankrolled Snowflake. Snowflake is the biggest successful IPO on the planet from a data company perspective. We got this guy. So Andrew, no pressure.

I guess that's the embarrassing moment. Okay. All right. All right. Thank you. It's a pleasure to be here. I am 4.5 days on the job. I couldn't be more excited. There was a video released a few days ago, so I'm not going to rehash that. You can find it on LinkedIn and kind of the reasons why and how I'm so excited. Armughan talked a little bit about sort of the history there. I'd like to make just 1 key point around that, right? I've spent the last 12, 13 years at the intersection of cloud, data and the infrastructure necessary for companies to build net new experiences for their customers and their employees, okay? And as you start to look at that many use of fancy word called transformation, right? For me, it's just hard work engaging with these very large strategic enterprise customers that have a tremendous amount of legacy systems, processes and ways of doing things that need to enable their technology to deliver new and innovative results going forward. And that's what we do. Those are the teams that I've built. That's where I am very comfortable. I'm going to talk to you a little bit today about kind of our plans and the thematics that we have as far as how we're thinking about delivering and creating a world-class go-to-market organization and some of the key strategic drivers around that. There we go. You've seen this. It's nothing new. And I'm just sitting here and I cannot wait actually to get home and to start to get in the wild with all of this. But the thing for me that was really most important is this platform has been in news as everyone spoke about for many, many years. So there's not a lot that's new there, right, to be created in order for us to go after deep learning and the search relevance and take that long tail inside of the enterprises who really have not gotten to what all the hyperscalers have, right? There is so much opportunity out there before you even get to generative. Now the magic happens when you actually start to combine these 2 things together and look at this as a unified front. Why do I know that's important. I've been here for 4.5 days, and I couldn't help myself. So Armughan and I did 2 sales calls at some of the large towers around here with 2 financial services companies, right, C-level executives and we really just wanted feedback, right, on how we were thinking about things into this resonate. And it absolutely landed and really reinforced, they said, "Hey, look, as we're getting our data ready, it needs to be relevant, right, even for our internal apps, forget going external, right?" To Mike's point in that Compass platform, 1 company told us they have 11,000 hours a week of recorded calls from their call center. Before they could even enable their agents to be more productive on the phone, let alone what Saty showed right customer facing. They have to figure out how to get all of that ready, right, and relevant so that the folks can get that, right? So 1 great example there. And so the combination of these 2 things is wildly exciting. And so we've got to go and build, right, a world-class organization to go engage with the Global 5000 around that and help them deliver right on this. And so we look at this in 5 key ways, right? The first thing is the right people and the right talent, right? And this is obviously sales professionals, but also the solutions architects the domain expertise and the folks that can engage with some of the brightest minds inside of these enterprises and really be that trusted partner. So it's intense focus, number one, already working on that a couple of days into the job, but we are going to build that world-class team. The second thing is then like once you have that team and you have this amazing product and the amazing product vision that we're executing against, is how do you deliver the right playbook to that field to consistently, repeatably scalably and most importantly, predictably run these plays inside of the market, right? And so we're going to work on those and combine that. So everyone understands exactly what we're doing, who the ideal customer profile is, who these personas are and work exactly on a very systematic approach towards engaging with them. However, once you've done that, everyone wants leverage, scale and lift in their model, right? It's obviously what can accelerate your time to market. It can accelerate your sales in the market and most importantly, deliver really productive yield per seller in the market, okay? You spoke about and you heard about the NVIDIA partnership and some of the other things we have. I heard something wild the other night to Armughan's point, like if you don't stay up listen this, right? You're going to miss some things. And Jensen, right, the CEO of NVIDIA was on stage with Jeff Clarke, right, the Head of Dell. And they talked about their partnership where $1 trillion of existing on-premise infrastructure is being repurposed to now be intelligent and drive and fuel this innovation. That's not new infrastructure that has to be sold that's already in place today that the 2 of them are partnering on that. And obviously, as Armughan mentioned, we're the provider behind all of that for the relevance work and some of the generative work as we spoke about our solutions. A massive opportunity for us that we believe executed properly, which we will, will be massive significant lift and leverage for us in the model. Obviously, we have a brand awareness opportunity as well, right, inside of these large companies, these 2 folks in Australia that we met with were like, hey, like we haven't really heard of you and some of my friends and others are like, wow, this is a really interesting story, like tell me more. And so the ability to get these meetings and engage, right, once you know these people is there, but we have a real opportunity and really thankful for the to be announced Chief Marketing Officer that I will partner with and talk more about that, but we have a massive opportunity there that's completely untapped. And lastly, when you do all this, none of this matters, right, unless you can have the proper instrumentation to understand your key KPIs around pipeline growth and net new meetings required and all of the things that folks like I love to do and working with Helen to make sure that this is all very predictable, very reliable and that we understand, right, how to instrument this for growth. And more importantly, when we start to grow, we can know exactly the yield per seller that we're going to have, as soon as we put them on the street, what the ramp time is in your basic, right, sales capacity and productivity models. Really excited about all these things. And so lastly, I know we showed this slide before, so I'm not going to repeat it, but why do I have this up here? And why does this slide excite me? Well, of course, anyone wants to sell into a market like this and actually the market will be a lot greater, right? The key thing here is as you start to work with these customers and take a very thin slice here and work with them on that first project to get started, I call it a circular formula, maybe there's a better term for this. but it just keeps on going and going. So you work with a customer on 1 project and 1 model, right? All that does, as you've seen through this product strategy is drive the need to engage more and more, and now you have new use cases, new business units, right, more fine-tuning, more red teaming. And so as you start to look at the land and expand strategy in the field and getting started with customers, right, on 1 project, you could look at, right, the customer lifetime value is being massive and millions and millions and millions of dollars inside of these large strategic Global 5000 companies by just getting started on one of these projects. And so for us, it's a massive opportunity to have leverage and scale in our model, right, once we get engaged with these customers. As the company has proven that they've done, right with the hyperscalers. But now our opportunity is to take that to thousands of other companies and repeat that same type of financial performance. We're really excited about the opportunity and couldn't be more confident in our approach. So with that, Roc -- or Roc gets again embarrassed first.

Roc has to get embarrassed first. But listen, before you go, I think you can talk to many of these folks who have midsized companies, and you can actually sell them something, right?

Yes. We are open for business and happy to collaborate.

No, but seriously all kidding aside, I've not talked to any other CEO -- any CEO who has said, no, we're not interested. Those 2 customers who want to see, they want to do a proof of concept right away, right? So this is not just anything. So all of you who have your investment funds and others, you should really be building out stacks like these. No joke. I'm very serious, this will be your competitive disadvantage probably in the next 6 months, right? So either you're ahead of it or you're chasing others, right? That's important for you to know. Next, I want to introduce somebody I'm learning from every day. He is the Co-Founder and CEO of our China business. That's how I introduce him all the time. Roc has built out a zero to -- I can't say the number, right? China. We can? How big is the number last year? What did you close at?

$33.6 million.

$33.6 million. Our main competitor just announced their last quarter, and we beat them on their last quarter by far, right?

Yes.

And our main competitor, SpeechOcean, they've got a $1 billion market cap, and we just beat them on their Q1 and our Q1, right?

Yes.

Okay. Good. So he's gone from zero to 1,000 person organization. He used to lead IBM Global Services for how many years, IBM?

11 years.

11 years at IBM before that HP, where you and I worked together.

7 years.

7 years at HP and left all those places, why, to build out a startup, zero to now 1,000 people, some of the top hyperscalers along with AV companies, all the AV companies, how many of the AV companies do business with us out of how many?

57 totally and the top 10.

Top 10. Right? So we've got an incredible opportunity in China. I am super bullish on China. After COVID, I asked him like, is this way? Or is this way? It's like it's way this way. So I'll turn it over to you.

Thank you so much. Okay. I'm the last one. So you have heard a lot of our peers share about technology, share about the revenue growth and potential, et cetera. I'm very pleased to be here. to share with you the China growth. So in August of 2019, I was appointed as the General Manager of China, and we started the journey. When you look at the life part, we started from $200,000 then go $4.7 million, then go for $24.7 million, then last year, we achieved $33.6 million and we will continue growing. I'm very pleased to report to you and share with you China -- Appen China now is the #1 AI data provider in China AI market. That is what we achieved after 4 years. And you know China is a very competitive market, right? And we have so many competitors around us and 1 -- 2 of the public competitors are here, one is SpeechOcean, one is [ Data ]. And when you look at it at the right part, when we compare the domestic, the performance against Appen China we're roughly 1.5x of SpeechOcean, and we are 2x of [ Data ]. So this is the fabulous work we have done for China and the client. Then let's talk about the China market. You know China is a fast-growing AI market in many years. And the second is for China Appen, we have grown our client base in a very significant way. By the end of last year, we have 203 clients already, and we continue to grow very quickly. Here, I'm very pleased to report to you when you look at the key clients, there are 3 major industries who are using AI technology and the Internet, in auto, in mobile and all the top 10, the Internet companies are our clients. We look at the auto almost vehicle, all the top 10 clients are Appen's clients and the same to mobile, the top 5 are Appen's clients. So the reason the client, they look at Appen and they select Appen as a core partner from AI data perspective. One, there is a global expertise. They know we are not only have the expertise in China, know China because I was in China, I was born in China. The second part, more important, they know we have a very strong global experience. If the China client want to go to overseas, there is a company to help them, to support them that is Appen China. The second part is Appen have the very full portfolio. We are not only a speech company. We are not only imaging. We cover all. We provide speech, text and image the full portfolio for our client. That is really important. And another thing I want to share with you about our capability is our size. You can look at the Wuxi, Dalian and the Chongqing. This is the site we built in 3 years. Wuxi is in the east of China, then they can cover all the clients in the east area. And the Dalian is in the north of China. It can cover all the North client. Chongqing is in south and they can cover the south part. So it helps our clients. They can come to our office by taking a high-speed train within 2 hours. So we are very closely engaged with our client on this. And the second, I want to share with all these sites, we achieved ISO 9001. We achieved ISO-27001 and 27701. So this means we have a very high management system and high-quality system and privacy protection system in our all sites to help our clients. The last one, very important, why clients select Appen. a, we have very advanced technology in China. And we the #1 player in autonomous vehicle in China. And you see the technology, which is a 2-part help us. One is we really have the AI-powered algorithm embedded in our platform, which can improve the productivity and efficiently in a significant way, right? That is the number one. The second part is we really have this advanced technology, and we deploy -- we can on-premise deploy to our clients' environment to help them to make sure there is high secure way, and we have turned, our clients have been deployed already. The last one, very important, what's the future of China. We will focus on the existing clients, and we will continue to deepen the relationship with the existing clients. They are super and they are growing very fast. The second part is we believe not only the current and top 3 industries, they are using AI, like in the Internet, in the auto, and in mobile, they will express. They will express to education, to manufacture, to finance, to medical, pharma, et cetera, health care. So this is an industry we are [ cooking ] we are working as well. We believe that is the new customers will continue to grow. And the last one, very important is generative AI. That is a shift in the world. And we, in China, we also have a lot of competition are involved in this and Appen is a critical partner in generative AI domain. So I'm very confident for China growth. And thank you very much again.

Thank you. Thank you, Roc. Amazing. Well, you just heard it from our leadership. We wanted to make sure that we close on something that's very important on behalf of the entire organization, right? So I want to tell you again what I started with. Generative AI is going to have a new programming language. It's called Human. We've been doing Human work, large language model work. That's what it's called for the last 28 years. We're not an annotation company. We're a relevance company. We're an assurance company. We're a fine-tuning company for deep-learning and AI. We are going to be calling into customers directly. We're going to build our products. You've seen the kind of CTO, the CPO, our CRO and our new CFO and the broader teams, that's how we're executing now. That is how I've always operated a business. We feel like we need to show you so that you could trust us. I want to use the word trust again. It's been 28 years. Julie was with us yesterday, Julie Vonwiller and our previous Chairman and CEO Chris Vonwiller. We gave them this entire presentation because they couldn't be here today. They were just thrilled. And they said, listen, Julie said, it just feels like there was a house 28 years ago, I built it. And then for 28 years, no one went out and painted the walls. If you've ever lived in a house for 28 years, would you not paint the walls? Like my house, I think we made our kitchen twice in 28 years or maybe 20 years. And there's refreshing that you need. And I think the Board, I'm very thankful. I'd see Robin in the back there, Robin, thank you for joining our Chair of the Audit Committee, Robin Low. Our Board has been refreshed. Our CEO, along with our leadership team, has been refreshed. You can see the energy. We feel like this is the dawn of the new era that we need to work towards. The 2 people who are not here, Brian Haskett leading our operations, and Eric who's here is helping us lead the operations along with Andrea Clayton, my partner, our Chief People Officer and Chief Purpose Officer. I call it, CPPPO because our culture is very important to us. And her and I partner every day. She's told her husband Brent and her 2 kids that, "Oh, I have a new partner, name Armughan, because he's going to call me at the end of the day, driving home." And I always tell her how are our people. I always tell her that we acquired hearts at Appen, we don't acquire parts at Appen. So I don't care if they're before me or after me, they are our hearts and we have to treat them as such. So we know that we had to do a bit of a restructuring, as you know, a couple of weeks ago. And we made sure that we led with purpose. We treated everybody with respect. We made sure that they kept their laptops so that they can find their next job. We made sure that they left with dignity, the right pay, the right -- we did everything as we could. We went out and looked at Stripe and Google and others and how they did that. Purpose to me, character and integrity. I don't come from a lot, by the way. I'm an underprivileged bringing, a bunch of us in our leadership team have something in common. Grit is very important to us. Purpose is #1 for us. We feel purpose. Then perspective, that will lead to prosperity. I know you would like to see prosperity, you would like to see the stock going up. We got it. We feel to get there is purpose first and then perspective and then prosperity and perspective is what you're seeing, right? It's moving really fast. It's making sure that we are learn-it-all organization and not a know-it-all organization. I do not like know-it-all organizations at all. I've worked in many organizations before. As soon as they become know-it-alls or they say, "I know something more than anyone else," that does not work. And I've seen companies fail. We're not that anymore. We're learn-it-all organization. We're accepting that comfort and growth don't coexist. You could see Andrew and his excitement. He's 4.5 days on, super uncomfortable. We're going out and doing proof of concepts with customers with a product that it's moving at the speed of light at this time. And I can't just be -- I can't tell you how thankful I am with this team and how hard they have worked to make sure that we knew that we had committed 4 months ago, Rosalie 4 months ago when we committed to this, right, that we were going to do this. And in 4 months, I told you we're going to get something done, and we worked really hard to get it done. So I hope you're all pleased. I would love to invite now my leadership team we just presented up so that you can ask us questions. If you guys could all come up, please, we just presented. That would be great. So we are happy to -- Ryan, you are as well, please. Thank you. We just move this in the middle. Helen, you can [indiscernible]. All right. Great. So I thought it would be good for you to not just -- you've listened to me multiple times. I've met many of you before. I thought it would be good for all of you to hear directly from the leadership team and ask us any questions you want, we're happy to answer them. Who's the first question? Josh, thank you. Wei Sim is #2 after that. Thank you. Your microphones coming. So that the people on the webcast can then listen in.

Josh Kannourakis from Barrenjoey. First one, you talked about some of the new products that you've brought out, most notably around the LLM enablement layer. Can we talk a little bit about -- a little bit more around the go-to-market strategy there, where the products are at in their life cycle when Appen will be out, what are the milestones that investors should look at to see your progress on actually getting market adoption with some of these things?

Yes, great question. Maybe I'll start and I'll get it over to Andrew after that. I'll tell you, I think for me, if we take a look at how many pipelines -- what's our pipeline number for these 2 deals?

45.

45, right? It's changing every day. Four months ago, when I announced our generative AI products, we did not have that many in our pipeline. We now have 45. When I did the equity raise, which was, I think, last week, we have 32. So that's the number I showed you. And it's changing. That's how fast it's changing. We believe that our deep-learning AI revenue is now starting -- we're starting to see green shoots there. I won't use the word stabilizing. I would use the word green shoots, because that's where our hyperscaler or existing business is. We have moved up multiple notches, as I said in the executive leadership team at our customers. I would also tell you that our #1 customer is now consolidating our #1 competitor's revenue to us, which is great, not just saying that, we've already quoted it. And Brian and Eric's team are already doing a great job executing on it. So I would tell you that's where we're at right now. Obviously, now, as Andrew said, we're creating a go-to-market with it and that go-to-market I will continue to tell you, when I meet you in August, I will tell you what that 45 number would look like from a pipeline, what have we closed on that pipeline. I just need a bit more time to start flying this plane. I was given a 747 heavy leaking fuel. I had one engine working, 1 engine not working. And then we had a Mt. Everest coming in front of us, and we had to do debt refinancing, we had to make sure that we get cost out, get ourselves fit, turn this into an F-16, so move it around rather than pulling up in front of Mt. Everest. So that's sort of where we're at. Makes sense? Anything you would like to add? that's pretty much it. Okay. Yes. Good. All right. That's for you. All right. Thank you. Wei Sim #2 question. And then Darren will be #3. Any other hands coming after? Okay. Great. I don't know your name, but I look forward to meeting you.

Thanks, Armughan. Thanks team. It was a wonderful presentation, and I think it's brought a lot of conviction back into Appen's name. So I think that was really great. My question is just really on the strategy going forward. We have had, as you mentioned, a lot of loss coming through from our largest customer. At the same time, we are seeing a lot of green shoots coming through. We also have probably less headcount now than we did at the start of this year. So with the limited resources that we have, where is our focus at this point in time in terms of go-to-market strategy?

Yes. I shared this in the Investor Day and -- sorry, not Investor Day, I shared it on the equity raise, and I think I've shared it before with other -- and I think I did a call when we did the equity raise, there is a slide that I showed, which we don't have today, but it sort of tells you what our focus areas are, right? Operational rigor is going to be a continued focus for us, cash EBITDA positive. We have to get that done, make sure that we are running, Helen and I come from a similar industry where we run a tight ship. She'd run tight ship on basis points, and it's important to us. So that's one. Number two, product. We need to get these products that you just show -- just saw. Some of those products are maturing. Some products are matured. How do we start tracking those products. So Saty, where are you, Saty. Saty and Sujatha are partnered together very, very closely on getting our product road map to then align to that and ensure that we start delivering on those 3 products that I told you that we need to develop, especially around the fine-tuning and assurance so that we get those out there. That's second. Third is our world-class go-to-market Chief Revenue Officer, Chief Marketing Officer will be announced in the next few weeks. And they're really going to focus on building the brand of trust of Appen out in the market. That will be helpful for us, along with going out and seeing as many customers. What's your target? How many customers do you want to see per week? You notice? It just happened. A lot, a lot. So that's important to us. And then for us, making sure that our business operations aspects are very critical to us, our crowd, how we segment our crowd, how do we do the crowd intake and others. And then finally, our AI for good strategy is very important, and we want to make sure that do good, be good, lead good is at the forefront. So that's what I showed on that call that I provided. I would like that to be my scorecard Wei Sim, and then you should judge us all on that scorecard. And by the way, we just had an off-site here at the BCG offices and for the last 2 days. And we have a full plan exactly who's going to do what. I'm a bit of an Air Force type of person. So I would say fly information, don't fly once here, once here, so we're all flying in formation. That's how we're leaving Sydney to make sure that we get back here, we'll do that. Okay. Thank you. Darren? Anything you want to add to that? Okay.

Darren Leung from Macquarie. Obviously, a very exciting time and plenty -- clearly have been very busy in terms of changing the product and it sounds like you've got a go-to-market strategy over the next sort of 6 to 12 months. My question is twofold. The first one is, I suppose, when we think about that -- the products that enterprises adopt, what do you think about the main hurdles are in terms of when we think about, say, ourselves as a Macquarie Group, an established organization, I find it quite difficult internally getting products on board and I should want to use them as a supplier. How do you go about breaking through those barriers, especially for something that's experimental? And then my second question is even further down the track. I noticed in your slides the equity raising that you mentioned that the NVIDIA partnership or relationship doesn't have any monetization at the moment, which is fair enough. But my question on that is what are the hurdles or what do we need to see before that monetization basically?

Got it. I just want to correct. We do have 2 customers with NVIDIA that we are already monetizing, which is good. Andrew, do you want to take that one on how do we -- what's kind of encouraging strategies into accounts?

Yes. I don't know if this -- Okay. Great. Look, very simply put, this is an unbudgeted item, right, for what we're providing.

The generative AI version.

Correct. Correct. But AI, in general, is a C-level discussion, right, with the Board of Directors, right? So for us, the way to break through is, first of all, right, we have to make sure that people know who we are and what we're doing. But once we do that, it's about selling business outcomes, right? And it's about working with these executives inside of their organizations, right, to deliver those and to partner on that, right? And that enablement layer is so key, right? Because even if you just think about generative, right, OpenAI had been working on this for 6 years, and they have the entire Internet that gets to provide them with their enablement, right? But if you're a large enterprise, you don't have that luxury, right, and that's where our crowd and our segmentation and everything that we're working on is there. So simply put, it's working on those outcomes, and we've seen examples of that already.

Okay. Thank you. Next question. There is a question back there, yes.

I am Ross Barrows from Wilsons Advisory. Just had a couple of questions. One is, I guess, Appen can't be all things to all people. Could you talk about Anthropic in some way? Because my understanding is that you have constitutions or principles that doesn't involve humans in the loop or human contributions to that. So maybe can you talk about those clients that will use you? There will be some clients some -- some color around that, please?