DoubleVerify Holdings, Inc. (DV) Earnings Call Transcript & Summary

[Audio Gap] to find reconciliations to our non-GAAP financial measures. And with that, let me just wish you a very good afternoon, and I hope you find today's event very useful. Thank you.

Up next, DoubleVerify's CEO, Mark Zagorski.

I'm going to read the entire disclaimer to you all now because I know you want to hear it. No, there it is. Look, read it really fast, memorize it, you're analysts, you can do this stuff. Thank you again, as Tejal noted for spending a gorgeous afternoon in an entirely dark room with us. Over the next several hours, we hope to enlighten you a little bit about not just how AI is changing the world -- in the digital media world, but how it's going to change DV and why that matters to you, to the folks that rely on need for information and to the folks that give you money to invest in us. So we'll cover a lot today. And as Tejal noted, just a quick -- a few quick thanks, obviously, to you for joining to the team that put this together. So Annie and the marketing team; Brady, who's the voice of God. You'll hear her above giving orders, running around and making things work; and then a special thanks, and I've heard this from all of you before, to Tejal and Ryan, the best IR team in the business hands down, amazing team. So with that, today is going to be an exciting day, and I said, we'll cover lots of things. And what I'm going to cover here in this very dramatic sounding presentations, the perils and profit in the age of AI because there are both, and I'm sure you've heard about all of them. Before we start there, though, we do have some announcements that we're really excited to share. And the headline is perfect. We keep growing. First thing we'd love to announce, you saw a press release this morning, the DV has launched an industry-first MFA solution. If you've been reading anything about digital media lately, you've heard about MFA, made for advertising content. We've launched a solution to enable our advertisers to avoid MFA content and to put that into the mix in the way that they choose. It's a big moment for us. We're excited about the launch, and you can read more about it and you'll hear more about it later today. Secondly, we're announcing expansion -- additional expansion into additional languages across TikTok. This is a direct outcome of our investments in AI. Jack Smith, our Chief Product Officer, is going to talk about how our classification, our ability to classify content has been accelerated by using AI tools. So we're launching more languages and announcing TikTok -- additional languages for TikTok today. Third, and this is real stuff. We've got some great new deals that we're announcing today. Some brands that you all know, big advertisers and huge wins. First off, Discover -- Discover card, got their business, nailed it. We're announcing it today. NFL, just in time for kickoff, NFL has become a DV client. Rolex, which I'm sure many of you are wearing right now because you're very well paid financial folks, they become a client. And then a really big one GM, and GM has been in the news a lot. We know that they're having some challenges with labor but they're a huge advertiser. Winning these businesses are great indications of how we continue to lead in the space and how we keep growing. And then finally, why we're all here today. Today, we're formally announcing that Scibids has closed. We've closed the deal on Scibids. You'll hear from the founders, the CEO and the COO of Scibids today. We're officially announcing that the deal is done, deal is closed, super excited about how that's going to change our business. Quick check in and some housekeeping on the agenda. Think of this day or this afternoon being split into 2 sections. The first is going to be all about what I just noted, Scibids. We're going to talk to the founders. We're going to see how it impacts our clients and the benefits they're getting out of it. And most importantly, why it's going to differentiate us and how it's going to change the way we approach our clients and how it's going to create growth opportunities for us. We'll take a little break. And then the afternoon, we're going to talk about broader impact of AI on our business, on our clients' activities. And you're going to hear not just from us -- and I promise, you've all heard from me and Nicola ad nauseam. Today is for really smart people to get up here, our CTO, our CPO, our clients, our partners. So folks like Trade Desk and Diageo, our agency folks like friends from Dentsu and industry experts from the ANA who are dealing with the challenges that AI-driven content are creating every day in the marketplace. Finally, we will get you outside, the folks that are here in person. It's a wonderful afternoon. We're going to have a cocktail hours here. We know sometimes you don't get enough time to get out and spend some time, especially during the earnings season. It's a gorgeous afternoon. We'll have snack we'll have food and drinks after we're done here today. So hopefully, you can all join us there. So with that, let's get rolling. When we think about AI and this idea of parallel profit, there's -- what's going on in the marketplace, which I want to cover today, just to get a sense of how this is changing the way advertisers work. There's -- our approach and how it's changing, we're approaching that market. But with all challenges, there's always an opportunity. And I think that's the most exciting thing that hopefully you'll take away from today, which is this is a massive opportunity that AI is creating for us. It creates challenges, but with every challenge, there's an opportunity. And we think that opportunity results in better ad investments for our customers, better return for our tech investments for DV and hopefully a better return to all of our stakeholders and shareholders as well over time. So let's start with the market. There's tons of challenges. I've been in this space for 25 years, more than probably almost anybody else in the space. I started in digital marketing at the home of the first Internet banner ad, which we ran for AT&T, had a 44% click-through rate -- amazing, amazing banner, if you remember, I don't know if anybody were around. It was 1996. It was an ad for AT&T. It was at a 44% click-through rate. It just said, do you want to learn more about AT&T, click here, people did. Now only if digital marketing was that simple today, the click-through rate is 0.05% today for ads, right, which means that not only do advertisers have to navigate a very crowded universe, but one in which advertisers have become -- consumers have become totally immune to ads. This marketplace, though, is now being even more challenging for advertisers than the world of cluttered ads. It's a world in which things like content quality, fraud and scale and complexity are only be exacerbated by AI. How is that happening? Content quality. Let's talk a little bit about MFA. You've heard a lot about it. MFA is nothing new, made for advertising websites. You know those websites where it's like, what do the Brady kids look like today? You've all clicked on them. We know you have and you've gone down the rabbit hole of 85 slides of Jan, Brady in her prime. Now that is -- that's one example. And these have been around for a while. But now think of the content creation across MFA, not just being a factory, but a factory that's run on steroids. Whereas sites used to be in the hundreds, they're now in the tens of thousands. From dozens of articles a day being published by sites to thousands per day being driven by sites. Think of that. A single website creating thousands of articles a day of junk content for the most part. There's been estimates, and there's been researchers who have actually come out and said, by 2030, 99% of all text-based content, digital content on the Internet will be AI generated. Think about that. That's not that far away. And you see this all in your world -- as people in the finance world. There are people who are scraping your hard work, pumping that into financial articles that exist out there today. It happens all the time. So MFA content is proliferating. And there was a recent study that just came out a month or so ago that showed that 19% of the programmatic bid stream -- so 19% of the bids that go through places like the Trade Desk or Google DV360, are being bid on MFA type content. It's a massive increase. And that's up from 7% in 2020 in just 2 years. It almost tripled in 2 years. The reason why because the content is proliferating. So there's a huge amount of MFA content out there. Advertisers are concerned about it. It's not all bad, but it's not great either. It's creating complexity. What else is AI doing to the content quality. Well, inflammatory news and disinformation. There's not only more of it out there, but it's being driven by a bigger microphone -- a bigger megaphone. Last year, we saw during election season, a 25% to 30% increase in incendiary content and a lot of that content is being generated by bots and AI that can perpetuate over time and even worse for advertisers and kind of for humanity too. Mis- and disinformation generated by AI is more convincing. Recent study to show that it's several percentage points more convincing when tweets were created by bots than when they were created by actual human beings. So AI is actually getting better at convincing people of things than we are as humans, which is pretty scary. You add deep sakes, synthetic news and lots of other types of content that's being created by AI, and now this is becoming really challenging for advertisers as well. So content quality being attacked by AI. What else? Fraud. Fraud is something that, again, is nothing new to the digital ad ecosystem. We've been battling it for over a decade. But the type of fraud and the scale of fraud that we're seeing just in the last year being accelerated by AI is pretty incredible. You've seen twice as many individual mobile app schemes year-to-date than we've ever seen before. We've seen 137% more audio fraud schemes year-to-date and 92% more CTV fraud schemes year-to-date. We actually had to name one cyclone bot because of the veracity and speed at which the fraud scheme that was being delivered against servers was insane. A lot of what this has to do with AI getting really good at imitating devices, looking like something that it shouldn't be, looking like a connected television set, looking like a valid mobile device. So AI is getting really smart and is creating even more challenges for advertisers. And then finally, I talked about the amount of MFA content that's going out there. But if you think about it, all content, even if it's okay, is being accelerated by the fact that it's easier to generate content than ever before, especially UGC content. Short-form video is eating the ad world right now. If any of you had a chance to go to the Meta Upfront, which was a great event, wonderful event, 90% of what they talked about was short-form video and reels. You would think that the Metaverse didn't exist at that upfront. Why? Because this is what consumers are engaging with. Every minute, there's 167 million hours of TikTok videos being streamed. Think of that. Every day, there's 34 million videos uploaded to TikTok. It's massive. And this content is not only at massive scale, which advertisers have to figure out what works for them. But it's pretty complex from a brand safety and suitability perspective. Think about it. When you're looking at a short-form TikTok video, you're looking at not just video, like text that goes over it. Images, audio, all of that makes the analysis and the determination of whether that's brand safe or brand suitable, really complex. All right. Have I scared you enough yet? If you're willing to take the iPads out of the hands of your children's because of this terrible universe that AI is creating online? Well, there is hope. And as you can tell, these are AI-generated images. But the good thing to note here is, actually, can anyone see what's wrong with this image. His eyes are weird. He also has no legs, which is just a bit a slight miss, but the interesting part about this is, remember, AI is only as good as the person who's training it. And you're going to hear about that from Jack and his team and our technology team that AI is a great tool, but you have to have people behind it. I unfortunately was the person behind generating this image. And my prompts weren't that great. And as you can tell, the eyes are a little screwy and the sword looks like it's out of ex caliber. It's kind of weird, but look, AI-generated images are only, only one thing that we're dealing with. So how do we deal with it? What is the hope? Well, this summer, I went to Sicily and I got to hike Mount Etna, which was amazing. And I heard lots of stories. We had a local guide, he told stories. First off, my wife was freaked out. She was like, "This is an active volcano, we're hiking on it." I'm like, "Don't worry about it. It hasn't really erupted in 20 years." 2 days later erupted. So always listen to your wife. That's just rule #1. But secondly, what I learned was volcanic eruptions, particularly like Etna type eruptions aren't as scary as they used to be. For a simple reason because they told the story that this was where the lava [ winched ] this town and it killed all the people and the lava is everywhere. The reason why it's not anymore is because they use explosives now to stop the Lava flow, right? So that's a really long way of saying, the way we address and the way DV addresses these challenges for advertisers that AI is creating is using AI to address them. So we fight fire with fire. And what is that ending up with. We attack these issues, and when we do so, we're leveraging tools that will make us faster, smarter and more efficient. Let me talk a little bit about each one of these things. And I'm going to touch on them really briefly, but for the rest of the afternoon, you're going to hear about these all in more detail. So first off, I mentioned content classification and our ability to move faster, for example, in analyzing TikTok videos. The ability for us to develop models is now 3x faster with AI tools than it has been in the past. So our ability to create classification models has been accelerated by 300% just in the last year. And those models are 20% to 40% more accurate. So they actually flag brand safety violations or brand suitability violation is more accurate than any models we've created in the past. And because of the speed and the effectiveness, we've been able to do things like analyze massive amounts of video at 20x cost savings than we had before. Why? Because when we look at the frame by frame of video, we're able to not have to see every frame, but use predictive AI to determine what's going to happen next. That cuts our cost because it takes computing power and computing cost to analyze video. As I just mentioned, as the world moves more towards short-form video, our ability to analyze that and do so effectively and cost effectively is critically important. So we're moving faster in areas like content classification, and we're getting smarter every day. Why we're here today, a big reason why we're here today is to talk about Scibids and what that means for us. There's been lots of questions about that. To say that we're enthusiastic about the acquisition is an understatement, and you'll hear that from the speakers today. But why this makes so much sense for DV? Is it's perpetuation and evolution of what we've done since day one. On the Prebid side, to take garbage out of the system and leave what's there better. If you think about how we started that process, it was with our standard segments, standard brand safety, standard viewability, which basically gave advertisers a binary decision. Is this good or bad? Is this viewable or not viewable? It worked, but it was the meat clever aspect. And it still works for some advertisers, particularly around areas like fraud and safety where it's truly binary and there is no -- there's only black and white. We adopted that to a variable model with ABS. Now Hopefully, you're familiar with ABS. ABS is authentic brand suitability. It is our most powerful fastest-growing product that truly differentiates us. Our competitors don't have anything like it. The market loves it. It grew over 50% last quarter. And it's because it puts more power in the hands of advertisers to have variable criteria for determining brand suitability, right? We launched that product in 2018. It continues to grow and continues to grow with brands. The challenge with that, however, is it still creates a binary decision? Do I buy or not buy based on this variable, suitability? What's so exciting about Scibids is it opens up our data to an entirely new application, which moves from binary to a spectrum and allows us to determine what that decision is based on multiple variables. Things like price, things like a KPI on effectiveness on the other side. So think of taking a data point like viewability and rather than saying non-viewable or viewable, saying, I'm willing to give up some level of viewability because viewability is a definition. If it's cheap enough, and if it actually drives a KPI on the other end. So it has the ability to do that for not just things like viewability, but areas like context, areas like attention, newer areas where we're going into a data set. So it increases the applicability of our data. So if you think about, it's no longer an either/or. It's in either and, right? We are able to do all of these things to not only clean out the system but make that system perform better. So we talk about this growth and protection and performance. It's really both of those things. And when we think about how we're getting smarter and how Scibids is going to help us in this activation and evolution has occurred, it's allowed us to drive value and also volume, right? It allows us to differentiate our offering, create more sticky value prop for our clients because we're creating more value with those data sets that we've created. But it also provides a broader opportunity to leverage data across more impressions and open up an entirely new customer set of performance-driven advertisers. Again, we're going to talk a lot about that over the next several hours, and you'll hear more about how we're doing it, customers we're dealing with today and what that could mean for the future financial position for the company. Finally, the part of the AI evolution and revolution, like it's a little less exciting from the outward perspective, but it's super exciting for us inward. The idea of being able to not only create client efficiencies, but operating efficiencies as well. So thinking about easing and we will be launching conversational AI tools for client support that means as we grow, as we grow globally with many, many customers, we don't need to have as many client service people to support them because we've got tools that are going to help. Also, our ability to create code has been accelerated. We use tools like Copilot and our engineering team now to actually help them rate code faster. And what we've seen so far is about 80% of the code that's generated using prompts that our teams are using is usable right out of the gate. So think about that. If you could increase your productivity by 80%, how amazing would that be, right? We're seeing those types of results by using these tools. So we're delivering client and operating efficiencies driven by AI and you'll hear more about those today, too. So where do we land? We've talked about a complex hazardous and challenging ecosystem -- an advertising ecosystem, which has never been more fraught with challenges for advertisers. But we know that transparent independent AI solutions can solve those challenges. But ultimately, it comes down to outcomes, comes down to outcomes for our clients, and it comes down to outcomes for DV. What are those outcomes looking like today? Well, we know through the AI tools that we've launched, we've been able to drive stronger attention. As a matter of fact, 63% greater attention using our algorithmic optimizer and our attention data together for leading Fortune 500 clients. We've been able to decrease brand safety incidents using our Prebid tools for advertisers by 48% on average when they use our Prebid tools and those Prebid tools and models are getting better every day. So the increase -- the decrease in brand safety instances is growing. And finally, using Prebid and now our AI-generated fraud detection tools, we've been able to reduce fraud and IBT incidence for customers using our solutions by 79% over the last year. These are real results that are helping our advertisers become stronger, safer and more secure, which has been our mantra since day one. We're doing it better, and we're doing it faster with AI. So finally, what does that mean for you? What does that mean for your investors? What does that mean for us as a company? Well, our investments in AI are providing more value and driving more volume, which provide opportunities for growth. If we become more valuable to our customers, we drive more volume, that drives revenue for us. We're able to move faster and more efficiently in developing tools and getting the product to market, like we've talked about with our TikTok language coverage. And we're able to -- when we do those things to solidify the essential nature of our business with those customers to not only protect and perform -- but to perform as well. So I'm going to leave you with this. When you think of DV and you think of what AI is delivering, it's the ability for advertisers to not only protect their spend, but increasingly and particularly driven by tools like Scibids and our algorithmic optimizers to help that spend perform even better. It's been an incredible journey so far in this AI world that's been filled with tons of opportunity. We couldn't be more excited to jump in and take advantage of it and have you all go along with us for that ride. So with that, I'll close my beginning and bring out the real folks who do the real work here, starting off with Doug Campbell, our Chief Strategy Officer. Thanks, Doug.

Up next, Doug Campbell, DoubleVerify's Chief Strategy Officer.

All right. Hi, everybody. So I should start probably by explaining this, just in case anybody is curious. And don't feel too badly for me. I broke my ankle basically whitewater rafting in North Carolina about 3 weeks ago and had surgery. So that's why I'm going to hobble up and down here and you're going to see me 2 times because I'm going to get the -- or have the privilege of introducing Scibids' founders. And then I'll come back up and talk a little bit more about where DV fits with Scibids. So we'll kind of start there. I'm going to start out though with just a little bit of context or exactly where the idea came from and talk a little bit about where it fits into our strategy. So about 1.5 years ago, our product team came to me and said, "We have an intriguing proposition for you." They found a way to activate our attention data which is one of our clients' biggest asks in a very elegant way. And they told me that it would basically increase the value of our data, decrease the costs that our clients paid and increase the KPIs or the key performance indicators for our clients. And I said that sounds amazing, let's do it. And they came back to me and said, we can't do it because we don't own the company. It was actually given to us through a small, but brilliant team in Paris, France, and they are called Scibids. So we said, that sounds great, let's go explore that, and that is what started the path the whole way to hear today. And what we're going to demonstrate is exactly what they do, how they do it and why it's important. As part of that, I will start by explaining where they fit into DV. In some ways or hopefully in all ways, you all recognize the circle in the middle. This is the core value proposition that DV offers its customers. We have a very large measurement footprint and then we take that measurement data and we activate it for our clients. Now to date, we have 2 ways that we activate. We have static segments, which are inside -- sit inside of a DSP or demand-side platform. And essentially, that's -- an example of that would be fraud, where we take a set of data, we put it into the DSP and then our clients come along and activate against that segment and essentially for fraud as an example, they say, "Is that fraudulent? If it is, we don't bid. And if it's not, we do bid." With our dynamic segments, oh, it's even better because we have the ability to have our clients make decisions or dial knobs on the data that we put into a segment. So an example of that would be authentic brand safety where we have the ability to have our clients make decisions about, do I want to be on violent content? Do I want to be in gambling content. And so they get to make those choices and then we take that custom data set, we send it to a DSP, and then they can action against that. Now, we have Scibids and Scibids allows us to do full algorithmic activation and what that means effectively is that we can take our data, we send it into Scibids, who layers on cost data and any other third-party data that a client wants and then sends that into the DSP. And so as you can see, effectively, what we've done is we've just increased the number of activation points that we have along our strategy. So nothing has really changed, we've really just augmented our strategy in the ways that we activate our data. And the reason why that's so important is because it drives more customer value. And that customer value then drives investor value, which is why you all are here. And it helps us just be much more efficient when we think about how we offer our solutions to our customers. And I'm going to talk a little bit more about that. But before we do, let's get to the real stars of the show today. With that, I'll hand it off to Remi and Julien, the founders of Scibids.

Co-founders of Scibids, Remi Lemonnier and Julien Hirth.

Okay. So I'm Remi Lemonnier, President of Scibids and [indiscernible] very excited to be here today. As you might expect from co-founder of a successful global startup, we are both speaking perfectly English. But nonetheless, today with a hint of French accent just because deep down, we believe that no English should be spoken. So we'll start with showing you a short video that will explain that we introduce you to Scibids and explain how our AI works. [Presentation]

[Technical Difficulty] AI works and now we are going to dig a little deeper. So 7 years ago, we founded Scibids to solve a key problem for advertisers, how to optimize for unique ad performance. And in the context, we are actually poetic advertising, you have to take billions of decisions every day. And how to do that while leveraging unique business intelligence and while also very importantly, regarding consumer privacy. Now, we all agree that the goal of advertising is to generate a desired business outcome for the advertiser. But every advertiser and every single advertising campaign serves for a different business outcome and is trying to leverage unique business intelligence. So let me give you a few examples of that. A digital only e-commerce retailer might be looking to, for instance, optimize for the sales of high profitability product, but low turnover and might be trying to do that by factoring in the margin generated with each product. A nationwide CPG may be trying to maximize their offline sales by putting in share of shelf, in-store revenue [indiscernible] or point-of-sale data, whereas an insurance advertiser might be looking to maximize the lifetime value of our customers by supplying user level scores based on the characteristics of the very best customers. In fact, and this is very important. It is this diversity of objectives of approaches and other data that actually allow advertisers to create a competitive advantage. On the contrary, imagine Coke and Pepsi were using the same DSPs and optimizing to the same standard KPIs. They would be buying the same second and third-party data. They will be competing on the same inventory, going head-to-head on every impression competing solely on price, and this would be suboptimal for the both of them. And actually, that's why DSP is recognized that in order to allow advertisers to create a competitive advantage, in order to allow them to approve their unique business intelligence, they might need to open their bidding APIs and to allow for more than one [indiscernible] bidding algorithm. And that's why the leading DSPs and the most advanced DSPs have actually opened their bidding APIs. And they have created capabilities that they call bring your own algorithm or BYOA, that allow advertisers to actually upload customizable AI and billing engine and also the advertiser unique business intelligence in order to inform the bidding process.

So [indiscernible] Media sums this up nicely in a report titled how to train your DSP that I encourage you to it, by the way. This report is stating that you DSP does what you tell it to do. So tell it what you wanted to do. This is brilliant. I love it. I would even add and feed it with the data assets that differentiate yourself. So the point here is that despite the sophisticated bid optimization offered by DSPs to advertisers, there is still a large opportunity for advertisers to significantly improve the return of their campaigns by: Number one, customizing the outcomes they want to generate; and number two, customizing the data inputs to inform inflation valuation. And guess what, they can do both seamlessly using Scibids AI. So how to define Scibids AI. In a nutshell, Scibids AI is a technology which is trained to understand the real-world business outcomes of our brand and take the billions of daily bidding decisions accordingly. It does so by activating the advertiser data assets through custom building scripts, which are then executed by the DSP in the DSP building infrastructure. So to be crystal clear, the media buying process is still 100% executed by the DSP. It's just that now the DSP is informed by Scibids' independent inflation valuation and bidding role. Having said that, I want to highlight something which is important. This value provided by Scibids to both advertisers and DSP is delivered without the usage of any third party cookie or PII for profiling or targeting purpose. We all know how important this is in our industry nowadays. So let's move and talk about results. Once deployed, campaigns using Scibids AI routinely deliver more than 40% improvement in achieving the actual business outcomes compared to a twin campaign, but running without Scibids AI to inform the DSP. 40% improvement, this means millions of incremental revenue for large brands, but I also want to highlight that it's also very good news for the DSP and the entire supply chain actuary. Indeed, over the years, we've observed that such an improvement leads on average into a 10% increase in ad spend, 40% improvement in KPI, 10% increase in ad spend. What does that mean for you? This means that the more Scibids AI will be deployed across the DV's customer base, the more the ad spend and the more the impressions, which of course, will benefit DV's activation business, which will also benefit DV's measurement business. So it's time now to move from COE to practice. And at this point, I wanted to share with you a few examples of both stored out and custom KPIs in order to really grasp this important notion of custom KPI, which is really parament when dealing about Scibids and custom bidding in general. So here, you can see a few typical examples of standard KPIs available in the DSP for optimization purpose. They all make a ton of sense from both a performance and branding perspective. The thing here, however, that's taking alone, these KPIs don't always fully tell the full story. And that's why, increasingly, sophisticated advertisers tend to blend this KPI together into custom KPI. So let's see an example. Here is an example of a custom formula, which is a Quality CPM or qCPM. This is a custom KPI frequently used by CPGs and brand advertisers in order to better approximate the brand impact of advertising. The takeaway here is that there is no button in the DSP to directly optimize towards such a formula. Direct Response advertisers also rely on customer KPIs. They are often a bit different. They are often deeper in the funnel or less easily captured by the DSP. So with this background in mind, let's now watch a demo of Scibids platform and how it works in practice. [Presentation]

Let's now look at a few use cases where Scibids AI was applied. So the first is Spotify. Spotify was trying to generate incremental in-app registrations as measured by their external source of Truist patches. So this is a very common setup for app editors because they are using so many marketing channels. They want to make sure they don't pay twice for the same conversion. So they use like deduplication partners like Adjust. And what it's meant that actually the KPI was not measured directly inside the DSP, DV360 and the KPI was also computing using custom attribution. So when applying Scibids AI to this company, the 20% increase in In-app registrations, while also decreasing the CPC by 60%. Another very interesting use case is Colgate. So as a lot of other CPG advertisers, Colgate is measuring the effectiveness of their media buying through a well-tuned qCPM, quality CPM like Julien explained earlier. So this is a fairly typical use case for Scibids. However, in this case, we had an [indiscernible]. This is the first time we actually partnered with TV, and we use their authentic attention data in order to inform even the bidding process in order for the bidding engine to be given more accurate. And this resulted in a 52% increase in quality CPM, so this means that actually Colgate Media was working more than 2x harder with Scibids then without Scibids. And this also reduced to a 28% decrease in attention. And last, but not least, POKERSTARS. So POKERSTARS is launching hundreds of YouTube companies every year. And in '22, they decided to scale Scibids across their YouTube operations. And what they witnessed was there was 66% decrease in CPM. So this means on the flat budget, 3x more impression to buy and measure. But more importantly, they generated $2.5 million of Media Efficiencies and when they completed the return of our investment of Scibids, the competitor for each dollar invested in Scibids they generated $5.33 of income to revenue, $5.33. So just to summarize in a few words what brands can expect when [indiscernible] with Scibids AI. It will optimize to custom KPIs that actually correlate to their true business shortcuts. Second, that it will activate the leakage at the top of unexploited data that is so important for company performance that usually see dormant and does not affect the inclusion variation process. And third, that it will automate the workflows. It will lessen the burden on traders and company operators and give them more time to focus on highly strategic task instead of doing manual and tactical complicated optimization like, for instance, pulling reports from DSPs. But I want to be very clear. All that is possible only because we are standing on the shoulder of the giants. We are using the most advanced APIs that DSP has built for the exact purpose of helping advertisers succeed on their platform. So this really is a symbiotic relationship. And if you think about it, it's not very different from the dynamic that happened when AI [indiscernible] durability or bond safety, for instance. It's not very surprising. But if you take -- the most challenging problems that our company to offer, like, for instance, helping advertisers generate field outcomes or helping them -- preventing them from spending on harmful content, it takes a combination of 3 great companies and their partnership, to brand, to DSP and independent domain experts like DV or Scibids. And now to elaborate on that, we are very pleased to welcome on stage, Cory Grever, as the Senior Director of Global Trading Strategy at The Trade Desk.

Cory Grever, Senior Director, Global Trading Strategy at The Trade Desk.

Welcome to be here. We are delighted to have you on stage with us today. In order to warm it, could you please tell us a bit more about yourself and your role at Trade Desk.

My name is Cory Grever. I'm a Senior Director of Trading Strategy over at The Trade Desk. I've been at The Trade Desk for over 7 years, and prior to that, I actually worked in finance so worked on the investment management side over at JPMorgan. So I understand your walls as well as this side of the table. When I started out my career, I was sitting really in working with the day-to-day traders and teaching traders at agencies and at brands, how to actually use the Trade Desk platform. This is everything from industry education, who is DoubleVerify? How do verification metrics works. To what levers can you pull on our platform to drive performance, look at optimization, reporting, you name it. I now sit on our newly created trading strategy team, and this is where we partner with our, I'd say, top 200 clients at The Trade Desk. So we are a strategic overlay and really help with joint business plans. So those are kind of executive and senior level engagement plans that we have with our top clients. And then also really focus on any sort of custom integration or solution, whether that's something that's felt within The Trade Desk walls or something where we are partnering with someone like Scibids or DoubleVerify on a custom solution for our clients.

Thanks very much, Cory Can you maybe explain to the audience why you are allowing advertisers to bring their own algorithm into Trade Desk? And what do you think of a custom AI company like Scibids.

Yes. Great question. So everyone in this audience likely knows The Trade Desk and what we stand for. So we were founded on a few core pillars of objectivity, independence, transparency and openness within the ecosystem, which is why we can even sit here up on stage and talk about our partnership. We've recently announced our new platform called Kokai, which is coming out this fall for a few people who might also be following The Trade Desk, which really means open for business. So this is a concept where we are trying to work with our partners, whether those are advertisers and agencies on more of that brand side for us as well as ecosystem partners such as Scibids and DoubleVerify and the rest of the industry on making it easier to integrate with The Trade Desk. So a lot of that is really kind of how do we then think about applying that and working with Scibids as well as our clients given this kind of new ease of integration, a lot of clients are bringing us new business problems such as how do I optimize towards a custom KPI. With this new increased level on access of integration with the Trade Desk, we're able to help them with new business challenges such as custom KPIs or whatever business goal might be.

That's awesome. Did you see the course of own observations about the need of openness and customization for ad advertisers.

In order to make crystal clear, the complementarity between on one hand, Trade Desk powerful off-the-shelf solutions. And on the other hand, your API offering for customization purpose. Would you mind illustrating with a few concrete examples or success story you may have in mind?

Yes. That's a great question. The Trade Desk, we have a number of algorithms that are powering performance and KPI metrics within our platform. So why do we need to bring out another algorithm on top of this? Really, when we think about how The Trade Desk is optimizing campaigns, it's really focused on the data that we currently have access to or visibility into. So cost impression data oftentimes sales data from a client. However, a lot of times, there are clients who have additional custom needs or custom KPIs where there's an external data set that The Trade Desk doesn't necessarily have access to at that time. And so when a client has that goal of saying, "I want to optimize towards a data set that Trade Desk has as well as an outside data set that could be attention metrics from DoubleVerify or any other sales-related metrics at Scibids might be receiving. An additional algorithm can connect those 2 different data pools, build a model on top of that and then activate it back in The Trade Desk. So that's where we're really seeing those clients who have sophisticated goals who have data sets that wave in The Trade Desk or a DSP live outside, how to bring it all together.

That's very illustrative. Thanks, Cory. You mentioned earlier that you started your career very close to campaign optimization. Could you please explain in simple words, how does Scibids help automating the deteriorative process of optimization.

Yes. So The Trade Desk and Scibids, we launched our partnership, I believe, around 2019, but kind of going back before that, starting as a hands-on keyboard trader, you were spending a lot of your day pulling excel reports, looking for insights, figuring out how to optimize the campaign and really in a manual way. Of course, the algorithm in the industry has come a long way where a lot of that is automated now. However, if you're still trying to drive performance to that KPI that isn't just in The Trade Desk, that custom KPI we've been talking about, you really need that automation. And so what Scibids really allows is it gives a lot of time back for those day-to-day traders, whether the agency teams, at the brand teams so that they can focus on a lot more meaningful work and more strategic work. So instead of standing through an Excel report and looking on ways to optimize the campaign on removing sites from a campaign to drive performance. They can focus on how do we make our media plan stronger for next quarter? What were the insights that we pulled out of this campaign that we can apply to future iterations of campaign. So it's really about giving that time back and the additional benefit is with these new custom algorithms, we are hitting performance goals off the bat. So you have that ability to know that your campaigns are performing at or better than benchmark, and you are giving additional time. So that's really exciting. There's an added layer especially when thinking about The Trade Desk, where we have our entire team that is working to train agency teams, brand teams, how to use our platform, a lot of those office hours and those day-to-day meetings that we're having is how can we look into a campaign and better optimize that campaign. Since a lot of this optimization is happening at really that platform and algorithm level, we are getting time back at The Trade Desk as well to go think about how can we help our clients up-level their skill sets, up level and think about how can we add in other metrics to this custom algorithm? How can we further advance the custom algorithm opposed to how can we get a campaign to spend in full?

That's very insightful. And maybe to another forward-looking question. Can you tell us a bit more about how The Trade Desk and Scibids are working in concert to create advantages for Trade Desk and its customer for the next few years?

Yes. Great question. So just building off of what we've been talking about, I think the biggest takeaway is brands are able to and agencies are able to really easily hit some of these performance goals which ultimately is going to drive business results. A lot of the custom KPIs that we're talking about today and want to talk about in the future are really related to sales. So as we are helping brands drive the overall growth of their company, that can help protect their marketing budgets and also grow those marketing budgets. When they see that the performance that they're driving for the sale of their product or service, whatever it might be, is happening in The Trade Desk with Scibids, they will invest more into these technologies to ultimately help further advance our business. So I really see it as kind of the 1 plus 1 is 3 situation here where we are ultimately driving innovation, we're driving growth and everyone in this room is helping to drive increased ad spend on our products.

That's a win-win-win relationship.

Exactly.

Thanks a lot, Cory, we're delighted to have you on stage.

Great. Well, congratulations on the acquisition, and great to see everyone today.

Please welcome back, Doug Campbell.

All right, I'm somewhat awkwardly back. Okay, recap. So I know that we've just offered like a ton of information and a lot of it is complicated. We will be around afterwards to ask questions about all of that, so feel free to. But essentially, we went through effectively exactly what Scibids is, how they work and then why it works within The Trade Desk. And thank you, Cory, did a great job. But how it works inside of all of the DSPs and so now I wanted to talk just a little bit about our strategic rationale and why we think that Scibids plus DV makes an incredibly good combination. I'm going to be sure not to get too much feedback, so I'm going to stand away. So there are 4 reasons. The first reason is volume. It enables direct action on DV data, and we've talked a little bit about this. Two, value. It actually increases the value of our granular data, and I'll get more into each one of these. Three, velocity. You heard already how it increases or decreases the time that traders have to spend thinking about optimizing their campaigns. And then four, vision. We acquired the DNA of a very brilliant team that is deeply steeped not only in AI, but also inside of the trading systems and the ecosystem in which bidding occurs. So we're going to go through each one of these quickly. So first of all, we have something that is essentially a set of actionable data and we action that data in 3 different ways, and I mentioned this in my first introduction, we had static segments, which are very, very good for things like fraud or brand safety binary decisions. The second is that we have dynamic segments. Again, that's when our client can move some levers and dials in put together a custom segment that we then send to a DSP. And then last, best for performance data. This is where we take our individual, our granular level data and send it directly into an algorithm that then gets put into a DSP. And so that direct actionability is something that is very important to our clients and is incredibly important to us because we have a ton of data, we have incredibly detailed granular measurement data that can be made more effective by sending it directly through models and then into the DSP. So I want to talk just a little bit about our -- how that granular level of data becomes more valuable. And again, I move away. And I'm going to use attention as our first example. So we have essentially our authentic attentions data, which is our measurement data, and it's comprised of 50 points, 50 different data points. And then we -- and those data points are essentially bisected between exposure and engagement. So as an example, did you see an ad. So that's exposure. How many other ads were on the page. We do a lot of analytics around exactly what happens with that ad placement. Secondly, we have engagement. So did a person engage with that ad in some way, did they mouse over it? Did they turn their phone, did they somehow turn up the volume. We understand all these things, all these data points, and we bring that all together into an index and then we provide that measurement data to our clients. And our clients, of course, say that's great, but we want to make it actionable. So the way that we make it actionable is we created a universal attention segment. And that segment is the -- essentially the aggregation of the 50 points of data. And what we do is we look at the least performance with the least attentive data set, and we essentially filter that out for our clients. But now what we can do is we can take both all 50 -- both the attention data and all 50 points of that data and the log level data, the cost data, the KPI data, the outcomes data and bring that all into a model and optimize that model across all those different multivariate fields. And that is effectively what you just heard about and what Scibids does. And that's how our data becomes more valuable. It goes from aggregation to being able to use it very, very precisely. We have another data set that this will be used in contextual. And I think many people will recognize this. We have essentially a truck buyer segment, which sits inside of a great universe of inventory and what typically you would do is you would go into a DSP and you would action against that segment. But now with algorithmic activation, we can take that segment plus lots of other segments, all around, say, truck buying or repairs, and we can find the precise and best inventory within each one of those segments and send that into a DSP for actionability. And the best thing is that because it's a machine and it iterates and learns over time, it doesn't just have to be truck segments -- it can find it in lots of different segments that might or might not have been intuitive to say a trader or anyone for that matter. We're looking for signal within all of this great bigger, larger set of data. And that's what we bring to the table. And that's why we think this can be very valuable. Okay. Number three. Workflow automation, which is really just a fancy way of saying we make things easier and we make it less costful or we lower the cost. And you'll recognize that we have agencies and advertisers who have traders that sit on trading desks, those traders schedule ad campaigns, they choose data sets, they choose KPI optimizations and then push it into the DSP. And as you can see, data segments sit at the top and they come down and then the DSP buys inventory on publisher sites. Now if you look at what we can do, we can directly inject optimization data into the bottom part. And it will allow the trader to spend less time thinking about what they need to do on a daily basis and let the algorithms essentially accomplished what they were doing before. So this is a really significant change. And you'll see kind of when I talk about at the end, the increasing complexity of trading is going to be offset by the AI that we create over periods of time, and this is just the beginning of it. All right. And then number four, we acquired an amazing team. You all saw some of that today. We have not only like an incredible set of founders and leaders, but we have a very diverse group of data scientists and engineers that will not only help us build what we've talked a lot about today, but build a lot about what we believe AI can do to help bidding in the future. And so that's why we're super excited about the team. And I hope that is very, very obvious. Okay. So in closing, I want to say just a few things. I'm going to echo a little bit of what Mark was talking about earlier today. Well, we sort of find ourselves at an inflection point in the ecosystem. Complexity is on the verge of outstripping the human understanding of how to make that complexity useful. And if you kind of think about it, AI not only will help that, but it will help drive other solutions that will improve not only optimization and bidding, but lots of other parts of our lives. Kind of bringing it back to our ecosystem. The machines and networks that we all refer to as DSPs and SSPs are really truly amazing sets of extremely sophisticated software and incredibly advanced hardware that essentially trade trillions of ad impressions on a daily basis. In fact, if you think about it, the entire global ad tech stack trade something on the order of about $1.5 billion on a daily basis. So these are -- and honestly, like this is just the beginning. This is before we bring in linear TV spend. This is before we have out of home. This is before lots and lots of parts are truly, truly digital. And that's why Scibids is so important. Scibids plays a really important role in simplifying and aiding what we believe will help our customers take advantage of future opportunities. And most important, as you've seen, each of the examples that we gave, our clients are benefiting from 30-plus percent based on the non-AI-enhanced capabilities or KPIs that they have, and just that should help drive an incredibly strong client uptake, advertiser uptake, which in turn, will help us provide more and more value to our investors. And all of these opportunities are literally just around the corner. Thank you.

Up next, DoubleVerify's Gian LaVecchia, Managing Director, Americas.

Hello. I hope everyone is doing well. Again, my name is Gian LaVecchia. I'm the MD of the Americas. And I'm absolutely thrilled to be able to introduce our next featured panelist. Perhaps one of our most insightful and strategic partners out there. Joshua Nafman, VP of Data and Digital Operations at Diageo. For those folks that are not familiar with Diageo, it is one of the world's largest alcoholic beverage and spirits companies, leveraging a portfolio of more than 200 iconic brands across 180 countries. Kind of a big deal. So please join me in welcoming Joshua Nafman today. Josh, do you mind just starting with a little bio a little bit of your backlog?

Sure. Hi, everybody. I'm Josh Nafman, Vice President of Data and Operations and more specifically in marketing operations. So how are we using our data and technology in order to basically make the best business decisions possible across consumer data, commercial data and media quality and marketing effectiveness data.

Excellent. Excellent. So starting off the conversation today. Obviously, we're living in a very dynamic data-centric and increasingly complex marketing world. Do you mind just providing your perspective on the state of programmatic advertising and what you believe some of the biggest challenges for marketers are?

Absolutely. So marketers are increasingly under pressure to have a lot of considerations in what they're doing. So efficiency, effectiveness, reputational considerations like safety, suitability, attention, optimizing to sales, et cetera. And all of these complexities are basically for every single campaign, you have to optimize to them. And it's kind of like trying to have basically good, faster, cheap and you can only pick 2. You can't have really all 3. You can't have your cake and eat it too. That's one of the bigger challenges, but you're still expected to meet those. And so, in the case of Diageo and programmatic more broadly is there's a lot of walled gardens that will while providing cost efficiency, they're not necessarily aiding you in the reputational side these days. So I would say, as of right now, Diageo and a number of other marketers are actually increasing their spends within programmatic because it gives you control over efficiency effectiveness and a lot more of those I'd say, custom KPIs and considerations rather than it being just a blunt object of you're optimizing to a media metric, you can then optimize to an actual business outcome.

Awesome. Awesome. As part of the due diligence process, Josh was one of the partners that I actively saw consult for to get some insight in this market intel on Scibids and the broader custom algorithm space. And he shared with me perhaps my favorite quote ever, which was we trust Scibids with our sales data. And I remember just hearing that, is kind of music to my ears. So Josh, with that, do you mind just talking a little bit about the role that algorithmic optimization AI solutions play within your marketing mix and kind of what your expectations for [ AI ]?

Yes. So the usage of -- like I said, there was a number of considerations across how we need to optimize our marketing, et cetera. And the challenge that we run into is sometimes those values or those considerations are compete with each other. So let me bring this to life a little bit. Diageo wants to be the best performing and the most trusted and respected CPG company out there. Now with that said, we're obviously going to support certain progressive agendas, LGBTQ and other underrepresented communities within media. Because they're underrepresented, they're probably underfunded and not able to meet safety or suitability or some of these other requirements for my marketing, which means less money if I'm not using a custom algorithm or optimizing it might drop them off of the list of places where I'm spending money. So you can immediately see how reputational considerations that might be more long term might compete with short-term considerations because if I advertise with them, my efficiency might go down, my effectiveness might go down and while my reputation might go up, Diageo is always looking to balance reputation and performance. So when it comes to AI, it aids us in not only clarifying how we want to wait those things, but considering all of them rather than just one piece of that equation at any given time.

So that's great plan. Sets up the next question quite well, which is understanding that you do have multiple priorities that also evolve over time, right? Attention may be relevant today supporting your progressive values tomorrow carbon emission management down the road. Can you just talk a little bit about how you actually manage -- manage the tension or dynamic between all of those variables. Like what are you really looking for? What's that core signal that tells you this is something that I want to pull that lever a little bit more firmly.

So the way that I visualize it for my exec is, there's a number of levers that you can pull at any given time. Let's say, you pull the cost efficiency lever that might drive your effectiveness lever up or down or it might drive, let's say, suitability or safety, et cetera. And so there is always a little bit of a give and take between those things. So the core signals that ultimately that we're going to look for, it really depends on the campaign by campaign basis, and it depends on what data we're putting into that conversation. So in many cases, we're starting to use our commercial data as a signal within that. So to give you an example would be how are we utilizing our sales depletions data in order to only advertise places where our product is actually available to be purchased. And how do you optimize that in real time, if all of a sudden, let's say, a zip code in Manhattan sells out of Ketel One impressions wise, how do I ensure that I'm not advertising or wasting the impressions in that particular area or the halo area. And so it really is at any given time, working with Scibids, DV and kind of automating the idea of when something changes, it is automatically adjusting while still within our priorities across that basis.

Great. Shifting gears just a little bit. As I mentioned earlier, and as you kind of already able to tell from this conversation, Diageo is one of the more sophisticated marketers in the digital realm. So I would love -- this is a two-pronged question. First, just laying out some of the KPIs that an organization like Diageo would focus on and optimize against? And what are some of the data sources that you're leveraging? And maybe even taking a broader step back and giving a perspective on the broader CPG category.

Sure. From a broader, I would say, data source standpoint, consumer data is a really hot topic because there's a lot of signal loss in the vast majority of places. And so it's all about first party. How am I collecting, what is my value exchange with that. So consumer data is a major factor. For Diageo, our scale of our portfolio and our distribution makes our commercial data and our sales data a competitive advantage. So that is definitively a factor within it. Diageo also uses MMM and MTA, a lot of CPG companies do this factoring in what is going to be a higher ROI. In our case, would be dollars per spend or dollars per volume at a ZIP code level, so we can do these minute changes and automations based off of that. And then finally, very importantly, is media quality. So safety, suitability, attention are kind of some of the data points that we're using across the board. In terms of KPIs, it really depends on the market need in certain, let's say, DMAs or cities, it might be we want to increase footfall into bars or restaurants for the purposes of premiumization going from a Smirnoff to a Ketel One or a competitor over to a Ketel One as an example. So using the data points of what is available there, what is the sell-in, what is the sell-out and also factoring in the media quality aspects of it or the ROI aspects of it. How are we leveraging those. So it's not necessarily one KPI per say. It really is more of a how are we balancing those things along the way.

Got it. A question we didn't talk about, but I know you've got a deep reservoir here. So you've talked about this idea of investment optionality, right? As kind of a core driving strategy for Diageo short term and long term, right? Can you just provide a little bit of texture to how you define that? And I bring that up because you just mentioned navigating between the portfolio itself. When you think about investment optionality, is it tactical? Is it at the brand level? Can you just provide a little bit more color to that because I think it's quite interesting.

Sure. I would say there's investment optionality and also inventory optionality. Diageo looks to maximize its investments across the portfolio and across channels at any given time. And then from an inventory optionality standpoint, we don't want to be overinvested in one particular channel. And candidly, we found ourselves in that area very quickly just based off of cost efficiencies driving effectiveness, but not necessarily balancing out the reputational considerations. So one of the reasons why we're investing heavily in programmatic is inventory optionality the reason why it's not just because we want cheaper media and cheaper reach in order to do that. It's -- we're looking to expand inventory in order to drive our costs down for the media, so we can layer on the important data points that we're looking for. Because we view it as every single time we use a data point, we are probably increasing the cost because it is more targeted. It is more targeted as a smaller group. Therefore, the cost is probably going up. So we drive down our -- we open up inventory. We drive down costs -- we layer on data to drive effectiveness improvements. And some of the examples that we've actually worked with Scibids on is we saw a 1.7x ROI and also a 40% efficiency -- cost efficiency by using this method.

Excellent. And as we close today's conversation, what I would do -- what I would love to hear from you is 2 things. Number one, what do you see as the role of AI in the future, right? How do you see the space evolving for Diageo, maybe at a more macro level? And what guidance or advice would you give this community and maybe the marketing community more broadly about the next about 24 months?

I would say -- well, the role of AI, that's a loaded question. I would say right now, Diageo's thinking about AI in two different ways. One is there's obviously operational efficiencies that can be driven from it. But on the flip side of it, it's -- how is AI going to impact the consumer experience, which is what Diageo is mostly focused on. So we're really starting to think about how do you market to an AI that markets to a human essentially. So originally, we were kind of thinking about it as how do you market from Diageo to millions of people than it was, how do you market to an algorithm that markets to millions of people. Now it's how do you market to essentially whatever AI you have in your pocket at any given moment because it's going to give you an answer rather than optionality for a consumer going forward. So we're really thinking about all those considerations, how is our AI and algorithms working with, although I would say, consumer algorithms and AIs for the purpose of influencing purchasing or availability in their mind or physically. In terms of advice, just across this category is a lot of this stuff is coming off as tactical, I'll call it, fairy dust, if you would, of tech for tech sake, I'm using AI. It seems to be dropping as same every single place. We're viewing it more as a -- view it more as a strategic discussion. So the best advice that I can have for here is it's not a check the box, I need to be using AI. It's not a check the box. I need to be safe, suitable, attention-oriented, et cetera. These are strategic considerations about not only the short-term health of your brand, but also the long-term health. And if it's only in the conversation of compliance your debt in the water to begin with. It really is a strategic conversation on where is the future of your brand and your marketing going.

Outstanding. Thank you so much, Josh. This was a great conversation. Appreciate the time.

Thank you.

Thank you all. We're going to take a short 10-minute break. Please join us back in your seats to continue at 02:40.

All right, everyone, we're going to get started with the second half of programming today. Please take a seat when you get a chance. Thank you so much. So before we get into the speakers, we're going to share 2 video testimonials from two of our core strategic partners. First will be a testimonial from Keith Bryan, President of Best Buy Ads. And it's a really interesting partnership where we support them both from the brand of Best Buy as well as the Retail Media Exchange. And then secondly, we're going to feature Joshua Lowcock, who is the Global Chief Media Officer of the iconic media agency, Universal McCann, has been a longtime partner of DV and for many years, actually led global brand safety for IPG as a whole. So he's very familiar with our proposition. Thank you very much. Enjoy.

I'm Keith Bryan, President of Best Buy Ads Media and CRM. Thanks for the opportunity to share a few thoughts about DoubleVerify from a client perspective. For context, Best Buy's media strategy and investments to the demand side and retail media business called Best Buy Ads, the supply side, are a single, highly integrated end-to-end team, which has many strategic and operational advantages internally and for our partners and clients. From my perspective, DoubleVerify's x factor is threefold. First, they're collaborative, both with my team and our key partners from Google to our media agency, Starcom. Second, they're proactive with relevant opportunities and responsive with their support. Third and my favorite is that DoubleVerify is highly compatible with our model, specifically the fact that our demand side and supply side are integrated and synergistic. I bring that up because we significantly over-index among RMNs and the amount of off-site inventory in our Best Buy Ads campaigns. I can't give you that number because it's likely bigger than you'd guess. For that reason, DoubleVerify is uniquely important to Best Buy both from a media investment perspective and ads supply side perspective because optimization, ad verification, fraud avoidance and brand safety are important to Best Buy as a brand and the Best Buy Ads clients. The DoubleVerify team anticipates the needs related to this duality and sided is a logical extension, bringing AI-based flexible bidding optimization to their clients and their clients. and that fits squarely within DoubleVerify's trademark, trust and transparency. Personally, I admire DoubleVerify's Scibid's acquisition because it anticipates and serves clients like Best Buy, and defines what business DoubleVerify is in dynamically to avoid marketing myopia and stagnation. DoubleVerify understands that its value to clients is dependent on understanding and evolving its value within the ecosystem. Thanks again.

Hello. My name is Joshua Lowcock, and I'm the Global Chief Media Officer at UM Worldwide. I had the pleasure of working with DoubleVerify DV almost a decade now and see them as a leader in the measurement space, their solutions in brand safety, fraud detection and overall media quality are industry-leading and measuring these things is increasingly critical in a world that's driven by AI-generated content and the MFA based advertising websites -- they can take dollars from brands and use it for inappropriate purposes. I also like DV's investment in technical innovation, leveraging [indiscernible]. For example, their recent developments in retention measurement and the acquisition of Scibids puts them in the optimal position to better serve our brands now and into the future. I look forward to a long and successful partnership with DoubleVerify.

And now, please welcome, Nisim Tal, Chief Technology Officer at DoubleVerify.

All right. Thank you, and good afternoon. We continue this innovation day with more accents from across the Atlantic, and I'm delighted to present this next section with less charming than French yet very authentic, Israeli accent. In fact, the founders of DoubleVerify were Israelis as well. And I had the opportunity to work with them back in 2010 when I joined the company and embarked on a 13-year journey of innovation of scale, of growth, of continuous adaptation to emerging media types and of technological leadership. We invented the verification category. We were there when mobile emerged as a meaningful advertising outlet, when programmatic started gaining more and more volumes and budgets when social scaled up, when video became more and more popular and now we are here when AI is disrupting our space. I'm excited by the opportunity that AI is presenting to DoubleVerify. And in the next 10 minutes, I'm going to share with you the vision and the steps that we are taking in order to seize the opportunity. So Mark already talked about how AI, the rise of AI will amplify the need for verification, the proliferation of sophisticated invalid traffic and of fraud will lead to new challenges for advertisers. But the impact of AI is far beyond than just industry tailwinds. AI is embedded deeply within our culture, within our products and within our operations. We build AI, we sell AI, we leverage AI internally. And most importantly, we do it right. So let's look at our AI investments in 3 different angles. One, AI as a growth and scale catalyst for DoubleVerify. Two, value creation with AI, improving operations and getting efficiencies. And then last, we'll talk about how to implement responsible, AI that you can trust. So let's break it down even further, starting with growth and scale. So DV invests in 4 -- I'm going to go over 4 main areas where DV invests in AI for growth and scale. Starting with Scibids. We already heard about Scibids, our most recent addition to our basket of goods. And we heard a lot about Scibids. But let me emphasize a few things. By having access to the technology of Scibids and to the talent of Scibids. And when you combine this with enormous amounts of data, proprietary data of DoubleVerify, the sky is the limit. And from an R&D perspective, we now have access to talent and to technologies that are specializing in 3 areas. One, this is predictive analytics. Two, optimization and scoring. And again, when you are combining these technologies with our data, you can do a lot more for the customers. But we talked a lot about Scibids. You probably want to hear about other types of AI that we have in DoubleVerify. This brings me to this second item, a system that we call multi-modal, language agnostic, content classification platform. This is a big name because this is a big technology. This is the technology that powers double verifies brand safety and brand suitability products across measurement and activation across the board actually. So let me break down this long name into this component, starting from the end, content classification platform. As the name suggests, this is a platform that allows us to analyze content, for safety, for suitability, for contextual, identifying competitor logos and so on and so forth. So that's the content classification piece. Language agnostic means that this technology can work in any language. Today's AI allow you almost seamless translation between any language. And we are leveraging these models as part of our technology stack in order to grow and expand into additional markets. And then multi modal. Multi modal means that this technology works across all of the modalities of the content and the modalities, for example, can be the video element of the content. It can be the audio elements. It can be Metadata data, it can be text. Stickers that sometimes you see on top of TikTok videos or Emojis and so on and so forth. So we are analyzing each type of those modalities and combine all of the analysis together into one determination. This technology is already live in production. It has been growing, and we changed it over time according to the changes of the content available out there on the web. And thanks to this technology by the end of this year, we are going to expand to at least 25 more markets and languages with our brand safety and suitability product. So, and let me repeat it, if it wasn't clear, yes, our technology is analyzing video frames in order to determine safety and suitability. All right. Let's move on. The third system is totally different type of AI. It's in the first one, we talked about AI for optimization, for modeling, for scoring. The second one was about content classification. The third system that we have built, it's about traffic classification. It's about anomaly detection. It's about pattern recognition. And we have experts in this field that help us generate value by detecting fraud by detecting by MFA, by detecting invalid traffic thanks to this technology. In fact, talking about MFA. MFA is a recent advancement that we released to the market by combining this third system with a second system. In order to determine MFA, you need to know 2 things. You need to know the quality of the property, whether it's a site or an application and you need to know the quality of the traffic that goes into that site or application. And by having these 2 systems combined, we have uniquely positioned to tackle the MFA problem. And then last, workflow automation with Generative AI. So this last one is at early stages, but we are leveraging and we have a path forward and plans for leveraging generative AI as part of our user interfaces and using it to create more value for our customers. For example, our system in the future will generate recommendations and insights that are customized to each customer. We are planning to introduce conversational interface as part of our analytics dashboard so that customers will be able to ask our system, please compare my campaign in this month versus last month or whatever question they want to ask. It will be very intuitive and it's going to be a productivity gain, both for the customer and also for DoubleVerify as we expect that this will reduce the amount of involvement by our customer service team. So that was growth. Let's move on to value. On this slide, I gathered a few examples that show how we leverage AI both homegrown AI and licensed AI in order to drive efficiencies, to drive productivity and to drive cost savings. On purple, you can see our anomaly detection system, which we are using as part of our production operations. Our support team is using this system in order to identify all kinds of anomalies. And thanks to this technology, we were able to staff down or scale down the team by 25% and reallocate some of the support team members into other activities. On green, we are seeing how our fraud detection AI is helping our fraud lab team identify new fraud schemes. So in fact, 40% of the fraud schemes that we are detecting are thanks to the AI and leads that we are getting from that system. [indiscernible] it's a nice example that I like where we are using AI in order to build AI, so -- and it comes in 2 ways. One is we are leveraging foundational model that are available out there, whether it's open source model or a licensed model, and we are incorporating them into our system. But more interesting, we are using this technology also to create training sets for our homegrown AI. So rather than having manual people look at content and label them, we can give it to Generative AI to label it for us. Of course, we don't trust AI, so we need to put at least one analyst or labeler that will verify that AI gave good results. But in the past, we used 3 of them, and we wanted the consensus among the 3 of them when we did the labeling operations. Now we can reduce the staffing of the team by over -- it says 66%, I would say, between 50% to 60% efficiencies that we are gaining by reducing the staffing of that team. And then last, in pink, teams productivity. And you know AI is all about productivity. Generative AI is all about productivity. So we are doing a few things on that front. First one is Mark had already mentioned, the AI-based coding assistant. So we equipped our team and our developers with access to these tools and now they are implementing code faster. In fact, 90% of the participants of that program reported material productivity gain, thanks to that AI and more than 80% of the code generated by AI was usable. So it's not that we are going to get rid of all of our developers. We still need developers. There are certain tasks that fit AI, certain taks that don't fit AI. And we are doing it smartly, trying to leverage AI as much as possible. And then the last example on teams productivity, again, we equipped our employees with access to knowledge management platform that is based on Generative AI. Actually, we have a private instance of Generative AI based on open AI's implementation, and this instance is connected to all of our knowledge sources, so that our employees can access to that system and start asking questions. It helps with onboarding of new employees. It helps with existing employees. And overall, it drives productivity. And the results or the reaction from the team is superb. They are extremely excited, and we are seeing increased usage of that internal system. So we talked about growth, we talked about the value. Let's talk a little bit about trust. So as we roll out these programs, both internal and external at an incredible pace, we remain vigilant to the great responsibilities that come with great power. Not only it is our duty to our employees, to our customers, to our shareholders to responsibly manage risk and compliance, this diligence is also a competitive advantage as we balance speed to market versus quality. To that effect, we've rolled out multiple programs and comprehensive compliance program consisting of many policies, many procedures and many processes to ensure that we balance these 2 things, as I said, speed to market and quality. And and we tackle it from multiple angles, whether this is domain expertise that we have in AI, domain expertise that we have in content classification. Whether it is the policies that we are creating for each category. So when I'm saying content policies here, each category that we are providing, whether it's violence, whether it's adult content, has a clear policy of what does it mean this category. No out-of-the-box AI can do it. You must have your own policies and your own calibration. Of course, when talking about AI, we have antibias policies as well as security, privacy, copyright infringement, and IP protection and last quality controls. Every market and language that we are releasing whether it is TikTok, whether this is short, whether this is Facebook, we ensure that our solutions are calibrated to that specific market. It goes through testing, and it goes through a process where we adapt to the cultural nuances of that market. Translation is easy. What makes it more difficult is this. And the only way to succeed with AI is when you can trust it. And this is what we do a trustworthy AI. All right. So in summary, what do we need in order to succeed and how do we capture this opportunity? First, we have access to proprietary data sets at scale, whether it is fraud, attention, suitability and more. Second, you need domain expertise and access to talent. We have experts in the domain of AI and experts in domain of content classification. And we have strong teams all across the world, a big team in New York, team in Israel, in Belgium, in Berlin, in Helsinki, and most recently, we got a new team in Paris, France. Then third -- this is the responsible implementation that we already talked about. And last is our ability to impact. And even this strongest technology and the most advanced technology will be powerless if nobody is using it. If this is not integrated into the ecosystem and our technology is integrated into our customers' workflow, into DSPs, into social platforms. We have all 4 of these, and we are set to succeed with an impactful and trustworthy AI that drives growth that captures efficiencies and reduces risk. And with that, I will end. Thank you.

Up Next, Jack Smith, Chief Product Officer at DoubleVerify.

Right, show hands, how many people when they first use ChatGPT felt that it looked like magic. Anybody? Me too. I work in this world. And if you were to drop 2007 me working on natural language processing in a time machine into today, I would have -- it would be hard for me to comprehend the changes, right? It's kind of the equivalent of having a doctor from 1900 being dropped into a modern operating. And Arthur C. Clark said that an efficiently advanced technology is indistinguishable for magic, and it feels like it's more true than it ever has been. But it's not magic, right? It's people that you see talk about and the places that Nisim talked about. It's all the data that we've been talking about all day, and it's computing power that's available now in ways that it never has been before. And we need all that power because consumers are changing. And it's not just the consumption of text like it was in the early days of the Internet. Last year, Cisco said that over 80% of traffic was video, and that's a lot of consumers consuming video. TikTok is streaming 167 million. That is a video every -- I can't remember the exact stat that Mark talked about this morning, but it's a significant amount of video. And then if you think about the shift from text to video and you think about metadata as an example, there are different quality of metadata depending on the platform. So if you were to only use metadata, only about 10% of the brand suitability incidents would be detected on one platform. Now it varies from platform to platform. And some platforms, it's quite good. But those changes in consumer behavior have really driven our need to think about how to scale up and change the way we've done many different things. And that's what I'm going to talk about today to try and demystify the magic and why it's important. And it's really important for two audiences, support for our customers who are the most important things that I think about as a product manager. They care about things like sentiment, they care about avoidance and brand suitability. They also care about positive targeting, and they care about the kind of optimization that Remi Julian talked about earlier. And for you, you care about us doing it cost effectively and efficiently, and we have to scale it globally as well. So we're doing it all around the world. So I'm going to show you a demo. And one of the first things I learned when I became a product manager was never do a demo in front of more than three3 people. And there's nothing that kills a [indiscernible] faster than hotel ballroom connectivity, but we're going to try it and see how goes. So I'm going to try and diversify a little bit about how some of this works. So I'm going to select a video here. You see the video come to dinner with me. I'm going to go down my cursor is yellow, so it will be a little bit easier for you to follow. And I press play. I'm going to let it play for a little bit, and then I'm going to stop it -- oh the audio is going, so I'm going to pause the audio. And I'm going to look for some interesting spot. I'll stop it when it gets to something that's visually interesting. I'll just let you take it in for a second, and then I'll explain kind of a little bit about what you're seeing and what some of the outputs are. [Presentation]

Okay. That's a good spot. Okay. So there are a few things that are happening here. So you see the video, obviously, over here on the left, and you see some boxes. I'll talk about that in a second. You see the title of video, and there's emoji and some hash tags. And then you see this box. This is the most important box for clients because it's the outputs they care about. It's a combination of a lot of different things that are represented here, but it's combined with other things like our ontology, right? We've been investing in ontology for 10-plus years. And that gives us a real advantage when we're thinking about how to classify video because as Nisim pointed out, it's multimodal, right? It's not just video. It's not just looking at a frame. It's looking at frames of the video. It's looking at text overlays, it's looking at all kinds of different factors, the audio, et cetera. And so these things, let's start here, and I'll back into some of the individual models that are represented here. So you see the sentiment is positive for this video. Obviously, it's starting in a restaurant. And in the avoidance category, there's a lot of alcohol. So if you watch the video, you see a lot of wine being poured, wine glasses, people are drinking wine but it's a low-risk alcohol, right? So some brands might be okay with low-risk alcohol, but they don't want high-risk alcohol and it's very important for us to be accurate when we do that. So again, we're using all these modalities combined together to determine the risk level and then on the other side, we're trying to determine the contextual categories, these are IV categories here. So there's travel, dining out luxury, et cetera. And then some of the individual models are represented both on the video itself and then in the detected elements on the right, you can see a wine bottle here, wine glass, a handbag, they're outlined blue. Those are called bounding boxes. It's just an easy way for us to show what the technology, what the models are detecting in the video. And you see green here. We talked earlier about stickers. So 20% to 50% of platforms, platform traffic and videos have stickers on them, that could be text or emojis. There are other kinds of video or there is other kinds of tech on screen as well. You see a handbag. And then you see down here in the corner, Chanel. Now these are all different models. So this is notable object detection. This is on screen text detection. We actually run different models for this because it's not just text, could be an image, it could be an emoji, there's Chanel, that's logo detection. And if I could show you a little bit about some of the output that we get here in and why it's meaningful. So you can see, you saw the Chanel go. If I click on the Chanel pill, it takes you to the first point at which that Chanel logo appears. That's also the same -- it's also the first time you see a handbag too. And in notable objects, we saw wine glass, we saw a wine bottle shot cocktail glass. And if I click on knife, we see a nice down here in the bottom right. Not just because there's a knife in the video, it doesn't mean that it is a violent video. I mean, this might be a butter knife, but if you're detecting a knife, you may not actually know that it's a -- that it is a butter knife, right? It might be just a knife among different kinds of knives. And we're using these multimodalities in things like the settings in the background, the fact that it's in a restaurant, gestures and action. So the only gesture we see here is poring drink. I can click on that, and that's the first time you see pouring a drink. And we're also using the speech that we're detecting to make a determination that the video actually isn't violent. Even though there's a knife for the video, it's not violent. And we need to make sure that again, we're using all of these things together because any one model is not 100% accurate all the time. right? So what you'll see is if you look at many different videos and the outputs of models running for, let's say, speech detection as an example, you will see mistakes in that. And it's important to ensure that we're using, again, all these modes together to make a determination that it's -- the sentiment is positive. We have a correct assessment of the avoidance categories and the contextual targeting positive segments are also accurate. Now, let's look at it in a slightly different way. Going to load video here. It's boxing match, obviously, sorry about the audio being allowed. I'm going to pause it for a second, just at a random point here. It doesn't really matter where it is. Now this is a medium risk of violence classification. So you can see here, violence is medium risk, positive contextual categories are sports and boxing. And now the reason we know this medium risk is that no one is dying here. So it's not high risk. So there's no obvious blood, we can see the settings of the background. It's a boxing ring, right? That's where we detect that it's a boxing ring, we see the gestures punching, right, which in some cases could be high risk violence. In this case, it's not because we're also using these in combination to look at -- and speech detection as well, in this case, English to make a determination that it's boxing and its medium risk. Now that's one of the things that's pretty interesting here is that everybody talks about frame-by-frame analysis. And we don't think of analyzing individual frames one at a time. And the reason we don't do that is that if you take a video like this one. It's 58 seconds long. It's -- they're 30 frames per second, so it's roughly 1740 frames overall. What's important is to understand changes that are happening in the video. So I'll just highlight this. So this is a scene. So when we get a video in, we actually parse all of the frames, and we break it apart into individual scenes. This is a machine-generated description of the scene. Sometimes it's more accurate than others, but it's just a reference point for this. But we know that it's C4 and you see a representative frame on that scene. And obviously, it's taking place in a ring. Now if we were to think about a violent video, for example, like someone pulls a gun, right? So someone pulls a gun and they're going to commit a violent acts. In a video, they pull the gun that gun is probably going to be on screen for a minimum of 4 or 5 seconds. So you have many different opportunities to take that gun. And if you're understanding different scene changes, you can ignore the videos that , for example, don't have signals. So there might be lighting issues where the screen might be completely wide. There's no need to analyze that video. And I think also what's important too is that part of our scalability is using all these modes together and understanding the video and using advanced scene analysis so that we don't have to analyze every single frame because we think that's wasteful. It also slows down our speed to classify. So we're doing this. I think we're classifying now roughly 60 years of video every single day. And -- that means that we need to do it quickly, and we need to do it accuracy. And what we've discovered is that when we do this, there is 0 impact on the accuracy of classification. So we tried muscle methods. We use humans obviously, to build the training sets. But all of this is important in us scaling this up, cost efficiently, quickly and doing it for clients. I think also there's a really interesting example of, let's say, you have a 5-minute video that has two scenes, but both scenes are of a sunset. There's really no need to detect or to analyze in-depth every single scene. If you know that there's really not that much difference from one scene or one frame to another. And we can go back to the slides now. I talked about all this already. So I'm not going to go over again, but I think there's one really important thing here. We're driving a 20% cost savings, we're analyzing each frame again with no impact on accuracy. And where that's meaningful is when we put it into practice. So when we put it into practice, really taking all of the science and all the technology, all the data that we have and putting it together so that when we apply it in a client's case, so in this case, this is -- I think we used an example earlier of Colgate, Remy Julie were talking, but this is a different example where Colgate combined brand safety, suitability and fraud together in order to lower their block rate and reinvest those dollars in higher-quality media, and that led to a 77% reduction in the quality CPM. So again, all of this is great. It's fantastic. It may seem like magic, but it's not. It's not magic at all because the results are real.

Up next, Julie Eddleman, Global Chief Commercial Officer at DoubleVerify.

Hello, everyone. Again, my name is Julie Eddleman. I'm the Global Chief Commercial Officer here at DoubleVerify. Thank you so much for your time, investments, your monetary investment. Whether you've been with us for 2.5 years or you've recently joined our investment. We sincerely appreciate all of your support. Go ahead and sit down. Thank you very much. I'm going to introduce our partners here in a second, but I've been at DoubleVerify almost 3 years now after almost a decade at Google and 2 decades at Proctor & Gamble. And I have the great fortune this morning attending the Forbes Power Women's Summit up at Lincoln Center. And there were two of my idles who talked today. One was Sue Bird, a WNBA and 5x Olympic Gold Medalist, who now owns a production company; and Katie Couric, who is a journalist, former today show anchor and also now the head of a production company. And there's something that they reminded me of this morning, and that was the power of purpose and the power of a mission and everybody being catalyzed around that. And at DoubleVerify, our mission is to make the Internet safer -- stronger, safer and more secure and to drive better outcomes for our advertisers. And I can tell you every single person on this management team that you've heard for. All of our leadership teams and the nearly 1,000 people that work for DoubleVerify get up every single morning and think about that. We think about how to do that better for our clients. We think about to do that better with our partners and very importantly, as a mother of five children and a grandma, a proud grandma. We think about how to do that for our families. So again, thank you for being here, and we wouldn't be here without you today. So I'm going to introduce a couple of our partners today. Our panel today is on protecting media quality on an AI-powered Internet. I've got two amazing panelists here today. The first is Deva, who is Dentsu Media. She is the EVP and Global Head of Brand Assurance. She's been in the industry for 20 years and has worked across all verticals, including tech, CPG, quick-serve restaurants, telco, literally everything. In her role at Dentsu Media, which if you don't know Dentsu, it's one of the largest global media agencies in the world. She leads global brand safety and suitability across the world. So a hugely important role. And then we've also got Bill Tucker. Bill is the Group EVP of Data Tech measurement and addressable media at the ANA. The ANA is the association of national advertisers and the pre-imminent industry body for advertisers in the United States. Bill's career media stands 4 decades. He was the CEO at several media agencies and has now spent the last 10 years in industry associations. In his role right now at the ANA, he oversees data technology, digital supply chain industry initiatives measurement for marketers practice and very importantly, the ANA B2B marketers community. So again, thank you for being here. We really appreciate it. So we've talked about kind of the good, the bad and ugly of AI. There's a lot of good that we have seen in terms of productivity and machine learning and very importantly, our recent acquisition of Scibids and the outcomes that it's going to drive for our partners and our clients. There's also a lot of ugly and a lot of scary things that we've shown you is on the Internet. So Bill, I'm going to begin with you. The ANA has studied programmatic media supply chains very long time, and you've been really deep into that. That's part of your responsibilities. What impact do you think that generative AI has on the media supply chain right now?

Thanks, Julie. The study we just released was our first phase, really groundbreaking programmatics supply chain study, 21 advertisers, substantial amount of media spend. One of the things that came out of there was the prevalence have made for advertiser sites inside the supply chain. We estimated 15% of all impressions, potentially 20% of all ad dollars going to these sites. And that was a surprise to our community, right? AI, of course, has the problem with these sites as they described is they offer low-level content, click-bait content through pop-up ads and other devices that would not be suitable for most brand management decisions, yet the media buying systems are lured by the content into serving ads to them, right? So generative AI is absolutely going to exponentially produce more of these kinds of sites make the job harder and harder to avoid them, which is the idea of most marketers want to avoid them.

So Deva, how are marketers navigating brand suitability and brand safety in this AI-generated rules, like what are some of the challenges that they're facing on a daily basis?

So this is the point in time where is no longer acceptable to take a set it and forget it approach to protecting our brand. So my title is brain assurance. So okay, if you don't know what that means, we made it up. It really has to do with many areas of media responsibility. So in addition to safety and suitability and front protection, we're also doing things like brand reputation management, giving energy to the diversity discussion in AdTech, sustainability, loss in lots of different areas that basically make it safe and responsible for our clients to activate. So within all of these spaces, we run into issues like copyright infringement around diversity around audience identification. The net-net, when we come to AI is we must remain vigilant, and we partner very heavily with DoubleVerify in that space. to make sure that our clients' campaigns are protected, but also that their brands are protected. That's the most important thing for us right now.

So Bill, you said those statistics, 15% of budgets, 21% of impressions. So when you and I worked together at P&G 20 years ago. We collectively add about a $2 billion budget. So just to put that in perspective, that could be between literally between $200 million and $400 million that we would be "wasting" every year on advertising. That is a mind-blowing figure I think you released that study the Monday of CAN, and it was the talk of CAN. Every single meeting that I had, everyone was talking about the MFA issues. So for the ANA, how do you define MFA. And just because a website is MFA, does that mean that it's low or bad content.

There is no clear definition at the industry level of an MFA. So there's behavioral descriptors, there's judgments made certainly is an action area that the industry needs to take on, first of all. The issue with MFA is beyond just the consumer experience, it's feeding into the broader transparency and waste in the programatic system, right? And Ad [indiscernible] marketers, they don't understand where they had dollar is going and exactly who's getting paid what, so it amplifies the problem.

So Deva, what were the marketers -- you knoded your head when I said that it was a talk CAN, it was the a talk CAN. So what are marketers reaction to that 21% number? And if sites perform even though they're MFA sites, how does that affect the way that you approach your media planning and by...

I absolutely love this question. I think it's super important to make the distinction. I think the first reaction and the first reaction are still rolling in as new sort of spreads around. The first reaction was panic, which is correct rightfully so. This is a problem that's been growing for quite some time, but this is the first time we've had a number on it. So obviously, brands worry about the consumer experience. what consumers are seeing and doing and feeling when they experience an ad from their brand. If it's crowded site, if it's poor quality content, if it's just generally a poor experience. that there is an immediate correlation to that brand and how consumers feel about that brand. On the flip side, though, you mentioned performance. So Bill was talking about winning bids about some of these MFA sites winning bids. That is one of the truth of the matter. So that's a conversation that we're having to -- that we're having to engage our clients on, what are you willing to sacrifice for the health of your brand? Yes, some of the costs are low on these sites. But if it's a core experience and consumers are getting a bad feeling about your brand, they're not going to continue to be loyal to your brand in the long run, is it worth it?

So Deva, in terms of the reaction of the brands, what extent do you think that AI is going to solve brand safety and brand suitability.

So I know we're a little doom and gloom for the most part these days. I'm actually one of the few folks that are pretty optimistic about the opportunity that we have around AI in general, not speaking about one specific area. The truth of the matter is AI has and we'll continue to bring tremendous workplace efficiency to the industry. And we're really excited for what we can partner with DV on in this space. An example here is my team takes a unique approach to policing the -- what we call the inclusion list of the sites that our clients are allowed to run on from a programmatic perspective. I have a person on my team who twice a year, visits all 10,000 sites on our list and manually checks for certain criteria that we use as proxies to identify MFA site because as Bill observed earlier, we don't have a set of standards. So her name is Jessica. She spends a lot of time. I do send her reads gift cards every time she does it because it's a really long task. But we're really hopeful that, that task is an example of something that AI can really help us with getting closer to and getting more efficient. So that, frankly, we can police our risks more often than twice a year.

So we talked about this a little bit a couple of days ago, and we did do a press release this morning. That's a great lead into our announcement on our new MFA product.

We're so excited.

We are very excited, and I've got to tell you, I've had a lot of e-mails today on this, and I think it was only at least 4 or 5 hours ago. But DV is using a unique blend of human and AI-based auditing to create an exclusion list of MFA sites. And advertisers then can choose whether they want those sites or not wanted sites. Advertisers can directly enable that list in their brand safety and suitability profiles for measurement and monitoring and then indeed, the authentic brand suitability for prebid avoidance. So we're super excited about that and very excited to work with both the industry and Dentsu. So Bill, when you think of some of the key issues with the programmatic supply chain, including waste and supply chain inefficiencies, which, again, we've talked about a little bit today. What opportunities does AI give marketers to increase productivity as it directly relates to media buying?

Well, I think, two areas come to mind. One is doubling down on kind of the management of safety and suitability and valid traffic and better tech to manage that. Another area, which has been a focus of the marketers is getting log level data that can be understood so that the costs through the waterfall of the programmatic supply chain can be understood, clarify and waste can be identified. And there's hope that AI can help enable not just availability of log level data, which has been extremely challenging, but also the stitching together of that data through the supply chain, so that the right observations and then decisions can be made about the facts.

And Deva, turning to other AI-related issues, and you talked about it a little bit. You and I have done a lot of work on DIB together in the industry. How can advertisers navigate some of the ethics of AI as it comes to a bias intellectual property and some of the other challenges that not only advertisers have but agencies in general.

Yes, absolutely. So this is a really important. As you said, this is a really important issue, very close to my heart, very close to our hearts. And Dentsu as well. So I think the most important thing that we have to keep in mind is we are in the build phase. We are at all of these, all of our metrics, all of our feelings, all of our guidelines are coming together now. So as we're building, we have the opportunity to bake in a code of criteria, a code of ethics, a way of moving forward that is inherently more inclusive and is more transparent. We had -- it's a wonderful opportunity. As you know, and as you know, we've been around for many, many years. We're kind of trying to retro -- I have it here -- we've been trying to retrofit ethic and trustworthiness into the systems that were created, which is a lot more difficult than baking them in at the beginning of the process. So super important that we not only build these things in, but that we also nurture them and continue to help our AI to learn probably more inclusive moving forward.

And the final question that we have today is a little bit of a forward-looking question. So we're not going to record it and ask you again in 3 years, but maybe we will, maybe we will do that for a fun meta post. As you look forward in the privacy-focused future. So a whole new world that we're entering into, how does AI and generative AI fit into that privacy-focused future, which DoubleVerify is completely built around? Bill, I'll let you go first.

So look, we obsess about privacy at the ANA, both from a legislation perspective. We just saw the California [indiscernible] delete largest is past the third-party cookie depreciation is coming. So whether it's the targeting use case that drives audiences and sales or the measurement use case, new solutions need to be built that are privacy enhancing, right, for doing those use cases. and it ranges from contextual targeting solutions informed by AI that can predict to virtual IDs that need to be trained by panels. I mean the world is moving into big data sets and calibration panels and AI is going to be essential technology, I think, to make that happen.

And Deva, final words?

We're -- I don't have much to add to what Bill was seeing here. But yes, constant scrutiny, I think, informed approach to adopting AI models to further protect our consumers' privacy to further protect our brands. It's all possible and really excited to work with our colleagues at DV with our colleagues at that trade organization to make the future great.

Yes. And two things just to add on that. We're going to be working with ANA, IAB and all the other industry organizations on that privacy-focused future. And I think it's really important that we continue to push. What is the definition of on MFA on industry-wide basis so that we have more standards, along with another really important topic for DoubleVerify and really for our advertisers, which is around attention. So those are just a couple of other things I'll drop with you. And thank you very much, again, Deva, Bill, for not only today and your time, but your partnership throughout the year. Thank you, everyone.

Up Next, Nicola Allais, Chief Financial Officer at DoubleVerify.

Okay. Great to see everybody in person. I am particularly delighted to be coming up after the whole team. I feel like one of my jobs is to kind of do justice to the innovation and the forward thinking that happens in the team and you finally saw it in person, which is great. I also realize that I am probably the last person standing between all of that and the reception. So I will get straight to it. We see three3 financial takeaways from what you heard today. The external one is the AI innovation and the products that we bought with Scibids. The second one is the internal innovation that we're going to be able to put at play around efficiencies that you heard from Nisim and from Jack. And then the third one is a general statement that AI is basically going to continue to power all the growth vectors that we have been very consistently talking to you about, and we'll get to the numbers at the very end of my slides. So today, we talked about products that are tied to the programmatic space. And so I just wanted to resize the opportunity here. Programmatic revenue is $95 billion this year, say to be $95 billion going up to $123 billion. That's 85% of all digital ad spend. If you take out search, where we don't play, and if you take out social. It's a very large opportunity. The 2023 growth is 16%, slate 16%. Our activation revenue grew 29% last quarter. So large growing opportunity. We grew faster than the industry, and this is why we're very excited to have found an acquisition in this space. Now the beauty of the Scibids acquisition is that it is an actual product already. A lot of people talk about AI, what it's going to do to the environment, to the media space. For us, one of the exciting things about Scibids is that it is actually already a product that's integrated to the DSPs, which means that it expands our addressable market straight away. And the way we think about how it expands it is pretty simple. We look at the open Internet that we just discussed on the programmatic side of things. and assuming that 10% of all bidding will include a customized algorithm tied to then the take rate of the Scibids current model, which is about 10%, gives you a $1 billion addressable market that comes with us immediately by having acquired Scibids and are going to what we think the actual revenue opportunity is for us. But before I do that, let me once again explain where Scibids fits into our product mix. So this is a flyway that we're already all familiar with. This is what we do today. We have measurement with the DV authentic ad and the authentic attention products. And that data feeds into activation the programmatic activation that we do today, which is static and dynamic sets, right? The dynamic set is authentic brand suitability, which is really a great product for [AutoGroup]. 50% last quarter, certainly not mature yet. And with that flywheel, once you have information on the activation side, you can feed it back into your measurement and that flywheel continues to go for our advertisers. The beauty of Scibids for us is that it creates a second decision track in activation. So essentially, it starts again with measurement data and the advertiser is now able to use that data tied to third-party data, which is very specific to the customers. You heard, for example, from Josh at Data today, talk about specific products within specific ZIP codes, right? Very detailed information at the customer level that can be combined with our DV data to then create an AI optimization through the Scibids platform. essentially a complete second track as to how you can then get into activation and back into measurement. That track exists, Scibids is a product. It's out there already in the market. And what I'm going to talk about is the opportunity that we think that creates, right? Just by having us acquire the company, having our sales force and the ability to reach 1,000-plus advertisers, there's an opportunity there to just grow what the opportunity we're not going to discuss today, which is potentially greater than this is what Doug talked about, which is the velocity, the volume and the value of our data. This TV data box here, all of this sudden you can see how authentic attention could become one of the data sets, which makes it a lot more valuable and a lot more likely than an advertiser will actually buy it on the measurement side. That's the overall opportunity now that we have this second flywheel. But just on this side, what the opportunity is for us that we see around the Scibids product that we've acquired. We think that, that opportunity is probably over $100 million by 2028. We already mentioned the $15 million to $17 million next year on a second quarter call. We see this going to $100 million or so by 2028. It's a 58% CAGR just on the fact that we have a larger customer base, and we're able to sell it within our own product suite. To understand that $100 million, I think it's useful to put it in the context of our other programmatic product because Scibids is just the evolution of what we already do in the activation side. Doug talked about going from static to dynamic to now AI activation. And this slide explains the evolution of our products. In 2015, we launched a static statement -- the segments. So this is a non-ABS part of our business. in the last 12 months to generate $133 million even though this was launched way back in 2015. In 2018, we launched a dynamic activation segment, so authentic brand suitability. This is our most successful product. It was $152 million in the last 12 months. We've achieved 175%. And just as a reminder, over 80% of that growth last quarter came from existing customers that continue to use their product on more and more of the volume. So really not mature yet and an excellent growth rate. DV activation, the $100 million, you can see it there, by fiscal year '28. This is a projection that we have. And the reason we're showing it here is you can kind of see it based on the other numbers that we have and the other products that we have. This is based on their current model and based on again, a fairly conservative view on how much of the business we think will go through custom algorithmic bidding. It's early days. intuitively, it feels like this is going to be something that a lot of customers are going to want to use, and you heard it today that it's something that customers see a lot of value in. But just thinking very simply over the next 5 years, the $100 million compared to the other numbers is how we think they will stack. And again, that's without the value that we think Scibids is going to add to that usage of DV data. more DV data is probably going to be used. If you think about authentic attention, for example, that can become one of the data points that goes into the DV data along with third-party data to create an AI optimization. All that growth that we anticipate on the measurement side is not included in this $100 million. So while I'm up here and I have the floor, I'm going to take the opportunity to remind the -- of the five growth vectors that we always talk about, you're familiar with these, but it is worth sort of reiterating so that you understand that Scibids is not a complete pivot. It just kind of fits into the growth vectors that we always talk about. And I'm going to start from the far right, the strategic M&A, we just completed the acquisition of Scibids, international expansion only 27% of our measurement revenue is international, and over 50% of all spend is international. So there's a gap there that creates an opportunity to close current and new client growth acquisitions, 61% of the top 800 or so advertisers, we do not cover, right? The other one -- another provider or they're actually greenfield. And basically opportunity for us to go after. The acquisition of Scibids honestly just really enhances the value provisions that we have for our clients to go after clients even go after a takeaway. The channel expansion, 60% of DV's revenue is derived from social. It's well over 60% of all spend that goes to social. If you look at what advertisers are spending the most immediate opportunity that we see here is obviously the launch of brand safety on the Meta platform. That is going to be one other opportunity for us to kind of just grow that channel in the social side. And then I'm going to double-click on the last one, which is new product introduction and upsell. About 60% of our top 700 customers are using less than four products. So this is the power of our model, right? And we talk about it a lot and so I'm just going to go into it a little bit more. One of the stats that we mentioned in the second quarter call is that 80% of our growth in ABS and -- over 80% of our growth in ABS and over 80% of our growth in social came from existing customers. The customers that we already have that are continuing to use our products on more and more of their volume. That's the part of our model. Once we have a customer, you can see all the breadth of the products that we can sell to them. Historically, we would start a relationship with the customer on the post-campaign measurement and then upsell to activation. We've talked about in the past few years, we now start wherever the customer wants to start. Some customers actually start on authentic brand suitability because there is no equivalent product in the market, and then we're able to upsell them to measurement. For those of you who are here in 2022, there were five bars in 2022 at our Investor Day. We now have seven and then if you add Scibids, you end up at eight already. So Innovation is a key part of our growth strategy, right? The more we can put products out there, the more we can upsell and cross-sell once we have the customer in-house with us. And so this will continue to be part of our strategy. This will continue to be part of types of products that we look to launch. I'll say one word about authentic attention, which is here the second bar here, that is the type of innovation that we are funding, right? This is a product that we launched even before there was a standard in the market. It's a product that we launched even before there was real demand for the product. And as you've heard today, the Cybit acquisition actually creates that demand, right? It creates demand for the measurement piece on authentic attention to be used for free campaign activation. So that's the theory around creating more and more products and being able to sell them to our existing base. Now, a word on the efficiency side, there was a second takeaway, right? AI is going to create a lot of efficiency in how we do our business, right? It's going to make us do it faster, smarter and more efficiently. The obvious outcome here is cost savings. There will be cost savings that comes from us being able to use AI. The way we think about those cost savings is additional opportunities to redeploy investments into other parts of the business, right? We have superior top line growth. This is not a new strategy. We always have said that we are reinvesting in the business as we see the opportunities within the range of our EBITDA margin, which is around 30% at all times, and that's what we expect to continue to do. Again, authentic attention is the kind of product that we were able to invest within the confines of a 30% margin and be able to fit it out there even before others in the industry have thought about. And so that's kind of what we're hoping to do with sort of the AI efficiency that will come into the business. Now to the numbers. three slides on numbers. We closed Scibids mid-August. We closed Scibids on August 14, and we expect the contribution to be $1 million on the revenue side. for Q3, that's the partial quarter, right, post acquisition, $3 million in Q4. And then from fiscal year '24, we already discussed it, the $15 million to $17 million range, which is a 35% to 40% growth rate. The financial profile, it is currently a percent of ad spend model. We are not going to tweak that model until we kind of learn how it works and how best integrated into our own set of products. Our intention with Scibids is to help it scale in the way it's working because it's been a successful model. And before we actually start thinking about tweaking the model just to fit it within the MTM, MTF model that we have for the rest of the business. So we're likely to keep it that way until we see really the potential and the scale of the product. The gross margin profile of Scibids equivalent to DV's overall margin, gross margin profile. And this is a business that was running at a near breakeven. There will be obvious synergies that are around G&A, but we've acquired a team. You saw it today, this is a talented team of data scientists. It's hard to find really good data scientists. We are likely to actually use that as a basis for us to invest further into data science. So this is not a cost-saving play at all. This is more of a starting point for further investments, still within an overall 30% margin play for the whole business. The guidance update is essentially what I just discussed. It's $1 million more of revenue in Q3. It's $4 million more of revenue for the full year guidance. That gives you a 24% growth rate at the midpoint in '23 and 25% growth rate for the full year. We did not adjust the EBITDA margins or the EBITDA targets. So it's 29% in Q3 and 31% for the full year. This is the data that we give today on guidance is literally just including the impact of Scibids now that it's closed. And finally, I'm going to end on shows our track record. So we have a track record of delivering superior revenue growth add this 30% margin. This is always a number that we keep in mind that has allowed us to invest in the business, getting products out there before anybody else has in the market. AI is going to help us continue on the same growth vectors that we've been discussing all along. This is not a pivot in strategy whatsoever. This is just going to be accelerating our ability to get into new markets. into new verification modes and continue to deliver the kind of numbers that we have in the past 5 years. So it was short and sweet. And with that, I am going to ask for the rest of the team to come up for a Q&A session.

All right. We've been swarming you with information this afternoon. So feel free to fire away with questions about what you've heard or what you didn't hear. Andrew first and more.

I will all take the easy first question. So the $100 million target, just talk to us about what assumptions are built into that? And like help us do a build in terms of better understanding what needs to take place for you guys together?

Yes. So the $100 million number is what I just described, which is truly the existing Scibids product. being deployed to a larger set of advertisers, which is what we bring to the table, along with a larger sales team. This Scibids a great technology, it's a small team. They have a few large brands, which is very promising to us to be able to sell within our own set of clients. This does not assume any sort of changes in the revenue model at all, which we might do in between now and then, right? But this is truly just taking what they're doing and being able to do it to apply it across all of our existing customers.

Before we continue, I'd just like to remind our viewers on the webcast. You can submit a question on the upper right-hand corner of the Ask-a-Question tab, if you would like to participate. Thank you.

I want to start with a market one. And that is, you guys are part of an ecosystem. You're an ecosystem infrastructure play. And it feels to me like my takeaway today is if the world is going to generative AI. This is really going to accelerate your product introduction and cut your costs, maybe not near term, but over the next 3 years. So you're tying into Trade Desk, you had on stage, you're tying into Google, you're tying in TikTok and Meta. Is this an ecosystem that's about to become five guys that are globally scaled. And we're going to put all the small guys out of business. I mean what is this industry design? What is this implications for the industry design of programmatic advertising.

It's a great philosophical question. Look, I think there is something to say for a world that's powered by data for those companies that can have access to data at scale and technology that can employ that data at scale are going to have an advantage. Now I think there are certain sectors that will become more consolidated in digital media. But as we've seen over the last 2 decades in digital media, there's always something new that pops up and there's a whole new sector that creates new start-ups, and there's always new start-ups that come out of it. I mean DoubleVerify didn't exist 15 years ago. A lot of the companies that we work with did, but we came up, created the sector. There were many other companies, obviously, in the space when we started, many of those have gone away because they didn't have the technology or the chops to go out there and get. So I think although you'll see consolidation potentially maybe on transaction platforms like ASPs or data companies like verification, companies like ourselves, there's always going to be some new technology that comes up that will emerge from small companies and continue to proliferate and grow. But I do think you're right, Laura, that there's an advantage to companies that have data at scale and the tools to employ it.

Okay. And then my second question is -- it's a DoubleVerify question. And that is regarding data and what's happening, like you mentioned California and with the GDPR and the cookie application, does this movement towards Scibids, which is a nonpersonal ID help you? Does it help DoubleVerify specifically?

Yes. I think we've been pretty clear on the fact that Scibids helps us employ our data in a much more broader way for customers who want to have a spectrum of decisions. And that data that we're talking about in many ways is data that will benefit from the lack of cookies out there. So data around attention, around context, around viewability, those are data sets that have never relied on the individual level data aspects. So as those proxies go away as the ability to target a user or an individual goes away because of those laws. These other proxies are going to become more important. And if we -- the data becomes more important and the way we employ it becomes more powerful and more fluid, I think that's a winning combination for us. And that's why -- that's a long answer, to the short answer, which is yes, it will help us drive additional growth with these data sets.

Youssef Squali. So a couple of questions. One, maybe on the MFA opportunities, congrats on the launch today. Can you maybe just speak to how you see it scaling, how do you price it? How big do you think it is? Maybe Nicola, maybe you can give us estimate as to how big do you think it gets? Is it part of your 2023 estimates? And then maybe a question for [indiscernible]. Talk a little bit more about custom AI or competition in custom AI. Who's out there this challenge, which is parked on by Trade Desk. Do they have any advantage within the Trade Desk family? Or my understanding they're smaller than you about how do you guys compete and what's the overall market there.

Yes. I mean that's an excellent question. I think we created our category 7 years ago. So the category is still quite new. And I think, as you said, we are the clear leader in the category. And so we -- like most of our in our greenshield, actually, the question is more to deploy custom AI now why the varieties better with competitors. And with respect to the investment of the Trade Desk, think thing is you found this is a trade discount. I think I can speak for them, but like the part about the Open and some sort of independence of having independent dominant expert and discuss and for the DSP platform are open to this experience, I think is more important for the trade to control at this time.

Yes. I will add that comparing Charles to Scibids now that Scibids a part of the DoubleVerify family. It has access to more controls, more data to the feedback loop that [indiscernible] doesn't have, and then that's a big differentiator between the two companies.

Yes, I think on MFA, we haven't really thought about how we just announced it, so we'll think about what the scaling of the opportunity is. It's going to depend on sort of uptake from the clients more than anything else.

Yes. I think also one of the things that we're monitoring carefully, too, is that we talk about MFA, we deployed this. And as we launch this product, we're trying to figure out how to assess the value that consumers are actually getting out of this content because I know that I visit sites like there are certain recipe sites, I go to and Guitar specific sites like [indiscernible]. That have high audacity, but I still get value out of that. So we're trying to understand that balance first and give clients a choice on how to deploy and how to think about that as part of their overall marketing.

The one thing I'd note is on the MFA, and we didn't call this out specifically today, but we've talked about how fast now we can move in AI tools and machine learning allows us to move quickly in response to market activity. I mean MFA content has always been there, but it really started becoming an issue when the ANA talked about this summer. So we're talking something that happened in June, July report came out and advertisers is really starting to get wound up about it. I mean, within a matter of weeks, we were able to spin up an entirely new product set and entirely new classification. And that has to do with the power of our models and how we've been able to supercharge them using these new AI tools. So that's something that hopefully we'll see more in the future, which is being more agile. And it's not just speed, but it's agility, how quickly can we respond to market activity, to build classification to build products, et cetera?

And then the last thing that I relate about MFA, that I think it contributes the stickiness of clients rather than like being an incremental product. As you heard today on the stage clients are filtering out MFA. They are spending money on this, whether in-house or whether they're using somebody else and it was important for us to deliver this value to our customers so that they will not look for other places to achieve that.

Brian Fitzgerald from Wells Fargo. Maybe for Rami first, as you were building this business independently, were there any key pushbacks or obstacles or hurdles you had to overcome to get traction with this the BYOA model maybe complexity, maybe the incremental take rate maybe they hadn't fully maxed out their runway with the indigenous DSP algorithms. Anything there that as you grow the business, you overcame?

I think -- I mean that's the case for a lot of greater technologies. I think you can have your ID when you start, it may be a very sophisticated mathematical model. And what's actually been green products is constructing your technology with hundreds and thousands of use cases and thousands of small improvements that can only happen over the years. When you do conferences with lots of different use cases, you industries and verticals and DSPs. And I think that's what's basically the most important for us in building a competitive advantage was ready to be able to be exposed to all these use cases and have each time decent feedback loop in improving the technology plan.

Got it. And then a follow-on question is just for DV clients, who decide for whatever reason, we're not ready yet to go down the "bring your own rhythm path". Are there ways you can integrate the benefits of Si bids AI into some of the core DV products and customer experiences?

Yes. I mean we just closed the deal. So we're working through that. But yes, we -- there are a lot of ways that we can do that, that we're actually actively talking about that integration now.

One thing to think about, and we've talked to this...

I don't know how much I am allowed to say so...

Data sets are all about learning, right? And now we have the opportunity to learn about what's working and what doesn't work and what drives. So think about the impact on measurement and how we create measurement solutions, wouldn't Nielsen love have real-time feedback of how effective their data is and be able to change that and change their audiences, et cetera. So we do see longer-range implications of this, not just on growth but on the efficacy of our other products as well.

Yes, if you think about this Scibids primarily or mainly or only on programmatic, I see a future when the same solution is applicable to other bidding platforms such as a social network, will even be direct publishers and so on. That's in the future, but there's many ideas on that direction.

First, on Scibids, as you think about kind of integrating that acquisition and getting it ramped. Where do you see kind of a near-term low-hanging fruit? Is there a type of advertiser where this fits better than others? And how do you kind of direct that your go-to-market resources for that?

Yes, I want to say that. So in general, almost all of our customers can benefit. As you saw, it exist inside of the workflow of the DSP already. So any customer that is using that benefits. What I would say is we will begin certainly with our largest sophisticated lines because they will have more of an appetite to known and test or learn. So we'll start.

Yes, I was going to say the initial thought is when we look at what kind of market this opens for us is, first, performance-driven advertisers SKU are a bit more sensitive to the binary nature of viewable, nonviewable, et cetera. So that's like the no-brainer. So there's lots of folks out there that we may not have got products into because they limited their scope or reach. That's the initial like rent. But what always makes me like we have Diageo up here today. And Josh, you don't think Diageo is a DR or a performance advertiser, but they looked at this looking to balance things like brand suitability and reach along with cost and put those all together. So I think, as Doug noted, this touches every customer set. And the way our go-to-market is going to be is we're going to every customer with it.

Makes sense. And then on the data side, do you have a sense of what type of data is going to be most helpful to marry that customers can bring themselves? And is that going to be. Is there a complex implementation? Like is that going to require resources on the developer, engineering side? Or is that relatively self-serve.

You can talk about...

Well, I'll start as to -- the most efficacious data is certainly outcome data. So the idea is how do we draw direct lines or from proxies KPIs, the outcome data. So as more and more update becomes available, this solution power. To the extent that it's easy or hard to get that in, I'll go that [indiscernible].

I think the reason why we are so excited about that and the reason why we partner even before having this acquisition discussion is that there is a natural fit between bidding engines that we have built and it's actually a very hungry for that data and all the data that is available at the fiber or outcome that context so that like any other kind of data. And I think that's also the port thing we have to explore with obviously, we have data plans. But having just that I said is looking at exactly how is -- can it pivot into [indiscernible] testing what right at best opponent is part of the.

Yes. I will -- knowing the SIB technology going through the due diligence processes. Some of this data is already existing in the DSP. So if you think about [indiscernible], the Trade Desk, marketers are already using this platform to push the data out there and Scibids is integrated into this platform. So retrieving the data from there is almost seamless. Of course, there can always be a customer that has special data in Excel that they want to upload, but they have mechanisms in order to deal with that special cases.

Alan Gold, Fluke Capital. I've got two questions for you. First, Mark, Julie, congratulations on those new customer wins that you listed earlier in the day. Are those refilled customers? Or did they come from competitors? And the second question is the Scibids model is the first one of yours that takes a percent of revenue. I know you've always prided yourself on independents. Is there some conflict of measuring a company where you're taking a percent of revenue?

So first, on the customers in a mix of greenfield and some competitive wins there will be more specific, I think, in the quarter. But they're great brands, and we're excited about them, and Julie gets all the credit there. So thanks for putting in that combination. But really, she did all work. On the percentage of media side, I think it's important to note like where you are in the transaction and what that percentage in media actually means. Ultimately, the side of this business model is about driving an outcome. And although the model is based on a percentage of media, it's really -- if you think about it, it's a percentage of return. And that meaning if they're not driving actual performance that pays for that percentage of media, they're not going to get paid. And the ultimate goal of that return usually is to drop CPMs, so if you thought about what you would really want to be biased to do is not drop CPMs and not drive a return, which is antithetical to what Scibids actually does. So I think it aligns pretty nicely with the fact that the outcome that the advertiser is trying to drive is lower CPM, higher outcomes, better terms return. And the percentage of media that they take is really based on their success or non success. Now as Nicola noted, we're still going to evaluate the model ahead to see if we can align it up with how we approach customers. I think there's lots of opportunities there. We talk about a basket of goods and how we arrange the costs and how we sell those basket of goods, Scibids is part of that now. And beyond the fact that it's something we're going to be able to sell, we have already seen it in markets be something that differentiates that basket, which can take a customer to us. And that's the hard thing to measure, which is, yes, we can measure how much revenue we make from Scibids and how it may or other data sets. There's that other bonus that we have here is this is something our competitors don't have. We think it's going to be very hard for them to have. And when we go pitch a new customer like some of the ones we just talked about, it can tip the entire deal to us. So the Scibids is maybe 10% of that deal, but if we get 100% of a multimillion-dollar deal because of the differentiator, that's huge. So we're excited about that. I think that is it. We went over for time. We will all be hanging out here outside having coctails and food with, hopefully, all of you. Feel free to ask us questions then. I just want to thank everyone for joining us today. I know it was a long afternoon, but I hope you found it for filling, exciting and are geared up as we are about AI, what it means for us what it means for our customers and ultimately, what will mean for the people that invested in us, the people that you represent. So thank you so much. Thanks for joining us.