Salesforce, Inc. (CRM) Earnings Call Transcript & Summary

January 30, 2024

New York Stock Exchange US Information Technology Software special 58 min

Earnings Call Speaker Segments

Peter Sherman

executive
#1

Okay. Welcome, everyone, to Tableau AI: Propel Your Data Culture with Generative AI. So if you guys are here in this session, I'm going to assume that in some capacity you've been working on building a data culture within your organization. And this might be a newer role for you, or maybe you've been at it for a number of years, but hopefully, you've actually found some success with this. And no matter where you are in your journey to building that data culture, I'll bet that the first time that you heard about ChatGPT, you probably felt something, right? Maybe it was excitement, maybe it was eagerness. In my case, there was a little bit of skepticism. But also, I felt a little bit of pressure to figure out what exactly that meant for the journey towards building a data culture. And because this isn't changing that end goal of creating the data culture, it definitely changes how we actually get there. And that's exactly what we're here to talk about today is how generative AI is actually going to change the way that we approach that journey to building a data culture. Now before we get into it, I have to remind everyone that what we're going to be talking about here today is forward-looking. And so just as a reminder, reserve purchasing and investing decisions for what is actually GA in the product today. And of course, I want to start out with a sincere thank you. I'll speak for Harveen and Nate as well, but it's really an honor to be able to speak at events like this with our customers and have real discussions. You guys make all of this possible. So sincere thank you for that. So before we get in, let's start with a quick run of introductions. My name is Peter Sherman. I'm a Principal Solution Engineer, and I'm based out of Chicago. I specialize in a specific area of our product, and that's our augmented experiences. Harveen, would you like to introduce yourself?

Harveen Kathuria

executive
#2

Thanks, Peter. Hi, everyone. My name is Harveen Kathuria. I lead the Core Analytics and Tableau Public PM team, and I've been with Tableau for about 5 years now.

Nate Nichols

executive
#3

Hi, everybody. My name is Nate Nichols. I joined Tableau about 2 years ago with Peter through the Narrative Science acquisition. I was Chief Scientist there. Now I'm working on Tableau's generative AI strategy.

Peter Sherman

executive
#4

Thanks, guys. Now I want to kind of kick this off. And I mean those intros were nice, but I want to help the audience get to know us a little bit better. So I want to do a little bit of an icebreaker here. Now I'll ask you a question. When ChatGPT first came out at the end of last year, what was the first thing that you did when ChatGPT was launched, when you kind of played around with it, what have you? Harveen, again, I'll start with you to share your experiences there.

Harveen Kathuria

executive
#5

Absolutely. When ChatGPT was announced, just like everyone, I gave it a try. I asked some questions; did a lot of reading; made sure my team was getting educated on it as well. Thanks to Nate, we did that. But at home, the ChatGPT conversations were very prevalent as well. When I'm not leading teams at Tableau, I'm led by 2 very opinionated middle-schoolers. And so one evening, we started trying our ChatGPT. And my daughter and son are both into music, and we decided to ask some silly questions with GPT and we asked questions about, can you convert an English song into a Bollywood song? And to my surprise, the result was in Henglish, which is a Hindi version of English. And then I was like, okay, let's take this a little bit further. Let's ask some more questions, and can GPT be creative enough to just compose a song with the piano chords? And to my surprise, we gave GPT a few words, and there was a whole song with the piano chords as a result, which was just amazing. What I -- my biggest takeaway from that was if you tried, I think GPT and GenAI, in general, can be a path where it can be used for learning purposes. So yes, that was my key takeaway from that.

Peter Sherman

executive
#6

That's awesome. Yes, I didn't even realize that it would be able to come up with like an informal language or a slang. And then also like the language of music, too. It's just something that I never would have thought of. But yes, I think I would have been pleasantly surprised by that as well. Nate, what did you first do when ChatGPT came out?

Nate Nichols

executive
#7

So I was a little -- I don't know, I was a little unimpressed with the buzz around ChatGPT. I've been following -- this is -- my PhD was in machine-generated content. I've been in the space for almost 20 years. I was like following GPT-2 and GPT-3 and trying them out. And like it was super impressive, and you could see that it was getting better. But there's also a lot of limitations and it was very easy to trip it up. And so when people got all excited about ChatGPT, I was sort of just imagining it as it's that kind of stuck in a chat interface, and I was like, oh, this is cool that other people are getting a chance to use this and you don't have to hit the API and stuff. But I was kind of not expecting too much from it. And then one night, I was laying there in bed and my wife was asleep and I was watching American Dad!. It's like this kind of silly cartoon show, and there's a character named Jeff Fischer, who is like not on there for 3 seasons and then he came back in sort of silly story reasons. And I was like, I'll ask ChatGPT if it knows like why this character was not on the show for 3 years. And so I said, why was Jeff Fischer not on American Dad!? And it came back and it said, "Oh, it could be because the writers didn't know what to do with the character or maybe the actor Jeff Fischer was doing a movie." Maybe he was on another show or something. And so I felt all smug, right, because it was confusing the name of the character, Jeff Fischer with the name of the actor. And I was like, oh, yes, it's like GPT is cool, but there's still a lot of ways to go. And so I called it out. I was like, wait, are you talking about Jeff Fischer, the character or the actor? And it said, "Actually, the character is named -- has the same name as the actor, Jeff Fischer. And in fact, that's really common with cartoon characters." And I was kind of -- first of all, I've never googled this because I don't want to know if it's right or not. It's like important to me to not ever find out this is true. But I was like, I couldn't think of any characters named after their -- any characters named after their voice actors. And so I said, what are some examples of this? And it came back and said, "Oh, you're right. I misspoke. Normally characters aren't named after their voice actors' cartoon characters, but in this case, it really is. They're both named Jeff Fischer. And it liked blew my mind that it said it misspoke. And so I said, what -- like what do you mean you misspoke? I understand like when I misspeak sometimes, I kind of have this model of what's going on. How do you misspeak? And it said, "Well, I thought there was a lot of characters named after their voice actors. And then I scanned my memory and I couldn't think of any more examples. And so I knew I misspoke." And it was -- like I remember this. Like -- again, like the lights are off my bedroom. I'm sitting there typing to this like thing on my laptop, and I said, I know how you work. You don't have a memory. Like you're a probabilistic language model. You're just -- you're predicting tokens and there's a lot of math going on. It's very complex and very impressive, but you don't have a memory. You didn't scan anything, like that's not true. And it said, "You're right. You understand how I actually work. Most people don't, and I tell them that I scan my memory, and it like makes sense to them. But for you, yes, you're totally right." It was like a mathematical model being updated. And it's like, even now, it was such a surreal experience. And so it really got me thinking about these things as just almost like aliens, right? They're like really good at some things, but they're really not humans and the things they're bad at, they're really bad at. And so it's been a lot of fun working with people here and others at Tableau to try to figure out how we take these things that are intelligent, but not human intelligent, and really kind of build the right scaffolding and structure around them so we get a ton of value out of it, but we also get things that we can rely on and our customers can rely on and is secure and governed and all that kind of important stuff.

Peter Sherman

executive
#8

Yes. That's an interesting story. And further reckon, I would have googled that immediately because I would have had to know. But it is like your experience is an interesting one because you like conceptually know how it's working, but then you actually come across a real example and you're still racking your brain trying to figure out how the heck is it actually working on this one thing. And it's like, yes, it's kind of uncanny how it works. Yes, that's a really interesting one. Mine is not nearly as interesting. I had kind of put it on my to-do list. I was reading about it. I listen to like a daily news podcast, it came up, and I was like, right, I'm going to try this at a certain point. I didn't have anything to try it out on. So I waited a few days. I had my daughter's first birthday like that weekend, and I had to write thank-you notes after it. And I hate doing thank-you notes. I'm not sure if it's just because I'm like an ungrateful jerk or whatever, but I hate doing thank-you notes. And I was like, all right, I'm going to have this thing help me do the dumbest task of all time and write thank-you notes. And so I did it, it actually came up with a reasonable response. I had it adjust the tone so that it sounded more like something that I might say. It took a little while for me to almost describe like what is my personality, like how do I speak, and get the system to like match that. But I eventually got to something usable. Then it came time to write the next thank-you note. And I realized, okay, I want to run this back, but I want to like switch out the gift and the recipient. And I didn't realize it at the time that it was called that, but like we called that grounding with data. So I was doing this initially, and I thought that, that was interesting. A couple of months later, I figured out that, that's what I was actually doing and kind of connected the dots on it. But I just remember at the time thinking, okay, first of all, I got the task done so that's nice. But also like this can't be it. Like I know that there's a lot more cool stuff that you can do with it. And so my takeaway was really, okay, I proved out like a very basic example of how this can help. But I was really just more eager to figure out what this actually means in the context of my work and in the context of analytics and how we can actually like really use this technology and see what it can do and see if we can tackle some of the challenges that we've been facing for the last number of years in our space. So that was really my takeaway from that. So anyway, thanks, everyone, for sharing those experiences. I always think that, that's a fun question to ask people. I either learn something new about the technology itself, or at a minimum, something interesting about the people that I'm talking to. So if you guys get a chance, maybe that's a good conversation starter or an icebreaker if you're talking to somebody and you need to kill some time. But anyway, so everyone knows now what we 3, I guess, did individually with generative AI. And what we're really going to be talking about today is, what was the first thing that Tableau did with generative AI? And what did that turn into in terms of our generative AI strategy and kind of how we're building this stuff into our products? So that's what we're going to dive into. And yes, we can get into it. Okay. So before we get into what we're doing with generative AI, let's first set the scene a little bit. Tableau came onto the scene about 20 years ago and really revolutionized the way that we work with data. We've gone from messy and unmanageable spreadsheets to beautiful interactive visualizations. We've gone from these long reporting cycles to self-service analytics and data as an afterthought to data as an actual strategic asset. And we've really pioneered AI capabilities in the data and analytics space. We've come a long way, and all of these augmented analytics capabilities have helped us get closer to those self-service analytics. But something really big has happened recently. A really powerful technical capability has emerged in generative AI, and that hype is real. It's really been a huge catalyst for us on the product development side, and it's really allowed us to usher in this next wave of innovation that allows us to bring analytics to everyone. Now what do I actually mean by catalyst for change? I'm going to have to ask you guys to bear with me because I'm going to tell a story that may be boring at the beginning, but I have a point to make. So I actually came to Tableau via an acquisition of a small company that really focused on natural language generation. And we historically, right, had used templated NLG. So one of the first projects that I did was actually a daily story about how bond yields were changing, and this was for one of the major credit rating companies. So we would say something like yield on a 10-year treasury -- on 10-year treasuries were up 12 basis points, yadda yadda yadda. So it's not exactly the most exciting type of reporting. But doing this job, there were 2 real challenges with building projects out this way. The first challenge is that I had to understand the business rules. So when to say the sentence for bond yields was increasing versus decreasing versus staying flat since yesterday, all of that is data driven. I had to know what those business rules were. And then the second real challenge is that I actually had to figure out how to talk about bonds. And that part was templated. So I literally had to write out the words, but I didn't really understand the terminology of the jargon. So at one point, right, I had to fly out to New York and sit down with some bond analysts to figure out how to speak their language so I could write it into those NLG templates. And it may sound surprising today to hear that because we've heard of all of this hype around generative AI, but that's actually how NLG worked, at least up until the end of last year when generative AI came onto the scene. And it really changed the game. So with generative AI, I can now prompt it with the numbers and the calculations and the increase, and I can just say something like, write me a report using the typical jargon that a bond analyst would use. And it actually does a pretty good job. So here's an example of any sort of tool that's using natural language generation is now automatically getting vastly upgraded by generative AI. So that's really what I mean for a catalyst for change and for advancement. Now this is one example. This is an example around natural language generation specifically. But generative AI actually isn't limited just to language generation, even though that's maybe one of the most important or kind of biggest modalities that we've experienced so far. But as we've actually seen over the last few months, there is actually a ton of really incredible things you can do with generative AI that go well beyond natural language generation. This is a technological capability that is opening the door for us and for a technologist to really innovate and create really wonderful new experiences. And in the data and analytics space, it's no different. It's really being this kind of catalyst for change in allowing us to innovate. So with that, let's get into our generative AI strategy at Tableau and how we're using AI as a key piece of our product innovations moving forward. Our strategy really aligns to our 3 primary personas. It's empowering business users with smart personalized insights in the flow of work. And for analysts, it's making them faster and better at building data assets. And it's also allowing admins to enable generative AI in a secure and trusted way because Tableau AI is built on the Einstein Trust Layer. Now let's talk about terminology here. So Tableau AI is a brand that represents all of the AI-related capabilities in Tableau. And you'll hear the term Tableau AI a lot, but I think what you as users really care about are the net new experiences that we're building. So first, we have Tableau Pulse, which is a reimagined analytics experience built specifically for business users. And second, we have Einstein Copilot, which is an AI-powered assistant, helping data professionals become better analysts. So you can think of Einstein as the Iron Man suit, all of the Tony Starks in our Tableau developer community. Now again, this is all powered by the Einstein Trust Layer, which in and of itself isn't a net new experience, but rather how our platform architecture supports these new experiences while maintaining the highest levels of data security and privacy. So how are we using generative AI to empower the business user? Tableau Pulse is a reimagined analytics experience built specifically with the business user in mind. It's personalized to the individual consumer. It's delivered proactively and in the flow of work and it's smart and accessible, thanks to AI. So let's get into a demo of Tableau Pulse. Okay. So let's pretend that I am a store manager here. And this is the view that I now have with Tableau Pulse. What we're seeing here are the KPIs that I individually care about the most. And more importantly, what I see here at the top is a summary that's helping me understand what are the most important salient insights from those metrics or from those KPIs that I'm tracking. So here it's calling out that device sales had an unusual spike since the start of the week. Inventory fill rate is actually underperforming. But when we actually look at revenue, it's steadily climbing and it's kind of in this normal range. So it's really kind of cutting down on the noise. It's looking across all 12 of my metrics and telling me the things that I actually need to care about here and now. So if I want to analyze this a little bit deeper and explore what's going on with device sales, I can click into this card to get a more detailed view. And as a consumer, I want to make sure that I have a good understanding that this data is up to date; that I can trust it; that this is that single source of truth. So here, I can make sure that I can see how this metric was built, what it's based off of, and I can feel confident that this is the same number that my boss is looking at and that the rest of my team is looking at as well. But let's say that I'm looking at this -- these device sales numbers, and I want to really understand what caused this increase. The system is going to prompt me with a question on where I might want to take this analysis. So maybe I want to look at this by product. So I can just click on this question, which products drove the sudden increase. And it's immediately going to break down that delta by the different products that I have. So it's calling out e-phones as being the primary driver of that increase in sales. Now I saw on that other page before that it mentioned that my inventory was maybe an issue. So now my next question is, am I actually going to be able to fulfill these orders. But if the system is not actually prompting me with the question that I actually want to take this analysis, what I can do is I can ask any question up in the search bar, so I can say, "Hey, are we going to fulfill our phone orders?" It's going to make a connection that I actually want to do some analysis on inventory fill rate, which is another metric that the system now knows about and has made a connection to between sales and inventory fill rate. So I can click on that, and I can get a deeper explanation of what's going on with fill rate. And it's telling me, as expected, that increase in sales is actually causing somewhat of an issue, right, with our inventory. And this is maybe something that we now want to take action on. So I might share this out, but in fact, this is something that I actually want the rest of my team to be following on an ongoing basis because this isn't going to be fixed overnight. We want to make sure that we're staying on top of this. So I want to make sure that we're all following this information so that we can continue to be updated on this and that this can be monitored as we proceed. Now let's see how we actually create and sort of support this experience. So that will kind of be the end of the experience that a business user would experience, but how do Tableau developers actually support that, right? So this is what it takes to create a new metric. First, you choose the data source. You connect to it. Let's say that I want to make a metric on customer satisfaction. I would choose that field in the data; I would choose the aggregation; I would choose the time dimension, in this case, it would be like the survey date. And right off the bat, these are like the key sort of core ingredients for what defines customer satisfaction. So I can save this off. And from here, we can have different followers following different versions of this customer satisfaction score metrics. So maybe we would change the time frame or the comparison period depending on the individual who's consuming this, they might want to personalize it in whatever way is most meaningful or impactful for them. Let's say, though, that I want to create another type of metric where we don't actually have all of the data in the data source. So maybe we're trying to do a calculated field. So one example of that might be churn rate. So I can -- for more complex metrics, I can actually open the advanced editor. And if we wanted to make something like churn rate, first of all, we would be tracking this by disconnect date, and then all that we have in our data is total subscribers and lost subscribers. So to calculate something like churn rate, we'd make a calculated field here. And we would define it as lost subscribers divided by total subscribers. So we're doing the math here in the metric layer, adding in this business context so that we can actually track this metric that we care about. And here, we've created a pulse metric based off of this calculated churn rate. So something that wasn't inherently in the data source. Now you might be thinking, okay, we're going to be building up this metric layer to be able to support the Pulse experience. That could take a lot of time and effort to build that out and to add in all of that business context to create and produce this robust sort of consumption experience. Well, one exciting thing that we're doing to incorporate AI is we're actually using it to help what we call bootstrap, that metric layer creation process. So if we try this out, basically, what it's doing is it's going to scan across all of the metadata from our data source. It's going to look at column headers and its "understanding" of the world, and it's going to make connections. It's going to connect the dots between specific measure fields and the related date fields. It's also going to present different dimensions by which you might want to analyze something like subscriber churn or customer acquisition cost. So it has a good understanding of how these metrics should be analyzed. And all of that stuff is defined in the metric layer, but you're able to really speed up that process and give yourself a head start by using generative AI to kind of bootstrap that process. Now of course, there's always a human in the loop, so it's really just making these suggestions, and you can choose to accept them or toss them out if they're no good. But what you've just experienced is Tableau Pulse. And now I'll hand it over to Harveen to talk about Einstein Copilot.

Harveen Kathuria

executive
#9

Thanks, Peter. For the next few minutes, I'll talk about how Einstein Copilot will empower every user in their data analysis workflow. I want to start with The Wall Street Journal article that was published in March this year about tech skills that are expected by many employers today. As per the article, proficiency and productivity tools isn't enough anymore. Data analysis is one of the top skills desired by many employers. Let's admit not everyone in the organization, including myself, for example, has the word "analyst" in their job title. That said, we use data almost every day to make business decisions. While data professionals are the backbone of an organization that drives the data culture within the organization, they're responsible for capturing business requirements, prepping data, creating content for their end users. There are data-savvy end users who do not have the time or the skill set for deep analysis, but are curious and want to ask additional questions about the insights shared with them without learning all the ins and outs of Tableau. Einstein Copilot for Tableau is an assistant for data professionals and data-savvy end users alike. It will be integrated into the Tableau suite of products to make all users successful in their workflow. Tableau is a deep product. It has many features. With Copilot in product assistance, users can ask a question in natural language about a specific feature and get step-by-step guidance on how to use the feature. This is truly powerful. Data-savvy end users and new data professionals will benefit from the Copilot. New data professionals will learn by doing. Einstein Copilot will be a true in-context assistant for data professionals. Repetitive, time-consuming task can be tackled by Copilot. From data prep to formatting worksheets and dashboards, Copilot will automate many such repetitive tasks. Pulse metrics generation, Slack integration for sharing insights with others and many more can now be tackled by the Copilot. All right. I'll admit, calcs in Tableau are still hard for me. But with Copilot, you can create calcs in a matter of seconds. The possibilities with Copilot are endless, and we are just getting started. Let's dive into a demo. For this demo, let's assume I'm a novice analyst at a pet retail company. I have access to customer purchase data. I want to use this data to create personalized experiences for my customer that will drive incremental revenue. As a novice analyst, when I see this blank canvas, I have no idea where I should start my analysis. Einstein's Copilot has got me covered. Using generative AI, Einstein is able to understand the context of my data and presents me with a set of recommended business questions. I am interested in understanding the pattern over time for sales across product categories. So I'm going to go ahead and click on the first recommended question. I can see that the sales of outdoor sporting goods is high in summer, which is expected. I'm now curious about the location of these customers who are purchasing outdoor products. I'm going to go ahead and type my question into this text field. I am going to submit my question. And just like that, Einstein Copilot created a map verse for me. This is the power of Einstein Copilot. Even without deep knowledge of the product, in a matter of few clicks, I'm able to kickstart my analysis. The best part of all this is, I'm learning the product by doing my analysis. I had no knowledge about map layers, no knowledge about creating map visualizations in Tableau, and Einstein Copilot made things very easy and simple for me. Over to you, Nate.

Nate Nichols

executive
#10

Awesome. Thank you, Harveen. Of course, none of these really cool new capabilities that Peter and Harveen have shown are actually worth much if you don't feel comfortable turning them on for your users or your legal and security and compliance teams block you from using it. So it's super important to us that not only are we able to deliver value from generative AI, we're able to do it in a way that's consistent with Salesforce and Tableau's reputation for trust, security and governance and all those things that are really important to customers. And from talking with customers, we've identified 3 main sort of buckets of concern with generative AI. I want to be clear that these are all exactly the right buckets. These are the correct things to be concerned about. The first is around privacy and security. And in particular, what kind of training is happening off of the customers, prompts that are sent to the language models and the responses that come back. I'm sure a lot of people on the call have heard what happened with Samsung, where some employees shared confidential information with ChatGPT and ChatGPT trained off of that. And now you can get confidential Samsung info while chatting with ChatGPT. That's not anything we want. So figuring out how to deliver all this in a way that's private and secure is super critical. The second, of course, is accuracy. We polled 500 senior IT leaders, and I think 60% of them were hesitant to use generative AI because of perceived issues with accuracy. And if you've used ChatGPT or Bard or some of the other models, you've undoubtedly had the experience where it hallucinates something, and it says something that's totally wrong, but it says it in a very confident and accurate-sounding way. And so how do we make sure that as Tableau, as Salesforce that we're able to deliver all of the trust and accuracy that customers expect from us? And the third one is around bias and toxicity. These large language models are generally trained off of the open Internet, which if you've been on the Internet, you probably realized that it's full of bias and toxicity. And so we need to make sure that the content that's coming from large language models is something that you're comfortable putting in front of your employees, in front of your customers and users and without any risk to your brand. And we're tackling all 3 of these issues really holistically with the set of services and components that we're calling Tableau AI. And really, the key part of Tableau AI is built on top of Salesforce, and in particular, the Einstein Trust Layer. The Einstein Trust Layer is a huge investment that Salesforce is making. Everything that's happening with generative AI at Salesforce and obviously at Tableau is always going through this trust layer. And it's really the middle ground helps us marry the security and governance models we know from sort of typical column data, sort of data cloud or in Tableau data sources where we understand how to do role-level security and permissioning, that kind of stuff, with this brave new world of large language models that have a lot of sort of different properties and different things to think about when you're thinking about security and trust. And it's really because of the Einstein Trust Layer that we're able to bring these 2 worlds together and deliver them together in a way that customers can get a lot of value from and also trust. And in particular, how the trust layer works against the details here, but I want to call out the 2 things that normally is the biggest relief to people, to customers who were having these conversations. The first is that no matter what language model provider we're using, and we -- I'll talk a little bit about the different options we have. But no matter what, there is never any training that happens off the customer's data or the customer's prompts that are sensing the model. There's no training from the customer's data. So there's no risk of getting Samsung and nothing is ever stored outside of Salesforce's trust boundaries. Those are the 2 things that are really important. And then generally, you can see a little bit on the right side here is that we've got a lot of different language model providers we have available to us. And so when -- as a feature team, when we're building a new capability that's powered by generative AI, part of what we're doing is prompt engineering and work against the net space, but a part of it is also identifying the right model provider and the right model to use. Balancing a lot of things like performance, latency, cost to serve, those are the kinds of factors that we're balancing. And really, we're supporting models from 3 different types of model providers. The first are commercial APIs like open AI. The second is open source models that we're hosting internally like you've heard of Llama2 or Stable Diffusion models like that. And the third one and one we're really excited about is models that we're developing in-house. Salesforce research was actually -- did a lot of foundational work on large language models. We've got some of our own running in production. And so we've got -- as future developers, again, we get to use the right model provider and the right model from that provider. But again, nothing is ever trained from. Nothing is ever stored outside of Salesforce's trust boundary. And to go through this trust layer just a little bit here. We've got Tableau, which like everything at Salesforce, is sitting -- only does generative AI through the Einstein Trust Layer. Here we're coming up, this is where we're grounding the prompt. So we're taking the prompts that we're building as part of the features, pulling in, in Pulse's case, pulling in the insights, for instance, that we want to summarize that's here driving the data and doing what's called prompt grounding. Then there's a data masking step where the system is looking for any PII in the prompts, any personally identifiable information, things like names or social security numbers, replaces those with variables. So those never leave Salesforce's 4 walls at all. Then it goes through the secure gateway. And this is where it get sense of the different model providers, again, with 0 retention and 0 training happening from that. We've -- our security teams have double checked that nobody in open AI ever is able to look at these. It's not being stored or anything like that. So this really is 0 retention and 0 training. Then it comes back, we do -- we demask any data and get the PII back into it, so the names are carried through. We've got toxicity detection that happens on every response coming back where we developed some of our own in-house models that can detect bias or toxic language. This doesn't come up a lot in the Tableau context, but it's really important to have in there, again, because it's -- the language is coming from us, and we stand by it. And then finally, the audit trail. So we do store the request and the response for 30 days, that's for compliance purposes, but that all happens within Tableau Cloud and is not something that's ever -- that ever leaves our trust boundary. And then it comes back to the user. And so this is the round trip that every bit of generative AI takes at Tableau and really across the whole sales force ecosystem. Taken together, Tableau AI is built on top of the Einstein Trust platform, which is great for us at Tableau because there's so much heavy lifting in security and processes and trust built into that, that we get to really be a boat that flips on top of the rising tide of Einstein. It's great for us because we can focus on the parts that are valuable to our users. Nothing is ever trained or stored outside of Salesforce's trust boundaries. And as you saw with the trust layer, I think we're really able to provide all the awesome new capabilities of generative AI that Peter and Harveen were showing, which we're super excited about, but also in a way that Salesforce and Tableau feel comfortable with. Trust is our #1 value, and we really expect customers to be able to trust us and what's coming out of our systems, like they would anything else in Salesforce or anything else in Tableau. So thanks so much for your time so far, and now we're going to open it up to any questions. Thanks.

Peter Sherman

executive
#11

Thank you, Nate. Can you hear me? I'm talking.

Harveen Kathuria

executive
#12

Yes, I can.

Nate Nichols

executive
#13

I can hear you, Harveen. Peter, I think it's just you, buddy. Do you have the little circle? Like on the presenter tab underneath your face, there's 2 circles for camera and audio. Is that one on? That one took me a second. All right. Let's -- Harveen, let's tag Peter out. Peter, if you can fix your mic, come back in. But otherwise, let's start going through the questions. Awesome. So let me scroll back down here to some of the first ones. Industry-specific models available or can they be added? So that's a thing we're looking at in the future. Right now, we don't have -- there's no industry-specific models. Again, as I mentioned on the call, where there's a lot of different models we're supporting with our gateway, we have -- sort of it's easy for us to support additional models in the future, and in particular at Salesforce and our background at Salesforce research, we'll be investing more heavily in our own models. And we've obviously got a lot of domain experience and domain knowledge to build out more models that are specialized for particular industries or verticals, but that's not anything we're doing now. When will it be released? And is it priced as an add-on? So Pulse will be announced at World Tour next week. It is super exciting. The pricing as an add-on, I don't know if we're quite saying that. Yes, I think we're -- I think we're saying this. We'll say it now anyways. Pulse will be available to any -- it will be available at no charge with any SDKs and licenses you have already -- excuse me, with any licenses you have already. You will get Pulse viewers, creators, explorers. Those will all come with Pulse included. And again, we're launching that at World Tour next week and part of a big -- would be a big part of the keynote. We're really excited about. I guess, answer that with question two, it comes with Tableau Cloud, I think that was mentioned in the call, but Pulse is Tableau Cloud only. Peter, I may hear you now.

Peter Sherman

executive
#14

Can you hear me?

Harveen Kathuria

executive
#15

Yes.

Peter Sherman

executive
#16

Sorry, guys. I had to refresh. All right.

Nate Nichols

executive
#17

Do you want to take over?

Peter Sherman

executive
#18

Yes. I'm reviewing these questions as they're coming in. So we can just take the next one. So how does Pulse or Einstein connect a specific metric to a specific question with different words, for example, knowing inventory? Basically, yes, Nate, I don't know, do you want to comment kind of on how we're using vector search for this and not actually generative AI?

Nate Nichols

executive
#19

Yes. So briefly, if you've heard about -- if you've been sort of in the generative AI space, you may have heard about vector search or semantic search is sort of a -- my cat's here and talk about it as well. It's sort of an ancillary technology that's used with generative AI, but isn't strictly generative AI itself. But the really important thing is that rather than -- typically Google and everybody else was very specific about the words. And so sales was a completely different word than revenue even though for us, they're often synonymous, right? But to a traditional search engine, those are completely different words. And the exciting thing about vector search and semantic search that we're using in Pulse, and it's also being used in Copilot, is that sales and revenue are very close together in that semantic space, and it gets math-y. But the important thing for us is that going from like fulfilling inventory to a fill rate, that wasn't a thing that a user had to do. It's not a data dictionary with a bunch of synonyms where you have to type all that stuff in by hand. We're using the knowledge that the -- basically the language understanding of the large language models to make that a lot easier. And that required manual effort.

Peter Sherman

executive
#20

There's a couple of questions about like rollout and licensing. So I think I'm just going to try to address a couple of these all at once. I'm going to note that this answer is specific to Pulse, and then I'll let Harveen kind of talk about the latest thinking on Copilot. But that one is a little bit further out. So in any case, with Pulse, it's going to be GA in February, as current thinking, obviously, forward-looking statements apply. We're going to be in an open beta though next week. So anyone will be able to try it, you'll just reach out to your account team and we'll get you set up on it. Will it be licensed as a separate product? So Nate kind of touched on this. No, it's just going to be included as part of the existing kind of Tableau Cloud offering. And then, I guess, similar kind of question. Will Tableau Pulse go to Tableau Server? Pulse is going to be released on Tableau Cloud first. We generally have kind of this approach from the product side now where we are definitely cloud first, especially on most of our newer innovations. Obviously, we're still going to be supporting Tableau Server moving forward. But Tableau Pulse is going to be on Tableau Cloud first, and we still need to kind of figure out if and when we will push Tableau Pulse to Tableau Server. So that's definitely one where you want to reach out to your account team and kind of talk about the specifics of your own situation on that one. But yes, I think -- yes, okay. So I think I just covered that last question about Tableau Cloud in the future. So again, the door is still kind of open on that one, but definitely reach out to your account team for more details on that. Harveen, do you want to talk about just like the kind of timing and rollout of Einstein Copilot as well? And I know that licensing is still up in the air on that one.

Harveen Kathuria

executive
#21

Yes. Licensing is still up in the air. Thanks, Peter. I responded back to some questions in the thread directly. For Einstein Copilot, we are going to have a pilot release coming up in spring of next year, and GA is summer of next year. As Peter suggested, please reach out to your account teams, feel free to ping me on LinkedIn, if you want to participate in the pilot program. There's a question about desktop users, I was going to respond back to that as well. We are exploring ways in which we could potentially bring Einstein Copilot for desktop users, no commitment yet. I just want to call that out that, that is something that we're actively exploring. I want to answer one more question by [ Shauna ]. Will Tableau Public be involved in any of the beta testing? Another area that we are actively exploring, if we can bring copilot on public because that would be a great way of testing with our existing users. It will be a closed beta testing approach, but that is something that we're actively exploring as well. Back to you, Peter.

Peter Sherman

executive
#22

Yes. So just 2 quick questions from [ Katie ] and [ Jordan ] on -- basically on versioning. So because it's on Tableau Cloud, it will just be on the latest version of Tableau Cloud as it's rolled out. So again, open beta in a week, and then it will be GA probably around like the February time frame. So everyone will just start to get access to it. Okay. I'm trying to -- so I think there were some questions earlier on, though, about setup. So from [ Sherry ], does it require setup? I think this was referring to Tableau Pulse. The answer is, yes, it does require setup. So what you saw me demo as part of like metric creation, that is the setup that's required. So it's really just a couple of clicks to get it going, but you do very much need to define -- like these are the metrics that I care about. This is the data field that represents the measure. This is the time dimension that it should be analyzed by and then all of the relevant dimension insights -- or sorry, insight dimensions that we should be calculating things by. So that is kind of what we mean by the setup in a Pulse world. It's really defining that original metric definition. Okay. Let's see. Will certain features of Pulse be able to be limited by admins? Yes, that will be coming. So in beta, basically the way that we'll kind of activate it is we'll be turning it on at the site level. In GA, the plan for that is to create more controls for admins to be able to turn it on at the site level or possibly have like user-level permissions on who can do what, specifically around like metric definition creation and how you set up business users to be following certain things. Yes, more to come on that.

Nate Nichols

executive
#23

We also just to call out all of generative AI is gated behind the toggle that starts off. So there will have to be an admin who comes in, accepts terms about what we're going to do with that, which is details on what we laid out here, and then check in the box. And until that box is checked, we will not do any generative AI.

Peter Sherman

executive
#24

While we're on the generative AI topic, Nate, do you want to just touch on PII and kind of how we're sort of managing data governance around that within the product?

Nate Nichols

executive
#25

Yes. So PII, as you, I think, saw on the slide, it's part of the trust layer. So it's a -- basically, all that trust layer is that's all real. Everything in there exists but it's all sort of V1s of that. And so the current V1 solutions of PII is basically different pattern recognition things to look into the prompts for things, to look like names, to look for social security numbers, phone numbers, that kind of stuff, and then those can get replaced. I was just in a meeting this morning about improvements we're making today, which will allow for more individual control admins who'll be able to turn that on or off. So there's a lot of road map coming there as well as on the toxicity detection. But that's -- yes, the basic idea is that pulling up things that look like names. Right now, there's no control over it. There will be increasing customer control available there. And again, that's all sort of a cherry on top of the fact that nobody outside of open -- outside of Salesforce, like open AI, it's never stored at all. It's not even like -- it doesn't even hit their logs. So the PII is on top of that no training and no retention policy we have.

Peter Sherman

executive
#26

There's a question from [ Amelia ] on kind of like data cleanliness. And so similar to other like analytics and BI tools, Pulse and Copilot are going to -- I guess the performance of them is going to depend on the quality of the data. So obviously, higher quality data is going to yield better and kind of more valuable content. And there is nothing about those products or those features that will sort of like detect mistakes in the data? I mean they might just shine through in what you're reading and it might make you think, hey, like that doesn't sound right. Like maybe that was a data mistake. So the AI isn't going to like comb through, though, and tell you where you have nasty the data. There may be other products on the market that they kind of focus on that sort of thing. But like with Pulse and with Copilot, all of the data best practices still apply with Pulse in particular, like you literally have to set up metric definitions based off of published data sources. So the assumption is that those will be kind of curated in clean, complete data sets.

Nate Nichols

executive
#27

One other thing I tap there, like using Tableau AI and generative AI to help people with their data cleanliness and help people manage their data. That's the thing we'll be talking a lot more about in the first half of next year. It will be a huge focus for us. Right now, we're sort of serving the consumers with Pulse and Copilot. We'll be moving more into that sort of data stewardship space. We saw a little bit of that with creating calculated fields for you for generating descriptions of a data source automatically but things like catching outliers in the data. There's a whole bunch of stuff we want to do there, and we'll be talking a lot more about next year.

Peter Sherman

executive
#28

Question about the difference between current Tableau metrics and Pulse metrics. It's a good question, honestly. So the way I kind of think about it is Tableau Pulse is sort of an evolution of a number of different augmented analytics features that we have today. So Tableau metrics, the existing version of it, is one of them. But you can also kind of think about, explain data and data stories and ask data where across those features, it covers natural language generation, natural language query. In the case of Tableau metrics, just the sort of UX and I guess, like visual design components of like what a metric is. So all of those concepts have been taken into account when designing Tableau Pulse, and it's really like a combination and kind of a reimagination of all of those features. So that's kind of the way I think of it. It's sort of like the next evolution of those features. Copilot have any automation features to help reduce repetitive tasks? Harveen, do you want to comment on that?

Harveen Kathuria

executive
#29

Yes. Yes, great question, [ Brandon ]. That's the goal of Copilot, right, for existing customers, creators and explorers of Tableau today to automate some of the repetitive, tedious tasks that we have, including things like creating calcs, formatting, like I mentioned, and a lot more. So it's truly an assistant so that you can -- the time to insight is faster. So that's the goal of Copilot. I hope I answered your question.

Peter Sherman

executive
#30

I think we have worked through the queue, but we'll give it -- we will give it another minute or so and see if there's any last questions here while we're on. Would the metrics created by Pulse be a content type and reused like public DS? Trying to think about how to answer that. So the metrics and sort of all of the content for Pulse has been architected in a way to be reusable in a number of different ways. So what we've seen in the web app is basically like our built-in version of how you can actually see and interact with Pulse content. But everything under the hood is based around APIs that can be accessed and used in really a number of different ways. So like the e-mail delivery and Slack integration, those are other things that we've built out to be able to get that Pulse content in other places. But you can imagine a world where -- and actually, this will be available at GA. If you wanted to build your own front end using Pulse content, you could use Tableau's embedded API to access Pulse services and get Pulse type of content there and be able to kind of do whatever you want with it. So I think that's maybe what this question was getting at. Hopefully, that was a helpful answer. But the idea behind that is we wanted to architect it in a way that it can be very extensible in the future so that we can build things like integrations with Teams and current Tableau dashboards and Salesforce and really anything. And so yes, we're excited about the future there and what customers will do with that. All right. Well, I think that's all the questions. I guess I'll just say I appreciate everyone's engagement, really, really good questions. Obviously, we at Tableau, we're super excited about the Pulse launch. And we're super excited about what's to come with Einstein Copilot, the beta around that and the launch next year. But this has been really, really fun. I don't know, Nate or Harveen, any closing thoughts before we end it?

Nate Nichols

executive
#31

That's great to see the interest and we're super excited to have it announced next week and launched, Pulse.

Harveen Kathuria

executive
#32

Yes. Exciting work ahead and truly excited. Thank you for joining and all the engagement today. Yes, keep the questions. Reach out to us via LinkedIn, reach out to your Account Managers. We're just getting started with AI.

Peter Sherman

executive
#33

All right. Thanks, everyone. Have a good rest of your day.

Read the full transcript via the API

You're viewing the first half of this call. Get the complete Salesforce, Inc. transcript — plus 248,000+ transcripts from 12,000+ companies, speaker segments, AI summaries and full-text search — through the EarningsCalls.dev API.

Get the API View API docs →

This call discussed

For developers and AI pipelines

Programmatic access to Salesforce, Inc. earnings transcripts and 248,000+ others is available through the EarningsCalls.dev REST API. Plans from $24.99/month — full transcripts, speaker segments, full-text search, and the recently-added /api/v1/transcripts/recent polling endpoint for ETL pipelines.