Amazon.com, Inc. (AMZN) Earnings Call Transcript & Summary

Well, whether it's acquisition or retention, we're just happy it's working for you so far.

Yes. Yes. We're excited to get into a more rigorous pilot through February to have some more defensible data on conversion metrics.

Looking forward to being put Through the wringer.

Yes, can wait.

It's very cool. And Jenna, I'm sure if any of my team members are out there -- are there any Commerce Cloud members out there? There we go. As triggered as I am by hearing the words experiment, change, holiday season, December, why and how is it possible to actually experiment with something so close to such a critical time when most retailers are just completely locking down their e-com infrastructure?

Yes. I mean my perspective is actually twofold. I've been broken over and over again. When I was at SNIPES, we're selling Hike sneakers, right? 1,000 people -- 1,000 -- 1 million people showing up for 1,000 pairs of shoes, right? So dealing with Black Friday, 3 days a week, really put us in a position to build the appropriate infrastructure on Salesforce, by the way, to manage that level of volume and velocity to keep everything safe and thriving. So I'm a little maybe riskier when it comes to pushing the needle. We never went into code-free at SNIPES. We just kind of blaze through. So between that and the fact that it's a JavaScript project that layers over the site requires a little bit of testing, but it was pretty seamless. Getting this live really wasn't that big of a risk for us, and we took the risk and it paid off, so.

I'm still glad it worked. Michael, how do you think about the Salesforce integration with Buy with Prime?

Well, I love it. The thing -- what a surprise, I made the joke to Peter when we first started working together that as an Amazon customer, myself and a frequent user of Buy with Prime in our home life, I feel like Commerce Cloud and Amazon Buy with Prime coming together, it's like it's kind of cheesy, but we're like 2 best friends who somehow just haven't met yet. And the reason I love it is as you can see here on the screen is that we're bringing the power of Buy with Prime directly into Commerce Cloud. And as Peter mentioned, it's not just empowering a checkout method and some fulfillment options, which are very powerful. It's actually deeply connected to our order management infrastructure so that our customers can actually interact with the Buy with Prime orders and manipulate them using our order management system. They'll see the Buy with Prime information enriching the Customer 360 profile that every Commerce Cloud customer can use to look at who's coming to their site. And this is why Commerce Cloud is a part of the larger Customer 360 from Salesforce. So customers like Solo Brands that use more than one of our applications can actually see the Buy with Prime experience from a merchant perspective, traverse the whole life cycle. Service agents can actually look at and manipulate the orders. They can help issue with the refunds, and they can actually see the transactions just like something that processes through the normal checkout options. And that type of unification from the commerce experience through the service and even at the top line through the marketing journey is a really powerful multiplier of Buy with Prime plugging into Commerce Cloud. So I think we're just at the beginning, Peter. I think there's so much more we're going to do together, and I'm just so excited that we're getting to start our partnership today. So thank you.

Yes. Me too as well. It's great to work with great people too. Really fun to get to know you. And a little bit of you. We're starting to get to know each other, which I'm happy about.

I appreciate that.

What about your thoughts on what's next for Buy with Prime?

Fantastic question. So glad you asked. I mean, listen, I was super honored to get to see this integration before it was GA. What I will say is that one of the most complex parts of the customer journey, the thing we cover the most and protect the most is the checkout, right? It's the cart. The benefit of this integration is pretty interesting. It really helps you jump the line on the mixed cart use cases. And so being able to actually treat individual SKUs differently within a cart, this cartridge really allows you to kind of springboard development and to create an experience where you can recognize the fact that this item is eligible for X or Y or Z. X could be Buy with Prime. Y could be BOPIS. Z could be shipped to home. And so that's always been something that's been a little bit nebulous and hard to navigate. So knowing that through this integration, we can have a more seamless go-to-market strategy when it comes to mixed cart, I'm pumped to test it out.

It would be really interesting. Are you thinking about inventory any differently in this experience or?

Yes, definitely. I think as we manage our marketplaces, we're always dealing with inventory, right? Those -- sorry, dreaded storage fees, right? We have to be very specific in what we put into Amazon from a marketplace perspective. We know that Buy with Prime acquires us to have inventory that's with Amazon because they're the ones actually shipping these items out, right? Transaction takes place on D2C, gets routed to Amazon for fulfillment, all is good in the world. So what's interesting though, and I kind of flipped in and out its head a little bit, what I noticed is that we might be able to take some bigger swings on our assortment in Amazon by utilizing Buy with Prime as a way to derisk that inventory position. So if I know that I'm going to put core products in marketplace today because I know we're going to drive that sell-through, maybe I can drive some more accessories. Maybe I can create some more unique bundles with some more accessories. Maybe I can actually try to widen the assortment within marketplace to test that in the marketplace environment but know that in the moment that, that sell-through isn't hitting the volume or velocity that I anticipated, boom, I can turn on Buy with Prime, and I can all of a sudden use my direct-to-consumer selling power to actually drive throughput on those units and de-risk that storage fee or have to pay to actually bring those units back into our distribution center. So I know it's a little bit of a nerdy perspective. But I think from a logistics standpoint, it's a cool way to derisk that inventory.

It's those little nerdy things that actually make e-commerce win. Listen, Jenna, thank you so much for beginning this journey with us with Buy with Prime. Michael, thank you so much for being a partner with us. We're really excited to work with you both. Thanks for showing up here today. Really appreciate it. For all of you, that's what we had to talk about today. For those of you who are interested in the sale -- in the Buy with Prime for Salesforce Commerce Cloud integration, you can go to the Buy with Prime site and sign up for early access. You can also see a sales force and Buy with Prime demo in the Buy with Prime booth, which is in the AWS booth, which is 60-20. And immediately after this, we're going to take that concept of free, fast shipping, and we're going to go over to the Salesforce booth, 51, 25 for a free fast sipping happy hour. So please join us. Thank you very much.

Thanks.

Of course.

I'll start off by introducing myself. My name is Jon Jenkins. Everyone calls me J.J. I work at Amazon and my team at Amazon builds the technology that is in Just Walk Out stores. So if you've seen that store out there at the center or at the front of Javits Center or where you can just like walk in and not go through a checkout. My team builds those. And so I'll turn it over to Bill.

Appreciate it. So my name is Bill Toney. I don't have a cool name like J.J. So Bill Toney. Head of Global Market Development for Avery Dennison. And my team is really responsible for helping retailers or brands all throughout the world, understand how to take emerging technologies and particularly sensor technology like RFID. How is it really going to help you transform their business process from an ROI perspective, technology perspective. So that's what my team does globally. I've been an emerging tech and retail for 28 years, 4 years, running retail stores, about 25 years doing emerging tech and trying to innovate around retail. So happy to be here and look forward to the discussion.

Cool, yes. So we want to start off by getting you guys fired up and activated, so I'll see if I can get people excited here. So everyone who loves standing in checkout lines, raise your hand. Okay. I've largely failed. We'll see if Bill can do better, you can fire your questions.

All right. How many people have actually abandoned the purchase because the line was too long, Raise your hand.

Okay, Bill wins, I see how it works. So Bill and I were talking yesterday in advance of this session. And you mentioned this is your 23rd?

Twenty-third.

Twenty-third time at NRF. I've only been here twice. So you clearly have an idea of what the trends have been here over the years. And I guess I'd be curious to know like what are your perspectives on the trends and even some maybe micro trends about how shoppers frustration sort of factors into retail?

Yes, it would be great. Appreciate it. First of all, it's great to see so many people here at NRF. I mean it's back this year, and it's really exciting. I think there's really 3 micro trends. And going to different presenters and then kind of walking around the NRF this year. I think there's 3 key things. Number one, consumer experience. I think post-COVID, people are coming back in the store, and I think people are really trying to understand how do I create a frictionless consumer experience to really help drive more brand loyalty, what retailers called NPS or Net Promoter Score. And that seems to be a really big kind of theme as you kind of walk around the show and kind of listen to different executives talk about retail. I think the second one is really around labor optimization. So how do we actually take labor and save that labor and plow it back into things that have a little bit more value for the customer or in some markets, they have a labor scarcity, how to actually run light my operations with trying to optimize my labor. And the third really is around theft. I think a lot of people have heard some themes around theft. And 3, 4, 5 years ago, you would hear this theme around loss prevention. So people would put hard tags on products and try to prevent theft. And obviously, that's still a strategy, but a new theme emerging is what we're calling kind of loss detection. So how do you take analog technology like EAS technology and make it a smart sensor technology where you can get data analytics and sort of predicting certain things. Start to understand in our retail operation, what actually left your stores so you can replace that item and make sure it's there for the next customer. So I think those are kind of the 3 big things I would say that we typically see.

Like from a technology perspective, is there a technology or a way that you're starting to think about addressing some of these trends?

Of course, we think of RFID because it's the...

Oh, so hang on here. So -- like it's not 1994, RFID is not cool. We're in a big idea session, right? Like we can't possibly talk about RFID at a big idea session. What's up?

Well, it's interesting, right? Because I think some people still think of RFID back in the 2000s where it was kind of case level and the technology was just starting to get kicked off. But a lot has changed since then. I think there's really 2 key things. First of all, the technology itself has gotten from a performance perspective, smaller and a lot more higher performing. Obviously, price has changed a little bit since then. But the big thing is that in the retail enterprise, we've discovered that the right application really shouldn't start a case level, it's just start at item level. And you'll see a lot of retailers talk about utilizing RFID for inventory management kind of where we started back in 2010, 2011. Because when you look at RFID and you look at a portfolio, there's really 2 key ways to think about it, and JJ and I were talking about this a lot, which is when you look at sensors, you've got a one-to-one sensor and a one-to-many sensor. And one-to-one sensors are like NFC RFID, so everyone that's ever used pay on your phone, you tap, and it's kind of a one-to-one sensor, or a 2D barcode or a 1D barcode, you kind of scan one-to-one sensor. RFID, we're talking about UHF RFID as a one to many sensors. So you collect a lot of data very, very quickly. So as retailers discover this technology can actually help them get their inventory accurate in their supply chain to make sure they have the right product in the right place at the right time for in-store shopping or omnichannel shopping was kind of the foundation. But then you start building out the enterprise use cases as you look at things like giving data analytics for tests, looking at point of sale and making it a lot more faster and convenient. So RFID, which people might think is this 2,000 technology has really become this critical foundational sensor technology for the enterprise and the next generation of retail. And we'll talk a little bit later, I think about AI and how important RFID sensors can be to get your inventory and product data accurate for AI in the future. So we see this as a really big idea and really foundational not just now for retail enterprise, but the future of retail enterprise.

Yes. So like my team was sort of looking at RFID and they came to me with RFID and they were -- I was like RFID, like come on man, it's just like prevents you from going through exit gates at stores. And I was pretty skeptical when they came to me with that. And so I was like, how does this fit in with sort of the way we think about the world? And so the team was like, well, right, we're customer-centric at Amazon, right? And so RFID traditionally has been used for the benefit of the merchant. And my team is like we've figured out a way to make this for the benefit of the shopper, not the merchant. And so what they explained to me was this notion of, well, we have these computer vision-based Just Walk Out stores, a bunch of cameras on the ceiling, right, some sensors on the shelves. And they said, but that doesn't necessarily work in every case. There's -- T-shirts get all rumbled up and the stores get messy and stuff. And they said, we figured out a way to take RFID, tag the products with it in the store and enable people to walk out without a checkout for like things like T-shirts, concert merch, other sports things that you find in stores. And so I was like, all right, well, this is kind of interesting. It sort of fits in with the bigger story we're trying to tell about frictionless checkout and here we are now. So we can sort of take a look at how it works to give you guys an idea. [Presentation]

All right. So I've got to say, it is someone from Dallas, who has been a long-time Rangers fan, I don't know if there's a correlation of winning the World Series and using Jawoll apparel RFID, but we'll take it. So it's a really unique experience and have him going through the stadium. Again, something that's really unique. You just walk out, don't stop and pay and what a great experience. So I think -- are there other locations, JJ, that besides Globe Life?

Yes. So you'll find it at Globe Life, where I think I'm PR approved to say that if you deploy, Just Walk Out technology, we'll win the World Series every year. But you'll also find it at Hard Rock Stadium where the Miami Dolphins Play. You'll find it at Lumen Field where the Seahawks play. And we actually prototyped it originally at the Climate Pledge Arena, where the Kraken hockey team plays. But like we've got -- these are real stores, right? This is not a prototype technology we're talking about here, like we're rolling this thing out and a lot of people are using that.

And how long does it take to get one of these set up? I think that's got a big question I get from time to time. How does it take? Is it easy, complicated?

Yes. That's, I think, a big difference between RFID-based stores and maybe what people know with some of the computer vision-based Just Walk Out stores. So all the tech for this store is just in the exit gates. We don't have to hang cameras. We don't have to run network cables. We don't have to do any of that stuff. And so Just Walk Out RFID stores are much faster to set up than the existing technologies. We went from like concept to launch in Miami in a period of about 6 weeks, like from the first time a salesperson had a conversation with someone at that stadium. And so we can retrofit existing spaces by just plunking down a gate where you want people to check out, we can set up temporary spaces like they did at Lumen Field, where they set up just basically cordon off a store with stanchion around the edge of the store. So you can get up and running like really fast with this technology, which is a big benefit over some of the more complicated solutions. But like it might seem like our sort of path -- or the path here was easy. Like in reality, we've been working together for better part of a half decade, I think, on trying to come up with some way to use RFID and Just Walk Out. So like I don't know, do you want to talk about a little bit about like how did we get here and...

Yes. I think it's interesting. I think the power of coming together around this. So we've been working on this about 3 years' time frame overall, really to understand the right sensor tech, how do we get the right sensor tech to work and all the use case variants. How do we then work closely with JJ's team around the analytics capabilities they have and the deep science capability they have to really make this work because it's truly a unique combination of really high-performing, high-quality sensors that are meant for purpose around this technology and then how we integrate it with the team.

Yes. I mean it's definitely -- it's been an adventure. And I think like this is something I'm really proud about the partnership between our 2 companies. This was not the first thing we tried with RFID and Just Walk Out at all. We've been trying to figure out how to integrate this stuff into grocery stores to solve unique problems there and other kinds of sort of complex merchandise that we had been having trouble with, with computer vision-based stuff. And we sort of kept just plugging away at it and plugging away at it. And that's sort of an example of, I think, the long-term thinking of both of our companies. Like we knew something was here, right? But it wasn't sort of immediately obvious what that was or how it would ultimately sort of emerge.

Agreed. And I think, too, what makes it unique is, again, it's all around the customer and working together, we're always challenged on how do we get the best experience for the customer. The one thing too that's interesting about apparel, it's a very multidimensional SKU, right? You've got size, color, style, all these dimensions. And with RFID, based on the serialization of RFID, you can really understand all those dimensions relatively quickly. And I think the analytic unlock around some of that is quite interesting.

Yes. I mean like certainly, like if anyone who's been to an apparel store versus, I don't know, a stadium food and beverage store, like you know it's different, like it's like controlled chaos in one of those merged stores, right? People are throwing stuff everywhere, stuff completely, disorderly. And that's a real problem for a computer vision-based solution, right? Like the cameras need to know what they're seeing and just not possible all the time with merch. And so like we're pretty excited about some of the new sort of possibilities that this brings to the merchants that are selling this type of stuff. So the first thing, as I sort of mentioned, is like cost is decoupled now completely from the square footage of the store. So if you have a giant department store, like the deployment cost is really just in these gates. You don't have to modify any of the rest of the store. And that's a big deal, right? Like you don't want store downtime as you're trying to move to a new way of merchandising. The other thing that's kind of cool about them is they're basically portable. So if you have one of these gates and you want to set up a temporary sort of merchandising space somewhere, you can move these things around. Like when we set up the first store at Lumen, they just took a corner of their arena that was sort of unused, plonk the gates down and bam, you got a store. Obviously, that's pretty cool. The other thing is that it really does enable a whole new range of selection, right? Like -- and I think this is something I didn't know about RFID that I learned working with you guys. I was like, oh, these tags only work in certain environments, right? Like the there's a lot of stuff where it just won't work and I think you've proven to us that they can do a lot more than you can think like in terms of the types of merchandise you support and stuff like, can you talk about that a little bit and the future?

Yes, you bet. I mean today, people think of RFID because there's been a lot of growth, and a lot of retailers talk about apparel. But really now, you look at general merchandise, there's a lot of things that are happening with general merchandise space, enabling all these items because really those items have the same need from an inventory accuracy consumer convenience, right? Theft and analytic perspective and all the enterprise retail applications. So we're working very hard to make sure that every item in a store can be tagged and identified because we really, again, see this as a big enterprise shift in retail transformation that's occurring to enable things like this where you can literally walk in, have it a total frictionless checkout and walk right out. So very focused in the science around that. And again, it was interesting in the early days, people will say, well, this doesn't work here, this will work here and I don't think that's really a question with the Amazon team much anyone.

Yes. And I know like in some of the early days, we were like one of the things we were trying to get the RFID stuff to work for was we have these grocery stores, right? And we -- I don't know, steaks that have variable weights. And we were trying to slap RFID tags on steaks and things like that. And I mean, I think like you showed us a lot about what the art of the possible is there, even like types of stuff that you would have never thought possible are really quite possible when it comes to some of the new RFID tagging technologies that you guys have.

Yes. Yes, we're really excited about it. And I think one of the other things, too, I really want to talk through is AI. So I know that's been one of the topics you hear. From a technology perspective, I'm sure a lot of people have seen a lot of AI out in the show. So can you talk a little bit about how you're thinking about AI and the sense of Jawoll?

Yes. It wouldn't be a conference in 2024 without talking about AI. And so I think like a lot of people probably have some maybe misgivings or misunderstanding of how the RFID technology works, right? Like you're used to seeing it at like loss prevention gates at stores, right? And you think it's like magic like, okay, I walk through with the tag, sirens go off, that's bad. To charge people's credit cards correctly for the items they are taking out of a store with RFID requires a lot of heavy lifting, I think that's not necessarily obvious to someone who's just sort of walking through the store. Those gates that we put in our stores, they're reading tons of tags off the shelves all the time, right? Like RFID signals like propagate everywhere. And so what we need to do to make that store possible is we need to say this person is walking through the gate with this and only this stuff, right, and charge them correctly. And there might be another gate right next to that gate, where someone else is walking through with stuff or one directly across it, as you'll find at the Dolphin stadium. And so to do that takes a lot of pretty heavy lifting on the AI side. So we have, I'll just say, lots of antennas in those RFID gates, all of which are reading a bunch of different signals, all of which are tuned at very specific way, aimed at very specific direction. And so those things are getting massive numbers of signals coming through them at all times. And the ML and AI, what we do with that is we've trained a model that can say, of all of these signals that I'm reading of which there might be hundreds or thousands, which ones just passed through this gate with this person at this time. And that person might be holding some of those RFID tags under their armpit as they're walking through or they might have 2 tags sandwich together, like the world is a very complicated place, when you start to actually deploy this technology and like AI has been really sort of important for us as we try to actually build these real working stores to make that happen.

Yes. Interesting. And you talked about someone putting something beneath their arm. How do you think about theft with Jawoll? What's your perspective on that, that makes it a little unique?

Yes. Theft is a big topic as well, right, this year and last year. We didn't design Just Walk Out either the camera-based stores or the RFID stores to stop that. That wasn't the goal at all. The goal is to make a seamless checkout experience for customers. It turns out though that the side effect of doing that is that we have some pretty positive benefits in terms of reducing shrink. So in a camera-based JWO store, Just Walk Out store, it's our job to know what you took off the shelf and to charge you for it, right? So if you go into a camera-based Just Walk Out store, grab a candy bar, stuff it in your pocket and walk out, it's our job to get that right. Now in a traditional store, if you did that, that would be theft, right? But in a Just Walk Out store, an action that would have been theft becomes a purchase, right? And so it's kind of a neat side effect of it. And the same is true in these RFID stores, right? If you had an RFID store maybe that was enabled with bins, right? You put yourself into a bin, well, people might forget to put something in a bin, right? They might have stuffed something in their pocket, they forget to put in a bin, in that scenario, they end up stealing from the store, right? But the same type of store when you walk through one of our RFIDS gates, it's just a purchase, right? And so I don't know that we're -- we haven't talked about any branding where we're maybe making the world a more honest place just by default the way this stuff is implemented. But I think like a thing that we have learned is that RFID is not magic, like there's a lot of operational sort of knowledge that a store owner needs to make this stuff work. And I think that's an area where we've been really impressed with what you guys bring to the table. Do you want to talk about sort of how do you operationalize this stuff in real life and actually get the value from it?

Yes. Yes. I appreciate that. Again, a couple of things that we've been really working on collaboratively is when you install the system, how do you look at it from an end-to-end perspective? So it might be things until you get the source tagging where all the brands are putting RFID on its source, how do you put printers in the store to be able that you actually get the right data on the tag and get the tag on the right products. So we're putting together systems and solutions that enable the operators in the store to make sure they get the right tags on the right product from an operational perspective. And then thinking about things from an inventory perspective, right, along with the Just Walk Out perspective. So I think those are some of the things that we're thinking through collaboratively with JJ, about how do we put a whole solution together that allows -- right now, these stadiums actually implement RFID Simple Gates putting in, as JJ talked about, quite easy to put in. And then our team helps on the implementation around the printer systems, all the other ecosystem solutions that make sure we're doing the right thing, the right tag, right printer to get on the right product, so we can make sure we have a good floor product in the JWO stores.

And you guys have been doing some research around this and how this is sort of affecting the purchasing decisions or the store choice decisions. What have you guys found there?

So we just commissioned some research that we just released actually this week, and it was quite interesting. Two real key things. I think when you look at the aggregate research that we deal across all consumers, roughly 45% of consumers would actually choose a self-checkout that had an automated way of checking out versus scanning a barcode, which is interesting. The second thing is for Gen Z, I mean they really like kind of a frictionless checkout. So Gen Z-ers are telling us that roughly half, a little bit over half, would actually switch to a store that actually had kind of more of a frictionless checkout that didn't require a barcode scanning. So you can see that from a different generational perspective, there's still a big contingent overall that wants to have frictionless checkout, but you can see that proliferate a little bit more from a Gen Z perspective, to the point where they would actually potentially switch to a different retailer based on frictionless checkout. So very interesting.

Yes. And that's certainly one thing that we want to prom -- or try to make a promise to for the people we're working with, like increased sales is certainly an angle that for us is really important. We see retailers getting value out of the RFID-based stores in kind of 3 ways. So one way that we see it is increased throughput, right? So if there was a 10-minute long line to buy a t-shirt at a stadium and now that line is just like moving real fast, they sell a lot more stuff. And so retailers are pretty thrilled about that. The other thing we see is, as we talked about, that reduction in shrink, right? I mean it just is harder to make a mistake and not pay for your items in a Just Walk Out store. But then the other thing we see is that you might have had to staff up for the peak load during the store's operations at a stadium, right? And so you might have a -- during a game, you've got 2 periods in hockey or 2 intermissions in hockey, and so you got to staff for the giant load of people that are going to come into that store for those 2 intermissions, right? But with a Just Walk Out with RFID store, I can have a couple of traditional checkouts if I want for people that don't want to try the new thing, but one or two of these gates can put a lot of shoppers through the exit gates and can really optimize the labor in those stores. And even more importantly, free up the people in those stores to help me find the size I need or to answer the question if this is an authentic jersey or what this weird strap is for on the back of a hockey jersey, all those sorts of things, right? So we see that the store associates are doing a lot more like interesting interaction with customers that the customers like in these stores. Because I mean a lot of us have been to a store where you couldn't find the size or color that you wanted. And that's really a lot easier in these stores when you free up the staff to do that kind of thing.

Yes.

Yes. So I was going to ask the neat thing about this, and I didn't know this either at the beginning when we did this, is when we apply RFID to these items in the store, we're sort of assigning each item a unique ID in the store. We're not saying -- there's not one RFID tag on all medium men's, Kraken jerseys, right? Each one has its own unique sort of ID. What type of stuff does that enable? And like why should I care about that, I guess?

Yes. I think, again, the beauty about bringing serialization together with a sensor like UHF RFID is that you're able to differentiate things that you can't visually see. So when you go into an apparel store like a JWO store, you look at, okay, is that a small, medium, large? You can actually differentiate between all the dimensions of the SKU to make sure that, a, you've got the store merchandise correctly, right? You don't want to have a bunch of mediums sitting out in the store when your average big flow item is a large. So you have the right mix, you can replenish with the right mix. So imagine even today in a stadium, we have a high flux of people coming through one of those stores, and you run out of mediums. This will happen that the mediums are flying off the shelf. You now know those mediums are off the shelf. You can immediately replenish that. Because in the stadiums in particular, you've got a very short window in which you got to maximize your sales. So high velocity, high churn, how can you make sure you're replenishing appropriately, to make sure, again, the holistic consumer experience is good. So I think that's a really interesting part. The other part is, as we talk about the future of AI, if you think about like we've done a lot of studies in the apparel industry where the average retailer is roughly 65% accurate at the SKU level, now at the unit level, say they're 90%, 92% accurate when they do their physical inventory. But when you drill down the SKU level, size, color, style, et cetera, they're 65% accurate. But if you're 65% accurate in your inventory position and you're going to build AI on top of that, right? You're going to build AI on top of 65% accuracy, forecast planning, replenishment, right, the whole supply chain. So we see this from a product -- you can build great product attributes in your AI strategy, but do you really have the right product, is really something that we're very focused on around why this matters. Because again, you can have the right unit but not the right SKU. Customers are not going to really buy anything. They're not going to be satisfied because you didn't have their size, didn't have their color. And people have a lot of choices today, right? They actually look elsewhere.

Yes. And that's been something we've definitely learned is that, if you put garbage into your machine learning algorithms, they are going to perform really badly on the backside. And so having really high-quality data going into these algorithms is absolutely essential if you're going to try to build something that is trying to get the receipt right 100% of the time.

Agreed. And this is why we think it's a big idea, JJ, going back to your original 1994 question of RFID.

Yes. I mean I don't think we're anywhere near the begin -- the end of where we're going to take this either. Like, I mean, I think there's a lot of possibility here.

What do you see as next for JWO?

Well, so I think there's a lot of really cool things that this is going to enable. One is pop-up stores are going to become a piece of cake, right? I mean pop-up stores are hard not only because you need to set up a store really fast and get it running; you've got to hire people to staff them. And like in this country right now, it is extremely hard to hire people. And so I think that's going to be one thing. I think new types of selection, especially with some of the innovations you guys are working on, on the tagging side, are going to become possible with our RFID, where people are going to go, oh my gosh, I had no idea that you could apply an RFID tag to that thing and make it work in real life. And then like I think some of the really exciting possibilities around like the providence of items, right, to say like this is an authentic item, and its RFID tag proves it so. You can imagine in a sporting world that that's a really interesting thing to start looking at. Really anywhere where there's like collectibles or sort of unique one-of-a-kind items, it's going to become really interesting with some of these RFID-based technologies, so.

Agreed. And one of the interesting ideas we were talking about was like this concept of pop-up stores, taking the solution, taking the store to the demand, right? You have unique events that are happening with large populations. How do you have the flexibility to actually take the store to the demand, right, versus the demand coming to the store? I think that's a really interesting perspective from a pop-up store perspective.

Yes. And I think another interesting thing that we're going to see is like, right now, we have this sort of RFID-only store out here. We've got a vision-only store down at the front of Javits. There's no reason that we can't combine these technologies and potentially other technologies as well. So we might find really interesting blends of both RFID coupled with the computer vision stuff to enable like pretty amazing new retail experiences. Like I think, like for us, we want to create that sort of magical moment where you're like, I have no idea how this just happened but I just purchased this thing and it feels awesome. And so I think together, there's a lot of possibility there. So it's an exciting time. And I think, I don't know, we'll see, we may come up here with some new innovations on the big idea days next year with you guys and say, holy c***, we never thought that was going to be possible when we were sitting up there last year, and it was a pretty exciting time.

Looking forward to it.

Awesome. All right. Well, I think we're out of time. We won't keep you guys from beer and wine. So thanks for coming, and yes, let us know if you have any questions. We'll be at our booths and up here afterwards. So, cool.

Yes. Thank you.

Before we get started, hey, I'm Justin Hannemann. Let's all stand up, meet somebody you do not know around you. I'm giving you 2 minutes. You have to stand up, introduce yourself, name and company or role, someone you do not know. Come on, everyone stand up, yes. Everybody stand up, please. Me included. I'm going to come down here and say hello to somebody, I don't know. We got about 60 seconds left. All right. Perfect. 40 seconds. Perfect. You can start to grab your seats. Everyone can now say you've met somebody new at NRF. When you go home, talk to your company, you've met somebody new. All right. Perfect. Some are just going to continue on. I'll just keep presenting, and now we've created a bold social hour. So welcome. My name is Justin Hannemann, and I'm really excited to be here today. I work for Amazon as part of AWS. How many of you are shoppers on amazon.com? Show of hands. Okay. How many of you work with AWS? How many of you have no idea what AWS is? There's one hand in the back. Okay, great. Well, just in case you're not familiar, I'll give you a bit of an overview in a few moments. You get me for about 10 to 12 minutes. And for those of you that are watching on the camera at home or in your office, after today, welcome, we wish you were here with us. We're going to be talking about a topic I'm sure nobody is interested in, Generative AI. Anybody interested in Generative AI? Yes, a lot of hands, right. Anybody already testing Generative AI? You've got models, you're doing use cases. Anybody on the front end, you're still trying to figure out what it is? Understand it? Okay. Good. How many of you know what artificial intelligence and machine learning are? Okay, good, vast majority. Well, here's what our plan is, our road map for today. I'm going to give you a little bit of background on this space. I'm going to give you some -- a little bit of insight into what we're doing at AWS around retail. And then I'm going to dive into some of the excitement around Generative AI. And then you're going to get a really great use case and story from one of our biggest customers in Latin America. Oscar is going to be joining me in just a moment, and he's going to dive into some application with their business, and I'll tell you a bit more about that in a few minutes. So we will have time for Q&A at the end. We have some canned questions. We'll ask a couple of each other, and then we'll have a mic in the front of the room here. So if you've got questions as we go, just know at the end, we'll take a few. And then we'll hang out afterwards and we can talk more in detail. Sound good? Okay, cool. So here we go. Let's jump in and give you a little bit about AWS and Amazon. If you already work with us, this may not be new news. If you do not work with us, this might be new news, and that's okay. So at Amazon, our mission or vision is to be, first, most customer-centric company. And it's nice to have a very clear vision and mission, makes it very clear what we should be rolling up to at AWS. And within retail, actually, a quick note, retail and consumer goods at AWS are part of one industry vertical. So many of you work with other technology companies, right, that are verticalized from an industry perspective. And just so you know how we go to market as we've got both retail and consumer packaged goods like in one team. And many of the team members are here. So welcome. In retail, we like to say we're born from retail. If you come to our booth after, you'll see that it is very clear that many of the services and solutions and capabilities that we offer came from retail. And so we like to say we're born from retail and built for retailers. We work with the vast majority of retailers worldwide from all kinds of different perspectives, from core cloud technology, to marketing, sales, allocation, planning, analytics, data, supply chain, right, fulfillment. And I'll share with you a bit about some of those before we jump into Generative AI. The second area, frictionless commerce, unified commerce, the theme of -- again, if you come to our booth, is really unified commerce, physical store technology, and also digital tech. In fact, a year ago, we were all -- how many of you were here last year? Yes. Okay? How many of you are first at NRF? Wow, oh my gosh. Well, welcome. I mean this is the largest retail event of the year, and what a great place to learn. But a year ago, we weren't even talking about Generative AI. Some people were. And then we were really talking a lot about immersive retail, immersive commerce, right, unified commerce, new e-commerce platforms. For us, that is a huge part of what we do at AWS, and now have added in the components around Generative AI and, of course, artificial intelligence that supports. And then the last area, the third area is we really focus on operational efficiency. You might expect that we're really good at supply chain Amazon, and we've taken many of the learnings from that part of our business and brought those over to these services and capabilities that you can use as a customer. And so we talk about in retail the allocation process, the planning process, inventory visibility, right? Where should you put inventory to meet the demands of a retail store and the consumers that shop that store? We're very embedded with many of our retailers and helping them to solve that. And it's been especially important in the last couple of years, right, as we went through COVID, supply chain issues, and then the ups and downs that we've been facing in the marketplace in different geographies. So that's a little bit about AWS. That's the end of the AWS overview. Generative AI. So while this space has been there for more than a year. It really accelerated with us and our customers around March or April last year. And it's funny, coming out of NRF, we're talking about immersive commerce, stores opening, the experience in store, loyalty, new commerce platforms, and then it became all about Generative AI. And many of you are sitting here I've had calls with. And it was, what is it and why should we care? And for the first time, line of business people and CEOs cared about AI. In fact, many people at these types of events will get up here and talk about AI/ML, but they really wouldn't know what AI or ML were or how they interplay together, but we're doing AI/ML. What Generative AI really did, first and foremost, was help our customers and many of the line of business leaders to understand what it is and what's the potential for it, right? And I think that's what was so exciting. And there's a lot of news. I put some on this page here, and a lot of excitement, and also a lot of questions. So we'll dive into some of that in a few moments. AI is not new. Some of you have seen this slide. If you work with us, you see me show this slide. But AI is not new. In fact, it's been around for many years, all the way back to the '50s. And it's already built into many things that you're using today, whether it be with us at Amazon or even some of your other platforms that you work with, in areas like product recommendations, in areas like fraud for e-commerce, your contact center solutions, the next best offer solutions. This morning I met with the customer, we were talking about personalization around what do you -- how do you recommend the next best offer online or in-store or loyalty, right? AI is not new. It's actually been around for many years. And it's nice to know that it's already been there. On the supply chain side, demand forecasting, right? And at Amazon, many of you work with many parts of Amazon. There's AWS. There's Amazon Advertising. There's Amazon Pay. There's Amazon Buy with Prime. There's Amazon Logistics, distribution, fulfillment, Whole Foods, Amazon Fresh. I can keep going down the list. Many of you work with many parts of the Amazon ecosystem, and AI is part of many of the things that we do at Amazon. Where you might see this play out, is if you visit one of our fulfillment centers and you see the robots moving product around the fulfillment center for picking and packing, that's powered by machine learning. If you come over to our booth and you see what's called the Just Walk Out technology, how many of you have been into an Amazon Go store, Amazon Fresh store? Okay. For those that haven't, if you have an Amazon account, you would just go to your app here. Hopefully, we'll have some good WiFi, and pull up an in-store code. Yes, here we go. When I pull up this in-store code on my amazon.com app, I walk into an Amazon Go store, there are several around New York. I scan my code. Gate walks open. We're using computer vision and sensor fusion to build a virtual cart, right, for you in store? And then what do I do? I just walk out. My Amazon account is charged and I get a receipt in my e-mail. That is powered by AI/ML. Alexa powered by machine learning, et cetera. So for us at Amazon, many of the core AI or machine learning capabilities built into what we do every day in different parts of our business. Of course, today we're going to talk about Generative AI and we'll talk about some of the differences. We think of Generative AI as a true game changer. And some of you may like, I don't want to say last, but it seems odd that we'd be talking about like the printing press or the light bulb. But we at Amazon view this technology is that significant. And what's interesting here is that if you look at actually the pace of change, 400 years from printing press to the light bulb, 100 years from light bulb to personal computers, 20 years from personal computers to the Internet. Amazon, AWS is the first cloud provider launched in 2006 as the platform on which amazon.com was built, right, 16 or 17 years, there's no other cloud providers there, 2006. And then here we are and we're talking about Generative AI. We view that as it's that significant. And some of you have already found this. And some of you already found this in terms of how you're using the models today on the data that you have. So what questions are we being asked? So I mentioned all of the meetings that we started having last March or April. It became all encompassing. Every day what is it, tell me why it matters. Boards, we're meeting with the Board telling them like what in the world is Generative AI, and like why should we care? And what's Amazon doing about it, right? It's fascinating. And so these are the types of questions we get asked every day. And these are the types of questions like, especially here, it's what are we doing? What are our models doing? How are others using it? What are the real use cases and the business value add? Is it significant and a game changer today? Can I use this in supply chain yet? Can I use it to pull things from different systems? What about the hallucinations and when it comes back with bad response or bad data, very confidently comes back wrong? Like how do we think about that? And these are all the questions that we're getting now from customers. The majority, if I had to separate out our retail customers, there is a segment that is on the bleeding edge. They're testing it out today with text around e-commerce listings, titles, bullets, ad words, descriptions, right, with text models. They're testing it out with images. You can take images today and put them in all kinds of products and put them in all different environments without using the agency. Today. It's like available. And it's being used. It's happening today with our customers, right? With coding. The other third use case is coding in terms of how you can use natural language to code. But also it's powerful to do like QA on coding, move one code to another code set in minutes and seconds. It's just, when you see what these models can do and the power of what can be generated from them, you start to think about and realize the potential for this. You're also starting to see this built into many of the services that you're using every day, on your phone, with other technology companies and how they're embedding it in their solution. You'll hear us often talk about our models and the data, right, and the output from that. But many services are just going to have it embedded, like Bill said, you're not going to select the model on data. So a lot has changed in only 6 or 7 months. If you think about, again, back here, back to last year, around August, September, we went from this everyday meetings on what it is and why should I care, to how can we start testing, how can we start exploring this with our teams, where do we find value? How do we assign people? Is it going to change roles? Do we need to hire new people? Should I be worried about the people I have? Right? And then the other area that I would just say that really started to become in the spotlight at the end of last year was the data. And in our model, the way that AWS works is we provide the models on top of the data in AWS. But guess what, if the data is not good, what is your output? Not good, right? Just interesting that if you think about like the quality of data being a big part of the story. Finally, this is my last slide before I hand it over to Oscar. We -- I get asked this throughout the week here, actually, this has been one of the big topics, like where are we really seeing this today? And the use cases today are primarily in the center. I mentioned that some of those a few moments ago, marketing and sales, and around e-commerce and customer experience. Those are the 2 big areas. Now where there's a lot of interest is how can we use this for better -- let's say, a planogram. Many retailers do not -- raise your hand -- put the same product set in every store, a vast majority of stores, and do not think about who's shopping in their stores because you just can't manage the data to figure it out and then get the right product there. Great use case for Generative AI. How do we improve our forecast? I mean we've been working on forecast for years. But the potential to be able to grab data sets from different data platforms and bring that together to provide almost a dynamic forecast that ties back to inventory, powerful. Not quite there yet, but guys this has moved in like 6 or 7 months from 0 to everyone testing. Hey, imagine 6 or 7 more months. So it's super exciting. There's a lot to be explored here. There's a lot to be put in place in terms of governance, right? That's been also a big focus, is how do we ensure that this is used for the proper means and for providing the right insights to the right user within our business? And so I know you all are eager to see so how this has been brought to life with Mercado Libre. So I will introduce Oscar. Let me just tell you a little bit about Mercado Libre. How many of you know the brand, Latin America? Wonderful. I mean this is significant. 144 million shoppers or consumers on their platform. They operate across 18 countries. They've got 50,000 employees. A significant e-commerce presence in the vast majority of LatAm countries. And so you're going to hear today how they've taken these ideas and brought them to life. And so with that, I'll introduce Oscar and we'll be back in a few minutes for questions. Oscar.

Thanks. Thanks, Justin. Hi, everyone. Can you hear me okay? Good. So it's very nice to be here. As Justin mentioned, my name is Oscar Mullin. I'm a Senior Director of Engineering working with Mercado Libre. And the idea is to share with you today some ways in which we use AI and Generative AI in production externally and internally in our company. As you can see, this is not a Gen AI image. We managed to create it ourselves. So basically, one of the things that we all have in common is that we have a constraint that is time, right? We can only do a certain amount of things with the time we have. And when you are in business and you rely heavily in technology, one of the trade-offs that you have to make is a trade-off between speed and quality, right? So if you move very fast, most likely you are going to hit the quality level of what you're building. And if you are focusing too much on higher levels of quality, you're not going to move as fast, right? And when I say quality here, I talk about it in a wide sense, right? Reliability, efficiency, security, privacy, and so on. So usually, we move at some point within this curve, right? If you're building a prototype, most likely you are moving fast, you don't care that much about quality. If you are building mission-critical products, then you're probably focusing a little bit more in quality. So one of the things we like to think about this in Mercado Libre is how can we move this curve. And we think that with new technology like AI and Generative AI, and platform engineering, and other sort of technologies, we are able to move this curve. So we can actually move faster with the same level of quality or achieve higher quality at a greater speed. As Justin mentioned, Mercado Libre now have 50,000 employees. 15,000 of those work in technology. We have 140 million active users in 18 different countries. But it wasn't always like this, right? When we started many years ago, we had a monolith and a single database, and that was it. All of our code was there, are released, I mean everything in at the same time. And we stayed like that for 10 years. Actually, we went public with that architecture. But then we knew that if we wanted to continue scaling and grow, we need to change something. One of those change was to built in micro service. So we changed to a micro service architecture, so people can work decoupled from each other and continue to scale our company and our business. So started releasing our first mobile applications. And then the curved grows exponentially, the number of employees grow significantly. But it's not a coincidence that it matches with that logo that you see there. That logo is the logo of our internal platform that is called Fury. Fury is a mix between an internal engineering platform that basically abstracts away basic creation or constructions like compute, traffic, networking, data services and so on. And also it's an internal platform that helps our developers to work seamlessly with a lot of collaboration and [ are capable ]. And the last thing that you see there that is a brain basically to -- thank you. What you see at the end that is a brain basically refers to Generative AI because we think that's a new technology that will allow us to keep growing and keep moving with the same speed as before. One key thing to mention here is that we actually offer AI and Gen AI through our platform to our internal development teams and the whole company. We have a part of Fury that is called Fury Data Ops. Basically, there, you can train any machine learning model you want, and also you can do serving of the machine learning models from there. We also have constructions, so anyone can use Generative AI on different providers directly from the platform in a way to democratize the access in a secure and efficient way for all the company. So this is the side and one view of our product in Brazil. Brazil is our largest market. How many people do we have from Brazil here? Wow, a lot of Brazilians. We can say it in Portuguese now. How many people from Argentina? Good. So I'm from Uruguay, so I'm not going to enter in a Maradona-Pele thing. But basically, this is how our page looks. And this is all the places in which we use traditional AI, right? If we want to call it somehow. If you can see there, we use traditional in search, because we want to make sure that we understand the intention of your search and not just keyword matching, that will provide more relevant results for the users that are searching. We also use traditional AI in text moderation as well as in image moderation. We need to make sure that whatever the seller uses to publish are things that are related to the product they are selling, they comply with our thermal conditions, they are not offensive, illegal or even fraud related. We also use this to calculate the shipping time. So we need to understand the address of the user as well as the location of the product. We also need to understand the shape of the product and the type of truck that can transport this product. And we also use models that consider the historical traffic data of the cities where we deliver the products to make sure that we do the best promise of delivery, because that's a key decision for the buyer, right? If the product is arriving tomorrow, maybe I want to wait. If it arrives in a week, maybe I don't want it, right? And we need to be as accurate as possible. We also use it in price optimization. Basically, we have all the prices, of course, on our own platform, but we also understand the pricing in the competitors and the rest of the Internet, so we can recommend and predict the pricing for this kind of items, so the sellers can improve their price and also sell more products. We also use it, of course, in categorization and recommendations. Those are the most typical uses of artificial intelligence here. We use it to detect fakes. Fakes in e-commerce are a big problem. So in order to make sure we validate the authenticity of products, we also use AI there in checking the images and checking the products themselves. We also use it with fraud prevention. Mercado Libre is not only an e-commerce company. Mercado Libre is an e-commerce fintech and logistic company. We have one of the largest fintech solutions in Latin America. And fraud prevention is critical for us in many ways. So we have models that take thousands and hundreds of thousands of parameters to understand fraudulent activities and try to defer that from happening. And we also use it in credit scoring, of course, to understand which kind of loans we can give to our users and with which conditions. Of course, that was just one page of one of our products. We use it internally extensively in our logistics centers. We use it in our legal teams. We use it in human resources. We use it early -- also, sorry, in our internal optimization of infrastructure management. We have, for example, in AWS, more than 100,000 EC2 instances. And in order to improve the efficiency of the compute that we use, we use a lot of artificial intelligence as well. So that was traditional AI. And then Gen AI came, right? Basically, everything kind of got excited by 2022, right? That's when we all knew or most of us knew about ChatGPT and the things that we could do with it, right? And if you think about it, this is a game changer in terms of technology, but also is the technology that was adopted faster in the history of humanity. In only 5 days, more than 1 million active users. That's crazy. That's a lot of potential in that technology. If you want to understand the technology, you need to go to the source. So this is the original paper of attention is all you need that kind of opened up many of the things that we're seeing today. So if we want to really understand these technologies, we need to go deep here and understand how it works. And then you have the transforming architecture next to it. We're not going to go into the details of this, of course. But what I wanted to highlight here is that the paper was written in 2017, and the breakthrough happened in 2022. If we think about the slide that Justin show, going from the printer to the next evolution took almost 400 years. And here, going from the definition to the technology, it took only 5 years. 5 years may look like not a lot of time, but think about it. From having the way to do this to create a product that anyone could use, it took 5 years. So while we are very excited about this technology, probably it will take some time that we can actually incorporate it in a way that highly impact the daily life of users. This is an image that was created with a prompt, that kind of highlights the power of this technology that you all know. And basically, we put it here because we think that it kind of expose this new value that you can get out of this technology. That is the first time in history that you can create original content, human-like, that is difficult to differentiate from human creations. And we can use this in e-commerce a lot. We can use it to improve advertising, to improve how our images in the products look like, and many other things. And then the question came, right? What is Mercado Libre strategy about Generative AI? What is the strategy your company is having about Generative AI? And actually, in Mercado Libre, we don't have one, right? So that's our strategy. We don't have a strategy. We don't have a strategy for the Metaverse either, we don't have a strategy for NFTs, we don't have a strategy for any new technology because basically that's part of our culture, and that is one of the key things I wanted to share with you. For us, we have a culture of innovation. And that means using every new technology, even the ones that mean changing the complete landscape of technology, as a mean to create more business and user value. This culture also means that you need to partnership with the best in the industry because, most likely, when this breakthrough happen, you are not going to have the experts inside your company. So we were one of the first companies that made a strategic partnership with AWS to talk about and discuss about Generative AI, also with OpenAI. And this is kind of something that repeats in our history. It happens with platform engineering, it happened with mobile. It happened when we started our logistics. We knew nothing about logistics, and now we run one of the largest logistic operations in the world. It means building new capabilities based on new technology that's something that needs to be part of your culture, right? And that includes upskilling and reskilling your teams, because, most likely, they won't know or they won't have all the information to be able to use these new technologies. One other thing that happens when you define a strict strategy is that you cannot move as fast as the technology. So we have GPT-3.5, and then a few months later, we have GPT-4. Then we have Llama in the middle. Llama is a major breakthrough because now anyone can use these large language models, right? The thing is that the first version of Llama that was released by Meta was not commercially usable because of the licensing type it had, right? So researcher and academics could use it and could work with it, but were not able to use it commercially, nor were we. And then Llama 2 came, and with Llama 2, these restrictions somehow were lifted. So now anyone can use Llama 2 commercially, in your own infrastructure, you can use it in AWS Bedrock as well. So anyone can benefit from these models. What I'm trying to emphasize here is, if you define a very strict strategy then you're going to hit -- you're going to be hit with the speed of the technology. So the best thing you can do is create a culture of innovation. We also did a hackathon in our company where we stopped everything for one day. Everything means everything, not just technology, every area of the company. So they can focus on creating new ideas that impact our users directly with the use of Generative AI. And we got 200 ideas that were approved after the hackathon, more were created. And now we are currently implementing 20 of those in production. This is a more technical description of how we offer the access to Generative AI in our company to our technology teams. So all of this runs in Fury, the platform I explained at the beginning. Basically, what you have here is that any application in Fury, that is this platform, can access different LLMs through a Generative AI gateway data plane that we have there. The Generative AI gateway data plane is the thing that actually talks with the LLMs and can change providers in a simple way. But the key thing about this gateway is that it controls completely what data goes out. If we detect PII data going up to any LLM, we can prevent that from leaving our company. We also manage the rate limiting there. So we don't -- because cost is a big issue with Generative AI, right? If you still don't hit it, you will hit it, because LLMs are huge, right? And until we have smaller models that can be as performer with a lower cost, you need to be very conscious about the cost. And also with an asynchronous replication, we were able to evaluate all the prompts and all the answers that we got. So we understand which were the kind of things our teams were trying to achieve and we were the kind of -- which were the kind of results that we were getting. So we learned a lot of things during that. A lot of things that can be done. But we also learned a lot of problems that we face with this technology. Toxicity is one of those. Biases is another of those, and hallucinations is another one. So one of the main issues was hallucinations. And we don't even know if that's going to be fixed eventually or if it's part of the technology itself. If it works like the human brain, we kind of hallucinate, right? So which are the techniques that you can use to reduce this in your AI solutions were the first one. And it sound crazy when we first hear that prompt engineer is going to be actually a career in technology. But for LLMs, it's like it's currently the best way to reduce hallucinations and to improve the accuracy of the task is to do prompt engineering in a good way. If you really can craft a good prompt, you're going to get a much, much better response. Next, in terms of improving the accuracy, you have few short prompts that basically is providing a set of examples to the LLM of the inputs and the outputs you expect, and then you provide your prompt, right? This is kind of humans -- kind of how humans learn. Then you have retrieval augmentation. Retrieval augmentation is providing the ability to give a context to the LLM from which to answer. Basically, this is, for example, I don't know if you have experience with tools like AskPDF or things like that. Basically, this is you can ask your own data or you can use your own data and then use the LLM to answer on top of your own data, right? One of the key advantages we have on Mercado Libre is that we are owners of, of course, first-party data, a lot of data. So we can use this a lot in our internal solutions, right? And then you have fine tuning of the models that basically, to put it in a simple way, it's like retraining the model, changing the parameters that you use to train it. And this has been something very complex to do, right, you need a lot of compute to be able to actually fine-tune some of these models, and only very large companies with a lot of compute access can do these things. Now we have things like LoRa that allow you to do this with a subset of parameters. We also have quantization that kind of reduce the size of the models. But the thing is that this is, if you look at the amount of improvement that it can give you, most likely today for most e-commerce companies, this is not worth it, right? Yes. So now going into a little bit of detail of how we are using Gen AI in Mercado Libre in production and what are the things that we are also building. Well, the first one are assistants and copilots. We are building personal shopper assistant basically that will help you go through your purchase journey. My daughter, she is 5 years old, and she just discovered Nintendo. So she's in love with Mario for the last month. And the month before she was in love with Pokemon and Pikachu. And the month before she was in love with the Lion King. She kind of discovered new thing. And every month, she wants her birthday of a different thing, right? So one of the things you can do with this is give you your complex context, like the one I'm facing, and it will help you buy all the different things, take into consideration the things I mentioned like the shipping time and the best price and whatnot, to give you the least, and actually help you do the shop. In terms of content consumers, we are using a lot of acceleration of summarization. We -- one example could be customer experience. When your customers complain about something that didn't work, we go through a CX process. Of course, all of those process can always be improved. And one of the key issues we have at Mercado Libre is that when the case changes agent, we lose accuracy and we lose information that we already asked the customer. So we keep asking the customer the same question over and over. And that's not good, right? It's like when you call the airline and they ask for your reservation number and ID, and then let me put you through, and then the next person asks you the same thing and so on. It's very frustrating. So we are actually using Generative AI to summarize all the conversation with the user so that information gets passed through the differentiations, and we eventually make the process more efficient. But in the end, what's more important, we reduce the time for the user to get a solution. In our internal infrastructure, we use a lot of AI. I mentioned one example, that is compute efficiency and data storage efficiency. Those are 2 things in which we base a lot in machine learning models. But in terms of Generative AI, in particular, we have 2 initiatives. One is what we call the Fury Copilot that is in this platform that all the development teams use. You will have copilot that will be part of your team. For example, if you make a new deployment and the copilot start to analyze the data and see that you have new logs that you didn't have in the past and that your performance is worse than it used to be, it's going to go look at the code you changed in that specific version and give you, using code recommendations, of which things you need to change to make sure that you go back to the previous performance you have in your application. Another area is the enhanced experience. Here, when our users see that there were 1,000 reviews of this product, maybe they want to know which were the most interesting. So we are using Generative AI for that. And also answering automatically questions, right? Instead of having the seller answer the question, automatically answer the question with Gen AI. Like it is in Spanish, but it says what do you need to know the product and it says how long will the battery last. And the answer is automatically provided by a Gen AI bot instead of by the seller. The last thing we have in production right now is hyper-personalized experiences. Most of us in the past used audiences to segment our users and to be able to connect with them in a personalized way. But creating really, really personalized communications with your users based on the actual users of your platform was pretty complicated in the past as well as the interaction itself. Now with this, we are able to actually improve our click rate and open rate of all of our communications with our users. And the last thing that -- in which we are using Generative AI is in Amazon Code Whisperer and code generation in general. We use various platform for this. One is Amazon Code Whisperer. This one here is Sebastian Barrios. He's our Senior VP of Technology. Basically, what we saw is that 35% of all recommendations made by these assistants were accepted by the development teams. But that's not only it. It also accelerates and help us move the curve to the upper right because we have to spend less time writing test that now can be written by the Generative AI. And also we take bugs before they actually hit production that was more complex in the past. And to close, this is a sentence or a quote that we really like, that the best way to predict, and predict is a good word for this talk, the future is to actually invent it. We see Generative AI as a canvas for our imagination, right? We don't see it as a technology only. So there is a lot of things to create and invent in retail and e-commerce about the uses of Generative AI. And we think that the key to invent this new future is to be responsible in the usage of this technology. This technology can be super powerful, but also it has a lot of downsides still. So we need to be very responsible because of biases on how we use it in certain areas like human resources, for example, interviewing. Also when creating content, making sure that it's not being offensive. So those kind of things are critical and is part of our responsibility to create a future that uses this technology but in a way that is not harmful to others. So that's it. Thank you very much.

Amazing. Amazing. Thank you, Oscar. Thank you, Oscar. Thank you all for joining us. We're right at the end of our time. So if you've got questions for either of us, we'll be upfront afterwards, and really appreciate you all being here with us today. Thank you.