Moderna, Inc. (MRNA) Earnings Call Transcript & Summary

Ladies and gentlemen, thank you for standing by, and welcome to the Moderna Manufacturing & Digital Day webcast call. [Operator Instructions] I would now like to hand the conference over to your speaker, Ms. Lavina Talukdar, Head of Investor Relations. Please go ahead.

Good afternoon, everyone, and welcome to Moderna's first Manufacturing & Digital Day presentation. Before I begin, I would like to remind everyone that this conference call will include forward-looking statements. Please see Slide 2 of the accompanying presentation and our SEC filings for important risk factors that could cause our actual performance and results to differ materially from those expressed or implied in these forward-looking statements. We undertake no obligation to update or revise the information provided on this call as a result of new information or future results or developments. I will now turn the call over to Stéphane.

Thank you, Lavina, and good afternoon, everybody, or good evening, if you're calling from Europe. Thank you for joining us. We're excited with my colleagues to welcome you today to this first Manufacturing & Digital Day. So as you know, we started the company 9 years ago, asking the question: Could we turn this molecule messenger RNA, the software of life, into a medicine? From very first day, we've got excited about the idea that if we could do that, we could unleash a new class of medicines with very exciting features, like very large product opportunity, the ability to make secreted protein, transmembrane, intracellular, potentially, antibodies, potentially viral proteins i.e. antigens, and also potentially, the ability to combine molecules to make very exciting products for patients. So a large product opportunity. We also got very excited because mRNA is an information molecule, so we thought that we could have, over time, a higher probability of taking a success of our drug. We could accelerate research and development timelines because of a platform that show what we do, what we invest in process development, in robotics and in digital; and over time, a greater capital efficiency, again, because of our platform. And so if you think about these and also think for a minute on those last 3 features, those just come down to the science. mRNA is an information molecule, you pick the protein you want to design in the coding region. It's the same formulation for every modality, and that's what really got us excited of how do we build the company. As you know, because of a large opportunity ahead of us, since the very first day, we have been very focused with the team and we've evolved to manage risk. And 4 risks that we talk about on a daily basis are the technology risk, biology risk, execution risk and financing risk. And as many of you know, we have managed the technology and biology risk by building a portfolio, a portfolio across 6 different modalities. And in each of those modalities, to manage biology risk, we decided to not do only one medicine but do several. By managing this technology and biology risk, what we basically needed to ensure is that we didn't increase execution risk because it is one thing to take one drug at the clinic at the same time. It is a different game to take 24 drugs in the clinic at the same time with a new technology as we are building the new company, because you are literally flying the plane as you are building it at the same time. And so it became very clear to us in the early days that we had no chance to build the company in the right way, focusing on quality, making sure we could run this clinical experiment across 24 drugs without building our own manufacturing facility. And so if you step back and if you think about how we built the company. So the first phase, of course, was get to the clinic safely, invest in the science. And as many of you know, we entered the clinic with our first flu vaccine in December 2015. A few months after when we got the human data with immunogenicity and we showed that the antibody that we find in human was neutralizing, we started a discussion with our Board and very quickly moved into designing a new facility, the facility of a future that will enable us, through a full integration, through raw materials, making mRNA formulation, filling values, quality control, all in one physical location. So we will be in charge of our destiny, making sure we have the right quality, that we can move very quickly and we could control our cost and also know-how. It was very clear to us because it's a new technology, and as many of you have observed, it is very, very difficult to be the biotech company around a new technology. The challenges around technical development, process development, manufacturing, quality, everything has to be invented. And when you do it for contract manufacturers, we believe, at Moderna, it is a very risky proposition because the asymmetry of upside and downside between the contract manufacturers and the sponsors are just totally misaligned. And so we said we have to invest, because we're playing a very long game, we have to invest and own our own sites. And not only that it has to be fully integrated, we said it has to be fully digital. We have to really leverage the ability to access to information in real time so that we can execute properly and that we can learn very quickly. So the next phase of the company was, of course, running those experiments in the clinic and figure out, through clinical data, where was the mRNA technology of Moderna working best. And as you know, we believe that in the last few months of 2019, we went through a very important inflection point in the company's history. We were able to define 2 modalities that we view as core, where, in our opinion, the technology risk is off the table and still exploring the 4 modalities where we don't have enough clinical data yet to make such a determination positively or negatively. So the way we think about the company, as you can see on this slide, is basically, we have 2 businesses now. We have a business on the left of core modalities, where we believe that technology risk is off the table and where as we've communicated, since the J.P. Morgan conference earlier this year, our strategy is to double down and to leverage the uniqueness of this information mRNA platform to scale the company very quickly and to go very quickly from a drug concept around the meeting room in our scientists' minds to a development candidate ready to go into GLP talks and into the clinic. So this is around the infectious disease prophylactic vaccine, on the left in light blue and the IV systemic therapeutics that you can see next to it. On the right side of the slide, the exploratory modalities are basically us waiting for clinical data to decide what we do with those 4 additional modalities. As we said before, if we have positive clinical data, and we think the technology risk is off the table for any of those 4 modalities, we will move them as core, and again, we will double down and triple down into many more medicines. If we have negative clinical readout, we'll have to assess: Do we think, based on what we've learned, we have a way to recover that modality? And if so, we might decide to go back to the drawing board and go back to the clinic with an improvement in the technology, but as we said so, we'll be very disciplined with shareholder capital. And if we believe we've run fully the clinical experiments and this is not a good use of our time and capital, we will basically shut it down. And that's exactly what we're going to do moving forward as a company. Two businesses, on the left, acceleration investments, focus on BLA; and on the right, being very disciplined and very patient for the clinical readout to tell us what to do next. And that's exactly what we are entering as a company, this phase where we're focused on filing multiple BLAs so that when we get to this point, we'll be able to sell the company, and we are very excited about the next leg on this journey. So that's what I wanted to share with you for this introduction. What I would like to do is to review, very briefly, the agenda for this afternoon, where my colleagues are going to lead the internal discussions. So I'm going to hand over to Juan Andres. Juan runs our clinical -- sorry, our technical operations and quality. He's going to introduce himself in a minute. I've had the chance to work with Juan for now close to 18 years, that ages both of us. And then we do a small Q&A session on the topic that Juan will cover. And then we're going to move into the digital space. We are very fortunate to have Karim Lakhani from Harvard business, who's joining us today, to be able to talk to us about some important learnings and frameworks around competing in the age of AI. And then our colleague, Marcello, who runs all the digital and operational excellence operation in the company, will share with you a little bit of what we are doing there and what we've been doing recently. And then we'll be very happy to be joined by a member of Marcello's team, Dave Johnson, who is leading all of our AI and analytics across Moderna. After that, we'll do a short 15 minutes Q&A session before a couple thoughts for closing. So with this, without any further ado, I'm turning over to Juan.

Good afternoon, good morning, good evening, wherever you are. My name is Juan Andres, and I lead, as Stéphane said, technical development, manufacturing and quality for Moderna, where I have been almost for the last 3 years. I have 30 years of experience in the pharmaceutical industry. I started with Eli Lilly for 18 years in a number of different global roles, in a number of different countries, mostly in the technical area. Then 12 years in Novartis prior to Moderna, where, in my last 3 jobs in Novartis were leading technical development, leading global quality and leading manufacturing operations for all the different divisions at a global level. So here I am. We are in Moderna. So we are a platform. We are a new platform, as Stéphane said, so many things to be learned. But before I go there, let me give you a glimpse on how our process works. Our starting point is the DNA plasmid. And I would like you to think about the DNA plasmid as a template in that, that DNA template, together with nucleotides, with the right enzymes and in the right buffer conditions, form mRNA. This process is done without live cells in an in vitro way. Once the mRNA is formed, we go to downstream purification, typical through chromatographic columns, in order to ensure that the final mRNA product has right purity and quality conditions. The process does not stop there. Once you have the mRNA, we need to put it under -- or inside a number of different lipids. That is our lipid nanoparticle process that would enclose, cover that mRNA and -- in order to make it our product that would be injected with our drug product. Our drug product is an injectable form, so we need to go through the appropriate aseptic operations. And then finally, our quality control before the product can be released. Now as I said, we are a new platform. So 2 things here: new means we have to do a lot of learning; and platform means we believe that once we know how to make one, we will know how to make many, and these 2 equations work indistinctively. The more products you have, the more learning that you would have. And the more learning that you acquire, the more learning that you will be able to apply to a number of different products. This applies into a number of different things. So let's go back into history. At the beginning, we have more questions than answers. And we have to go and answer one of them. Can we reproduce the product? What is going to be the stability? Can we do this at GMP scale? And we have solved most of these questions, but this is in the journey. Every time that we solved a question, we knew that it was not only for the manufacturing process, we needed to make sure that, that new process would work in vivo. And we all know that making and developing medicine is a team sport. It needs to be having a number of different units, a number of different disciplines, scientists, people with experience in order to make that happen. The last thing: We had many questions, we had a few answers. And we needed to have the patience, as Stéphane said, to go one question at a time and solve them. So where would we start? The place where we started were with questions associated with: What is the structure? What is the level of purity? What is the stability? How to measure things? What is the right size of the LNP for the different products that we are going to be taking into the clinic? What about stability? How to measure, from a quality control, the properties of all these attributes? Now the first thing that we covered was around quality. It is very difficult to improve anything that is variable. We know variability is going to exist with us, especially in a new platform, but we focused on taming that variability. So quality needed to be first. So I wanted to highlight this as a control chart, and the first thing that you have to do is getting control first. Once you're in control and you are in good levels of variabilities, you can focus in other things, such as speed, scale, cost. We knew these problems were engineering problems. We know that these are engineering problems that we will solve as they come, but we needed to focus quality first. This will become evident in a number of different examples that I will go through in a few minutes. Let's go back to Norwood and let's start on how the process started at the beginning. So remember, plasmid, mRNA, LNP, fill/finish and the QC of it. And we needed to travel to many -- to different parts of the world to make that process happen. And so from one continent to another, back another facility, the other side of the U.S. Now we have Norwood. And in Norwood, we have vertically integrated all these different parts. So we are capable of making plasmid, mRNA, LNP, and we also have a small fill/finish operations where we can make aseptic drug product. And then we have the quality control laboratories in the QA release to be able to do that. Now we knew, because we were a platform, that we wanted to do these many times and we were going to be learning a lot. So our vision from the beginning was that we needed to digitally integrate everything that we did. We needed to go and be paperless. We needed to capture knowledge so that we could fast go from one place to the other. So the design of digital and automation was an obsession from the beginning. So Norwood is mostly a clinical development site. That is the vision that we had, and that is its main purpose. However, we'd like to think of it as 3 engines within this site. The first one is preclinical. So the first, the preclinical engine basically has made -- I had a lot automation for this, so let me go one by one. Let me center your attention on preclinical. Preclinical makes batches that are on milligrams level. And to this date, we have made over 23,000 batches of preclinical material. Scientists in the bench enters the code of mRNA, and it is sent electronically. The site of Norwood in the preclinical area receives that code and within days or weeks, there is a vial pack that arrives to the patient to do the research or to do some of the development activities. That is the scale. The second scale is the one in the personalized vaccine unit. And you have, in this area, we are making one batch per patient that we have in the clinic. It is an area that it is already in the clinical. It is -- it follows good manufacturing practices, and the scale is in the hundreds of milligrams. We have made over 90 manufacturing batches in that location. I will go, in one of the case studies that I will review later is around this area. The third engine is the clinical engine that is where scale-up starts and one batch can be used for many patients. In this area, we have a scale that goes from grams, 5 grams to to-date, 75 grams that we could go even higher as our need for scale are required. So I described the platform, I described Norwood and the different areas. So what is what we have here? We have similar processes, both preclinical, the personalized cancer vaccine unit with personalized medicine as well as the clinical areas. They make plasmid, they make mRNA, they make LNPs, and they are made in a very similar way, but at different scale. So similar processes. So we also have here the cell-free processes that will enable to have less capital requirements. All of this will impact quality, speed also our ability to scale and the cost. So 3 case studies that -- or 3 examples that we have prepared for you today. And these are examples that illustrate how much we've learned and we've been able to do in the last few years, but they are very recent. And as such, they will also give you an illustration on how we intend to go in the future. I always say, your past or your present is the best indication of your future. So let me start with the first case study. The first one is the personalized cancer vaccine unit. This is one of our programs that we have in partnership with Merck, where the patient gets a sample of their product, and we need to go back and make a vaccine specifically for that patient, and we would like to say needle-to-needle around 40 days. At this moment in time, the program is in Phase II, and it is up to 200 patients for proof of concept. And these patients, these manufacturing batch patients need to be done at Norwood. It is not only the speed at which we need to manufacture this, this clinical trial is designed to coincide with KEYTRUDA dosing regimen. So we need to also land the vaccine on a specific dose for KEYTRUDA. I believe it is the third dose. So that is the place where we land our vaccine. The third consideration that I would bring to you is as it is personalized medicine, you cannot scale up your product, you need to scale it up. So we will always have to make, in personalized medicine, as many batches as patients you have. So let me illustrate the learnings that we have had with PCV to date because our success rate, since we have started, has improved. Our cycle times has been reduced. Our yield has been improved, and as a consequence of all that, our capacity has improved. So we have made, to date, over 90 patient batches of personalized cancer vaccine, granted the scale is not as big as a big, big product. But when you make 90 products, in my experience, by the time you have made 90 products, so for normal product that it is commercial, you achieve that scale in your second, third or fourth year after being commercial. And this is the part where we have gained a tremendous amount of experience. Remember, while the process is the same, every batch is different from the code, from the mRNA that we produce. So it is very, very important to assess what is the quality and what is the learnings that we are going to be getting with this. Let me start, as I did before, with quality. So as you can see here in this slide, the first part or the first batches that we made, our success rate, was okay-ish, was good, but it was okay. It was around 69%. And as we have been making more and more product, as you would expect, as you get more experience, our success rate has improved significantly. And you can see there how our experience has made us improve our quality. And this is not the end. As we gain more experience, as we make more batches, this is going to improve. Next slide, please. In addition to that success rate, every mRNA is different, but it has the same specification. So think about lot 1 is 1 patient, lot 2, all the way into lot 90, the specifications are the same. And even with a different mRNA, we were obsessed with having the same specifications, the same quality, and they are landing in the same place. They follow exactly the same specifications. That will become important in a number of different things that I will highlight later. I also highlighted speed. So our speed has improved. And since the beginning, we have gone faster, around 40%, since 2018. Why is this important? Think about many -- making many batches at the same time with a schedule in which you need to arrive at a certain point in time to the patient. If you want to make many, many, many patients at the same time, being shorter in your cycle time will allow you to overlap one manufacturing with the other. Therefore, you are going to be increasing your capacity and ensuring that you're going to have your delivery on time, the shorter you are, so in this case, speed is important. Speed, quality, yields will give you cost reductions. And this has been our experience as well since we have arrived. So we have had the cost reduction per batch as we have gained in experience. So this is case study 1. Let me start with case study 2. That is the cytomegalovirus vaccine, we call it CMV. And this is a case that when I first saw this, with experience that I had in manufacturing, I thought this was going to be science fiction. I was making many batches, many proteins before, and my first reaction was, "This cannot be done. How are you going to put 6 different mRNAs into the same product and basically inject it; have those 6 mRNAs arriving to the cell, transcribed into protein by the ribosomes and 5 of those 6 proteins come together into a pentamer and then another one forming the vaccine?" And so we had as you can imagine, many, many questions and things that we needed to resolve from a manufacturing point of view. So lots of challenges ahead of us. We also knew that the CMV vaccine would be something that would have to scale to many, many patients, different than the previous case in which you had one patient at a time. So in this case, we needed to scale up, and we needed to do in a way that stability, in refrigerated conditions rather than frozen conditions, was going to be a good consideration. So we needed to do a number of different things where our creativity coming from platform and technical development and our ability to make product in Norwood would be very important. And these things basically get together in speed. One of the things that happened with CMV is within a year, we ended with a lyophilized form that basically increases our chances tremendously, is well-known lyophilization, but it was the first product that we lyophilized in our platform. And we were successful taking this to Phase II, where the product is right now. So we are at Phase II with a lyophilized form that we intend to take all the way into BLA, okay? So in this case, scale is important. So our scale at the beginning was 5 grams, and we used it for CMV. Now 75 grams was our next one, but we have never done this with CMV. But we had done this with another program, with VEGF, for instance. And we used the experience that we had in VEGF scaling at the 75 grams to export that knowledge and experience, our digital platform and our equipment conditions, to basically take this to CMV. So we were able to go faster. Next stage is going to scale, what we are thinking at this moment in time, is around 150 grams, but it could be another scale. I like my chances. In addition to this, obviously, as you scale, your cost reduces. But that is a factor of your scale and your quality conditions. At this moment in time, our obsession is reduced variability, be able to scale, and then things will fall into place from the rest of the things. Next slide for CMV is as I said, we have the lyophilized image that we intend for Phase III. Norwood can produce the Phase III material, and we have the potential to do the scale for a potential commercial launch. We will do the fill/finish at a CMO on this lyo. The image that you have on the screen are actual, real lyo products that we have on Phase II. This is CMV lyo product. This concludes the Case #2. So let me take you to our Case Study #3, coronavirus. Folks, this was unplanned. We didn't expect this. It landed in our lap at the beginning of January, and it required a number of different things that we have learned throughout the years. So it needed the convergence of a number of different disciplines, scientists, all the things that we have learned together in addition to good doses of boldness and a huge amount of collaboration together. We also used all the different parts of the engines that I have talked to you before, in -- so the preclinical, the personalized vaccine unit and a number of different things going forward are going -- we are going to be using all parts of Norwood in this place. We didn't think ever about using the personalized vaccine unit for something different than PCV. But then when we saw the challenge of going really fast, we said, "Why don't we use the learnings that PCV created on us on the same platform?" And we used this to attempt to go to Phase I in -- for the coronavirus vaccine very, very quickly. So let me take you through what has happened in the last few weeks here. 3 days, January 13, February 7, February 23. It started on January 13. We selected -- or we were given the amino acid sequence that we would need to use to design the plasmid template that would allow us to make mRNA, and we did that in a few days. We have the select -- we have the amino acid sequence. By day 2 to 4, we already had the digital sequence engineering and the DNA design. Now then between days 5 and 17, we had to do what normally takes weeks or months, which is get all the documentation together and all the things that you are required in order to make a clinical batch. Due to the fact that we are paperless, that we have the platform that we have, basically, we could do that at a very, very fast speed. So on day 17, we were ready to make the plasmid, make the mRNA and make the LNP at a clinical level. In parallel, we had used already our preclinical engine to make the mRNA that preclinically was going to be tested in vitro, and we sent several copies of that to NIH as well as to our scientist team site together with a number of backups. So the preclinical engine worked to perfection as well to supply that preclinical material. So February 7, we have made our vaccine. The vaccine was done, and it was ready to be tested. All the QC was done in a few days rather than in weeks, but we needed to wait 2 weeks for the sterility test. We cannot make bugs grow faster, and the test takes 2 weeks. So we needed to take 2 weeks and wait until the sterility test was done. And then on February 23, we shipped to NIH the product that was meeting the specifications. And at this moment in time, the IND is open. There was a huge amount of motivation in the system. This is how we felt in the company, each one of us. We channel a huge amount of motivation. People were talking to their families. I have not seen the level of energy anywhere before like this, and that was very, very motivating. So hopefully, I've been able to convey the 3 cases that can tell you how all the learning, the platform, the integration in Norwood have brought to existing examples, and you can make the connection of what is going to happen in the future. We can make the quality, speed and scale and cost that we are going to need as a company across the different products. As I said before, we can do this. I like our chances.

Thank you, Juan. And with that, operator, we'll go into the Q&A session for manufacturing.

[Operator Instructions] Our first question comes from Yasmeen Rahimi with Roth Capital.

I have 2 questions for you. The first one is can you walk us through what the purification cutoffs are as we go from preclinical batch to personalized cancer vaccine to clinical batch? What are other collection criteria that makes clinical batch more unique? And then the second question is can you walk us through how mRNA is combined with LNPs to yield the final drug product? What purification process ensures that we get a homogeneous drug product?

Well, the first one, the purification process, obviously, is different at different scales. And whether you're doing this in a preclinical setting or whether you are doing this in a clinical manufacturing with GMPs, but the process is very similar. What I would say is mRNA is a large molecule, and it follows traditional purification steps, chromatographic columns to be able to do that like you do in normal large molecules. For the mRNA to LNP is basically a nanoprecipitation process. Together with the mRNA, you take the mRNA in an equisetic setting. You take the lipids in a -- into alcoholic, where the lipids are in solution, and you bring them together under normal conditions. And then you go through a process with the right sizing to make sure that we end up into -- with the right quality. Normal scale-up processes are followed in terms of different sizes of the purification or sizing equipment to end up or to be able to process at the scale that you need.

[Operator Instructions] Our next question will come from Matthew Harrison with Morgan Stanley.

I guess, what I wanted to ask is can you just broadly talk about the scale that Norwood was built for? I think you've addressed that in terms of clinical stage batches. And what you've built there in terms of what you've learned, is this something that you can scale easily into a larger facility? Or are there additional steps or things that you would have to work out if you have to scale to a commercial facility?

So thank you, Matthew, for the question. I think the -- you're right. I mean, we need to scale. We need to scale up and our learning and scale has gone faster than we anticipated. I think we are at a scale level that -- in our ability to go and learn from one product to the other and the scale that we have achieved is good. As we go into commercial, the scale at which the site is designed will allow us to go into first commercial scale. The processes to go into a bigger facility with bigger commercial things is not going to be different than the scale that we have done from milligrams to grams to hundreds of grams. And at this moment in time, we expect no issue in doing the normal scale as we would do -- as you would have. This is less complicated from that point of view, that other type of scale-ups.

Speakers, I'm showing no further questions at this time. I'll turn the call back over to you.

Great. Thank you. It is now my pleasure to move on into the digital portion of the presentation. And I'd like to invite Karim Lakhani, Professor at Harvard Business School, to kick it off with his talk on competing in the age of AI.

Thank you, Lavina. It's a pleasure to be here with you. And I'm going to take some time to sort of walk you through a perspective that I've been developing with Marco Iansiti, also a colleague of mine at our business school, on how a new breed of company is emerging that is using data analytics and AI at its core and that these companies compete in a very different way. They operate in a very different way and then subsequently also have different types of market perspectives and valuations. This work emerged out of about 7 years of research that I've been doing with Marco, as we were looking at how the dynamics of the software industry, what was happening in the software industry around both ecosystems and platforms was basically spilling over into the rest of the economy. And as we began our research, what we started to see around 2015 and '16 was that this is not just a platform story, but really, a view -- a platform story or a business model story, but really a story of operating models and how you -- how changes in the operating model of a company imbued with data and AI would make a huge difference. So I'm going to walk you through that thinking. And of course, we'll have some time for Q&A as well. So the first thing I wanted to sort of show you is this image, this image, this painting which is known as The Next Rembrandt. And this is a digital rendering done by AI of a Rembrandt photo -- of a Rembrandt portrait. And the amazing thing about this portrait is that this emerged out of sort of mere mortals working in Europe, looking at trying to imagine what Rembrandt would do as his next portrait 400 years later. And they -- what they did is they scanned a bunch of images of Rembrandt paintings. They used some statistical algorithms, and then literally generated this painting by itself. Do we have the video? Great. Let me show you how this came about as a way to sort of ground you as to why we think this revolution is going on. [Presentation]

Great. So the point of that video is to show you how something magical can come out with a question in mind and then real emphasis on gathering data, doing the analysis and then using pretty much off-the-shelf algorithms to be able to pull this off. And for me, the big realization -- and for Marco, and the big realization was that the ways we think about AI in the popular press are really geared towards what we think about the Strong AI, the Star Trek computer, right, that you can ask any question and it gives you the answer. That's known in the computer science world as Strong AI. But Weak AI, which is a definition is that's also in computer science, talks about any activity that computers are able to perform that humans once performed. That's known as Weak AI. And what's important about this is that the algorithm that we saw, the algorithms that we saw the folks at The Next Rembrandt project do are geared towards very specific activities and are able to just generate The Next Rembrandt. They're not going to generate the next Renoir, right? For that, you will have to go back and redo all the data collection and so on and so forth. But the notion is can we industrialize this process in a way so we can do this at scale? And so the Weak AI goes a long way today, and that's the basis of how to start to think about how deployment of Weak AI algorithms can actually give us tremendous benefit. One of my favorite examples of this comes from China, JD Digits. So they were asked by the Chinese government to come up with products and services that can help the Chinese agricultural economy. And one of the areas they went after was insurance for farmers. And they were focusing on thinking about how do we insure livestock, especially pigs. And so what they realized is that a farmer may not be able to insure the entire -- his entire livestock of -- and they might want just 100 pigs instead of 500 pigs to be insured. And this becomes a problem because how do you know which pig has been insured. So they went ahead and they've used facial recognition for pigs to be able to identify which pigs have actually been insured, and that enabled them to then generate a whole range of insurance and financial services products for farmers in China because they deployed off-the-shelf algorithms with the new problem, got this data and got going. And what's interesting to me about this example is that JD Digits is not going to launch a pig counting or a pig facial recognition software and get rich off that. But what they can do is they can take these algorithms and deploy them in their processes and create brand-new services and products for their customers, the Chinese farmers. Now you can imagine what else you can do. Once you can do facial recognition, you can also keep track of health of pigs, you can keep track of what they're eating and change diet and all those can then become products and services that you can then go to the Chinese farmers with and enable them to succeed. So here's an example, again, of a brand-new type of a business being generated based on attention to data and taking algorithms to go and solve these problems. Now let me also show you one more example of this as well. So the other example I'm going to show you is something that I recently published in JAMA Oncology on an AI system that we developed in -- for radiation treatment planning. And this algorithm basically came about through crowdsourcing contests for algorithms that we launched in last year. And in the space of 10 weeks and about $50,000 in prize money, we were able to be as good as an average radiation oncologist at Harvard, but also importantly, better than any commercial software that was already out there. Now the thing to note is, I'm one of the co-authors of this paper. The team -- basically, the first author and the last author are radiation oncologists. The rest are physicists and data scientists with zero background in biology, being able to, again, attack this problem from a data-centric perspective. And that allows us to then go at this, and within, again, in a matter of weeks, develop algos that are solving a critical problem in radiation treatment planning. And the point, again, here being that these algos are available at scale. So let's now think about this from a much broader perspective, which is what happens when these algorithms are deployed at scale and how this, in fact, can start to change the nature of the firms. And my favorite example here is Ant Financial. Ant Financial, as some of you may know, is a company that has spun out of Alibaba. Initially, it started as an escrow service for payments so that you could build trust between consumers and merchants in the Chinese marketplace. And then since 2013, since they became -- and finally spun off, they have grown massively, to the point now where they serve 1.2 billion consumers on their platform, and they do this with 10,000 people. So if you sort of think about the levers that they're getting, it's pretty incredible. And Ant Financial has not just a payments platform, just like PayPal, but they also have, as you see, other products and services based off on this core platform. So they also have Ant Fortune, which is the largest lending market fund. They have MYbank, which is their banking service provider. They have Cloud Service System and so on and so forth. The thing to note about some of these capabilities is that MYbank, for example, has a 3:1:0 objective. 3:1:0 says that they can -- if you apply for a loan for the bank, it takes you 3 minutes to apply, 1 second to approve and 0 human intervention, right? Compare that to your experiences with banks in the U.S., very different scale. The other cool example I found is that recently, they launched in China, mobile phone insurance plans geared towards young women who wear tight jeans. And you go like, "What? How are they able to do that?" Well, they had enough data on the consumers to note, a, the shopping habits; and b, the fact that young women who were buying tight jeans were breaking their phones. And so they could then use that data to come up with a brand-new product and launch that in the marketplace as a way to say what they can now do with the levels of data that they have. Now Ant itself had scaled rapidly. They're currently worth about $150 billion in the private market, and they're approaching the scale of the U.S. banks today. Now again, the joke I make is they, again, have 10,000 employees, serving 1.2 billion customers. My bank here in Boston will probably need like 1.2 billion employees to serve 1.2 billion customers just given how manual their processes are. So just to put this into a perspective, we want to sort of think about companies like Ant Financial really from an abstract point of view of business models and operating models. Business models are really about your strategy, how you create value and how you capture value. And operating models are really about how you think about how you achieve scale, how you serve more and more customers, scope, how you, in fact, serve them with many different products and learning, which is how you innovate and improve on that. The operating model is the value delivery mechanism for your business model, and they go hand-in-hand. And the prospective operating model has been around for quite some time. So for example, a traditional operating model that might be at a Ford, or Sears, or Toyota is set up in a way such that we are siloed. We're siloed in our data, in our infrastructure, in our capabilities, right? And so this siloed perspective is very useful because it allows us to focus and deliver value quickly to our users and to our customers. But then this also has its limits. The limits are that basically, over time, we get diminishing returns. These operating models peter out because of complexity in the organization, administrative cost, inertia that we basically tend to peter out with these systems. What we see happen instead with a company like Ant and other digitally native firms is that they are set up in a way to drive increasing returns to scale, scope and learning by incorporating data and analytics at their core. Their core operating model is set up in a way so that the different parts of the platform interact together through their systems. And as they get more data, they get better algorithms. As they get better algorithms, they get better services. As they get better services, they get more usage and so on and so forth. And the thing to note, though, is that what this is telling us is that AI analytics and data are now basically relieving the human-centered bottlenecks that have existed for years on end inside of most organizations. And as Satya Nadella has said, AI is, in many ways, is the new run time that shapes all we do in most organizations going forward. Now one of the concepts we talk about in the book quite a bit is that this operating model at Ant is enabled by an AI factory. The AI factory does 3 things, and does 3 things at scale. They -- the AI factory produces predictions about some future event or some future customer need or some future activity; the AI factory does classification, so it tells you patterns in data and tells you how those patterns are being achieved; and third, it drives automation. So those are the 3 things that are the core of any AI factory. The big thing though in the AI factory is building your data pipeline. I can't tell you how many companies still are not being able to build a data pipeline that can actually ingest data from all different sources, normalize it and put it in a place so that we can drive algorithmic predictions as well as have an infrastructure that can actually make this happen. Now one thing to note here is that the AI factory that you see here is the same whether that's at Ant Financial or at McDonald's, but their own AI factory, right? Because the hamburger factory is very different from a finance factory, right? But the data factory, the AI factory underneath, is going to be looking the same and that also drives convergence with the operating model. What happens in this kind of an operating model pool is that the data is stretched across the entire enterprise, software components and libraries and APIs, drive much of interactions across all the systems. And then applications sit on top to be able to take advantage of this going forward. Now the result of this AI factory is that we have 2 different things going on. Companies like a Google, like an Amazon, like a Facebook, like Alibaba have network effects. So as the number of users increase in their platforms, the value of the inherent service itself increases and that -- we sort of -- we've seen this in many consumer products. But also interestingly, as they get this usage data and as they get more and more of this data, they can sharpen this value curve even more because of AI. And we see both of these 2 things happening at scale in these companies. And it's no surprise to us today to think about this in the sense of why we see companies like a Microsoft or a Google or a Facebook leading the race in AI research as well, because they're faced with immense amounts of data and they've figured out how to deploy AI to capture even more and more value. Now what happens when this traditional model of diminishing returns meets the model of increasing returns. Well, we get a collision. And what you're seeing is this collision is around both business models, but really around operating models. And thinking about this is very important because what happened is that the traditional business delivers value early, right, and it keeps going and it peters out. In the digital business, you actually take a while for you to grow the network effects and the learning effects, so that takes a while to grow. And so a traditional business looking at a digital business will say, "There's no value. There's no value. I'm not going to invest and change by processes to set up this way." But as we see in the marketplace today, these collisions are happening left, right and center. You think about Marriott versus Airbnb, the auto companies versus Waymo and what Waymo is doing, right, or banks versus Ant Financial. And these are collisions of operating models, companies that are set up in a very different way to drive scale, scope and learning across the enterprises and to supercharge their business models. So I want to sort of zoom out and sort of tell you what we think in the book actually is the new rule set that has emerged for companies. The first one, it really relates to universality, which is universality capabilities. What we see happening is from a recent sort of software eating the world to actually digital and AI becoming the core of most organizations, right? That's going to be the core of most organizations. But whether you be platform or not, you will need to be able to plug into platforms and be able to play in this way. Then what we see is this universal capability around the fact that the digital operating model when, it collides with the traditional operating model will lead to a collision, and in fact, a separation in value creation and value capture for the digital operating model. But the scale-up times takes some time, right? And so we have to be cognizant of that and figure out how that comes through. But importantly, this skill set is universal, right? We need this skill set in all of our companies going forward, right, and that most product companies will have to also build this skill set inside of their own organizations. The next thing we see is that there is a ton of opportunity, a ton of opportunity everywhere, as you look through that. And what we see is that the model here is to take human labor and management off the critical path, right? We develop rules for decision-making and we develop algorithms to enable us to drive those rules, right, but we actually, in fact, have humans as the designers and operators of the algorithms of the factory, but not as the factory workers. The factory workers are actually the algorithms themselves. And the application space for all of this is immense and keeps growing. And that's why we sort of see separately sort of all this hype and boom in AI. This hype and boom in AI has come about because, a, the cost of generating algorithms have dropped. You need proprietary data, you definitely need that. But everybody sees opportunity into how we can change operating models to succeed in this way. The next thing to also note is that we also need new strategies, right? So as I sort of showed you that collision curve, we actually have to become smart about thinking about how does, how do platforms and AI actually allow us to change the shape of our value curves, rights? So the much maligned WeWork example is one of basically a leasing business with slightly better economics, right? So you can imagine that they were still going to be in the diminishing returns curve. We add a layer of software in digital and it moves us to more of this higher dash line. What the trick for us and for many companies, is to say, "How do I go from this diminishing returns to this increasing returns curve?" And what you see is that different companies act in different ways. Uber is one where there are benefits to increasing usage and increasing data. But because Uber faces lots of competition in local markets, right, it's not going to be able to get this massive benefit of a sharply increasing curve, right? Already, I heard recently that, once again, in New York, there's a new competitor coming into the New York market to compete against Uber in ride sharing, right? And so the barriers to entry are quite low. Facebook, of course, has a slightly sharper value curve. But the interesting story about Facebook is that the networks that deliver value are small there, but at the scale of 100 to 200 people, and what they've become really good at is adding a lot of scope in terms of products and services so that they can then build their advertising business from that. And of course, Ant Financial is very sharp because they have sort of become this fin-like juggernaut. They have set up themselves in a way to be able to provide a whole range of products and services around financial interactions in the Chinese economy, right, and then using that data to then generate new types of products and services for themselves as well. So what this means, though, is that this type of a firm also has new sets of value. So this isn't a very crude measurement, which is market capitalization per employee that I've set up. And what this shows you that depending on the firm, you get different levels of leverage in market cap per employee. But when we bring in these digitally data firms, like a Facebook or like an Ant Financial, what you see is almost an order of magnitude increase in valuations per employee because they can do more. They have data and analytics at the core that enables them to scale faster, to increase scope in a new way and to then continuously keep learning and improving themselves. So this -- all of this is based on a book that I just recently finished up, Competing in the Age of AI. And this book really sort of goes through the details around what we see emerging as a new type of a company and also importantly, what are the -- sort of the challenges and ethical responsibilities for us as we enter this new age of AI.

Thank you, Karim, and we now will transition over to Marcello Damiani.

Thank you, Karim. That was extremely amazing. I encourage everyone to read the book. Good afternoon, and good morning, and good evening to everyone. I'm the Chief Digital Officer and Operational Excellence at Moderna. I have joined Moderna 5 years ago, and I will be walking you through the -- our digital strategy. Before joining Moderna, I was the CIO at bioMérieux, a diagnostic company. And prior to this, I worked for a large-scale organizations such as Motorola in the high-tech industry. And early on in my career, I was software engineer supporting the aerospace industry, so this is my third industry today. As mentioned by Karim and reflected on here by The Economist, data has become the most valuable resources in the last 2 decade, and the most successful companies were mainly data-centric ones. Advanced analytics is what provides insight to data, and AI has become the holy grail of analytics as it allow us to analyze large amount of data sets that are impossible to analyze with other techniques. However, AI requires extremely large, structured and well-annotated data sets, internal and external data sources that are linked and made accessible and digital and data-centric organization to help capture the data from inception in an integrated fashion, whether internal or external. For advanced analytics and AI to take off, digital foundation has to be ready. At Moderna, since day 1, we have decided to build a data-centric company, and we have integrated automation and digital technology into everything we do as you can see here in our 10-year vision. Why building a digital biotech? First, our technology is information-based. We instruct the cells in the body to produce protein and those instructions are like software code, so the very essence of our science is digital in nature. Second, we have a platform approach centered around modalities, and learning insight in cross modalities is key. Third, our business strategy is to progress multiple programs in parallel internally and with our ecosystem of partnerships. So building Moderna in an analog world would have been very tedious, if not impossible. What are Moderna's digital building blocks? First, the cloud. The cloud provide us with the agility to operate cost effectively without the limitation due to long implementation cycle times and the capital investment required for traditional approaches. AWS has been a great partner in delivering those capabilities. Second, this is what Karim mentioned, is really integration. Integration is the glue that bring our processes and data together in a consistent manner, avoiding silos of information. We see many companies struggle with integration due to legacy systems, siloed data and processes. And therefore, what they do is they build inefficiencies across all the functions, and this is exactly what we want to avoid. Growing lean keen to us. The third piece is the Internet of Things. We view this as the next level of integration with real-time collection of data from instruments. The fourth building block is automation and robotics. Our strategy here is to automate when and only when the processes are mature. We do not want to automate too early as this will reduce our agility to evolve and change the processes, but not too late, as we want to reduce any manual work as early as possible. The fifth pillar is analytics. Once you have all the building blocks, you have a wealth of data that provides you -- your business insight to make informed decisions. Finally, AI and machine learning, as previously discussed, are the holy grail of analytics and the ultimate level of our digital strategy, using predictive analytics to help with real-time decision-making and predictions. There are 4 benefits that we see in our digitization strategy: first, quality and error reduction, which increase our probability of success; second, speed, and the way it accelerates our learning and increase our agility; third, scalability allow us to accommodate growth and demand way more efficiently and sustainably; and fourth, cost reduction driving efficiencies, reducing waste, whether it's cycle time reduction, more efficient use of skilled resources or else. The key enabler of this strategy are, first, the people. We have a digital culture. Everyone at Moderna embraces it, and we partner with them to ensure that the technology is serving their goals. The second piece is process. We make sure that we build our processes from the ground up for our digital world, and we use operational excellence techniques, like Lean Six Sigma or else, to build optimal digitized processes. The last piece is technology. We continuously explore technologies and innovations that help us resolve critical problems, and we fully embrace those new technologies when we see benefits. To achieve our vision, we digitized every function at Moderna. Research, technical development, manufacturing, quality, clinical development and the business functions that I won't go through here today. This has created a virtuous cycle of learning, where we run more experiments that generate small data, which help improve our algorithms and learning, which, in turn, help us develop better messenger RNA medicines and the cycle continues. Now let's dive into each of these areas. I'll start with research, where we advance ideas from development candidates -- sorry, advance ideas to development candidates. In research, our scientists start with an idea for a new messenger RNA medicine. To help them realize their vision, we built the drug design studio web portal that allows them to design messenger RNA and protein sequences. To date, our scientists have designed more than 20,000 unique messenger RNA sequences. Once designed, the scientist has the ability to order the sequences and formulation directly from our high-throughput preclinical production facility in Norwood. After the order is placed, we run automated algorithms to optimize the sequence for expression and ease of production. The orders are routed to our preclinical production, which is integrated with 30-plus robotic platforms and where we collect data at every step of the process. We use this data to continuously improve the process. For example, we have built a machine learning model that can predict yield and inform the operators on the actions needed to meet the demand. With those apps and automation, we were able to scale in preclinical production from 40 different orders per month to up to 1,000 a month, allowing scientists to do way more experiments. Once the messenger RNA have been produced, scientists use a set of digital apps to run their experiments. For example, the registry and inventory app allow us to create and track any material used and created in research and in preclinical production. This includes DNA, messenger RNA, lipids, cell lines, chemical and much more. The app manages consumption, aliquoting, material transfer and stock alerts. As of today, we track around 1 million items. To manage in vivo and in vitro studies, Moderna scientists use the in vivo and in vitro study apps to help them design, schedule and track progress of their study in real time. The goal is to optimize overall study outcomes, resources and cycle time. The apps also capture a structured data and detailed study condition for analysis. Finally, included in the app, over 20 data analysis and visual tools that automate tedious calculation and standardize the analysis, so the result are comparable across the platform. The second functional area I'll be covering is technical development. The technical development requires us to capture structured data while providing engineers enough flexibility to mature, optimize and scale processes. So how do we do that? In fact, we built 2 apps. The first one is the platform editor app that allows tech dev engineers to build digital templates of their processes, composed of multiple unit ops, which are reusable building block of a process. Each unit ops is composed of a collection of process parameters, material equipment and calculations. This allows the engineer to capture all the information as the process evolves with the goal to reduce the cycle time of tech transfer and electronic batch tracker design. The second app is development hub. It's a process experiment execution system that captures and document experiments, design and results. It performs automated real-time calculations, support multi-arm experiments and generate automated reports. The development hub integrates with our analytic development app to streamline submission. This has allowed us to reduce testing cycle time by 36%. The next function is clinical manufacturing. In our GLP manufacturing facility in Norwood, as Juan mentioned early on, we are able to operate the shop floor in a fully integrated and digital fashion. Paperless execution, electronic batch records, automated material flow, review by exceptions. And we are collecting around 7,000 variables in real time. Integrated electronic batch trackers instruct operators which raw material to use. All materials, including product intermediates, are bar coded, scanned to ensure accurate inventory. All consumable materials are replenished at a click of a button using custom-built inventory replenishment system that leverages AWS IoT buttons. Integrated electronic batch records instruct, as well, operators which equipment to use, verify that the equipment is ready for use, control operations and collect and analyze the data real time. With this fully integrated shop flow, we reduced labor by 40% per suite per shift and the average cycle time by 2 days. We built, as well, visual dashboard of all the suites in Norwood that allow operators and managers on the shop floor to monitor all operations real time. They can visualize and respond to alarm conditions, check the schedule for each suite and view equipment calibration status. In this next section, we will discuss how we have also digitized our quality process from testing to releasing medicines for clinical trials. Quality is designed in every step of our digital strategy. For example, the sampling of product and raw materials testing is done digitally through real-time integration with our laboratory information management system, the LIMS. Sample labels are printed and applied on the shop floor. When samples are dropped at the pickup location, all information required for testing is passed digitally to the LIMS system to enable QC testing. When QC tests are approved, the results required for batch record calculations are passed back automatically. Before releasing a batch, all batch record exceptions are captured digitally and can be reviewed by supervisors and the quality assurance team in real time. This eliminates the need to review thousands of pages of legacy paper records. Instead, the quality team reviews and approves exception, significantly reducing cycle time. We reduced manual batch records error by 85% and the review cycle times from 3 days to 3 hours. Data integrity is a key component of a digital strategy in a GMP environment. Therefore, we implemented, for our QC labs, a data integrity solution on each workstation that capture equipment data in real time, sent to cloud with audit trails and security controls. We have built also paperless environmental monitoring at Norwood. The biggest benefit we have seen is that it was built digital from the start. So all environmental qualification was paperless and audit-ready, in addition of being very effective and efficient today. The last section I'll be discussion -- discussing here is our clinical development function. In order to track the progress of the various development candidates, we have created a set of integrated applications. Workflows include time lines of regulatory filings, planning for IND-enabling GLP toxicology and clinical operations management. Our clinical operations applications allow us to track our ongoing trials by accessing clinical operations information in real-time from our CROs. It also has multiple tools and analytics to draw key insights. Among the tools we have is enrollment forecasting that track actual subject enrollment relative to plan, a detection of enrollment issues, enrollment trends across the portfolio of actual target and baseline forecast. We provide also a view on the various milestones across all Moderna studies. It helped us easily compare key metrics against industry benchmark, publish all type of metrics, categories, performance, quality and risk, allowing the team to track progress. You can see also here how we monitor site activity, including training and trip reports. We automatically flag reports that fall outside of contracted windows, and we have a heat map of the clinical sites in the U.S.A. I hope this gave you a quick but comprehensive view of how we are building Moderna as a digital biotech. And with this, I'm handing over to Dr. Johnson, who heads our informatics, data science and AI group; and Dr. Johnson will share with you how, as part of our digital strategy, we have progressed advanced analytics and AI.

Thank you, Marcello. It's a pleasure to speak to everyone today. And as Marcello mentioned, I'm Dave Johnson. I run informatics, data science and AI here at Moderna. My role is really to deliver digital and AI solutions to better enable research and clinical development. I have a PhD in Information Physics, and over 15 years of software engineering and data science experience. The last 10 years have been exclusively in the pharma and biotech environment. So I'm delighted today to talk to you about how we're leveraging AI and advanced analytics at Moderna. We use these all across our platform. What you see here on the slide, in the left column, is a selection of all the algorithms we're running in production today. The first 2 highlighted at the top here, were actually leveraged in our novel coronavirus vaccine, the first one to optimize our mRNA sequence and the second to determine the optimal DNA template construction. The next 3 algorithms that are highlighted there are used regularly as part of the personalized cancer vaccine workflow that Juan spoke about. We use an AI algorithm to design a personalized mRNA vaccine for an individual patient and then, again, use our mRNA sequence optimization algorithm. We also use Monte Carlo simulation to plan our complex PCV supply chain. Coming up right behind these in the center column are many new algorithms that are in development or final stages of testing and will soon be out in production. Across all of these algorithms, we leverage a wide variety of computational techniques from simple rule-based optimization to more sophisticated neural networks. There's really no one-size-fits-all technique. So it's critical to have a large toolbox at your disposal. So what I'd like to do today is -- for the rest of this talk is dive deeper into 2 of these use cases that are really good examples of the kind of AI work we do at Moderna. The first use case is in Sanger sequencing analysis. This analysis is used in our preclinical production process to QC our DNA templates. This is the very first step of our production process. I'll show you how we built an automated data analysis pipeline using an existing algorithm. I'll then walk you through how we extended that algorithm using a neural network to address a challenge with our poly-A tails. Finally, I'll show you how we built a radically new algorithm from the ground up that achieves even better quality performance. The second use case, on the right here, is in our research space, where we're using CryoEM images to study the lipid nanoparticles in our formulated drug product. Analyzing these images requires a sophisticated 3-stage analysis using 3 different algorithms. Let's first dig into the Sanger problem. Sanger sequencing is an assay that allows you to determine the nucleotide sequence of endpoint DNA. While Sanger's an older technology and newer next-gen sequencing tools have emerged, Sanger's still used as the workhorse in most DNA synthesis operations due to its low cost. At the bottom here, you can see an example of a chromatogram, called a read, generated by a Sanger sequencing instrument. Each peak corresponds to a different nucleotide based in the input DNA. The relative peak heights aren't very important here, but the color of the peaks are, because each of those corresponds to a different nucleotide base, A, C, G or T. So it is looking at color of each peak you can read off the nucleotide sequence. Now this assay is really important for us because we use it to QC the DNA templates we use to produce our mRNA. If just a single nucleotide base is mutated, from insertion, deletion or modification, that mutation will be transcribed into the mRNA, which will then be translated into the final protein, resulting in a nonfunctional protein, so it's critical that we catch these kinds of mutations very early on in this first stage of creating the DNA template. In practice, the analysis isn't quite as simple as just reading off the nucleotide sequence. Sanger data read is noisy at both ends and will only give you around 700 bases of high-quality data. So you need to collect multiple reads across your DNA in both directions and then combine the results. The full analysis requires a complex 5-step analysis pipeline. We've built our high-throughput preclinical reduction engine about 6 years ago. We didn't have much Sanger yet, so we didn't have the data we needed to build our own algorithm. Instead, what we did is license an off-the-shelf algorithm, which has been an industry-standard tool for the last 20 years used by thousands of labs. However, what we did realize is that we'd be generating a lot of Sanger data. It takes on the order of 100 Sanger reads per mRNA construct we produce. So what we did is we automated the Sanger analysis pipeline using this algorithm. The actual Sanger wet lab work is done at an external sequencing vendor as this work is fairly commoditized. However, the end of the sequencing analysis, the instruments at the vendor upload the data to our cloud storage environment. The moment that data hits our environment, automated processes kick off and spin our compute servers in AWS's spot marketplace. These spot servers are pulled from Amazon's excess capacity, so we paid pennies on the dollar. The servers run the Sanger analysis on the raw data and send the results to a custom-built Sanger app. When all the data has been analyzed, the servers are automatically shut down and discarded, and we only pay for the time we use them. This approach is an extremely cost-effective way to perform large volumes of computational analyses in parallel. This custom-built Sanger app not only captures the results of the analysis, it also is a powerful interactive visualization tool that allows the operators and scientists to inspect the data, the analysis and any problems that might have been found. In this example here, you can see a target sequence at the top and a couple aligned reads at the bottom, so the data from the bottom read is further along, so the peaks are harder to distinguish and lower quality. The consensus of the reads here show that there was a single G deleted out of the thousands of nucleotides in this sequence. The analysis found this needle in the haystack, and so this sample was discarded. Since we automated this analysis some 6 years ago, we've now collected over 3 million Sanger reads of data across hundreds of thousands of samples. Now as we process all these samples, we've unfortunately discovered that there are a number of very rare failure modes that are not detected well by the licensed algorithm. With all this data we've collected, we can now turn around and then build better algorithms to catch these cases and improve the quality. The first major failure moat that we tackled was in our poly-A tails. As you might know, our mRNA sequences end with a long stretch of As that gives the mRNA stability. We've build these tails into our DNA template so that they'll be transcribed into the mRNA. We use Sanger to analyze these tails just like the rest of the sequence. Here at the top, you can see an example of a high, good-quality tail. You can see the repeated A peaks in green. There's a little noise that's underneath the peaks. But overall, it's a very good signal. On the bottom, on the other hand, is a bad tail. In fact, it's an extremely bad example of one. You can see these large secondary peaks sitting underneath the tail peaks. And what these typically indicate is subpopulations in this sample with truncated tails that are shorter than they should be. Now the real challenge in this problem is that the -- while these sub-peaks are really obvious to all of us who are looking at them, the algorithm we licensed is not able to detect it. It calls that read a little bit lower quality, but it still passes it. So unfortunately, while we have automated the majority of this whole Sanger analysis process, we still have to have trained operators weigh-in at the end and manually review these tails. That's obviously very tedious and expensive activity. So we set out to automate this part of the tail analysis. The first thing we did is we had trained operators review thousands of tail chromatograms in a controlled setting and have them label it as pass or fail. In all, we generated over 20,000 labels. And the first things we did with this labeled data is to see how consistent our operators were at making these calls, and you can see the results here on this chart. On average, 2 different operators are viewing the same image multiple times only agree with each other about 70% of the times, and some of these combinations are as low as 42% agreement. But it doesn't stop here, you can see on the diagonal of these boxes, we showed the same images to operators more than once, so we can determine how consistent they were with themselves. You can see the self-consistency here. On average, operators only agree with themselves about 90% of the time. That means that if you show an operator a tail image from this set 10 times, one of those times, they will change their mind. Now in an analysis that's used for quality control, consistency is paramount. So this is a problem. So this next slide here, you can see the individual operator performance relative to the team averages using what's called an RSC graph. On the left, we have the true positive rate on the vertical and the false positive rate on the horizontal. A perfect performance would be in the very top left, whereas the diagonal line here in the middle is the performance of a random classifier, essentially a coin flip. Each colored point here is a different operator. You can see that, in general, the human operators are actually pretty good. They're very far away from that random classifier, but no one is perfect or close to perfect. But the distribution that you saw, the variability of operator performance is also apparent here where they're scattered around here. You can see some operators that are just, in general, poor performers. Some are more strict operators intend to fail more than average. And that's not a particularly good thing either because they're throwing away good samples. And then some operators are more lenient and let more samples through. So what we did is we took this training data and we've built a convolutional neural network, or CNN, that is predicting this overall team performance. This CNN is the same type of network used in state-of-the-art image analysis. The gray shaded curve there is where we can choose this algorithm to operate. What we did is we chose a point up on the top there that is a balance between too strict and too lenient. Now what you'll notice here, if you look carefully, is that this algorithm achieves superior performance any single operator. So this is an example of when Karim talked about Strong AI, where the AI outperforms the humans. We put this algorithm into production over a year ago and have saved countless hours of manual tail review. But beyond that, the algorithm is 100% consistent. Given the same tail we mentioned, it will always return the exact same result. But we didn't stop there. In our pursuit of higher and higher quality, we discovered numerous other failure modes, like mixed bases, contamination, mispriming, failed contigs, peak drift and so on. And it turns out that all of these problems are rooted in the same mixed base problem, where these sub-peaks are not well handled by the algorithm. So if we can solve the mixed based problem, we can solve the rest of this. In this particular example that you see here, you see a significant G peak underneath the C peak here. And what that means is that approximately 30% of the DNA in this sample has a mutation of C to G and that's hiding under the dominant peak. Now that means that 30% of the protein that we would make from this DNA template through the mRNA would also be mutated and nonfunctional, which would be a huge hit to the potency. So it's really important that we can detect these failure modes as well. We realized we could just layer another algorithm on top here because there's a fundamental limitation on how the existing algorithm handles these sub-peaks. So we decided to build an entirely new Sanger analysis algorithm from scratch. To do this, we reached for Bayesian inference, which is a great tool if you have a good understanding of your data generation model. There are 2 key innovations here. The first is in the peak detection phase, which is the image on the left here, where you need to locate the center and heights of all of the peaks, including these sub-peaks. Here we designed a mathematical model based on all the historical data we have and used the Bayesian map analysis to find the optimal fit. In this image, you can see how the solid lines here are the raw data and the dash lines are the fit to the model. There's a really great correspondence here showing an excellent fit. Once we have accurate peak amplitudes, the next innovation is to determine what nucleotide bases those amplitudes indicate. To do that, we used Bayesian hypothesis testing, which evaluates the probability of all possible bases and all possible mixtures of those bases. Here you can see the same mixture on the left from the previous slide. With just a single read, the algorithm is going to still call it a C because these kinds of mixtures are so rare, but the faded out color of the C indicates that it would be flagged as a low-quality or a warning C. However, a really nice feature of this technique is that it calculates the cumulative evidence. So if you see this pattern 1 or 2 more times, the algorithm is now convinced that this is indeed a mixture of a C and a G. And then furthermore, it's able to quantify the exact percentages of each base. So here, it seems that, that percentage is actually 26.5% of the population with the mutation. These kinds of basing techniques are notoriously computationally intensive. This new algorithm takes about 100 times the computational power of the previous one. But with modern compute servers in AWS, the analysis only takes about 10 minutes to run. So with Sanger, we've looked at using AI for this step at the very beginning of our production process. Now switched one in the very end of our process, where we're looking in the research space with CryoEM, we're trying to see more about our lipid nanoparticles. After creating our mRNA, as Juan mentioned at the very start of his talk, we combine the mRNA with small molecules and lipids to form lipid nanoparticles that encapsulate the mRNA. These LNPs are our final drug products. The exact structure of these LNPs is heavily dependent on the specific components that go into the particles in a specific process by which we combine them. CryoEMs are images from an electron microscope that take pictures of our LNPs. In this image here, you can see a number of darker circles, each of those corresponding to a single LNP in a sample. Now usually, these images are just viewed at qualitatively and people make judgments on them. What we're doing here with AI is collecting quantitative numeric measures from these images. This is important, first, because we want to gain insight into how variations of our LNP components and processes impact the LNP assembly. But second, we want to understand how variations in these assembled LNPs will impact downstream in vivo performance. To date, we've generated over 500,000 of these images and countless process variations. But unlike in the Sanger problem, there's not a pre-existing algorithm we can turn to. We have to build our own from scratch. The first step of this analysis is called segmentation. This is where we identify the parts of the image that correspond to individual particles. Now this is a great example of Karim's Weak AI because humans are very good at segmentation. Millennial evolution have resulted in human brains excelling in these kinds of image-related tasks. But while you can look at this image here and figure out which components of that are of particles or not is an immensely tedious job for you to go through and actually draw lines around each of those, and it's not particularly accurate. But on the other side, it tends to be a very complicated problem for computers to do. It's only in recent years with modern algorithms that we're able to actually solve this. The solution we used leverages the state-of-the-art convolutional encoder-decoder neural network depicted in the middle. This network maps out an input of -- input image of pixels to a probability map that each pixel belongs to that particle or not. Then we use that to create an image mask. And here on the right, you can see the output of that, where the pixel shaded in purple correspond to the particles and those shaded in orange are the background. So you see this model performs extremely well at identifying these regions and saves countless hours of tedious manual work. The next task we approached was classifying the type of these particles. On the top left, you can see another CryoEM image. In this image, there are nice round particles and then there are these other irregular particles with blunts. What we want to do is build an algorithm that can detect which particle is which. Again, this is another great example of Weak AI. This is not a task that the previous neural network can do, it actually requires us to build a whole new network to classify them. The distribution here, the histogram shows what would happen if you tried something naive, like just looking at the major access length. And we see that this is not well separated. We need a more sophisticated machine learning algorithm. We tried a variety of models that are shown here. And what we landed on was a random forced algorithm that incorporates the intensity of every pixel in each of the particles. This model achieves an incredible 0.98 accuracy and 0.99 precision, which is just unheard of and to see rates that high. And once again, through the use of this algorithm, we're saving countless hours of manual effort. Finally, we'd like to extract some metrics from the structure of the individual LNPs themselves. If you look very closely at this image, you can see a railroad pattern around the edges that indicate a lamellar structure. If we draw a line from the center of the particle to its outside edge and then sweep around, averaging the pixel intensities as we go, we can produce a graph like in the top right here. Each row on the graph is the profile of an individual particle in a CryoEM image with its outside edge aligned to the right. Then we average vertically across all these pixels and we've got a profile on the bottom right here for the whole image. What you can see come out of this picture is that these pixel intensity peaks show up what you saw with the naked eye, but additional peaks show up as well that were harder to detect. But then beyond that, we're actually able to quantify, in nanometers, the exact width of that lamellar layer. These profiles turn out to be very formulation-specific. In the bottom right here, you can see what happens if you look at a variety of different LNP components and processes, you see a variety of different profiles that come out. So using the quantitative metrics here on this lamellar width and other metrics like it, we hope to gain insight back into the process that we form these LNPs and then in the downstream LNP performance. So what I'd like to do here is close by highlighting the AI factory that we've built at Moderna that allows us to continuously mine and deliver AI algorithms, like these several that I've shown you here. First, you heard from Marcello about all the great lengths that we go to in order to capture and integrate structured data across the company. I just now spoke at length about the various algorithms that we have in research and development. I also walked you through the automated Sanger analysis pipeline example that illustrates the sophisticated cloud and DevOps infrastructure we have at Moderna, which allows us to rapidly deploy compute power. Combining all these together allows us to do AI model design at scale, leverage all of these to rapidly design, test and deploy AI models. Finally, this integration, this infrastructure and processes we built at Moderna allow us to rapidly move completed AI models into a production environment for daily use. This AI factory allows us to push these same goals to even greater heights with improved quality, increased speed, increased scale and reduction in costs. With that, I'll say thank you.

Thank you, Karim, Marcello and David. And with that, operator, we are ready for Q&A for the digital section.

[Operator Instructions] Our first question comes from Hartaj Singh with Oppenheimer.

On just the digital and AI side. So having been in drug development for a long time, whenever I think of drugs, I think of preclinical, clinical and CMC, basically the 3 kind of the holy trinity of the BLA or the NDA. So I guess my question about the AI and digital that Moderna is developing and utilizing is, can you give some concrete examples what -- which portions of those attributes, and again, I know you can probably spend a long time but issues that are relevant today, for example, how they might have helped with coronavirus, the vaccine or CMV, but could you give us tangible examples of how AI and digital are sort of helping you in kind of interfacing through regulators through these 3 sections of the IND and then what ends up being eventually the BLA or the NDA? And then I got a quick follow-up question.

This is Marcello here. Just to set the stage. Actually, we started AI based on the historical data that we have. So you would see more and more AI at that stage in our research and preclinical space than in our clinical space. But as we are progressing and building the digital infrastructure to capture all those data from all those engines that we have, we are building more and more AI. And as Dave mentioned, we have many algorithms that are, at this moment, in training to help us in those new AI models in the different areas.

The learning, specifically the learning that we have gathered on PCV, has helped us go very, very fast on the coronavirus vaccine.

Yes. It's Stéphane. Just to close what Marcello said. I mean if you think about it, as we build the company from having nothing to be a preclinical company and then being the digital and being beyond preclinical, that's why we have the most data, that's why the team has done a lot of work. The next phase that we did as we build the company was CMC at Norwood, which is why you start to see there's a lot of AI algorithm in the CMC space, because we are collecting so much data in Norwood. And the team, in the last couple of years, has been building and is finishing the clinical side of digital, including how to file the IND by basically putting our pieces together coming from tech there, where we have integrated the robotics is ready, too, an IND file, the top study and the preclinical data as well as clinical synopsis of the clinical study, all coming together through that system. So in clinical, we're still at a stage where it's mostly building the digital infrastructure to then generate the data to then enable us to do AI.

And if I can comment on coronavirus, we used an algorithm that we have developed for a previous construct, long construct because coronavirus is a long construct, to be able to produce rapidly the coronavirus construct. This is based on the learning that we had in research and in preclinical from previous ones.

Great. And then just -- I had a question actually earlier when Juan was presenting. And I just want to ask that, if you don't mind. So just as a real-world example, if you can just walk us through if, let's say, you had been at the equivalent of potentially an analog company and the coronavirus, essentially, problem had landed on your laps, like it did a few weeks ago, what would the steps you'd have to go through and would probably still be going through in order to get a candidate to the NIH and then kind of compare that against what is it specifically that Moderna did to get it to the NIH in those 42 days? Just broadly speaking, if you could just give us a -- kind of in that context would also be helpful.

Thank you, and that is a real good question. So there is there is a lot of examples, and of course, I mean, we have our own platform and technology, so it's very, very difficult to go and compare. But I can give you a couple of anecdotes here. So a typical manufacturing batch record in the biotech industry has around 10,000 entries. And it is thicker than one of those old telephone books and so just it would have taken a long time to prepare a batch record only to make the product. And having the digital platform, it allowed us to be able prepare that documentation in a much, much faster way. The same thing goes for equipment, equipment parameters that would allow us to go and set up the equipment and the automation that will allow to make us the -- to make the product. I went in the presentation in the slide, it took a day to set up a process like this. I have done something similar than that, it would take weeks or months to get ready to do something along those lines.

Yes. And maybe just about that a little bit. If you think about it, there's a research process, there's the process development piece with [ the puzzle ] and then manufacturing quality, regardless of how you do it. If you think about what some of the big vaccine manufacturers have announced in the last weeks, J&J has announced it would take them 10 to 12 months to go into the clinic, and Sanofi has announced it would take them 12 to 18 months to get into the clinic. I mean as you might have heard from Secretary Azar last night, the FDA opened the IND for mRNA-1273 on Monday. That was less than 10 days after filing the IND. And if you read it, it's less than 60 days from getting the sequence published by the Chinese government. And so if you ask the question, how did we do that versus the time line that I just shared to that being communicated publicly by J&J and Sanofi. I think it goes right into those 3 buckets. First is the research front, which is we did not need to have a virus, just a sequence on the computer was enough. And thanks to the work we have done before, we're able to very quickly, working with the NIH, to pick the sequence. So if you think about research without spending months and months, playing with a different model, a different candidate and optimizing, we literally did that on computers without ever having the virus in a matter of a day or so. Then on the process development. In a typical process, you lead an army of engineers to figure out the cell line optimization and that will have taken literally months. In our case, because an mRNA is an mRNA, the team was able to retrofit very quickly the personalized vaccine unit. Those robots that some of you have seen in Norwood have been developed for personal cancer vaccine to look, for the first time, our infectious disease vaccine. So it's just a little bit of tweaking to do. Again, it took days and not months because it was the first time we did an infectious disease vaccine on the robotic system. And then the manufacturing is another piece where we get massive advantage. It takes a few days to make mRNA. It takes a few days to formulate the mRNA. So if you think about it in days versus running a recombinant train, that would take you 4, 5 weeks in terms of making the API from a recombinant standpoint. So I think it's just -- all those pieces that had, but then you layer on top of that all the digital electronic batch record, quality control that the team has been talking about, and that's how you end up having a cycle time that we had in our 42 days from sequence to shipping a product having successfully passed sterility testing. And I even think that this is because it's the first time we have done it. I don't think this is the fastest time we will ever do, just because -- and Juan is starting already here next to me. Just because, if you think about it, anytime we do something the first time, you are running things, you have to optimize things, you have to develop new things. And as you're able to do it more and more, as Juan has shown when we discussed the cancer vaccine slide, as you do it more and more, you become stronger and stronger as you keep on learning. And I think that's really a key theme about how we're wired as a company from a cultural standpoint and how we use the digital backbone that we have built and the access to data, it's all about learning. As we've stated in the past, and we believe mRNA is a new class of medicine, we believe it is a 15- to 20-year S-shaped kind of learning curve. And we want to be the company that learn the fastest. We've always said, we won't be the smartest, but we will be learning the fastest. And this digital and these AI tools is really at the heart of our strategy.

Our next question comes from Yasmeen Rahimi with Roth Capital.

I guess, I have a question -- two questions for you. Can you help us understand how you can utilize AI technology to really help predict translatability from preclinical models to clinical models? Can you use AI technology to really identify patient selection and clinical studies? And as we go with additional data from CMV upcoming, can we really help us to determine who could be responders versus nonresponders? And then the last question is for Stéphane. Not only is manufacturing AI technology setting you apart versus your competitors, but how much is adding AI increasing the barrier to entry among not only other companies in biotech but specifically in the mRNA world?

Yasmeen, this is Marcello here. I'll start with your second question. Will AI help us with patient selections? I would say, definitely. At that stage, we are connected. We are connected to all the CROs and we are pulling data from those CROs. And as you know, we started our clinical trials at the end of 2015, and we are starting collecting all this data. And as we speak, we are looking at different AI models that can help us. And as we progress those, I think we will communicate or we will show our progress in that field. On the translational part, it's the same question actually, because you're moving from a different part of the engines that we are building, so you have the research, the manufacturing and the clinic. What I can say as of today is we are collecting data in every step of those processes, and we can trace back the data from clinic to research. So the question is, how this is going to help us to improve in the clinic will be, as we progress the clinic, to put some problems to our AI team, and they will help us build the models to resolve those problems.

And this is Dave here, just to add a little more color here. As I hope is clear from the slides that I shared, all of these problems are a journey. We start solving these problems with the data that we have, and we continuously evolve and continuously push these forward. Right now, in a lot of these spaces, we're using traditional pharmacometrics and pharmacology techniques. We will continuously evolve those as we get more data and inject more and more AI into those spaces.

Yes. And it's Stéphane for the last question. I mean as Karim said when he -- in his presentation, we are big believers that time is the only common denominator across companies in the industry. And so we believe that being able to learn faster is a key competitive advantage. And I think it is even more important as you are going through the S-shape curve that I talked about, compared to companies that are at the top of curve like I used to be in my previous companies, where when you grow and learn 2% to 3% a year, you feel great, it's a great year. Now if you think about the mRNA technology, as we've shared in the past, if you look at the technology we have today in the clinic versus the technology we got from the Harvard lab when started the company, we have improved 1000x, 2000x the performance of the mRNA technology. And as you know, and you saw it the last time, the last spring, and I welcome you to attend the next one with Stephen Hoge and his team in June in New York, is we keep on learning. I mean if you get the tools like the cryo that Dave mentioned, and it's just one of many examples. We are equipping all the teams across the company with tools to be able to learn fast. The big change management, I believe, and it is across all companies, and we are driving it with Marcello and Tracey or [indiscernible]. This is new tools, AI, machine learning are new tools. And the biggest challenge, in my opinion, is change management because you have most of the leaders in most companies, they have never done in AI before. You have people like Dave and Marcello who understand it and have studied it. And -- but most of the line teams, they have never done AI. And so what we've tried to do is to first build the infrastructure, the digital tools, the processes to get the data because we have a great data and good luck. And then what we are layering on top of that is in every group in the company is AI. And what we want to do and it's going to take us a couple of years to really make it 100% part of our DNA everywhere. We want AI in finance. We want AI in HR. We want AI in manufacturing, in finance, everywhere, and that's really how we are building the company. So it doesn't happen overnight, but we believe that if we are well capitalized like we are, with the scale as we are as a company, you can really build AI as a key component of our culture. It'd be really, really very hard to compete with us.

Yes. If I can just add, this is Karim here. If you want my observations, generally outside of the pharma business, right? There is a data advantage, right? So you need to sort of think about the data you're accumulating and both the data fusion abilities, right, and proprietary data. And I think you can sort of see that in how they were talking about from sort of research to manufacturing to clinical, and that this has to be completely be ingested all the time. I think the second thing is really the industrialization of analytics and AI, right? Most of the time you think about decision-making that happens in companies, right, the tools we use are Excel sheets, right. We'll be having Excel sheets, we cop our buddy to give us more data. We think about the data, we modify the Excel sheets to do that, right? This is a totally different beast. You want to industrialize the decision-making process. And just think about all the decisions they have to make, again, from R&D all the way to the clinic. And the third part, I think the really important part, is the change management part. Where I see these things die in companies over and over again is because the management team, a, either were not committed, or they think it's clear as a mud and they hope that we'll sprinkle some AI magic, and we'll go from there. And the change benchmark part, both like you have the sort of the AI data showing up now in the economy that they're working with. But there's a huge science component here that also needs to be retrained and shown the benefits of this approach, and that's a big mandate. And I think many companies outside of sort of the pharma-biotech sector fail miserably at this -- in this journey. So I think those are the ways you sort of can think about how competitive advantage is built.

Yes. And Stéphane, again, for last point. Last year, Marcello and I took a trip on the West Coast. We see a few companies, just look at AI and what are best practices and change management in those companies and tools. And as you know, you heard several times that we are partnering very strongly with Amazon and AWS. And the piece thing that was extremely exciting to me when I went to see a server with [ Mark Shepo ] is the customer obsession that Amazon has brought into the retail space, and Bezos has talked about it all the time. When you talk to the AWS team, and you see the tools they are developing, they are totally customer obsessed. They are developing tools every day that are available in the cloud so that all teams are able to go and to use those tools. And so we have this set up now, which is very interesting, which is the Amazon team building more and more tools to just make them available if you use the cloud. And then Dave and his teams, when they need something that is not available, then they build it. And so we have this, really, ability to grow very fast building tools, and that number is just increasing and more and more powerful tools. And then the team, being able to design the tools they need if they cannot find them in the Amazon store.

Our next question comes from Cory Kasimov with JP Morgan.

This is Matthew on for Cory. So for the first part of your talk on your personalized cancer vaccine manufacturing success rate, where I believe you said you're currently at around 87%. What are the key reasons for manufacturing failure? And then how, if at all, do incorrect sequences that you outlined in the digital part of your talk factor into this?

Well, I think it is a combination of factors. A lot of that is just the know-how. I didn't go into the presentation necessarily with what are the root causes, but it is a normal part when you do a scale-up. Obviously, they go around equipment variability, it goes about operators' variability, it goes around process variability. So there were a number of different things. At the beginning, we even started with the CMO until we were able to bring that into Norwood. So through the obsession in tackling one product -- one problem at a time, we started to increase the success rate as we continued doing it. I would have argued that that the fact that we are doing personalized cancer vaccine, and we are making as many vaccines as we may, the learning, as Stéphane mentioned before, is going faster because our aim is higher. And so we can connect the dots on a number of different root causes at a much faster way. And of course, digital was part of it from the beginning.

Got it. And then, I mean, is there a technical limit to how much you can improve upon this? Or how do you improve upon the rate even further?

I think the sky is the limit. And I think we will get it way, way, way into high 90s here, when -- if we go all the way into commercial, which is the normal thing. We will go into validation. At that moment in time, we would have incredible experience associated with -- associated with it. I also point, if you remember in the slide, there were different colors. Some of the first colors were basically failures that we needed to -- that we found out after the batch in some -- or in all of the recent failures that we had, we had more abort rather than failure, so it was not a surprise. In this case, it's particularly important because if something goes wrong for whatever reason, and as I said, it could be equipment -- as simple as an equipment malfunction, I don't know, a pump fails to produce or just something that is unexpected. You discover that immediately so it is not a failure later, it is an abort. When you're doing something for a patient in a small period of time, basically, it allows you to go and produce fast again and arrive to your destination. So those things are integrated, so -- even in the failures.

Yes. And it's Stéphane, just to the point that Juan mentioned during his presentation, which I think is very critical, which is the personalized cancer vaccine has been extremely enabling for us because we have made so many learnings. As Juan said, it will have taken us years after launch of CMV vaccine or Zika, to get to the same point of learning. And so the power of these personal cancer vaccines was it -- not only it was an extremely important enabler to corona vaccine, as we discussed earlier. But more important to me, for clinical batch. When we make big batch for, again, CMV or Zika or MMA or pick your favorite product in the pipeline, when the team makes that, all the learnings coming from the ATN that we have in PCV, it's extremely valuable. And if you think about it, if we did not have PCV, it will take us years from now to get to the same place in learning, and that's a competitive advantage that I think sometimes is not always fully appreciated.

We have a question from Matthew Harrison with Morgan Stanley.

I guess, what I wanted to ask is, how far do you think you've pushed the limits of the technology and is there a case here where you've tried to use AI and have failed? And what sort of evolution do you need to see to be able to push it further?

It's Stéphane, you're asking about mRNA science, correct?

No, I was asking about AI and sort of where you -- you gave some examples of some concrete processes where you've had success. I'm wondering, has there been something where you tried to push AI into a process or a situation and haven't been able to make it work and -- because I was trying to understand, is that a limitation of just technology and you think it'll catch up or what are the reasons where maybe you couldn't use it in certain situations?

Sure. This is Dave. I'll take the question. So I don't know offhand any kind of large failures that come to mind. What we do when we start AI projects is we focus on ones where there's clear tangible business value that we can achieve in a realistic time frame. And again, there's this journey aspect of it, where we try to focus on problems we can get a minimal viable model out that's delivering value. And then we run that for a while, and we continuously evolve and improve that. And I think this is something that gives us a much larger chance of success rather than going for some large moon shop. Probably that will take us multiple years. So we try to focus on these real smaller achievable problems

Our next question will come from Salveen Richter with Goldman Sachs.

One on AI. I guess, if you're integrating it across maybe different verticals. How much do you want to optimize one part before you fully integrate just because you may have to play with various inputs? So just trying to understand how you do that. And then the second question is as you scale up on manufacturing, how do you think that's going to impact COGs going forward?

Yes, I'll take the integration verticals. Actually, we don't look at the verticals. We look, overall, how and where we need to collect data to be able to improve the processes. If you look at verticals, you're creating siloed information, and this is the thing that we want to avoid actually. We want to make sure that we're collecting data from all those verticals that in some instances, we will use to solve business problems.

Yes. Let me add, and maybe -- this is Karim. The perspective to take is one of the architecture and the architecture of these AI-first companies is very different. Like even the setup of a silo is going to be antithetical to what you want to achieve, right, and so this is the notion that the operating model, the way you're structured is very different than in a traditional company. And there's a notion in the management literature called Conway's Law that the architecture of the technology reflects the architecture of the organization. And that's what -- that's what you're hearing that it's a completely different architecture. So these questions sometimes don't even -- are not even relevant in this model.

I can give you an example. We are collecting data from one of our vendors. I won't name the vendors. And we were building this data for just as an integration between us and them. And 2 years down the road, we saw some issues that we needed -- and problems that we needed to solve. And we were figuring how to do it. So what we've done is we looked at, holistically, at all the data that we had available and we detected patterns that helped us resolve the issue. Shouldn't we have collected all this data and started, we weren't able to do anything with this use case or problem that we have.

Yes, we were able, in that particular instance, to identify 3 out of about a dozen instruments at the vendor that were malfunctioning and are not configured properly. So we're able to take that data for a purpose that we're using entirely separate and identify challenges and tell the vendor to go correct that mistake.

Another example that comes to mind, and I will comment on COGs here in a minute, is things such as predictive maintenance, in-process controls, a number of different things that we would go. I mean, we can even go -- we have started thinking about going into continuous manufacturing as we move forward. So let me take the COGs piece. One element that's still with COGs is obviously the technology, which is not necessarily related with AI. But the fact that what we use is the body as the bioreactor rather than the manufacturing process, you're not having big bioreactors. Your capital intensity, the length and the cycle time of your process associated with the technology is completely different profile, which is going to give you something -- a completely different profile of COGs. Then artificial intelligence and all the digitalization applied to manufacturing. The first thought in mind is still with quality, very similar to what we have. In counter to what some people think, high-quality means lower cost because it means high reliability. That's why we are focusing on reducing that variation, increasing our reliability there. Then artificial intelligence and digitization automation applies -- applied to yields, to cycle time, as I mentioned, continuous manufacturing, all that would take our COGs into one direction, down.

Yes. Salveen, it's is Stéphane. It was another dimension on manufacturing, which I think is an amazing power of the mRNA technology because it's a platform is that we do not have to be precise on every launch forecast byproduct across the portfolio. If you think about a wonderful launch that has happened recently is Shingrix. Shingrix is just typical vaccine, billion dollars in first year of launch, that's just can't be very open body that they cannot make enough. And they cannot make enough because they never thought they will sell $1 billion the first year of launch. This is the best vaccine launch ever. And if you think about it, the product study showed the product is that Shingrix has the biggest manufacturing facility for every product. And so you have to guess way before you have your Phase III data, your ramp, your commercial ramp. And even you don't know your efficacy, it's just a big guess. And so in our case, what we think is an amazing power of a platform, which we play into maximizing top line, which when you do a 90% gross margin business, is the most important than the 5% improvement in the cost of goods is make sure you never miss a sale. And we believe by having this platform, we should be able to remanage the portfolio so that we only have to be right at the portfolio. And we start to have 24 products like today and sooner 30 or 40 products, that makes your product easier and easier to manage, and I think this is one of the biggest value. Juan is going to work really hard with his team to take the cost of goods down. And of course, scale and volume will help, but for me, the piece that is the most remarkable is we do not need to invest additional CapEx for every program. If you look at it, we just announced recently, EBV in pediatric RSV, we don't need $1 more of CapEx to for those products. It's in the same rooms with the same people with the same equipment, so that's just fantastic savings. And then it's on the maximizing sales. If one day, we'll come to you guys we say we cannot sell enough because we didn't do the forecast right, we are not doing our jobs right.

And we do have a follow-up from Hartaj Singh with Oppenheimer.

I just had one quick follow-up on artificial -- on AI, artificial intelligence. This is I think maybe just keep it on whoever can answer it. In some ways, I guess, the holy grail of drug development is finding -- understand the pathology of the disease and finding what the underlying problem is, finding a target, right? Then understanding whether it's druggable and then coming up with a drug for that. With mRNA, you're coming up with a different modality to do that. So I guess my question is in that context, within your 5 or your 6 verticals, is there any 1 that is more amenable for AI to actually help in that process, automate that process so it doesn't become this laborious kind of going to the petri dish and then going out of the petri dish, and going into the kind of process or not really? And in how much time could that happen?

Yes. It's Stéphane, I'll take a shot at this one. I think it goes even beyond AI, I'll go back to basics, which is you think about technology. We do not invent biology. And that's for me the most profound thing, which is if you look at traditional medicine, it's a guess work. And so in that world, you can see our AI helps you avoid spending too much time in the petri dish or into a mouse or whatever. But in our case, we use evolution. If you think about the coronavirus, why could we go so fast? We don't have to do any biology work. We just double a sequence, dropped it into the same cassettes as the CMV and we flew to the clinic. If you think about the MMA product, it's the same thing. We use a sequence that all of us on the call have, which -- because we don't have MMA disease, and dropped it into the same cassette as the [ ChIC ] antibody technology and you fly to the clinic. And so the piece that I already think is powerful about our technology that I think sometimes is underappreciated is we don't have to invent biology. We just use evolution, and that's the best AI system I know.

Speakers, I'm showing no further questions in the queue at this time. I would like to turn the call back over to Stéphane Bancel for any closing remarks.

Thank you. I'll be brief. So first of all, I would like to thank our guess, Karim, for his help, thinking about the business model of the company, and his help today in the presentation. I'd like to thank Juan, Marcello and Dave for their presentation, but even more important, our impressive progress that they and their team have achieved in 2019. As you know, we are very excited by the next leg of growth of the company, doubling down on our vaccine, doubling down on IV systemic products. As you know, we have 5 new development candidates in just the last 60 days. I'm very excited and looking forward to 2020 because we're going to have a lot of clinical data with us. As you might have seen last night, we announced that the CMV Phase II was fully enrolled ahead of plan which makes us very comfortable to confirm which we'll have data in Q3 of this year, given all the subjects have been dosed now. And as I said last night, we had the HHS Secretary Azar, who communicated that the FDA has opened the IND for the corona vaccine 1273. And so we are hoping that soon, the first subject will be dosed in the U.S. More than ever, we believe that mRNA is a new class of medicine. Our Norwood site is a strategic asset, as you get a good sense for it. And what we have achieved and what we will achieve looking forward could not happen without Norwood. Digital and AI is becoming, day after day, more and more part of the DNA of the company. And you have my full commitment and the commitment of the team that we will keep pushing it every day so that the company just becomes stronger and stronger. So with that, I would like to thank you for taking the time today, and I hope to see many of you on April 14 in New York when we do our Vaccines Day. Thank you.

Ladies and gentlemen, this concludes today's conference call. Thank you for your participation. You may now disconnect.