IQVIA Holdings Inc. (IQV) Earnings Call Transcript & Summary

Good morning, everyone. Thank you for joining me and my colleagues, David, Mayank and Mohsin for this webinar on the key success factors for getting external comparators right in oncology. Welcome to this highly anticipated session. My name is Leanne Li, Senior Principal with IQVIA Real World Solutions business unit. I'm currently driving strategy and innovation initiatives to advance business globally, including external comparator. So before we get started today, a few housekeeping items. One, if you are not on Chrome or FireFox, sign out and sign back in using one of these browsers to avoid technical issues. So if you have any technical issues during the webinar, press F5 or post in the Q&A section. [Operator Instructions] We will have several short polls during the presentation, so be ready to share your responses. At the very end of the webinar, we have a short survey and we'd appreciate your feedback. Recordings and slides will be made available after the event. So without further ado, let's get started. We're thrilled to have all you here today as we delve into a topic of utmost significance. Over the next hour, we will embark on a journey of exploration, learning and innovation together. We will uncover valuable insights and exchange ideas from our collective knowledge here at IQVIA and also throughout the industry. So overall objective of today's webinar is to highlight both the strategy and tactics in designing and running external comparators. Most of the concepts and the principles will apply to all therapeutic areas, but we will make sure today to highlight some of the special caveats in oncology. So there are 3 main focus areas during today's discussion. So first, what is an external comparator? And where do we expect to see its applications? And then two, what's the recent FDA draft guidance on real-world evidence and what's its implications for external comparators? And then three, what are the best practices for fit-for-purpose real-world data and real-world evidence. So before we get into the first topic, the first poll question is how much experience do you have conducting external comparators? We have 4 options here, so we'll give the audience 1 minute to answer this first question. You have 30 seconds left. Another 10 seconds. Okay. So it seems like most of the audience have no experience and some have limited experience and then 10% of the audience have moderate or extensive experience. So the second topic, what are your specific areas of interest within external comparators? Again, we'll give the audience another 1 minute or so to answer this question. Another 20 seconds and then we can move on. 5 more seconds. Okay. So we're seeing an evenly distributed response here. So statistical approaches, regulatory implications, so these are the top 2, and then we are seeing data source identification as the -- actually application across different use cases is the highest, and payer and HTA implications is the last option that we see here. Great. It seems like the audience have some experience with external comparators. And we can dive into the first topic. So what is an external comparator? So in the context of real-world evidence, an external comparator refers to a group or population outside of the study, clinical study being conducted, that serves as a reference or comparison. Real-world evidence studies aim to gather data and then generate evidence from real-world settings such as routine clinical practice, electronic health records or patient registries to assess the effectiveness, safety or comparative outcomes of interventions. So we know what an external comparator is now. So why do we need to have an external comparator? So several scenarios. When we consider a traditional randomized clinical trial, there are usually 2 patient groups or arms. So the treatment group and internal control. Sometimes it may not be ethical or feasible to include an internal control. So 2 specific scenarios that we usually are faced with, scenario 1 when effective treatments are not available. For example, if the treatment is an orphan treatment of a severe rare disease or oncology indications targeting small sub-patient populations with novel biomarker targets. So scenario 2 is basically current treatment options are suboptimal. So there are numerous examples in oncology drug development where the unmet medical need is so pronounced that palliative care would be the only alternative to an experiment treatment. So collectively, these 2 scenarios and these factors have resulted in a growing number of early-phase single-arm oncology trials and increasing attention to a hybrid kind of a design in later-phase trials that includes randomized controls augmented with external controls. So who can leverage external comparators? We talk about what external comparator is, and then why we need to use an external comparator. So external comparators can support different needs and provide values across the full treatment development life cycle. So for example, in early phases, external comparators can benchmark early trial results so we can assess signals in safety efficacy profiles of the treatment, which can later support discussions with investors and internal teams on product positioning and development. So during clinical trial development, external comparators can provide context to a trial so what you really hear about the benchmark to a trial and then some with a contextualization. So external comparators can provide evidence on the natural history of the disease, so really we can contextualize the trial data, and we use it as a supportive evidence for regulatory submission or confirmatory evidence in case of a single pivotal trial. So during the regulatory approval process, external comparators can support submission of trial data. And then during HTA submission process, external comparators can strengthen the clinical trial data. For example, external comparators give additional power to the trial data by expanding the existing control arm through a real-world cohort or provide a more appropriate comparator treatment not captured in the trial. And then lastly, during and after commercial launch, external comparators can provide the key differentiations from existing products in the market as well as some of the new competitors. So external comparators are evolving as an innovative approach to generate evidence in challenging context. So there's a strong need for reliable and high quality of data with robust and flexible methods and designs. In the next few sections, we're going to dive into the details into the regulatory space and a statistical operational considerations. So with that being said, I'll transition the floor over to my colleague, Mayank, to cover the specific regulatory implications. So before I do that, a very high-level recap of what we learned from the first section. So external comparator studies are used to build a control arm for a clinical trial; and then second, the external comparator can add value across the whole product life cycle; and then third, alignment between suitable data, applicable methodologies and the stakeholders' expectation is critical to the success of the approach. So Mayank, I'll transition the floor to you.

Thanks, Leanne. And I think as we transition, there is another poll question for you. What are your perceived barriers with running external comparators? And there are 4 options. I think we'll probably give 30 seconds to a minute for us to respond to this, and then I'll dive into my section, which focuses on the latest FDA draft guidance on this topic. A few more seconds. All right. All right. So I think this is really spot on, right? What are your perceived barriers with running external comparators is really understanding where this fit-for-purpose data sources, which is the top response. FDA engagement, which is one of the key tenets of how you should be working with the agency as part of the external comparator design process, and then obviously, the HTA aspect and budget and expense and the whole impact that it really brings in from that perspective. So this is very much in line with what our thinking is. And what we would do next is to really get into the regulatory space, right? So my name is Mayank Raizada, Principal in Regulatory Science and Strategy innovation team. Been in this industry for well over 15 years, focused on innovative and novel data methods and tools and implementing the innovative designs for regulatory purposes. I think one of the first things that I want to highlight is that the whole -- the guidance or the whole real-world evidence initiative from FDA does not change the evidentiary standard, right? So I think what the guidance -- and this particular guidance and the ones that came before it, right, really are trying to do is to provide you what are the key considerations from data methods and tools perspective and apply it on real-world evidence and real-world data. I think what -- as a precursor to this section, what we are going to do in this section is to take you through a quick snapshot of the evolution of the real-world evidence space, specifically from FDA's perspective, leading up to the guidance. This particular guidance, which is in focus, considerations for the design and conduct of externally controlled trials for drugs and biological products. Next, we would review the key elements and provide key concepts overview, which is from the guidance. Next, we would turn our focus on the [ slow want ] for it, essentially to look into what does it mean for the sponsors, right? When you are looking at this guidance and how -- and what sort of challenges that you are facing in, and then we will delve into a slide or 2 to look at a couple of approaches that helps you derisk the external comparator approach. So that's the next few slides, which is focused on my section, and then I'll pass it on to the next presenter. So I think I really wanted to give you a brief history. I know that a lot of folks are very familiar with it, but like external comparator has been in conversations for a really long while. I think the oldest reference goes to an ICH E10 guidance, which was to -- guidance to refer to essentially utilizing the controls and the controls group and related issues in the clinical trials back from 2006. But the real landmark was the 21st Century Cures Act, which initiated and snowballed into these activities. The 21st Century essentially provided the context and urgency for using real-world evidence for regulatory decision making, approval of -- I think the 2 big use cases that was identified as part of that initiative was to applying real-world data for approval of a new indication for an already approved drugs, AKA labeling expansion, and then obviously exploring the opportunity to apply external comparator for satisfying post-approval safety requirement. This led to a fundamental framework that was released in 2018, 2019, which allowed FDA to articulate the thinking. When they're thinking of real-world data, they are looking at fit for purpose. They're also looking for answers to generate real-world evidence that is adequate and scientifically sound to answer the regulatory questions at hand. And last but not the least, it does meet all the FDA requirements as they exist now. This particular framework was the precursor to preceding -- or the guidance that came out in 2021, there were 4 of those, which focused on specific aspects of real-world data and real-world evidence. The guidance around EHR and claims provided the industry with some of the key aspects of how the -- you can utilize the EHR and claims data for regulatory purposes. There was another -- there was a guidance which was on the registries, right, which focused on using of registries and how the registries can be used as a real-world evidence for regulatory purposes. Then there were 2 procedural guidances which focused on the data standards and other one which focused on the consideration for use of real-world data in general, providing you guidance as to what sort of requirements would be applicable. Now all these guidance essentially help and kind of support the concepts that exist in the considerations for the design and conduct of external controlled trials for drugs and biological products, which is the topic of interest today. So to really ground us, right, I think what we have done is to kind of just try and provide you a context where do we really stand? And where is the focus of this particular webinar and this guidance in particular, right? So if you look at the real-world use across the regulatory spectrum, it basically is essentially on the scale of less regulated input is needed to the increased regulatory input, right? So the -- there are uses of real-world evidence, which is to contextualize in use like disease progression where you are looking at aggregate data and you're comparing the outcome rates from a single-arm trial. So this is where you would consider an example of where the regulatory rigor that is needed is not as same as some of the other examples where we are talking about natural history and potentially external control. The external comparator natural history is a bucket where we are seeing a non-matched sort of comparison where we are indirectly comparing and not looking at the patient level data per se, and these real-world studies help with burden of disease and understanding the unmet need, defining the patient journey, and essentially, in cases, helping setting up a trial for an external control. External control, which is the most rigorous and the most conservative sort of -- in terms of what sort of requirements are levied on a real-world evidence, is where you are actually doing a patient-level data matching and utilizing the BLAT data, the patient-level data in the regulatory submissions. So that's where we would be focusing on for the rest of the presentation. So let's just turn towards the guidance in question, right? This guidance came out in February of 2023. The comments period closed in, I think, the second of May. IQVIA, who has been obviously at the forefront of exploring innovative and novel uses of real-world evidence and real-world data, has been involved with this guidance intimately. We have provided comments independently, but also in collaboration with Real World Evidence Alliance. Real World Evidence Alliance is an independent trade group where IQVIA is one of the founding members and essentially provides input to the agency and other regulatory bodies from the industry's perspective. The focus of the guidance is external controls from real-world data and historical trials. And I think I'll get a chance to talk about both these topics a little bit more in the subsequent slides. And essentially, one of the key concepts is comparability of treatment and external controlled cohorts is a topic that will come again and again is where you're trying to do the matching, right? Matching of patients and exchangeability of patients are the key concepts. So what guidance really included? It included these 4 key topics, right? The design considerations, the data source considerations, analysis considerations and regulatory review considerations. Needless to say, and I think it has been pretty obvious with all the other regulatory guidance as well, that regulators are encouraging and almost like really pushing the agency to ensure that they are taking a regulatory perspective -- regulator's perspective, FDA's perspective in this case, into account as early as possible. And I think this whole guidance really tries to lay out the key considerations that FDA is looking for you as a sponsor to explore, but also bring it to them early enough. What this guidance doesn't touch upon is external controls using summary-level estimates instead of patient-level data, so we are not going to talk about external comparators or natural history studies, examples where you're using summary level or aggregate level information. What it also does not do is to discuss reliability and relevance of using real-world data sources. And there are other guidances and -- which are going to cover these topics. Moving on. So this is the money slide, from my perspective, right? I think this is the slide which tries to capture the essence of this guidance, and I think what we have done on this -- to explain that is really highlight the 3 key aspects of the guidance. So number 1 is obviously the design considerations. The second is data considerations. And third is the analysis considerations. If you look at the design considerations, one of the biggest outcome or one of the biggest aspect that came out when we were understanding that and going to the guidance that FDA is actually expecting you to have a finalized protocol before initiating an external controlled trial, which includes selection of external control arm and analytic approach. So this is an FDA's mechanism to dissuade from selecting an external control arm after completion of the single arm trial. So this is something which obviously is new, and it obviously now requires much more planning from a sponsor's perspective. The other key concepts in the design section includes prespecification. So you need to prespecify your plan, how you want to measure the different aspects of the design and the data and ensure that how you would reduce -- analyze the confounders and reduce the sources of bias. The key objective of the design consideration is the reduction in the potential for bias in externally controlled trials, and it is understood that it is best probably addressed in the design phase. What we also have in this section is design elements. You look at the design elements, the 4 key design elements in focus in this section are study population. So when you're thinking of study population, you're thinking about comparability, exchangeability with the experimental arm, the eligibility criteria, inclusion, exclusion. The second aspect is treatment. I mean, obviously, the aspects around the treatment that you need to take into account. The third being the time zero, which is designation of index date. Some of the bias components that come from that perspective, something like -- something that is known as immortal time bias, and outcome assessments. So design considerations really provide you the framework, how you can reduce potential for bias in your external control. Data considerations on the -- as an extension, highlights the pros and cons of utilizing the external control data. And I think it has -- specifically tries to create some sort of distinction between the clinical trial data and real-world data. I think when you think of a clinical trial data, it's protocolized and it's collecting in a clinical setting, so it obviously had some benefits. But at the same time, real-world data, which is -- which could be more contemporaneous, right, has its advantages as well. So the guidance tries to delve into that aspect of the real-world data. In this context, even though this is a clinical trial data from a historical control, so it would be required to go through the same sort of analysis and design consideration requirements when you're comparing it with your experimental arm. What this guidance also does really well is to lay out 10 comparability considerations. And these are highlighted on the slide: Time period, geography, diagnosis, prognosis, treatment outcomes, follow-up period, intercurrent trials, missing data and other factors. There is really a nice table in there, which tries to elaborate these different sections and how they need to be considered as part of the data consideration exercise. Last but not the least, analysis consideration. Here, you try to prespecify your statistical analysis plan, which includes evaluation of comparability and effect size. One of the key components, how you're going to address the issue with the comparability? And if there are, how would you justify the effect size? Because external control by design are considered a good mechanism when there is a huge effect size. So how and what you would do to manage that effect size? The second is strategies to address missingness and misclassification. So with that, I'll essentially try and turn the page to talk more about, so what does it mean as a sponsor? Like what does it mean for you as a sponsor? I think the first thing is that we need to plan early, right? Early planning is fundamental for externally controlled trial. When you're planning early, you need to obviously look at your study design, external control study design, justification for why a proposed study design is appropriate. You would be looking into aspects like large magnitude of effect, ability to capture key prognostic factor, the definition and how you can satisfy the unmet need. From a data source perspective, you need to identify the -- and plan for data sources which are fit for use. You would also need to make sure that you have a planned statistical analysis, a plan to balance the cohorts and assess the impact of confounding factors and minimize the other sources of bias. And last but not the least, look at the plan for addressing FDA's expectations around data, which includes access to data sources and documentation. Because FDA is expecting a patient-level data, so you need to essentially ensure that you have sort of engaged -- you have your data set up in a way that it is directly like vendor is able to provide the patient level data. All right. So I think what I'm going to do with next couple of slides is to just kind of provide you a peek into how IQVIA and the team on this call, in particular, like really explores and supports the external comparator feasibility. External comparator feasibility is an exercise which essentially leads to developing the rationale and the justification that you would want to go with a regulator to, right? So in our view, it's a 5-step process, and what we have done as part of this evolution of this process, right, is to include the key regulatory concepts at specific stages to really ensure that you derisk your external comparator approach throughout the life cycle. So the first piece in this approach is the regulatory landscape, which is a key regulatory concept. This is an exercise where we draw insights from case studies and informed regulatory options. This essentially helps as a sponsor in recommending the evidence needs and design options that has worked in the past, and there are precedents for that, right, and how to really rely on those precedents which may be in a same or a similar indication, therapeutic area, or maybe an adjacent indication or a therapeutic area, which provides you a rationale why this looks like a feasible approach. A lot of effort and insight needs to go into data feasibility and study design. These 2 are -- run a lot in parallel in a lot of cases, and I think these 2 specific activities are the bedrock in terms of identifying the data sources that you would need to use for an external control trial and the study design. My colleagues would be discussing these 2 a little bit later in the presentation in a bit more details. So what regulatory team can support you is to really tie these 1, 2 and 3 into a fit-for-purpose assessment. I'll talk about fit-for-purpose assessment as a tool in the next slide, but it lets you assess the appropriateness of study design and data for an external control. And overall, it provides you a solid foundation, justification and the rationale that you would need to talk to a regulator and when you essentially go to a regulator, and I think this is where we are going to be working with the FDA through a Type C mechanism or whatever other meeting mechanisms that are available to the sponsors. This slide just highlights the tool that I was talking about. Again, I think the -- this is not lost on us that the regulatory decision support tool is not narrowly focused just on this particular guidance, but it is basically something that we have built over time, which utilizes the key tenets of the guidance from 2021. It is global in nature. We also include and keep it up to date. What it really is, and it is an input-output mechanism where your inputs are your data sources, which you want to utilize for an external control and steady design aspects that you want to utilize for a external control. And then on the right-hand side, it would provide you potential strengths and areas and opportunities from a data source as well as design perspective, which then informs your regulator interaction strategy. So we have obviously conducted a bunch of these engagements. Like IQVIA has been doing external control much before these guidances came out, right? So what we have highlighted here are some of the key areas or categories where we have seen regulators providing more significant feedback. So if you look at the study design, justification for using an real-world data derived from external control instead of an RCT. I think you are trying to -- as a sponsor, you're trying to justify use of real-world data, right? From a data source perspective, agency is looking at sponsors from identifying mitigation of potential heterogeneity across real-world data sources. From a variable capture and follow-up, it helps you address missing, duplicate or incomplete data sources. I think agency is expecting you to looking at some of the aspects in your planning and design and in your statistical planning, right, how you address some of these missingness and duplicate issues. From an endpoint perspective, agencies looking at sponsors in assessing common oncology endpoints in real-world data and making sure that they can be used and how they can be used. And last but not the least, comparability analysis essentially how you can reduce the bias and lack of comparability between groups, how to reduce that -- those aspects. On this slide, we have also laid out some of the key aspects in terms of the mitigation strategies, which you can -- these slides would be made available and you can obviously look into that at your leisure. Before I turn over to my colleagues David and Mohsin. I just wanted to quickly provide you some of the key aspects and highlight, right, and reiterate the importance of engaging early with the regulator. So when you're working with a regulator, you need to be prepared, right? So that's where feasibility, the regulatory landscaping and fit for purpose and some of the activities that have been identified as part of a feasibility exercise provide you an excellent groundwork to let you do the due diligence and understand regulators' positions and precedents, engage early, discuss with the agency early on, ensure alignment, conduct a transparent engagement, to avoid any misunderstanding, make sure what you're trying to do, how you're trying to do and why you're trying to do. The last but not the least, you've got to be flexible. There are going to be some sort of feedback which are not always going to be aligned with what your regulatory strategy is, so you need to be clear on what the deviations are and stay open to regulators' feedback and look at it as an opportunity to discuss with them and be strategic in your concessions. With that, I really want to turn it over to David and Mohsin for the next phase of the presentation.

Thank you so much, Mayank. Hello, everybody. My name is David Alsadius. I'm a Senior Medical Director and a Medical Strategy Lead with the Oncology Center of Excellence here at IQVIA, and together with my colleague, Mohsin Shah, we'll be going through operational success factors and study design within oncology and I will start off with highlighting it from a clinical perspective, and then Mohsin will take you deeper into the science. So just very high level and briefly the clinical perspective on external comparators in oncology. As you know, we know now that cancer is not just 1 disease. And in fact, not -- the cancers within an organ system is not 1 disease. There are many tumor types within a cancer such as breast cancer or prostate cancer, and within that cancer group, there are many, many subtypes differentiated by expression of biomarkers, different gene expressions, histopathological features, clinical features and so on. And they come with various molecular characteristic. And we also see, to add to the complexity, geographic variations, not only in the incidence of these diseases, but also in the treatment and also in mortality and so on. And also, which has been very much highlighted, now the diversity issues where we see both higher mortality, various variations in availability and provision of treatment in diversity in subpopulations and ethnic minorities and other minorities. And this also pertains to the inclusion in representation in clinical trials. So some of the key strengths with real-world external comparators here from a clinical perspective is that they can actually provide a description of an extended clinical population and expand to be more inclusive and reflect the actual clinical situation, if needed. We can also address treatment effects in rare tumor types and subpopulations that are not necessarily and sufficiently prevalent to be represented in a statistically powerful way in a randomized controlled trial. Also with the geographical variations in standard of care, we could address differences and adapt to specific regulatory or HTA requirements that are particular to a certain region or to a certain country, and we know that there are these variations. And we can also focus on specific and underrepresented populations in minority groups. However, this also posted some challenges which I wanted to highlight. So tumor classification is one thing. And there are several issues here, one being that the testing and validation of specific markers to design or designated tumor classification may vary. And in some cases, there are validated assays that are being used some places, and other places not. So this is something that needs to be taken into consideration. And also for the driver of therapeutic decision and safety management, which plays into that where the treatment landscape in oncology is rapidly changing. And actually, the drivers of the decisions can be very different in an external comparator that is a study that has started later than the randomized clinical trial. So this has also -- needs also to be taken into consideration and for safety management. We are continuously learning to address safety issues of novel agents and other agents. And another thing that I wanted to highlight is the archival versus newly collected tissues. So we're assessing biomarkers and classifying tumors accordingly, we need to take into consideration whether we can do that on archival or retrospective information or if we need to actually have prospective collection of tissue, especially in tumors where we know that these -- the expression can change, such as gastric cancer, where we know the HER2 expression actually changes throughout the course of the disease and then it can be very different in the metastatic setting versus the upfront setting. And the same for genetic and biomarker testing, where validated assays are sometimes available, and sometimes you could have local setups which are not validated, but -- and this needs to be taken into consideration when you're assessing the data you're using in the database. And finally, the point that I wanted to mention is the efficacy and safety endpoints, which, of course, are very dependent on the quality of the database but also needs to be considered to -- if we need to mimic the time to progression endpoints such as PFS survival or if there are other endpoints that could serve better in that specific setting, depending on which data you have and what you want to show. So with that, I'll hand it over to Mohsin.

Thank you, David. Good morning, everyone. I'm Mohsin, a Consultant Epidemiologist in the Epidemiology and Drug Safety Team here at IQVIA. Today, I will be discussing operational success factors of an external comparator from a scientific lens. At IQVIA, we've developed a modular approach towards building an external comparator. This strategy allows us to address key risks early on and allow for an iterative study design. And I would so break this down -- this strategy into 4 distinct stages. Stage 1 is what we call defining needs. This stage helps us gauge stakeholders' evidentiary needs and expectations and helps us determine acceptability of using an external comparator for approval and reimbursement purposes. Stage 2 is where we conduct a feasibility assessment to identify fit-for-purpose study design and data sources. Stage 3 or as we call it test and engage is where we confirm our approach with stakeholders and adapt as necessary. Finally, Stage 4 is where we -- where fit-for-purpose evidence is generated and delivered. On Slide 2, I want to discuss a key concept in external comparator design that is of internal validity. In other words, for a comparison to be considered apples-to-apples, the 2 populations, the trial and external, should be exchangeable with one another. This concept of exchangeability relies on several factors, including eligibility criteria, patient characteristics or confounders, mode of treatment, outcome measure, the time period and the setting. On the right side of the slide, I have summarized the process for the good conduct of external comparator studies in 4 simple steps. Step 1, assessing fit-for-purpose feasibility of the data source and study design. Step 2 would include adjusting for baseline characteristics and potential confounders, whereas step 3 is where we conduct the analysis using suitable statistical methods, and finally, step 4 would include assessing the threat of validity and bias via missing data, sensitivity or quantitative bias analysis. In summary, one would want to maximize the quality of evidence through careful study design and combination of covariate balancing, missing data and bias analysis. And here in this slide, I will talk a little bit about the target trial emulation framework, which is a 2-step structured process that allows -- offers early alignment between the trial and the external comparator. The first step in this process is articulating the [ covariate ] question in the form of a protocol of a hypothetical randomized trial that would provide the answer. This protocol must specify certain key elements that define the treatment effect, reflecting the clinical question posed by the trial objectives and the statistical analysis plan. The second step of this 2-step structure process is explicitly emulating the components of that protocol using the observational data, that is, finding eligible individuals, assigning them to a treatment strategy comparable with their data, following them up from assignment, which is time zero until outcome or end of follow-up and conducting the same analysis as the corresponding target trial, except that there is adjustment for baseline confounders to emulate random treatment assignment and accounting for missing data. In this slide, I have laid out several statistical approaches that we think are suitable for external comparator studies. The 3 most utilized approaches are ranked in terms of increasing strength of approach, level of data granularity and completeness required, and include, one, real-world benchmark; two, an indirect comparison, which my colleagues alluded to initially; three, direct comparison. And so talking about real-world benchmark, this approach describes the patient demographics, clinical characteristics, treatment patterns and outcomes for the treatment group and the benchmark patients. Typically, this is used when patient level data is not available. In an indirect comparison, this approach assigns larger weights to the outcomes of patients for the trial whose baseline characteristics are closer to the average baseline characteristics of the comparator population. Typically, we use this when patient-level data is available for the trial population but not for the comparator patients. And then finally, direct comparison is where -- in this approach, this provides the most valid comparison and allows us for more direct input into the modeling to support health technology assessment submissions. So typically, we use this when patient-level data is available for both the trial patients and the comparator patients, and adjustment methods can include and range from propensity score matching, propensity score weighting, or stratification, G-computation, which is a non-propensity method and other causal [ inference ] methods. So in this slide, I want to talk a little bit about the common challenges that are normally encountered in real-world evidence oncology. These can occur primarily at the study design phase or stage, at the data capture stage, or at analysis phase and can be addressed through careful study design and a strategic analytic approach. So challenges that one encounters in study design includes assignment of treatment based on physician assessment rather than randomization. This process hinders application of straightforward statistical methods as applied in a randomized clinical trial. Similarly, during data capture, misclassified or unmeasured confounding information may lead to residual confounding. And then finally, during the analysis phase, questions in specific settings is whether precise measurement is required for valid inference or if a proxy measurement or valid imputation is adequate. A potential to use invalid statistical or missing data assumptions may lead methodological issues and bias. And so in the next slide, I want to talk a little bit about some of the potential biases and how we can mitigate that. So bias, or the conscious or unconscious influencing of a study and its finding, can arise at every stage, therefore careful study design can help mitigate potential sources of bias. Biases arising at the study design can consist of confounding by indication, where real-world and trial patients are treated with different intent that may be palliative versus curative, just to give an example. So therefore, input from key opinion leaders and clinicians to understand the local context and check whether sufficient baseline covariates are available to systematically choosing one to correct for this potential confounding. The second bias I'm going to talk about is index bias or selection bias where the real-world patients have greater than one potential index stage and that issues with systematically choosing one. So in this approach, we recommend taking all eligible index dates. It is possible and this enhances statistical power. Alternative handling includes selecting randomly just 1 index date for line of therapy, selecting the first 1 and other methods. And then in immortal time bias occurs, if event cannot occur in all the time between the index stage and the outcome. In this case, evaluating whether a sophisticated study design is possible would help avoid immortal time bias. One could also consider using appropriate statistical analysis, ensuring the outcome can occur at any point during follow-up without interfering with eligibility. And now moving on to several biases that can actually happen at the data capture stage. These include passive versus active reporting bias, that is underrepresentation of adverse events in real-world data and differential reporting of serious adverse events and adverse events. Unmeasured inclusion or exclusion criteria and/or confounding in addition to outcome misclassification. Majority of these can be mitigated using a probabilistic bias analysis, which is informed with bias factors coming from the prevalence and strength of a confounder, sensitivity of misclassification proportions, which can be found in the literature, or from expert knowledge. In addition, one can use external validation study of outcomes for a subset of the patients. Continuing with biases, there can be informative censoring of outcomes, which is differential censoring from the competing event such as progression-free survival. Sensitivity analysis can be used as well as weighting by the inverse probability of censoring can correct this imbalance of loss to follow-up if post-baseline covariates are available. Misestimation of the index dates or line of therapy is hard to accurately determine in real-world data, therefore, we recommend data selection algorithm validation or again, a probabilistic bias analysis would be helpful. And finally, bias can arise at the analysis phase as well, and this can include misspecification of the population as well as confounding bias. These we can mitigate using sensitivity analysis and performing various causal methods such as propensity score or G-computation or other methods. In this slide, I'm going to talk about selecting prognostic variables that go into those models or the [indiscernible] models, so we recommend carefully selecting those prognostic factors for these comparator studies. This may include stakeholders taking on a systematic literature review or we call it SLR to identify potential prognostic factors a priori. And also the external key opinion leader input, who can review those SLR findings, they can provide an unbiased evaluation of the evidence. And we normally recommend at least utilizing 3 key opinion leaders in this approach. And so this is a really important slide, which I wanted to talk about. Line of therapy that I alluded initially -- about the line of therapy selection. This can be challenging considering the treatment of oncology patients typically involves the use of multiple modalities over the course of the patient's journey. There are several approaches for handling LoT selection and can be considered but additional statistical considerations are required. Sponsors can choose to select either one qualified line of therapy. This could be the first eligible or randomly select a LoT or decide to select multiple qualifying LoTs of each patient. In the event of a limited sample size, selecting multiple timepoints is a good approach for increasing statistical power. To avoid possible assessment bias and to increase scientific rigor, the sponsors may also consider external validation of study outcomes. This can include carrying out an independent central review or ICR where 2 independent reviewers, blinded to treatment and treatment physician evaluation with adjudication by a third, review radiology scan, for example. And concordance between the ICR and the treatment physician can then be assessed between the ICR and evaluation. Details including the review criteria should be included in a predefined ICR charter, and due to the significant budget and timeline implications, we normally recommend that sponsors may consider including ICR on a subsample, but that would vary based on sponsor to sponsor and their requirements. Finally, to summarize the key takeaways of my talk today, exchangeability between the trial and the real-world external comparator population must be maximized. Number two, parameter definitions used for the trial will likely require adaptation for the real world in a more pragmatic setting. Number three, there are key points throughout the study where bias can be minimized through thoughtful planning. And with that, I thank you.

That takes us to the end of the presentation. Thank you for all the presenters today. And also, we have about 5 minutes left in the session, so everyone, if you can leave your questions in the Q&A section, we'll get back to you after the presentation.