IQVIA Holdings Inc. (IQV) Earnings Call Transcript & Summary

Hello, everyone. I'm [ Jeannie Linge Northrop, ] Managing Editor of Special Projects for pharmaceutical executive, and I'd like to welcome you all to today's live broadcast: Connecting Protocol Design Complexity with Trial Performance Outcomes, sponsored today by IQVIA. IQVIA is a leading global provider of advanced analytics, technology solutions and clinical research services to the life sciences industry. IQVIA creates intelligent connections to deliver powerful insights with speed and agility, enabling customers to accelerate the clinical development and commercialization of innovative medical treatments that improve health care outcomes for our patients. With approximately 82,000 employees, IQVIA conducts operations in more than 100 countries. You can learn more about them at www.iqvia.com. In today's webcast, data science experts are joining us and will present design data gathered from a large sample of -- excuse me, protocols conducted over a 10-year period with associated trial performance metrics. They will also highlight results of the assessed statistical correlation of 28 protocol design variables, with 18 trial performance outcomes used to identify key outcomes correlated to complexity. Finally, they will share insight on where critically focused during protocol development to mitigate impact of complexity on trial time, cost and quality. Before we begin now at today's broadcast, I just have a few important housekeeping notes to review with all of you. First and foremost, our experts are eager to answer any questions you may have on their presentation. You can submit any questions you may have in the Q&A box, which we'll review during our Q&A session. The Q&A box is found at the bottom of your video player. You can enlarge the slide window by clicking on the small icon that's located in the bottom right corner of your media player. But please note that all slides will advance automatically throughout the event. If you do happen to experience any technical problems viewing or hearing this presentation, please click on the question mark help widget that's located in the top right of your presentation window. Now I'd like to introduce and welcome today's IQVIA speakers. We are pleased to be joined by Denise Messer and Steven Zhang. Denise is a Design Analytics Director at IQVIA. She has over 25 years of experience in research and clinical trials, including expertise in clinical trial planning and design. Denise has spoken at conferences. She has been published in industry journals covering different topics, including scoring trial, patient burden and assessing protocol complexity. Denise has helped develop the IQVIA Data Informed Protocol Assessment using data to highlight areas for protocol optimization before operation -- excuse me, long story this morning, but operations, including creation of patient burden algorithm and protocol scoring benchmarks. Steven joins us as the Global Analytics Manager at IQVIA, has more than 7 years of experience in health care tech industry, has strong expertise in data science and machine learning. As a data scientist for this IQVIA project, he was responsible for the data processing and exploration regarding both the enrollment strategy and operational metrics. Combined with insight on clinical trial protocols, Steven analyzed what factors from protocols contribute to complexity and their impact on study benchmarks, such as cycle times and other trial performance outcomes. So we're really eager to get into today's discussion. Welcome both of you, and please feel free to get us started.

Thank you for the introduction. So a little bit of background and to start, why would we worry about protocol complexity? I imagine that you're all here today because you do worry about it. You've probably experienced what we all have, what we believe to be the effects of protocol complexity on our ability to successfully complete our trials. Complex trials affect everyone involved. They affect the patient's willingness to participate, sites willingness to participate, and they affect the sponsors and CROs as well. It's frustrating for sites as they are presented with complex protocols, and they know that increasing number of eligibility criteria, visits, procedures training requirements, systems, management, all of that to recruit and retain those patients to pull that trial to the end, and they still need to maintain quality and minimize those operational errors. So those kinds of struggles to recruit sites to run the trial, to get patients to be in the trial and stay in the trial, and to overcome protocol deviations and the amendments that address those kinds of problems, they increase the time and cost it takes to complete the trial. So complexity effects at all. I'm sure most of you are familiar with Tufts CSTD. It's the Center for the Study of Drug Development. If you're not familiar, Tufts is an independent nonprofit academic research center, and they publish on all kinds of clinical trial concerns. They've been providing metrics on protocol complexity every few years since at least 2008, and they report on how complexity just keeps on increasing. So in 2008, they talked about the number and frequency of unique procedures. They found that it increased annually by 6% to 9%. And site work burden also increased by 10%. And at the same time, cycle times, procedures, protocol amendments, recruitment, retention, all of those things worsened. In 2011, they told us how the number of total procedures in Phase IV trials and site work burden again for Phase I trials grew the fastest. But that complexity and burden were also growing rapidly for Phase III trials. In 2014, they told us that Phase III trial showed a 10-year increase by that period in number of endpoints, number of eligibility criteria, number of procedures. And again, those outcomes, more protocol amendments, increased cycle times, higher administrative burden on sites, more countries and sites needed to run the trials, all of that costs more money as well. In 2018, they reported that nearly all the Phase I, II and III trials that they had looked at increased in complexity, including number of unique, number of total procedures, number of visits, more site work effort and again, that higher cost. So their recent work was published in 2022, and they found that Phase III trial showed a 37% increase in the number of endpoints and a 39% increase in the number of countries. And both Phase II and Phase III trials showed an increase in number of sites and cycle time. So clearly, we have yet, as an industry, to get a grip on the problem of protocol complexity. What are the sponsor companies we work with, we're interested in understanding the complexity of their protocols and whether that complexity was negatively impacting their trial start-up time. So that was a particular interest to them. So we worked with them to identify 105 protocols that IQVIA had executed with them over the previous 10-year period. So it had a final protocol date between 2010 and 2020. And our goal, again, was to determine among the design variables -- protocol design variables, which ones influenced trial performance outcomes and which outcome did they influenced. So we agreed on 42 protocol design variables of interest, quite a few. And these would all be variables and that reflect design choices that would be found within the protocols themselves. So then I took those 105 protocols and collected the data manually from each protocol into an Excel spreadsheet. It was quite a lot of work. Once I had that data gathered, before we ran the correlations with the trial outcomes, Steven ran correlations among those 42 design variables. And we ended up dropping 15 of them. They were strongly contributing to another more interesting variable in that set. One example is that we were capturing duration that subjects participated in the trial as well as all the parts of that. So the length of the screening period, the length of the treatment period, follow-up period, et cetera. But what we found was that the duration of treatment period was actually 0.93 correlated with the duration in trial. So it was really -- the duration of trial was really run by the duration of the treatment period. So when we did the correlations, we just did it against the full duration period rather than that in each of the parts. Another example, we were noting down the number of visits in the clinic, the number of visits that could be done by telephone, as well as the number of visits that were or could be remote in person. But once again, we found in this set of trials, at least, mostly it was clinic visits. It was 0.87, number of in-clinic visits correlated with total number of visits. So we kept total number of visits and in this case, we actually created a new variable that was the percentage of visits in the clinic. So after those adjustments, we ended up with 28 design variables that we included in our correlation analysis. And Steven then collected the trial performance variables for these protocols. Steven?

Thanks, Denise. So in addition to the cleaning of the 28 protocol design variables, we've also extracted the trial performance variables for this analysis. Specifically, we looked at 18 outcome variables of interest, which resulted in 17 out of the 105 total trials being excluded, either due to a lack of data or the study that we're investigating being discontinued. So we ended with a final data set of 88 trials with metrics that we use for the correlation analysis in the end.

Great. So I wanted to show you those variables. What you're seeing here are the 28 design variables from each protocol and the 18 trial performance outcomes that Steven pulled and cleaned. You can see that we cast in that fairly wide. We wanted to interrogate as many variables as we could that you'd expect would come up when you're talking about the design or complexity of a protocol itself. And we also wanted to be sure to include trial outcome variables that we're all interested in improving or making sure that we hit our targets for. So among those protocol design variables, not only total number of endpoints, but we also did end points of different types. So how many primary, secondary and exploratory endpoints, to see if that affected trial outcomes? Total number of eligibility criteria, total number of visits. But not just total number of visits, we also did, like I said, percent of visits in clinic, but then also a calculation of the frequency of visits per month. And we were able to do that because we had the months from duration and trial. So as I mentioned as well, we did a per patient days, expected to be in the trial. Number of procedures. We did unique procedures, so that would be each procedure as it's done once. So if you have MRIs in the trial, that's one. If you have a PRO or 5 PROs, you count each of those. But we also did total procedures, which takes into account how many times you perform each of those unique procedures. So really what visits, how many visits are performed at and you would do that multiplication? And then a calculation, the frequency of the procedures divided, a number of procedures divided by visits so frequency of procedures. As you can imagine, there are plenty of aspects of the study drug itself that can increase the complexity of the trial. So we looked at the number of study drugs or cohorts in the trial. We looked at the number of drugs that would be studied in the trial. The percent of subjects that received experimental treatment as well as how that study drug is administered, whether that's oral, subcutaneous or intravitreal, et cetera. Burden specifically, a lot of these things clearly cause a burden. There's -- when you talk about complexity, there's a lot of things that cause both burden and complexity, although there's things that cause each that don't cause the other. But here at IQVIA, we have a patient burden score algorithm. We score each protocol for the amount of burden that we expect that patients will see in that trial. And that's a result of what patients have told us -- or sorry, a survey that we did with participants on what would affect their willingness, what design parameters of a trial would affect their willingness to participate. So I actually conducted our patient burden scoring for each of those 105 protocols, and we put that into the analysis as well. We looked at whether there was an inpatient component that was due just to the trial, and we looked at whether there'd be a study partner that would be participating along with that participants. So all of these things, as you can imagine, increase complexity. And we were interested to know as well a few other variables. Was there a genetic mutation required to be eligible for the trial, is there a run-in period that has to be done before the treatment period, the age of the participants, the phase of the trial, what kind of blinding was required, whether the disease the trial was studying is a rare disease, whether that trial was long-term extension only, meaning that patients were just rolling over from another trial, and then the number of pages in the protocol as well. And then trial outcome measures. We looked at all the cycle times. So what we found was that, and we've known this for a long time, that IQVIA tends to define start-up and enrollment differently than some other databases and groups might do. So we wanted to take the IQVIA definition of startup, which for us is from final protocol to the first site initiated. We call that our start-up period. And then enrollment starts as soon as there's a site up. And then looking for a patient, then you're starting to -- that's the enrollment period. So for us, the enrollment period is the first site initiated to the first patient in. But there's others who define that differently. Tufts is one, where startup is actually up to the first patient in. So until you get your first patient in, you're still in start-up under that definition, and then your enrollment would be from your first patient into your last patient in. So we looked at both of those to see if either or both, actually, were correlated with design. And then the closeout period. So once you get your last patient in, you have a particular duration that you follow them that's specified by the protocol. But then once you've got that last patient's last visit, it's up to you how quickly you can close that trial. So we did that period to the database lock. We looked at deviations, not just total but again, types of deviations. And for global amendments, we looked at 2 different aspects. One was a binary of whether the protocol was amended at all, so yes or no. And then we also looked at the number of times it was amended, so a continuous variable. We looked at screen failure rates. So the percentage of patients who don't pass the eligibility criteria. But then also the screen rate, how fast are you actually screening patients, randomization rate, percentage of sites that end up not enrolling a patient, the dropout rate. And then also we looked at a number of subjects with at least one serious adverse event. I'm going to turn it over to Steven for results.

Thanks, Denise. So with these 28 protocol design variables and 18 trial performance outcomes, we test it for statistical correlations to identify key protocol variables for trial recruitment and execution that are related to complexity. So these are the exact same variables that Denise has just covered, so I won't be going through them again, but they are color-coded now according to the results after our statistical analysis. So we grouped the variables into 3 categories based on their correlation results: green for key correlated variables, orange for correlations that require further validation and gray if there were no correlation associated with that variable. So as you can see, in gray, we had very few variables that were just uncorrelated entirely, such as study partners needed or run-in period required. But we did have a few variables in orange, which did show correlations. But we see those correlations as questionable and that they would need further validation before any conclusions can be drawn. Some of these orange variables were lacking in statistical power either due to a low number of eligible protocols that actually contributed to their correlations. So for example, if we were breaking down the 88 eligible trials by study phase, so into Phase I/II/III/IV, that also had complete data through the follow-up stage of our patients. We did not have enough sample size within each Phase category to determine correlations with the outcome variables on the right, such as drop-out rate or number of amendments. So the variables that we do want to focus on are the ones in green. These 11 protocol design variables that you see on the left, together, correlated with almost all the chosen trial performance outcome variables that we selected. And most of these design wearables were even correlated with multiple outcomes, as you see in the next slide. So what you're seeing here is for those 11 key protocol design variables in green, which trial performance outcomes in the columns were correlated with each of these design variables in the rows. The darker shade of green here, the stronger the correlation coefficient strength. And in the text, the higher and lower or more likely and less likely, that indicates the direction of that specific correlation. So for example, in the first row here we see a number of total endpoints, and it's correlated with higher major, minor and total protocol deviations as correlated with lower screen rate, lower randomization rate and a higher dropout rate. And as you can see, going across each of these rows, most of these key protocol design variables are, in fact, correlated with multiple trial performance measures. So the number of endpoints and type of blinding are correlated with 6 performance outcomes, and the number of unique procedures and study arms with 5. Similarly if you see at the other way, going down each column, we see trial performance outcomes that are correlated with multiple design variables, such as the number of amendments, the screen failure rate and the number of protocol deviations. And for those of you who are more mathematically driven and would like to see the data, we also have the same slide with the correlation coefficients that we found for each significant correlation. So additionally, what's included in this slide are all the significance p values where the lowercase [ T ] actually denotes a p-value of less than 0.01. Otherwise, all other significant correlations that we found were with a p-value of less than 0.05. And again, we see that these 11 design variables are correlated with many trial performance outcomes, and many trial performance outcomes are also impacted by a multiple of design variables. So I'll pass it back to Denise to discuss what can we take away from this analysis in terms of protocol design.

Right, right. So for this particular company, concentrating on the variables that have the most correlations or impact. And among those, the ones that they have the most control over as they design the protocol is going to be the best way for them to improve their trial outcomes. So type of study blinding and number of study arms, those are probably more driven by the study drug itself or your scientific questions. But there's more discretion when we think about the endpoints of the trial and the required procedures. So being deliberate and intentional on those looks like a good way for this sponsor to improve screening and enrollment potential and to lower deviations. On the other hand, as Steven mentioned, amendments, screen failure rate and protocol deviations, these outcomes were correlated with so many things in the protocol design that if you wanted to improve those outcomes, it seems clear that the focus for the sponsor would be on managing endpoints, eligibility criteria, visits, procedures and patient burden. And remember, I did say that they were particularly interested in start-up. So while start-up and enrollment cycle timelines were both correlated with study arms, that start-up timeline was also correlated with a number of eligibility criteria and with the patient burden score while the enrollment cycle timing was correlated with the duration that the subjects will spend in the trial, and the number of procedures. So again, we see that highlighting the number of eligibility criteria, patient burden and number of procedures is a way to manage cycle trials times just keeping that trial on track. For this sponsor, we also took these results and we compared them to the correlations that Tufts has found. So in addition to publishing on trends over time, Tufts also published in 2022, a different paper on correlations between protocol design and trial outcomes. Their investigation, they reviewed 187 protocols, and they involved 20 sponsored companies and CROs in the trial. So they report that each gave about somewhere between 5 to 20 protocols to that total of 187, and it was for a 6-year period from 2013 to 2018 with their final protocol dates. So we can show you how this compares to our results for the 88 protocols from our sponsor over the 10-year period of 2010 to 2020. So first before I do that, just a few methodological differences between our work and what Tufts has published. While we both included a number of total endpoints, number of eligibility criteria number of unique procedures and then those calculations of procedures per visit and visits per month, and we both included outcome variables of start-up, enrollment and closeout cycle times, screen failure rate, randomization rate and dropout rate and total number of protocol deviations and amendments, we included, as I mentioned, the 2 measures for amendments, binary as well as continuous. And if I didn't mention, I can't remember if I said, we treat duration of the patient in the trial as a design variable as dictated by the protocol, how long a patient will be in the trial. Tufts does treat it as a performance outcome in this work that they published. We focused on design variables that were available from the protocol itself. So we did not focus on strategy variables, such as number of countries and a number of sites. And in addition to the 5 protocol design variables we had in common, we included, as you saw, 22 other protocol design variables that Tufts did not include, or at least they didn't publish on them. And Tufts included 2 protocol design variables that we did not include, number of data points per patient visit and number of interim analyses. In addition to the 8 trial performance outcomes that we both included, as you saw, again, we included 6 other outcomes in our analyses. And Tufts included 2 that we did not include, the percentage of patients with protocol deviations and a sum of all of the cycle times in addition to analyzing each one individually. So similarly to what Steven was showing, this table shows the 5 protocol design variables that we had in common as rows on the left, and the 8 trial performance outcomes that we included in common as columns. So in green, what you'll see here are correlations that we both found: number of endpoints with protocol deviations, number of eligibility criteria with screen failure rate and number of unique procedures with enrollment, cycle time, screen failure rate, protocol deviations and protocol amendments. Again, all of this certainly does make sense. It fits our intuition of what we would expect. In our work, we found 5 additional correlations that Tufts did not, those are in blue text. So we found number of total end points correlated with the randomization rate and with the dropout rate. This might be a proxy or a function of number of procedures. In some cases, the more endpoints you have, the more procedures are needed. Of course, that's not always the case. We found the number of eligibility criteria was correlated with the startup time by IQVIA's definition of that cycle, not the Tufts' definition. And it was also eligibility criteria were correlated with the amendments. And that might make sense to counteract the higher expected screen failure rate, which we both found. And then we found correlation that this is per month with run rate as that might be a mathematical anomaly. It's interesting where visits per month was correlated in a way we expected. But then -- I'm sorry, visits were correlated in a way we expected. But then when you divide it by duration, as the calculation gets smaller, then the burden actually gets higher. It was really interesting. So we're not so sure about that correlation. That's one of those orange needs further validation. So you can see also on this table that Tufts found 5 additional correlations that we didn't. For most of those that are sampled though had what we think are possibly related correlations instead of the ones shown here. So where Tufts found that the number of total endpoints was correlated with screen failure rate, we did find this correlated with screen rate, which is a variable that Tufts didn't include. So very similar there. Number of eligibility criteria with dropout rate. That one I found kind of interesting, unlikely to be direct causation since dropout happens later than screening and eligibility. It would be interesting to look into that one a little bit further. Tufts found the number of unique procedures correlated with randomization rate. We did agree, as I mentioned, on 4 other correlations with unique procedures, including the screen failure rate, which, of course, would be related to the randomization rate. Procedures per visit calculation with protocol deviations, although we both agree, as I mentioned, the unique procedures correlated with protocol deviations. And again, that visit per month calculation. For Tufts it was with -- correlated with enrollment cycle. For us, correlated to randomization rate. Again, those 2 things not unrelated. So that said, we want to keep in mind that our work is the result from one sponsor. It covers a wider time period. The protocols we reviewed for the sponsor also were mainly in one therapeutic area. So the differences in correlation here could be due to some or all of that. So that's the data. So none of this is surprising, of course. We all know it. But what does it take to move us from what we all know, to evidence that actually translates into actions that we take for each and every protocol that we design. We can do individual projects like this. So when Tufts does it, like I said they collect about 5 to 20 protocols per sponsor. Each sponsor actually gets personalized analysis of the data from that small set of protocols that they've contributed. And then Tufts aggregates all the data from the 20 sponsors and they publish it for all of us to see and to use. It's a snapshot, certainly. Ideally, we'd like to understand the industry as a whole, more widely and how that differs for each, say, therapeutic area for individual diseases, et cetera. So to do this type of investigation, the first steps would be identify the variables of interest, which elements of trial design and trial outcomes are you going to collect and compare. But the second step is to determine a data set from which or a set of trials for which to gather that data. If you're going to do it manually, it's likely to be a small sample, like we've done here, subject to bias. It would be great if we could use large publicly available or commercial databases as a resource. But we're convinced actually there are no data sources like that, that have the integrity, the data integrity to support this kind of work. And when I say data integrity, I mean, a source where the data is available. So that data set covers a large percentage of the relevant trials of interest. And accurate, that the data in it is complete and correct as well as relevant, so from that large data set, you could actually choose our filter to protocols that are most like yours to compare to. So the same phase, indication, patient population, et cetera. So we've looked for such a source where that kind of data would be available, accurate and trials would be relevant. But as I presented last summer at DIA, we did an investigation and found low data integrity in what most of us would actually consider to be the usual or natural sources for this type of data -- clinical trial data. So first, I reviewed 20 protocols that I had from the work I was doing. And I compared what was in those protocols to what was in clinicaltrials.gov, as well as Sightline and Cortellis. While sponsors are required to post their primary endpoints and their eligibility criteria in clinical trial databases, which would seem to make this data available and accurate, there is no assurance that full endpoints or full eligibility criteria are going to be present in these types of data sources. And in fact, the investigation that I did showed quite clearly that it's -- they're not -- while 70% of the trials I reviewed were accurate as far as the number of primary endpoints, that was likely because there's usually 1 to 2 primary endpoints in a protocol. The number secondary endpoints was way underreported and the number of eligibility criteria as well. It certainly seems like sponsors are providing perhaps what they consider to be key secondary endpoints or key eligibility criteria, nowhere near the number of endpoints and eligibility criteria that are really in the protocol. For scheduled assessments data, which is not reported in clinical trial databases, we need another source. So I investigated Grant Plan, which is an investigator database, it's for budgeting. So you would expect it to have variables, such as a number of visits and the number of procedures, and certainly, they are in there. So I took 43 indication phased pairs from a 3-month period of requests that came through my design analytics group for study design support. I found very low availability. So if I were to try to use that to compare the protocols that I was actually seeing to the, say, industry standard for a number of visits or number of procedures found in protocols like those, it wouldn't be possible. That database, at least as available to us, only provided low relevancy as well. You get top level phase and you get indication, but even if the data that you needed was there, you could search Phase II breast cancer, but you couldn't specify the patient population with any more detail than that. So it wouldn't be possible to see if, say, first-line trials or more -- or less complex than third-line trials or trials that require certain biomarkers or et cetera. So really, the best data integrity is going to come from gathering the data directly from protocols themselves. So the sponsor we conducted this project for will be using this information to press their protocol development teams as well as their senior leadership. You have to have that buy end to concentrate on the protocol design elements that are now proven to affect their study outcomes. Really they are bottom-lined. But they also supported us in sharing these results today to contribute to the conversation. Continuing to discover and share with one another protocol design factors that contribute to trial complexity and are the most relevant to achieving better trial performance outcomes, that's how we pave the way to standardizing the assessment and even maybe scoring of a trial's complexity from accurate protocol-based data. Really, I think there's one thing that we'd like you to take away from our presentation today, it would be this. The better that we get at designing protocols so that they are no more complex than they need to be, the better our trials will run, the better able will be to assess the safety and efficacy of promising drugs and get those new therapies to the patients who need them. And that's why we do what we do. So I want to end on the call to action. Let's see if that next Tufts impact report on patient complexity can show a decrease. Let's get a trend in the right direction. So thank you for the opportunity to share this with you today. And I think we can open for questions.

We're excited to hear with the call to action. Looking forward from here. So great research, great information, great data. Before we begin with our presentation, there are just a few important announcements I'd like to make one more time. For anyone who may signed in a bit late, both Denise and Steven, and are eager to respond to any questions that you may have based on today's presentation. I have been seeing some great questions that are generating. [Operator Instructions] So Denise and Steven, are we ready?

Absolutely.

Okay. Great. Pulling up my questions. Knowing that IQVIA chose one therapeutic area and that Tufts may have or did not, if that was the case, does this therapeutic area significantly affect complexity? And Denise, this is something that I'm thinking you may be able to touch on based on your insight here. If it did significantly affect complexity, what exactly did it affect? And it's kind of a loaded question here, so I'll start with that and then ask you to close out by just explaining further how the protocol developer would address such challenges.

Yes, sure. We were really happy to see that although the trials that we analyzed with the sponsor were mainly in one therapeutic area and the Tufts investigation covered 20 different sponsor companies in a wider range of therapeutic areas, the results were actually pretty similar. It does seem like we're narrowing in on those most impactful design elements. That being said, we'd be happy to talk to any sponsor about conducting this kind of in-depth investigation into their trials and concentrate with them on the variables that are most concerning or of interest to them. So if you're interested in talking further about such a project with us, please do contact me. One thing we're definitely going forward to is having enough understanding by therapeutic area to really be able to see the quantification of the impact of design variables on trial outcomes. For example, if we could say something like every 10% increase in the number of procedures in the trial, we expect, for example, a 20% increase in number of protocol deviations or a 10% increase, say -- or decrease in enrollment potential, these are just examples, that would be incredibly useful, right, not just directional, but truly allowing us to understand the impacts and the trade-offs.

Great. And I know that one of the concerns was protocol complexity is increasing over time. What are your thoughts there?

Yes, there's a lot of reasons why that could be the case for sure. Precision medicine, customized drugs based on biomarkers and genetics can certainly require more eligibility criteria. So you're looking for a specific patient population. A lot of drugs these days, those novel mechanisms of action, they might require more endpoints. Payers, of course, influence the endpoints that they require for compared effectiveness. There's a lot more use of patient-reported outcomes and focused on patient quality of life, so that increases endpoints as well. And even the increase in decentralized trials, while it reduces the burden for some patients, it can also increased complexity for sites. I think it's important to consider that not all complexity is bad. Some of it is actually necessary. So the key to success is going to be balancing the need for safety and efficacy data on the drug against the interest and ability of sites to run those trials and patients to want to participate and stay in the trial. And that's why we think it's really important to understand the impact that a trial's complexity has on outcomes.

And a follow-up to that is just what more IQVIA may be doing to address the protocol complexities.

Yes. Groups that have been a part of at IQVIA for more than 10 years have been working on developing a protocol algorithm or a tool. And again, what we found is it's really best to get the information from the protocols themselves. That's how Tufts CSTD has always proceeded, manually collecting that information from each protocol one by one, and that's how we did it for the sponsor. It's a lot of work, but definitely worthwhile. But one of the things that we're doing now at IQVIA is investing in protocol digitalization. So that would ingest a protocol and use AI/ML to extract key information that can be used in downstream systems. So the overriding purpose of this is to streamline operation of clinical trials, to automatically fill in EDC forms, et cetera. But once we de-identify that data, can help us better strategize and plan for clinical trials, because we can better understand the design and usual common ideal design for trials of that type. So this digitalization is definitely going to help us automate the patient burden algorithm that we've been using for 5-plus years. We have a new site burden algorithm that will be debuting soon. And this is also going to allow us to roll out automated protocol complexity algorithm. And that we could base it then on accurate industry, de-identified industry benchmark data. So as you mention here, too, that over the last year, we've set up programs to engage with sponsor clients to partner with us and innovate. And we do have these types of programs for both our burden algorithm and our complexity algorithm development. So for sponsors who are interested in learning with us and exchanging ideas and insights, that's available as well. And then these 3 algorithms that I mentioned: patient burden, site burden and protocol complexity, are being programmed into a new tool, IQVIA's Protocol Analyzer. And that's, again, going to use that protocol digitalization. It will automate those scorings, and we're definitely looking for sponsors interested to pilot using that tool with us as well. So if you're a sponsor company and you're interested in partnering on any of these innovations, again, please do contact me to learn more.

Now Steven, I'd like to bring you into the conversation of it, and I'd like to shift the focus over to talking more about some of the statistical tests that were conducted. What types of statistical tests were used to determine your correlation?