Palo Alto Networks, Inc. (PANW) Earnings Call Transcript & Summary
January 23, 2024
Earnings Call Speaker Segments
Carlo Tarantini
executiveSo today's session will cover the MITRE Engenuity ATT&CK Evaluation results for this year. This is going to be hopefully educational for all of you. We learned some, not all of that, something about this year's round about MITRE, about the framework, how we have done at Palo Alto, no other vendor Fed. It's now 10:32. So I assume -- I presume we can get started. Now good morning. My name is Carlo Tarantini, I'm Senior Product Marketing Manager here at Palo Alto Networks for Cortex. I'm based out of London, connecting from London today. And today, we're going to talk about MITRE, the framework and mostly Palo Alto's results on the last evaluation round for MITRE Engenuity 2023. So what are we going to get covered today? I'm going to quickly run through this slide. So we -- I'll share some opening thoughts on the state of the ATT&CK Evaluations, the general sentiment there is. We kind of came across the measured in the industry around MITRE and their results. I would like to share with you a background on the MITRE ATT&CK and Engenuity Evaluations and how relevant they are, how can -- how should they be read and generally, what the framework means, what it does, how it's structured, we'll dive into the 2023 results. And then finally, we'll show -- I'll share a few thoughts on what's next and share a few resources with you, so you can deep dive, if you like, and read more about the test scenario and general year results. Now our initial thoughts on this year's round. So basically, we have Palo as spoken to dozens, if not hundreds of customers over the last 6 months. And I personally get so excited about these evaluations because I think they are super valuable. But at the same time, myself, my colleagues at Cortex, we feel a bit disheartened by some of the conversations recently we had. Because we think that some of our customers, some of our partners, some of our prospects that consume the results, those that consume these evaluations still struggle to understand how to interpret them. And I think this has a lot to do with the way vendors are espousing their results and are communicating them. And I feel like we're in a state where the value is not entirely clear. So hopefully, we'll share with you some things that we are doing at Palo Alto Networks and some things that MITRE Engenuity have done, some brilliant visualization stuff to help sort out some of that confusion and make the value very clear and make the results very clear and allowing them to shine and really generate value for you. So people have kind of lost, they're warning what the validity of the results of the test is, but the test is very valuable. It's comprehensive, and we'll make sure that it generates highest value, the most value for you. In order to understand the test and kind of frame and kind of really have a test of the rigor that MITRE have put in their evaluations, we can look at MITRE as a company. MITRE started -- was created as a nonprofit organization. And some of the missions that it did initially was, for instance, researching mix. I'm talking mix, I'm talking fighter jets and they dealt with that they kind of specialized in aerospace defense to begin with. It's amazing, right? So keep that in mind. There's not a whole lot of comparative in that space. There wasn't a lot of bias in that area. It was very much factual, almost scientific. And this is kind of their ethos. So what they do is, they go here, are the results of the test. You shouldn't be biased. It is factual. And this is what they've done for the last 4 years. So this year, they have progressed their approach in kind of exposing a lot of that data. And what they do is they don't -- I don't think that it's a framework or their visualization, so their data is sort of designed to conduct any comparison. Because you can't really sort of cut any biased slices of data, if you look at MITRE. But nevertheless, there is a sentiment around these results. And we really want to unpack the sentiment. We're all feeling about how vendors are talking about these results. They do -- so MITRE do focus on the data and they give you a way to visualize that data that allows you to understand how different solutions compare. But I think that this leaves an opportunity for vendors to really drive the narrative rather than leaving the data speak for themselves. And this is what we're going to try to do today. So let's talk about the MITRE ATT&CK framework to begin with. What is it? Well, first and foremost, this has really become the language that cybersecurity defenders and probably even attackers use to talk about the tactics, the techniques, the procedures and the minutia of how attacks unfold. So I'll introduce it to you briefly. All of you or most of you know it at this point. But if you're joining us and allow me to help you just with a little bit of visual clarity around the framework, because it sets the scene and it's really helpful to kind of frame it, framing the framework. So what we're going to see here, I'm going to split up the MITRE ATT&CK framework. This is the enterprise framework that we're talking about specifically. And you can see this includes the pre-attack and the execution faces as well. So the things you see across the top, they are the tactics within the ATT&CK framework. And -- as well as the tactics, you actually see the underlying techniques and sub-techniques that comprise all different ways and adversary might compromise own organization. So I just defined tactics at the top and techniques, which are sort of the things on the side are kind of underneath each of the other tactics. And what do they mean? So when we talk about the tactics, it's really what the adversary is doing, what are they doing and why they're doing it. They're doing it maybe to establish initial access within an organization, maybe elevate privilege, move laterally, exfiltrate data, establish command and control. Again, those are the what the adversary is doing and why they're doing it. And then the specifics of how they're achieving those results, defined by those tactics are what we call the adversarial techniques and sub-techniques. A few thoughts on the relevance and kind of the evolution of the ATT&CK framework. So if you go back through all the valuations in the past, 10 years at the beginning of MITRE we had attack, right? Then pre-attack came, and now pre-attack and attack have been combined. So you see there are stages like reconnaissance on the left-hand side, and others that follow, the beginning of the chain, well, these are pre-attack stages that have been added on throughout the years. And the following ones are attacks. So what I'm trying to say is it's nice to have everything in 1 place. So the way MITRE has evolved the framework has combined pre-attack and attack stages and whether that's comprehensive and clearer to read. MITRE also sort of manage this framework in a way that it's a living document. So they do listen to us as a community. And what they've done is over the years, we've submitted a request for change as a community and adding additional technique, and that's exactly what they've done. This framework evolved over time, it became bigger and bigger and bigger, and we've made modifications to it. So it's really evolving its living documents. In fact, you remember, they added the impact stage only a couple of years ago as far as I remember. So it is evolving, and it's also granular. If you think about it, each of these stages of the ATT&CK framework, what it is, it sort of leverages or relies on the description of a specific technique. Maybe there's a specific API that you're calling, what the attackers are calling. Maybe they're doing a power shell. So each of those sub steps in subset -- sub steps describes the a specific capability, very specific capability. But yet, the framework is sort of high level. So it allows us to bubble up this technical -- all this technical information to a higher level so we can have a higher level of conversation. And a little caveat around the framework is that, well, let me say, attackers don't move left to right, right? What I'm trying to say is don't try and interpret the major framework as a sort of a time line. It's not always the case that attack moves sort of linearly from the pre-attack stages to the attack stages. And this comes in conversations a lot and there's a bit of confusion around this. So if you have been watching all attack, you might have initial breach moving over to lateral movement and then you have other tactics in between. So attackers tend to go move back and forth across the framework, and incidentally, this is what happened in these tests as well. So when you're looking at the framework, keep in mind that it doesn't go left to right. And in fact, MITRE Engenuity have designed their own tests so that they may make a real attack and kind of sort of swing across the matrix of the chain. So let's go down to MITRE ATT&CK Engenuity ATT&CK Evaluation. So what important is the subject of today's presentation is, the talk is to look at these results, look at that the Engenuity ATT&CK Evaluations for this year. Let me say that there are 2 organizations that we're talking about here and that we should keep in mind. So MITRE is the parent organization. There's MITRE again, and there's MITRE Engenuity. So MITRE Engenuity is actually running the show for these evaluations this year. And just a bit of history. So this is the 5th year of results and the 2 organizations kind of spun-off at year 3. So the first 2 years, it was MITRE that kind of managed the evaluation sort of the process. And lately, MITRE Engenuity has became a spinoff and it started sort of creating process around these evaluations. And again, you can see how the evaluations have evolved and have become more comprehensive over time. In fact, looking at -- looking back at where -- when MITRE was in charge, the first 2 years were focused on detection only. And detection only, as you might imagine, it's about visibility. So there was a bar that all vendors, all of us were trying to sort of meet and trying to achieve, which was 100% detection or close to 100%, whatever we could, give the most visibility to our users, to our customers. And I think the evaluations as they were sort of conceived initially really allowed us to progress forward as vendors and making sure we gave as much visibility into all those tactics and techniques. But then in rounds, I think 3 to 5 they were taken on by my MITRE Engenuity, and they, MITRE Engenuity introduced prevention. So they still kept with detections but now you have added preventions to that. And we've seen a greater focus on prevention moving forward here, especially in the fifth year. So it's something to keep in mind. But what are MITRE Engenuity enterprise evaluations and what are they not, right? Let's talk a bit more about them. So MITRE Engenuity's motivation is really sort of give the tools to understand the capabilities that sort of attackers are utilizing. They are a bit of a neutral authority and really being sort of an authority in the field and sort of providing a bias testing is where MITRE have started. And I believe these are the, by far, the most valuable evaluations that have been done in the industry today. But what are MITRE trying to accomplish, right? So if you put it in bullet points, what is it that evaluations do? So number one, they try and emulate an adversary. And this year, the adversary they picked is Turla. We'll talk about Turla in a second. Number two, they walk through the tactics, the techniques, the procedures that these adversaries, that the adversary they use within the MITRE context. And number three, they observe what the different endpoint security solutions can bring to bear against those steps in the form of detection or detections and also in the form of prevention. So 2 phases. But what are these -- what's -- where does MITRE fall short, right? What are they not? What are these evaluations not, if that makes sense. So they really weren't designed to address noise or false positives. And they're really not meant to be a vendor ranking of source. There's no rating like leader, strategic leader, visionary, et cetera. So it's really just data and that's the end. Its observations about how products and solutions prevent and detect. Now this year, let's get into the evaluation results themselves, which is probably what most folks are here to talk about. And before I do, let's talk about what MITRE Engenuity have done in terms of evaluating the adversary. So the adversary they picked is Turla. Now, Turla are our Russian-based threat group, part of the C16 of Russia's Federal Security Services, the mighty FSB. And there are lots of other names that they know by -- they're known by, so they're known by -- they might be known by Pensieve Ursa, IRON HUNTER, Group 88, a very fancy name such as Beluga Sturgeon, monster fish; Venomous Bear, many have heard of the Snake rootkits and the carbon malware. So these are sort of -- these came out of Turla. And there's a lot of things that Turla are famous for. Who do they target? Specifically, they target government agencies, embassies, military facilities, education and research and development bodies and pharmaceutical companies. They've been around for a long time, about 20 years. And we have seen, as a community, a lot of heightened activity since about mid-2015 up until today. So they're very, very active. We know they've impacted more than 45 -- or around 45 countries. They're very known for their stealthy ways and their intrusions. And there's just a myriad of custom malware that they've built. We will provide links to you in the resource section where you'll see that our Unit 42 team, they conducted extensive research about Turla and the whole host of resources that you can access to read about them if you want to know more about Turla and their projects. Now CISA, C-I-S-A, put out a really interesting advisory back in May this year about the Russian Intelligence “Snake” Malware. So kind of -- we kind of saw it coming. Which is, in our opinion, the most sophisticated rootkits that we've seen out of Russia's FSB. So you might think if I'm not a federal agency, if not government, I'm not a pharma company or a MITRE institution, why should I be concerned? Well, think about sort of the broader picture. These attacks have their own blast radius. Actors aren't always very good at containing the consequences and these sort of unintended collateral damages of their attack, right? So blast radius is always an issue, right? If the attack hits an organization it might spread across its supply chain. And the other thing to keep in mind is that these guys are very well funded. They're still developing new techniques and they're the tip of the spear. They also sold the most sophisticated malware out there. And frankly, we can use that technique so what we know about them as a resource to defend themselves better. So moving on to the Turla evaluation, what's happened this year in terms of like using the adversary to design a scenario? I think, well, I took this picture from the MITRE website. It might be a little bit hard to read, but we want you to focus on like coloring, if anything. The stuff in blue here represents the path of the enterprise attack framework that was emulated for 2023 for Turla, and it covers a wide range of tactics and also techniques and sub techniques. This evaluation spans -- unfolds over 4 days. I just wanted to talk to you about how it works. So day 1 is really setting the scene and the first day is about the carbon malware setting watering hole and whatnot as part of the detection evaluation of the MITRE Engenuity tests. So day 1, carbon malware detection focused. Day 2 is about the Snake rootkits. Both those first 2 days were focused on detection. So only vendors are participating as the vendors are asked to turn off the prevention capabilities and allow the malware to proceed unimpeded to maximize the visibility and kind of the -- and these are the rules that they set up for the test to really show what you have to bear in terms of detection and the visibility that you can -- as a vendor sort of provide to a customer or to a user. And the third day is similar tactics and techniques with a bit of change to make it unpredictable, but focused on prevention this time, not detection. Actually, what MITRE Engenuity calls protection, just a bit of confusion in other words. So you turn on your ability to prevent all those capabilities and keep track of what you were able to block and you weren't able to block. But day 3 is about protection, in MITRE terms, all prevention. So a bit of a few numbers about the scenario. So we -- in this evaluation, Engenuity grouped the 143 detection substeps into 19 major steps. Why do we talk about it? It's important later to understand sort of the ratings and kind of the success of vendors claim in these evaluations. And in the protection phase, there are 129 substeps grouped into 13 steps. A big improvement, again, done around protection. It's been kind of an afterthought in recent evaluations, but really evolved, really took a leap forward this year. And the reason I'm talking about these starts and giving you these numbers that you really need to be aware of what a vendor mean when they claim 100% protection as opposed to detection. Is it 100% of the steps? Is it 100% of the substeps? We'll talk about this a bit more later. So in terms of the participants to the test, let me shade 2. Basically, we had about 31 -- well we had 30 -- exactly 31 registered participants. But when you look at the results, we only see 25 -- 29. So 31 were registered, we only see 29. So a couple of vendors to chose not to publish their results. For 1 reason or another, I don't have the details on why, we'd like to know more -- learn more about them. We don't get to do it and we don't get to learn about the results this time around. Just keeping aware and this is a screenshot provided by the Engenuity site. Right. Let's get into it. Let's get -- talk about how the results are scored. Now this is my favorite slide. I'm going to have to warn you. And I'm just trying to explain to you those who are not aware how the graphic can be read in terms of how the individual sort of techniques and tactics are evaluated. So what I'm trying to say is for each of those, say, 143 substeps in the detection phase, every solution is graded essentially on what they have to bring to bear in terms of detection for each of the steps. So these gradings are applied to each of the 143 steps basically go from NA to technique. What do these terms mean? So starts with NA -- starting with NA, it means basically you don't participate in the evaluation of that specific tactic or technique. You will see that in this year's evaluations, there's a Linux evaluation, because the scenario entails some lateral movement or encompasses lateral movement from -- or towards a Linux server. Maybe a vendor hasn't participated in the Linux evaluation, they will be scored NA. So that detection will pop up as NA. Up from NA, you have none. A none detection means that you missed it. This is if the adversarial tactic was executed or if the technique was executed and there is nothing you have to show either on your data lake or in your product that amounts to any observation of that specific action. That's none. Up from there, there's telemetry. So what is telemetry? Telemetry is, if you're able to showcase that maybe somebody did PS Exec, somebody that was not -- maybe that was not Psql. If in your data lake, you can actually show information about that step happening, then you get to telemetry detection. So it's not something that's surfaced in terms of a human readable alert or an incident, but it's basically data that solution collected, and that's available for investigations for threat hunting. So if you've got an EDR or an XDR tool that you can search or use to search on after the fact and if you have that particular step occur, you could find trail of evidence in the past, but no alert whatsoever. And so the other thing that's worth noting is that in order to get a telemetry detection, really, you have to be able to identify the source of that data. So it has to be good quality telemetry. It has to include information, specific information about the source of the data that security professionals, SOC professionals can use to sort of backtrack and reconstruct the story of the attack. Up from none -- up from telemetry, excuse me, sorry, you have general. So let's talk to other detection types. Now general tactic and techniques altogether, they are talked about as analytic coverage. So when you see a vendor talk about the percentage of analytic coverage, they're talking about how many of the actual malicious activities where -- they were able to detect with an alert. So general means, I've got an alert. It's really just an alert I was filed. Doesn't have to -- doesn't have to have sort of context around the specific tactic or techniques, so it's missing a lot of information. So you don't really -- you're not able to tell why or how -- why the adversary was doing something or how it did it. But if the solution files an alert, then at least the user will know, it have something malicious is happening and that's where the detection is graded as general. Up from general, you have tactic and techniques. So step up from that basically, there's a tactic detection that shine lights on the MITRE tactic that the adversary has used for that specific step. So it might be privilege escalation, defense evasion, exfiltration, you name it. So you have a name that sort of step of the attack in the form of a MITRE tactic. The alert gives you a lot of context in that sense. You can identify a tactic. That's how you got a tactic and get a tactic detection. And up from there more -- with more granularity and so even more context, you can get a description of the technique that the attacker has used, and that's where you have a technique grade sort of detection. So you can even get an understanding of how the adversary went about sort of fulfilling that attack stage. And we at Palo Alto like to think that, when we talk about non-adversaries that have been around for a while, well, this is the bar, right, technique. We should all be able to identify what an adversary are doing, what they're doing and how they're accomplishing it. It's really the bare minimum. A few words about modifiers. So there are sort of some things to bear in mind to keep in mind when you're looking at MITRE results. You have tick boxes -- sort of tick -- ticks on the MITRE website. I'll show you in a second where you can actually turn on and off the modifiers. For these attack stages, you can have delayed detection. So what does it mean? So in these evaluations you call the MITRE Engenuity your red team sort of simulating the adversary and your solution trying to sort of capture the data and kind of generate alerts. But if you get evidence, your evidence, you're alert after 15 and 20 minutes, well, that's where you get delayed detection. So it's delayed evidence in a way. And I am personally a bit skeptical of these because anything could be happening theoretically in those 15 or 20 minutes. There are some legitimate delayed detections, like if you have a cloud-based alert, that requires analytics or if you have -- if you need to do correlation with another source that requires some computational power or there isn't sort of an in-built delay in that correlation, then it makes sense to account for a delay, if the data is stitched together after the fact. But still, it's not ideal. Ideally, we'd like to sort of detect in real time and kind of react and mitigate cyberattacks straight away. Another scenario is that, if you're doing batch-based analytics or using ML or AI, we might have a delayed detection. So it makes sense from MITRE's perspective to have it there. But from a defenders perspective, we don't want to have delayed that. And then last but not least, my favorite, you have config changes. So configuration changes are detection that happen on day 4, because day 4 is a bit of a rerun of the attack that sort of vendors do provide or generate as a result of configuration changes in their solution. So you can change anything you want in the product on day 4. You can change the way you collect data, you can push a new release, you can push an update. You can buy another company, you can buy a country, you can do what you want, whatever you want, and that will be qualified as a config change, and you have the chance to show your sort of reviewed results and the outcome of those configuration changes. Now I'm not 100% against config changes, but kind of 99% there. Look, if there are -- there should be exceptions, we should allow sort of vendors to provide the results and accounting for configuration changes because otherwise, they'll have to wait for a year to submit their MITRE results. But the comment to keep in mind is that as defenders, as blue team is, we don't get a second chance. So there are no configuration changes when it comes to using those products and the reality of our jobs as SOC analysts, SOC managers and CISOs. So let's get into the results. And let's -- the best way to talk about it is to compare them directly from the MITRE website. Now I'll show you a bit of a tour of a MITRE website and share my screen in a second, and try and kind of explain what the best way to read these results in. So you're going to see on the ingenuity page that basically the interface this year is designed to run comparisons against vendors, if you want, but it's not the primary purpose of these results. You can stack up to 3 vendors on the MITRE page. I chose us, Palo Alto as well as Microsoft and CrowdStrike, which are very valuable solutions that we do encounter a lot. And configuration changes are selected by default. So if you connect to the MITRE web page, you'll see something like this. So you will account for delayed detection and configuration changes. But again, do you really want to sort of rely on tests in a scenario that where vendors are allowed to change their configurations and kind of retest their solutions and rerun the detections? I suggest you tick them off, and that's where you kind of keep things on us and you can run a comparison against the vendors. So the interesting thing about this year's website and kind of interface is that you can go back as far as you want to basically, first, Engenuity Evaluation and compare vendor performance. I chose to stick to this year to Turla. And for this year, you can look at results for detection, again, carbon and snake as well as protection. This is really exciting. And they're broken down by attack stages. So in terms of the color legend on the MITRE website, the detection key is down here. I'll blow it up for you. So you've got none, which means, as we described in the slide, I talked about it extensively. You have no detection whatsoever and not applicable where the vendor hasn't participated at all in the detection of that particular technique. And above none and not applicable, you have telemetry, you have general, tactic, and technique. These are the analytic based detections. So anything that's not yellow basically, amber and above is a scenario where actually your solution gave you an alert and some context around that specific technique or tactic. So you can -- we will switch from carbon to snake in a second. And you can see that this is basically carbon snake is day 2 as well, so that the scenario went from sort of a watering hole prep to an actual exfiltration impact stage. So advanced stages of the attack. At the top of these evaluations, you have a scenario detection but an overall sort of summary. And guess what, Palo Alto for carbon detected 76 out of 76 techniques, so 100% of the techniques. And for the snake scenario for the advanced stages of the attack, we got to 66 out of 67 techniques. No other vendor fed, no other vendor have this sort of performance, right? And the reason why you see 1 technique will show to 1 technique, is just because in the lateral movement stage, well we addressed only 4 to or other 5 techniques. And for the fifth technique, we gave a tactics or level alert as opposed to a technique one. So really 100% on detection across the board on a MITRE level with full context around the technique and the tactic. Generally the technique, 66 out of 67 times for you to gain context around the alert and these alerts and attacks. No other vendor had the same performance. And it gets even more exciting when you look at the prevention page. So how does the protection actually in MITRE terms, preventional protection, how does it look? It's got 13 steps this attack. Purple is a block, yellow is a none. And only Palo Alto scored 100% in this scenario for protection. Compared to previous years, this visualization is actually pretty exciting, pretty granular. So it shows you the steps that are actually malicious. So it's actually 3 or 4 positives, and it kind of shows only the stages of the attack that have relevance in terms of protection, detection, red teaming. And when -- keep in mind, when you're looking at that view, you want to compare these results and get full visibility of the attack stages. So what happens is, it shows you the steps that were prevented as well as the steps that would have happened, had there been no prevention at all. So for instance, for the Linux compromised, right, Palo Alto prevented straightaway other gates in that first technique. If you prevent an attack from the outset straightaway, you might lose some visibility in terms of MITRE, right, so MITRE, basically, what you've done is you've mitigated in the beginning and you kind of you don't know where the attacker would have gone from there. But with this visualization, you actually can see what the following stages are for each of the tactics. And if you scroll down, you'll see images and you'll see exactly screenshots of scenarios and sort of output on the -- on your solution, and how that telemetry sort of brought to life. And again, Palo Alto sort of knocks over the park. The only event that have scored 100% for protection. Now the MITRE, again, is a very -- is an objective body. MITRE's way to present these results is incredibly unbiased. But as I said, it's not an ideal scenario if you want to compare vendors. And what we've done at Palo Alto is really -- we made your life easier as a technology -- basically user, as a customer, as a top professional, by providing an interface that explores all the data, all the interactive data for MITRE and explores their framework over the years and across all detections -- across all stages of the attack, across all vendors actually. So we have introduced the visualization on our website that is basically data-centric and allows you to view results across all vendors and across all years. What we've done is in the MITRE website, basically, I'm a techie at heart. So I just want to explain how it works. So in the MITRE website, you can download each of the results for each of the vendors as a JSON. And from there, we built a Power BI and visualization tool for you. Now for -- with this interface, you can see that it has all the vendors. It's a data-centric view. So you can see just data, all the vendors gives you flexibility to visualize, no spin, and you can see the truth. So how it works is you can go back to any of these valuations in the last 5 years, select as many participants as you like to the MITRE evaluation, select as many tactics as you like, and by default, it doesn't have delayed and configuration changes sort of activated. So it's an unbiased view of the evaluation in which you'll see the percentage of analytics detection for MITRE, the percentage of sub steps blocked. So the detection side of things, the prevention side of things and as well as -- I go back to the summary for you, as well as the sub steps missed. So this is really a fair comparison that takes into account the highest level of detection, the technique level, really allowing you to kind of compare performance across vendors and technologies. And it's really interesting that what you get from this evaluation, basically, it accounts for configuration changes. It tells you what happened on day 4 and the sort of things and the contracts that vendors have to sort of rely on to score that 100% to get more detection -- more detections, it's a score higher. So there's actually 2 sort of types of visualizations in this page. You have a detailed one. If you go on a website, you'll find a detailed button over there. And what it does, we need to blow up some of these graphs and it gives you an extra plot. It's a magic quadrant style where you have basically the percentage of detections versus the percentage of protection or start substeps block. So you have on the x-axis sort of the level of visibility you get from your solution and y-axis is the sort of protection abilities or the effectiveness in protecting your environmental, and pollution, and distribution provides. Now you can use our tool, you can use our visualization to basically get an unbiased understanding of these results. And again, still remember what the configuration changes in delay do. We really ask defenders don't get a second chance against attacks. We really want to keep it fair, we want to keep it honest and the only way is to kind of have a look across the data and across vendors and have a fair comparison. So to sum up on the state of the evaluations moving forward. I think there's confusion, there's a bit of mistrust around the MITRE Evaluations, this confusion can stem from 2 places. On the 1 hand, MITRE over the years have published results, but they haven't quite given an opinion on who did well and how. And there's a lack of sort of presence there of voice and vendors have sort of chipped in, and they gave their own version of the story. So the problem of providing transparency to the SOC community for organizations to be able to select the right solution for their needs is on us, on us vendors. And this is a step we've taken at Palo Alto by sort of building the Power BI and the comparison tool. I was in LinkedIn a couple of days ago, I saw a couple of -- 1 vendor actually so talking about having 100% detection, 100% protection, 100% visibility. But you saw the data. Nobody else has got to see those results, except for Palo Alto. What do they mean when they say 100%? But what they're saying is really to have 1 detection or 1 prevention in each of the major categories. But if you look at the details of 143 techniques that were used or the 129 steps for prevention, well, it's a very different story. And you have to basically write the story yourself. We kind of helped a lot by providing as much transparency as we can by building that sort of visualization tool. Other vendors, other things that vendors have done lately is to removing steps basically from evaluation or ignoring steps and kind of manufacturing their own rating. So you get 100 because you ignored your bad rating or your NA on 1 of the steps. Is it fair? It's up to you to decide. But at least we gave you an interface, we gave you a tool so that, again, as an organization, you can actually see for yourself how the different vendors have fared and have scored in the MITRE Evaluations this year. Another thing to keep in mind is that there is a difference between enterprise evaluations, which is the 1 -- the sort of time to test that we explore today and the MDR evaluations. So some vendors will use the MDR evaluations, sort of messaging the data and presenting them as enterprise evaluations. That's, again, not fair. So what the MDR test does in MITRE Engenuity test as for MDR, they go through a tough scenario and then they generate a report and have the vendor who's providing managed detection and response sort of again, generate that report and kind of measure the value of that reports to the end customer. So it's basically a measure -- it's an evaluation of a managed service as opposed to the protection and detection levels that the endpoint solution provides. So yes, everyone gets 100, is a bit of an Oprah game in this MITRE world, but it's not quite true. Only Palo has scored 100%. Now finally, I want to share with you a few resources. If you now want to learn more about Turla, if you want to know more about MITRE, we wrote a couple of interesting blogs. Peter Havens, from our team, wrote a blog about this year's results as well as Parker Crook. They're both are my colleagues in product marketing. You have Unit 42 as providing a wealth of knowledge against -- regarding Turla and the attack scenarios and the tools they use. And finally, do use our evaluation dashboard to play around with the -- build your own visualizations and explore the results and sort of write your own version of the story. That's it for me.
Carlo Tarantini
executiveThere's a question about XDR. I think I answered it. So it's -- when bringing up XDR endpoint, is it sufficient to detect and prevent 1 of the steps in attack chain to be successful within the prevention path? Not for us. I mean, we keep the bar really high. We want to achieve basically technical level detection. Some vendors will say that they achieved 100% by only sort of accomplishing or preventing 1 of the key steps, not for us. Are there no -- have got any more questions? I think we'll close the webinar for now. Thanks, everyone, for joining. And see you next time.
This call discussed
For developers and AI pipelines
Programmatic access to Palo Alto Networks, Inc. earnings transcripts and 251,000+ others is available through the
EarningsCalls.dev REST API. Plans from $24.99/month — full transcripts, speaker segments,
full-text search, and the recently-added /api/v1/transcripts/recent polling endpoint for ETL pipelines.