Analysing the Latest Mitre Engenuity EDR/MDR Evaluations: MDR Vendors Compared

The latest Mitre Engenuity ATT&CK® Evaluations for EDR (Endpoint Detection and Response) and MDR (Managed Detection and Response) services have provided critical insights into how well various vendors security offerings compare including the softwares prevention and detection capability, but more crucial how the vendors MDR Team react to the attack scenarios, and whether they provide actionable insight back to your organisation.

Overview of the Evaluation
Mitre Engenuity’s evaluations are designed to emulate real-world attack scenarios, allowing organisations to understand the effectiveness of their security solutions. In this evaluation, the performance of several EDR and MDR providers was tested against techniques employed by Menupass (APT41) and BlackCat (ALPHV). These threat groups are known for their advanced tactics, techniques, and procedures (TTPs), making the evaluation results particularly relevant for organisations seeking robust defence mechanisms.

Menupass (APT41) Scenario:
Menupass is known for its cyber-espionage activities and use of sophisticated malware. The evaluation tested how well security solutions could detect and respond to these advanced threats. High-performing solutions were able to detect lateral movement, privilege escalation, and data exfiltration attempts typical of Menupass attacks.

BlackCat (ALPHV) Scenario:
BlackCat is associated with ransomware operations, using advanced techniques to encrypt and exfiltrate data.
The best solutions identified initial access vectors, command and control communications, and encryption activities, showcasing their ability to thwart ransomware attacks.

Our Key Findings

Detection Capabilities:
The evaluation highlighted significant differences in detection capabilities among the EDR/MDR providers. Some solutions were able to detect the majority of attack techniques, while others struggled with certain stages of the attack lifecycle with the highest vendor detecting 42/43 techniques lowest only detecting 25/43.

Solutions that provided comprehensive visibility into system activities and covered a broad range of TTPs performed better. Effective solutions offered detailed telemetry and context, enabling quicker and more accurate threat identification.

Response Effectiveness:
Mean Time to Detect (MTTD) is the average time between when an attack is run and when the managed service provider triggers an alert on this attack. The timestamp on the first email relevant to the step in question was used. The average across the group was 41 minutes, with the best responding in as little as 4 minutes, and the worst at 93 minutes. As you can imagine time is critical in advanced attacks as they aim move laterally and spread persistence throughout their victims environment.

Alert Email Fatigue:
As any IT/Security professional will know being able to understand and interpret events both in the platform, via email/ticket and direct guidance provided by the managed service team is important. Having your service desk light up with 100’s of emails is going to be a real burden on the security team so being able to aggregate and prioritise alerts with meaningful insights is critical. If your utilising an MDR service you really want them to deal with the noise, and then report on positive mitigations that need to be implemented.

Actionability:
Actionability is a measurement of whether a SOC analyst is provided with enough information in the alert about What, Where, When, Who, and Why to take action on it. This is very important for Vendors providing an MDR Service as they need to be able to give a clear picture of why an event is being alerted, what the impact is and how teams can mitigate the issue.