How to grade your MDR?
Use an MDR, but not sure how well they are doing? Or, how to evaluate them. Well, read on...
After having spoken with several CISOs that use an MDR, one common theme emerges is that they are not extremely happy with their MDR.
What are the common reasons why CISOs are not happy with their MDR providers?
Based on my experience, it boils down to one of these things:
Data Collection: MDR provider does not provide complete visibility. It cannot collect all the data that my monitoring tools generate.
Tools Support: Most MDR providers have a set of tools they are really good at integrating with. Some they integrate with, but the quality might be low in terms of being able to fully leverage that tool for detection & response. And, there might be some tools the provider simply does not support.
Low Detection Coverage: Many MDR providers do not provide an easy to understand way to see what kind of detection coverage they have deployed. Are we not finding bad things because we are not looking closely enough? Or, do we not have enough detections built in?
Poor Escalations: While most MDR providers avoid sending a lot of noisy alerts to their customers, I have heard the customers complain about not enough details or clarity to tell what really happened, why it matters, how to remediate it.
No Response, just notification: Many MDR vendors will tell you how to respond to an alert or an incident, but they will not take on the responsibility to execute those on your behalf or provide a one click way to trigger the response.
Dashboards, Reports and Metrics: Does your MDR vendor report on KPIs like MTTR (not just to respond, but remediate), MTTD (mean time to detect). Do they provide metrics to see how good you are doing on monitoring and detection coverage? Can they provide custom reports and dashboards?
Portal Experience: Does the MDR provide a portal that is usable and capable? Can you see all the information you need to gain situational awareness and take the actions that you need to keep the enterprise secure?
Automation: Inevitably there will be tasks that you have to do even though you have an MDR. If and when those tasks become repetitive - does the MDR provider provide automations to help you save time and effort?
Quality of Service: Two big aspects of service that MDRs provide are adaptability and responsiveness. Depending on how complete the service is - you may have to ask MDRs to perform tasks that they should have proactively done. When you request such tasks - how willing and responsive is the provider in meeting those requests.
What did I miss? Is there an important aspect of MDR that you value that is not covered here?