Detection Isn’t Declared – It’s Proven: What Your SOC Actually Detects
Roman Sologub, CEO of ISSP, opened his SOC Day 2026 talk with a question that sounds almost naive: if your SIEM sees an attack, does that mean it will tell you about it? A log event, a rule tagged with MITRE ATT&CK and a confirmed detection of an adversary action are three different things. Between them sit data sources, parsers, schemas, rule conditions, exclusions and the configuration of the environment. That gap is where, according to both industry research and ISSP’s own measurements, most attacks get lost. Hence the thesis the whole talk was built around: detection is not something you declare. It is something you prove, and prove regularly.
An adversary with no malware and a stopwatch
Start with what a SOC actually has to catch today. According to the CrowdStrike Global Threat Report 2026, 82% of detections in 2025 were malware-free. The intruder logs in with a legitimate account, uses built-in administration tools and works hands-on-keyboard. An attack like that simply has no classic indicators.
David Bianco’s Pyramid of Pain explained why long ago. An adversary can change a hash, an IP address or a domain instantly and at no cost. Changing a tool is harder. Changing tactics, techniques and procedures, the very way they get the job done, hurts the most. That is why behaviour is what has to be monitored.
And monitored fast. The average time from initial access to lateral movement is 29 minutes, and the fastest case on record took 27 seconds. Mandiant’s M-Trends 2026 gives an even more telling figure: the median time between gaining initial access and handing it off to another group has shrunk to 22 seconds. Indicators of compromise are still useful, but at this pace they mostly protect against yesterday’s infrastructure.
The paradox: thousands of rules, very few detections
Behavioural detection looks like a solved problem. Thousands of ready-made rules are freely available with no subscription at all: SigmaHQ, Wazuh, Microsoft Sentinel, Splunk Security Content, Elastic. Take them and deploy.
Here is what happens in practice. According to CardinalOps, the average production SIEM runs 163 rules, 25% fewer than in 2022. Back then the company had already found that customers disable 75% of vendors’ out-of-the-box rules. Every year there are more rules in the world and fewer of them in production.
Is it a lack of data? No. CardinalOps estimates that the sources already feeding a SIEM would be enough to cover about 90% of ATT&CK techniques. Actual coverage is 22%, and one rule in ten does not work properly. The Picus Blue Report looks at the same picture from the attacker’s side: over four years the share of attack actions that reached the logs rose from 37% to 58%, while the share that raised an alert stayed between 12% and 16%. Organisations have learned to see more. They have not learned to detect more.
Two independent studies point to the same place: the gap is not in the data but in the detection content. Picus also shows where it breaks: half of the failures come from log collection and a quarter from rule performance. In 2026 performance became the largest single category of detection content problems, at 49%. A rule is formally correct, but it searches raw text or parses data at run time, and under load some events simply never get evaluated. A code review will not show this. It only shows when you trace the whole path from the action to the alert.
Three diagnoses for one system
The talk groups these symptoms into three “diseases” of a detection stack, and anyone who has run a SIEM for more than a year will recognise at least one.
The first is fattiness. The library grows after every advisory: indicators get written into rules and forgotten, and new green cells appear on the ATT&CK map. They have no effect. A classic example: exploitation of remote services is “covered” by a rule on network ports that was once written for a network worm.
The second is hypersensitivity. Generic rules that know nothing about the environment fire on ordinary business activity. The team treats the noise with exclusions and has no idea how much of an attack it has excluded along with it. According to the SANS × Anvilogic State of Detection Engineering 2026 report, 66% of false positives come from vendor rules.
The third is necrosis, and it is the most dangerous because it has no symptoms. A source schema or syntax changes, or a security control starts blocking the action earlier, and the rule stays “active”. Everything in the console is green. In one publicly discussed case a team spent two years trusting an EDR rule that did not even compile.
Detection as Code: the foundation
The industry has a mature answer to these diseases, inherited from software development: Detection as Code. A rule lives as code, with versions in Git, changes through pull requests, review, static checks, tests, staged deployment and rule health monitoring. It is a great approach, and it remains the foundation: it defines how detection code is built and maintained.
The foundation is not cheap: you need qualified engineers, and a team of them rather than one. The same SANS × Anvilogic study found that 62% of teams use version control, 58% do peer review and 42% have CI/CD, while 72% name lack of time as the main barrier.
And there are two things Detection as Code does not tell you. First, what to defend against first when there are hundreds of techniques and even more ways to execute each of them. Second, whether a correctly built rule will fire on a real attack in your particular environment. Two further levels answer those questions.
Threat-aware detection engineering: the focus
Threat-aware detection engineering adds priorities and focus to the foundation. It makes it possible to handle a large mass of varied techniques by concentrating on the likely ones. That is what makes Detection as Code meaningful: the team writes and maintains not everything in sight but what it needs against its own adversaries.
In 2024–2025 ISSP delivered this approach as a service. The focus shifted from content to cyber threat intelligence. For each customer an adversary profile is built from the sector, region and assets. Its techniques are laid over the SIEM’s existing coverage, which shows who the organisation is already protected against and where the gaps are. Out of thousands of rules and hunting queries, only what closes those gaps goes into the tenant, together with runbooks for triage and investigation.
Adversaries act in sequence, as an attack chain, and that became part of the approach too. When a rule fires, the system uses the CTI base of campaigns to predict the adjacent steps. A retro query looks for what was missed in the event archive, and a watchlist puts what is likely to come next under control. It is a hypothesis for the analyst, not proof, but together with risk-based alerting it cut the number of alerts by 72%.
Two years of this practice produced a telling result. Out of more than 14,000 content items, only about 9% turned out to be effective. The distilled core of 1,300 rules for priority threats, by contrast, works in full. By ISSP’s estimate, the standard approach yields a detection for roughly 16% of attacks, Detection as Code for 75% and threat-aware detection engineering for 91%. The remaining 9% are blind spots of limited visibility.
Threat-proven detection engineering: the evolution of threat-aware
91% is a good number. But it still rests on analysis, and there are questions analysis cannot answer. Will the rule fire on exactly the behaviour it was written for? Has false-positive tuning changed the logic it was meant to have? Will the telemetry arrive in time, and what changes after an agent or a parser is updated?
There is only one way to find out: execute the malicious procedure and look.
This is the next step in the evolution: threat-proven detection engineering. Threat-aware tells you what to focus on. Threat-proven lets you make sure the detection code really works. And not in lab conditions but in your own environment, with its domain policies, controls, security systems and configuration.
The key element of the approach is rule generation and validation through automated purple teaming. A procedure is executed; the system checks whether telemetry arrived and whether detection fired with the right parameters; and the rule itself is created from the real execution. This level of assurance used to come only from costly purple teaming, mostly in a vendor’s lab. Now it runs in the customer’s infrastructure, regularly, as a service.
The three levels do not replace one another; each builds on the one before. Detection as Code answers how to write and maintain rules. Threat-aware answers which rules are needed first. Threat-proven answers whether they really work where they have to.
One green cell, three ways around it
There is a tempting shortcut here. According to Red Canary, the top 30 techniques show up in three quarters of attacks. Why not write rules for those thirty and be done?
Because a technique is not a single action. It is anywhere from a few to a few dozen procedures. Persistence through the task scheduler, for example, can be achieved with the schtasks.exe utility, through PowerShell or directly through the API. A rule on the schtasks.exe command line sees only one of the three paths. Yet on many ATT&CK maps that cell is already green.
So the right question is not “is there a rule for the technique” but “what would the adversary have to change to get past it”. The answer comes from MITRE CTID’s Summiting the Pyramid model. It unfolds the top of the Pyramid of Pain into five robustness levels: from a file name that can change in an instant to the specific behaviour without which the technique cannot be executed at all. ISSP scores every rule on this scale and sets an achievable ceiling for every technique: the highest level the customer’s telemetry allows.
A library that knows how to attack
On this basis ISSP’s rule library stopped being static. It is now two collections linked by an execution engine.
The blue collection describes 701 techniques: detection logic, required telemetry and the robustness ceiling. For 342 of them a rule at level L3 or higher is possible. The red collection holds almost 2,000 actions and 307 attack chains assembled from CTI, real incident reports and emulation libraries such as the widely used Atomic Red Team, MITRE Caldera and MITRE Attack Flow. The attacks are not invented; their elements repeat. The execution engine runs these actions in the customer’s environment and creates a rule from the telemetry that is actually there.
What this means in practice was shown in a lab example using Wazuh with its 4,511 built-in rules. Of 515 prepared techniques and sub-techniques, 285 were executed. Out-of-the-box content detected 52 of them, or 18%. Another 212, or 74%, produced telemetry, but no rule fired correctly. And 21 techniques remained a blind spot.
It is the same picture the industry studies describe: most of the activity is visible, little of it is detected. Then comes the interesting part. In one run on the test bench, the standard content detected none of the four chain steps that were executed. After new rules were generated, three of the four were detected, with no false positives.
One more detail is telling. Of the ten steps in that chain only four executed: the other six were blocked by the security controls already in place. The outcome depends not only on the rules but also on domain policies, controls and configuration. So a lab bench shows the mechanism, while the real answer comes only from running the check in your own environment.
SOC detection: a process, not a project
These numbers have a shelf life: as new procedures appear, any coverage erodes. That is why threat-proven detection engineering is not a one-off audit but several cycles that never stop turning. The CTI cycle updates the adversary profile. The telemetry cycle checks whether the sources see the reproduced techniques. The detection cycle fixes and validates rules. Detection as Code takes every change through review and tests. Infrastructure hardening closes the gaps where the fix is not a rule but a configuration change. The point all the cycles share is TTP distilling: it defines what to reproduce, what to measure and what to write content for.
A process like this is complex and resource-intensive. It takes systems, data and process engineering, blue, red and DevOps teams working together, and the use of Git, XDR, SIEM, CTI and BAS. But without it a SOC cannot know what it actually detects. Coverage has to be measured by execution, not declared by mapping.
The main idea of the talk fits in one sentence: a detection rule is only a hypothesis of detection, and reproducing the attack is how you test it.
Watch Roman Sologub’s full SOC Day 2026 talk on video: https://youtu.be/Uc8vaCdMBfU
