Detection engineering is the discipline of systematically designing, building, testing, and maintaining the detection logic that a security operations center depends on to identify adversary activity. It applies software engineering principles to the problem of creating reliable alerts, including peer review, version control, automated testing, and structured lifecycle management, replacing the ad-hoc configuration exercise most teams start with. A mature detection engineering program measures coverage against known adversary behavior patterns, tracks detection fidelity over time, and builds feedback loops that continuously sharpen signal quality.
Most security teams write detection rules, but what is needed is a detection engineering program. The gap shows with the ratio of true positives to false positives, the lack of breadth of coverage across cloud services, SaaS applications, identity infrastructure, and endpoints, and how quickly the team can respond when an adversary shifts technique. That gap is the product of how the discipline evolved, from manual log review and static rule sets to what detection engineering demands today.
Why detection engineering is a discipline, not a task
The informal alternative to detection engineering is ad-hoc rule writing, which is when an analyst spots a threat, writes a query, saves it to the SIEM, and moves on. Without systematic testing, many of those rules fire on irrelevant data, flood analyst queues, and erode trust in the alerting system. Without lifecycle management, rules accumulate over years, firing on data sources that no longer exist, referencing fields that changed in a log format update, or duplicating logic already captured by a different rule. Without coverage mapping, nobody knows which adversary techniques the organization can actually detect and which ones would succeed silently.
Detection engineering addresses all three problems by treating detection as a product that requires design, quality assurance, and ongoing maintenance. Rules degrade without active ownership, and a program built from undisciplined, ungoverned logic eventually creates more operational burden than security capability.
The discipline draws from software engineering practices not because security teams want more process overhead, but because the problems detection rules face, such as correctness, maintainability, drift, and scale. Exactly the problems software engineering developed systematic answers to. Tests verify behavior. Version control provides history. Peer review catches logic errors before they reach production. Ownership models prevent rules from going stale without a responsible party noticing.
The detection engineering lifecycle
Detection programs that produce reliable output operate a consistent lifecycle that moves each detection from hypothesis through validated deployment and into ongoing maintenance.
The process begins with hypothesis formation. A detection engineer identifies an adversary technique, draws from threat intelligence or the MITRE ATT&CK framework, and forms a testable hypothesis, such as if an attacker executes this technique against this environment, what log events, behavioral patterns, or configuration states should become observable? Hypothesis quality determines everything downstream. A vague hypothesis (i.e., "detect suspicious PowerShell activity") produces a detection that catches too much or too little. A well-scoped hypothesis (i.e., "detect encoded PowerShell executions that spawn child processes making outbound connections to destinations not seen in the prior 30 days") produces a rule with defined scope and testable conditions.
Logic development follows. The engineer translates the hypothesis into detection logic, a query against available log sources, with field-level conditions, thresholds, or time windows that distinguish the targeted behavior from benign baseline activity. This stage involves selecting the detection type most suited to the hypothesis, mapping field names to the specific schema of the relevant log sources, and deciding what severity and routing logic the resulting alert should carry.
Testing comes next. At minimum, a detection should be validated against synthetic data matching the hypothesis (does it fire when the target behavior is present?) and verified not to produce excessive results on representative benign traffic. Stronger programs also run the detection through purple team exercises, where the hypothesized behavior is actually executed in a controlled environment to confirm the rule fires as expected and produce the right alert context for investigation.
Once testing passes, the detection is deployed with documented metadata, including the MITRE ATT&CK technique IDs it maps to, the log source dependencies it requires, the severity it carries, the analyst runbook it routes to, and the last validation date. Documented metadata is what makes detection inventories manageable as the number of deployed rules grows into the hundreds.
The lifecycle does not end at deployment. Detections require ongoing monitoring for false positive rate, tuning when the environment changes, and eventual retirement when a technique is no longer relevant or is superseded by more precise logic. Teams that skip this stage end up with detection inventories that grow in volume without improving in fidelity.
Detection logic types
Detection engineers work with several distinct logic patterns, each suited to different adversary behaviors and log source characteristics. Understanding when to apply each type is one of the core competencies of the discipline.
When a threat intelligence feed surfaces a malicious command-line string or a confirmed infrastructure indicator, signature detection is how that intelligence becomes an alert. Signatures match specific indicators exactly (a file hash, a command pattern, a network destination) and are precise about what they know while entirely blind to everything else. Once an attacker rotates infrastructure or modifies a command string, the detection stops firing. Coverage that relies heavily on signatures reflects yesterday's campaign rather than today's attacker, which is why continuous freshness from threat intelligence feeds is the operating requirement, not an enhancement.
Threshold detection fires when an observable metric crosses a defined boundary, such as more than five failed authentication attempts in ten minutes, or more than fifty S3 GetObject API calls in a single minute from an IAM (Identity and Access Management) role that normally performs zero. Thresholds work well against brute force, credential stuffing, and data staging behaviors, but they require baselining against normal operational patterns to set meaningful cutoffs. Thresholds calibrated on lab data rather than production traffic typically produce unacceptable false positive rates.
Sequence detection looks for a chain of events in a defined order within a time window. A detection targeting persistence via scheduled task creation (T1053.005) might look for a process creation event spawning a specific binary, followed within sixty seconds by a scheduled task registration event, followed by an outbound connection to a destination not seen in the prior seven days. The sequence encodes a tradecraft pattern rather than a single indicator, making it harder to evade by changing one element of the attack.
Behavioral detection operates at the level of established patterns rather than individual events. Rather than matching a specific command or indicator, it characterizes what normal activity looks like for a given user, service account, or resource, and alerts when observed behavior diverges beyond a statistical threshold. Behavioral detection covers techniques that blend into legitimate activity, including internal reconnaissance under the Discovery tactic (TA0007) and credential abuse by valid accounts (T1078). It requires sufficient historical data to establish meaningful baselines and is susceptible to slow adversary acclimatization if those baselines adapt too aggressively.
Correlation detection aggregates signals from multiple sources and detection types to build a compound alert representing higher-confidence evidence than any individual signal would justify alone. A single failed authentication is noise. A failed login followed by a successful login from a different country, followed by enumeration of cloud storage permissions, followed by a large data-read API burst, is a correlated sequence that warrants immediate escalation. Correlation detections require log sources to be consistently normalized so that events from different systems can be joined on shared identifiers like user ID, IP address, or session token.
MITRE ATT&CK as a coverage framework
The MITRE ATT&CK framework provides the vocabulary and structure that detection engineers use to assess what their program covers and where the gaps are. ATT&CK organizes adversary behaviors into tactics, representing attacker objectives, and techniques, which represent the specific methods used to accomplish those objectives.
The fourteen ATT&CK tactics span the full attack lifecycle, which are Reconnaissance, Resource Development, Initial Access, Execution, Persistence, Privilege Escalation, Defense Evasion, Credential Access, Discovery, Lateral Movement, Collection, Command and Control, Exfiltration, and Impact. Mapping deployed detections against this structure reveals which phases of an attack the program can observe and which remain blind spots. Gaps against early-stage tactics like Initial Access and Persistence mean attackers can establish footholds without triggering alerts. Gaps against late-stage tactics like Exfiltration and Impact mean damage can be completed before the team responds.
Some tactic categories are systematically harder to cover than others. Defense Evasion (TA0005), which includes techniques like disabling logging, timestomping, and process injection, often requires behavioral detection because these techniques are specifically designed to evade signature and rule-based logic. Collection (TA0009) is difficult in cloud environments because legitimate data access and malicious data staging can look nearly identical without contextual enrichment beyond the access event itself. Understanding these structural challenges helps prioritize detection engineering investment toward the areas where gaps carry the highest consequence.
Tagging deployed detections with ATT&CK technique IDs in rule metadata serves a second purpose beyond coverage visualization. It gives analyst runbooks and triage workflows structured context about what adversary objective the alert represents. An alert tagged to Credential Access (TA0006) carries a different investigation priority and initial steps than one tagged to Exfiltration (TA0010), even if both arrive at the same severity level.
Common detection failure modes
Detection programs fail in predictable ways. Most trace back to skipping one of the lifecycle stages above or treating deployed rules as durable artifacts that don't require maintenance.
Alert fatigue is the most visible symptom of low-fidelity detection. When too many detections fire on benign activity, analysts stop trusting the queue. The practical effect is that true positives get missed because the analyst dismissed the incoming detections as noise due to high volume. High false positive rates are typically a testing failure, such as the detection was deployed without validation against representative benign traffic, or it was never tuned after the environment changed around it.
Coverage gaps are often invisible until a breach exposes them. Organizations that have invested heavily in endpoint telemetry but have limited detection logic for cloud control plane activity, identity providers, or SaaS application audit trails discover the gap when an attacker traverses those environments unobserved. Coverage gaps typically reflect historical bias toward environments that were easiest to instrument first, combined with the assumption that the threat model hasn't shifted. An assumption that requires active verification rather than passive confidence.
Detection decay happens when rules are deployed and then forgotten. Log schema changes break field references silently. New services are onboarded without corresponding detection updates. Rules that once fired reliably stop producing results, and nobody notices because the absence of alerts is interpreted as the absence of threats. Regular validation, confirming that deployed detections still fire against current log formats and still produce appropriate output volume against current baselines, is the discipline that prevents this.
Runbook absence is an underrated failure mode. A detection that fires with no corresponding guidance for the analyst who receives it creates inconsistent investigation outcomes. Different analysts apply different judgments to the same alert type, producing variable response quality and making it difficult to measure the detection's actual value or improve it systematically over time.
How AI is changing detection engineering
Traditional detection engineering is constrained by analyst time. Writing, testing, and validating each detection requires judgment, environmental familiarity, and sustained effort, which means most programs prioritize coverage for techniques most recently seen in incidents and let gaps persist elsewhere. AI-assisted approaches change several of those constraints by making detection development, tuning, and maintenance more continuous.
Coverage generation is faster when AI can suggest candidate detection logic for a broader set of ATT&CK techniques, generate alternative logic formulations for the same behavior, and flag when deployed rules have likely drifted in fidelity based on output volume anomalies. This does not replace hypothesis formation or validation judgment, but it compresses the time between identifying a coverage gap and having a deployable candidate to test.
AI also changes the economics of signal creation. In a traditional SIEM workflow, detection engineers must be conservative because every noisy rule lands directly in the analyst queue. When detections are automatically enriched, correlated, and triaged before they become analyst-facing alerts, teams can afford to be more expansive with what they instrument. A lower-confidence signal does not have to become a high-priority alert on its own. It can become one input into a broader triage pipeline that suppresses benign activity, elevates suspicious combinations of behavior, and preserves useful weak signals that would otherwise be discarded.
Behavioral detection specifically benefits from AI-native approaches. Building statistical baselines for user and entity behavior across cloud services, identity infrastructure, SaaS applications, and collaboration tools manually requires significant data engineering work. AI-native behavioral models can establish those baselines across more dimensions and update them continuously as the environment changes, rather than requiring periodic manual recalibration that creates windows of degraded accuracy.
This is where features like Exabot Detect illustrate the direction of the category without changing the underlying discipline. Exabot Detect is positioned as an AI detection engineer that learns normal behavior, flags abnormal activity, and maintains coverage across IaaS, SaaS, identity, code, and collaboration tools. It combines machine learning and rule-based detections, broader coverage for sources such as Google Workspace, Slack, GitHub, cloud, and identity systems, and explainable alert evidence that includes context and MITRE technique mapping.
The more important architectural shift is the multistage detection pipeline. Instead of treating every rule match as an alert, AI systems can ingest low-fidelity signals, correlate them across users, assets, and applications, enrich them with business context, and filter them into a smaller set of higher-confidence findings. Exaforce describes this as combining behavioral analysis, managed or custom detections, explainable evidence, and automatic triage that applies historical analysis, expert analysis, business context rules, and analyst feedback to reduce false positives over time.
An AI SOC architecture extends this further by making detection an ongoing operational function rather than a point-in-time configuration. Rather than relying only on static rules deployed at a fixed moment, AI-native detection continuously analyzes behavioral signals, adjusts for environmental drift, and routes findings through triage before analysts spend time on them. The result is not rule-free security, but a different operating model: broader detection coverage, more tolerance for experimental or lower-confidence signals, and less manual tuning required to keep the analyst queue usable.
Building a detection engineering program
Most security teams practice some version of detection engineering before they formalize it. The first step toward formalization is typically a coverage inventory, including documenting every deployed detection with its ATT&CK mapping, log source dependencies, last validation date, and current false positive rate. This baseline reveals where the debt has accumulated and gives the team a concrete starting point for prioritization.
From there, the highest-impact early investments are testing infrastructure and lifecycle governance. Testing infrastructure means having a way to validate new detections against controlled data before they reach production. Lifecycle governance means assigning explicit ownership to deployed detections and establishing a review cadence so that rules don't silently degrade without a responsible party.
Coverage prioritization should reflect the organization's specific threat model rather than what ATT&CK techniques are most discussed in industry reporting. A company with limited on-premises infrastructure and significant cloud exposure should prioritize coverage for cloud control plane activity, identity provider events, and SaaS API behaviors. Even if those environments are harder to instrument than Windows endpoints with mature EDR (Endpoint Detection and Response) telemetry. Threat model specificity is what makes programs effective rather than merely comprehensive-looking.
Programs that operate this discipline consistently tend to compound over time. Each new detection makes the team's collective understanding of adversary behavior more concrete and testable. Each tuning cycle reduces alert volume and improves analyst confidence in the queue. Each coverage gap closed raises the cost of attack for adversaries who rely on the organization's blind spots. The value of the program grows with investment in the process, not just in the rules.
Frequently asked questions
What is the difference between detection engineering and threat hunting?
Detection engineering produces persistent detection logic that runs continuously and generates alerts when conditions are met. Threat hunting is a proactive, time-bounded investigation that looks for adversary activity not yet captured by existing detections. The two are complementary. Threat hunts frequently uncover new adversary behaviors or coverage gaps that feed back into detection engineering as new rule candidates, creating a feedback loop between investigation and persistent detection improvement.
What log sources do detection engineers typically work with?
Detection engineers work with endpoint telemetry (process creation, network connections, file system events), cloud provider audit logs (AWS CloudTrail, Azure Activity Log, GCP Cloud Audit Logs), identity provider logs (authentication events, MFA failures, directory changes, role assignments), SaaS application audit trails, network flow data, and email security events. Coverage breadth across these source types is often the primary constraint on detection program maturity, particularly for organizations that have instrumented endpoints well but have limited visibility into cloud control plane and identity provider events.
How does MITRE ATT&CK fit into a detection engineering program?
ATT&CK provides the vocabulary and structure for coverage planning and gap analysis. Detection engineers use it to map which adversary techniques have corresponding detection logic, identify where coverage is absent or partial, and prioritize new rule development. Each deployed rule is typically tagged with ATT&CK technique IDs, which enables coverage visualization across the matrix and gives analysts a structured triage context about what adversary objective a firing alert represents.
What makes a detection high fidelity?
A high-fidelity detection fires reliably when the behavior it targets is present, produces a low false positive rate against representative benign traffic, and provides enough alert context for an analyst to reach a confident disposition quickly. Fidelity is an empirical measure tracked through true and false positive rates in production over time, not a property assessable from logic alone. Detections never validated against production baselines have unknown fidelity regardless of how carefully they were written.
How do you prevent detection decay?
Detection decay (rules that stop producing accurate output because the environment changed around them) is prevented through lifecycle governance, which includes regular validation checks confirming that deployed detections still fire against current log schemas and still produce appropriate output volume against current baselines. The practical implementation is a scheduled review cadence (quarterly at a minimum for actively used rules) and automated monitoring for rule output anomalies, such as a detection whose daily alert count drops to zero after a log format change.
What skills does a detection engineer need?
Detection engineers typically need strong familiarity with attacker techniques (usually demonstrated through ATT&CK coverage work), proficiency in the query languages their environment uses for detection (SPL, KQL, YARA-L, Sigma, or others), solid knowledge of the log sources and data schemas they instrument, and the software engineering habits, version control, testing, peer review, that make detection programs maintainable at scale. Deep knowledge of the specific environments being defended (cloud, SaaS, identity, endpoint) typically matters more than depth in any single query language.
