This article originally appeared in Cybersecurity Dive.
The data-centric foundation for modern threat hunting
In cybersecurity, we’re told that AI is the future of threat hunting. But the dirty secret is that most AI is operating with one hand tied behind its back. Researchers have argued that AI models are only as good as their data pipeline. That principle doesn’t stop at academic machine learning. It applies just as powerfully to cybersecurity. Threat hunting powered by AI, automation, or human investigation will only ever be as effective as the data infrastructure it stands on.
Too often, security teams focus on building AI on top of existing data lakes or tuning new detection models without addressing the more fundamental issue: the data itself. When telemetry is siloed across disconnected systems, such as endpoint, cloud, identity, SaaS, and code repositories, analysts are left to piece together context from fragments. Throwing all of the data into the same platform, without proper transformation, can overwhelm both humans and AI. Even the most advanced algorithms cannot overcome incomplete or inconsistent data. AI that learns or operates on poor inputs will always draw poor conclusions. And human-driven, AI-augmented threat hunting is no different.
Why unified data matters
A unified and correlated data platform changes the game. Bringing all data into one place reduces noise and makes it possible to see patterns that individual systems obscure. Pre-transforming and correlating this information also makes it more usable by large language models and other AI-driven tools. Rather than wasting compute power and tokens trying to make sense of structure or context, which often leads to poor results when the context is wrong or too large, the AI can instead focus on understanding real behaviors.
Unified data also allows connected identities to emerge naturally. A single user might appear as an IAM principal in AWS, a committer in GitHub, and a document owner in Google Workspace, all with completely different names. Look at any one of those signals, and you have only a sliver of truth. Look at them together, and you have behavioral clarity. Downloading dozens of files from Google Workspace might seem suspicious in isolation, but if that same identity also creates a public S3 bucket minutes later and clones dozens of repositories to a personal laptop, the activity becomes clearly malicious.
Threat hunting through correlation
When data from logs, configurations, code repositories, and identity systems all live in one place, correlations that once took hours or weren’t even possible become immediate. Lateral movement that relies on stolen short-lived credentials, for example, often crosses several systems before detection. A compromised developer laptop might assume multiple IAM roles, spin up new instances, and reach internal databases. Endpoint logs reveal the local compromise, but without IAM and network data, there’s no way to prove the scope of the intrusion.
Similarly, an attacker using a compromised GitHub Action token to create a shadow admin account in the cloud would go unnoticed without connecting CI/CD logs to configuration and identity changes. And when a third-party app with overbroad OAuth scopes exfiltrates data through a compromised user account, only unified SaaS access logs and OAuth consent histories can reveal the true vector.
These are not abstract hypotheticals. The Salesloft/Drift breach showed how attackers initially gained access via a compromised GitHub account and then obtained OAuth tokens in Drift’s AWS environment, which they used to access hundreds of connected customer environments through the trusted Drift-to-Salesforce integration. Each platform’s logs likely appeared normal until forensic teams correlated activity across GitHub, identity, and cloud environments.
Fidelity and determinism
The quality of your data pipeline directly determines the fidelity of your threat hunting. If done right, the right data pipeline reduces duplication and therefore costs without sacrificing fidelity. AI-driven systems depend on that fidelity to produce deterministic answers instead of probabilistic guesses. Improving data quality has a greater impact on AI performance than any architectural tweak. The same holds for detection and response.
Threat hunting is fundamentally about asking precise questions and getting reliable answers. Without a connected, high-fidelity data foundation, every query is incomplete. A modern security architecture must prioritize clarity over volume, ensuring that both humans and machines operate from a single, accurate source of truth.
Strategic storage and AI readiness
Your threat hunting platform should also be strategic about what data lives in hot versus cold storage. Not every log, trace, or event needs to be instantly queryable. The key is ensuring that high-value telemetry of identity changes, cloud configurations, and source control activity is readily accessible, while historical or low-signal data can be tiered for deeper forensic use. The smarter your storage strategy, the faster your analysts and models can respond without wasting compute or cost on irrelevant noise.
When your data is all in one place, it’s also inherently more ready for LLM use cases. A robust data pipeline is a form of effective context engineering. As engineers at Anthropic have shown, the best AI outcomes come from platforms that feed the right data, at the right time, with the right context, but not too much. Giving a model a well-structured and relevant set of information allows it to focus on reasoning through a problem, rather than drowning in unnecessary detail or being starved of critical facts. It’s the same for humans: even the best analysts lose effectiveness when overwhelmed with noise or starved of context. When your data pipeline is designed for contextual precision, your AI threat hunting can truly scale.
Turning insight into advantage
When adversaries are moving faster than ever, the organizations that win are those that can see across their environments in real time. Building an AI-ready data platform for threat hunting isn’t just about detection speed; it’s about transforming uncertainty into understanding. Unified data means unified vision, and unified vision is the foundation of proactive defense. When the data engine is tuned for fidelity, scale, and AI readiness, your threat hunting becomes sharper, faster, and more precise.
