Separating Signal from Noise in the AI Era

How Security Intelligence Has Evolved

The fundamental challenge facing the intelligence community has shifted. Throughout most of history, intelligence was about looking for valuable and scarce signals when too little information was available. Today, it is still about detecting signals, but also separating signals from the deluge of noise, including misinformation and disinformation, much of it generated by AI.

This shift means that intelligence tradecraft needs to change, too. As the core premise of intelligence has evolved from scarce to excessive information, the challenge for the intelligence community is to adapt research and analytical techniques used to collect, structure, process and analyse information.

Over the past few years, this challenge has triggered innovation in applying data analytics and AI technologies to develop new tools for intelligence analysts. It has also opened a broader debate about the optimal model for security intelligence practice: Should it be tech- or human-led? How well can tech tools and AI models replace human analysts?

We firmly stand on the human-led, tech-enabled side of that debate. Not only because our values are rooted in the importance of human touch, but because we believe that the highest-quality intelligence, actionable and forward-looking insight that enables organizations to mitigate security risk, can only be crafted through expert judgement, deep contextualisation, and lateral and creative thinking that remains the sole domain of humans (at least for now).

Below, we outline how human analysts can use AI tools to become faster and more thorough in the collection and processing of information, without diluting confidence in their assessments and forecasts. In practice, this means embracing AI intelligence tools where they add value while keeping humans in charge of the final call.

AI solution to an AI problem?

At the heart of this challenge is a question about the human capacity to efficiently process large volumes of information. The development and widespread use of generative AI models since 2022 have produced too much information for humans to process effectively. AI-generated online content will continue to grow exponentially, and it will soon be impossible to sift, filter and analyse it or reach meaningful conclusions as an intelligence analyst.

AI tools seem like the perfect solution to the overwhelming information problem.

• They have impressive processing power and can ingest, analyse and summarize huge quantities of data in a short time.
• Their processing power and training rules enable them to detect patterns in large and diverse data sets.
• In this respect, AI tools have a distinct advantage over human analysts whose ability to absorb and analyse such vast quantities of data is very limited.
• There is, therefore, obvious appeal in the instant summaries and analytics produced by AI tools.
• That most AI tools used by intelligence teams can also draft concise and persuasive reports only fuels further adoption across teams and organisations.
• Increasingly, organizations are also trialing AI intelligence platforms to speed internal workflows.

However, there are good reasons to question claims that AI tools and agents can fully replace human analysts. Many arguments about the use of AI tools in intelligence, examining their limitations and weaknesses, have already been well rehearsed. We have previously written about the risks related to data pollution, unchecked biases and hallucinations.

These criticisms are all still relevant, although they are subject to growing AI governance and information verification efforts to mitigate them, among users and providers of AI tools alike. However, in the debate about the optimal intelligence model there is a more fundamental critique of AI tools: Does large data processing power always (or even often) lead to valid findings and forecasts?

The Enduring Value of Judgement

Intelligence goes beyond spotting patterns in large datasets or detecting events and incidents in real time. The core value of intelligence is judgement, balancing the weight of diverse evidence to derive considered assessments about risks and their future trajectories. When exercising judgement, patterns matter; but they cannot capture the full complexity of volatile and unpredictable events or the contingencies of individual decisions.

Intelligence almost always comes with residual uncertainty and a qualification of the confidence level in the analysis. This is not a shortcoming to be eliminated, but a mechanism through which intelligence analysts resist oversimplification of complex phenomena.

That is why it is essential that intelligence models remain human-led. The high processing and analytical power of AI models comes with an implicit promise to equip the analyst with certainty and full confidence about their assessments and forecasts. But without a layer of human judgement, the validity of the analysis and ultimately its actionability to the end user is undermined.

Below are three examples of how combining human expertise with AI tools can lead to better intelligence, tackling three key challenges of AI intelligence tools: over-reliance on quantitative questions; predictive analysis extrapolation; and the calibration of risk implications.

1. First, AI models are very good at answering only some intelligence questions. When it comes to spotting patterns or anomalies, AI is incomparably better than humans. Applying AI tools to assist with the detection of cybercrime or organized crime patterns makes perfect sense for security intelligence teams. However, when it comes to the nuances of political events, diplomatic disputes and leadership decisions, where so much hinges on the context and contingent human factors, analyst judgement remains superior and AI is best deployed in combination with the expertise of the human analyst. In a human-led intelligence model, the analyst applies AI tools selectively, where the strengths of AI tools can improve the analysis.
   
   
2. Another example is extrapolating from historical data to forecast future outcomes. This is not a new challenge for analysts,[1] but the widespread use of AI tools makes it more pertinent than ever. As AI models and tools analyse vast quantities of data about past events, the risk of extrapolation increases. While past precedents are essential in intelligence analysis and risk assessment, their explanatory value is anchored in the thorough understanding of the context and the nuances of the threat environment. Without human judgement and creative, contextual thinking, it will be difficult to identify the next 'tipping point' in a steadily escalating trend; or account for so-called 'black swan' events, for which limited or no precedent exists.
   
   
3. Finally, human input remains essential in calibrating risk models, weighing and balancing risk factors and indicators based on their importance for different users. In assessing risks or forecasting scenarios, the scope and severity of implications vary according to the vulnerability, mitigation and risk appetite specific to the organization. Sometimes, indicators for which very few data points exist can be disproportionately impactful to a small set of organizations. Risk models and risk rating frameworks need to reflect that exposure. Creative and lateral thinking, together with deep understanding of an organization's risk posture and strategy, remain key human strengths.

The Benefits of Adopting a Hybrid Model

In summary, despite the impressive data processing power of AI models and tools - and their ability to detect patterns and anomalies - a human-led hybrid model is at present likely to deliver better intelligence to organizations.

While AI tools are likely to continue to improve and help analyst teams work faster and more efficiently, they cannot yet fully replace human creativity, expert judgement and contextual knowledge, all key ingredients of good intelligence work. Even if AI models' processing power grows to match the expanding volume of AI-generated noise online, the true signal may ultimately lie elsewhere.

1. King and Zeng. "The Dangers of Extreme Counterfactuals" Political Analysis (2006) 14:131--159.