AI in the Situation Room: What Is AI’s Risk to Presidential Judgment?
Artificial intelligence could help U.S. leaders sort intelligence during fast-moving crises, but it may also reinforce their existing assumptions. A warning about that risk follows a reported error in an AI-assisted assessment of a Chinese vessel that nearly contributed to a military confrontation.
Artificial intelligence could soon help U.S. presidents assess intelligence in the Situation Room, raising questions about what is AI’s role in high-stakes decisions. The technology can sort large volumes of information quickly, but it may also reinforce leaders’ existing biases, according to a crisis decision-making expert.
A recent reported incident illustrates the stakes. CNN reported that a military analyst used an AI chatbot to prepare an assessment of a Chinese vessel and wrongly concluded it was carrying nuclear materials. American aircraft were reportedly airborne to intercept the ship before the error was caught.
The incident prompted debate about how an AI-generated mistake could move through the chain of command and gain the authority of vetted intelligence. The expert warned that AI could give leaders’ assumptions the appearance of objective analysis.
AI can reinforce assumptions
AI systems can process more intelligence inputs than a human team, detect patterns and anomalies, and organize complex information quickly. In a national-security crisis, that speed could help officials handle large amounts of information, including conflicting reports, when time is short.
But the systems are built by people and trained on data produced by people. The questions users ask also reflect their beliefs, the expert wrote, meaning AI’s output is not automatically objective.
That creates a risk of confirmation bias: the tendency to interpret evidence in ways that support existing beliefs and discount information that challenges them. Selective attention can also lead decision-makers to focus on vivid details while overlooking others.
Presidents and advisers must weigh uncertainty, not just reach a single answer. The expert argued that turning competing judgments into a probability can conceal which indicators officials considered credible and which they disregarded.
Judgment beyond a probability
The essay points to the night of May 1, 2011, when President Barack Obama gathered advisers to assess intelligence indicating that Osama Bin Laden was in a compound in Abbottabad, Pakistan. If the intelligence was correct, Obama would send in American troops.
The advisers, described as trusted and experienced, were looking at substantially the same data. Their differing assessments formed part of the judgment the president had to make.
The article’s warning is that AI-generated analysis could make those disagreements harder to see if it presents a complex assessment as a clear-cut result. The incident involving the Chinese vessel shows how an erroneous conclusion could acquire weight as it moves through decision-making channels.
