A million-dollar decision was sitting in front of me, the evidence was incomplete, and I could feel myself doing exactly what people do under pressure: trusting my gut, skipping the hard parts, and calling the result “judgment.”
I knew rigorous methods for making decisions like this existed. I also knew I would not assemble an expert panel, hire a red team, or spend several weeks building and testing a formal model before making this decision.
We do not lack methods for thinking better. We lack the practical capacity to use them at the level of rigor their designers intended.
Rigorous thinking is just too expensive - not only in money, but in time, staffing, discipline, and ego.
Consider Assumption-Based Planning (ABP). RAND developed the methodology for institutions making consequential decisions under deep uncertainty—situations where the future cannot be predicted cleanly and a hidden assumption can derail an entire strategy. It asks planners to identify what their plan quietly assumes about the future, determine which assumptions are both load-bearing and vulnerable, define signs that those assumptions are beginning to fail, and prepare actions before they do.
This was not management theater invented for an off-site. It came from an institution built around some of the world’s most sophisticated analysts, working on decisions for governments and militaries where the costs of being wrong could be enormous. That is precisely why its failure mode matters: even very smart people, facing very serious consequences, can design a rigorous method whose demands exceed what human teams can reliably sustain.
In a review of James Dewar’s Assumption-Based Planning, a 47-page planning document used the word “will” 470 times. Each “will” smuggled in a claim about the future: a condition the plan required, but had not necessarily earned [1][2]. If one short plan can contain hundreds of candidate assumptions, the bottleneck is not intelligence. It is the scale of attention required to apply the method properly.
But imagine doing that properly. Not finding five assumptions for a workshop slide. Finding hundreds, sorting them, challenging them, deciding which could break the plan, and designing signals to monitor them. The method is simple to describe. The work is enormous.
Rigor has a price - and it’s too high for almost everyone
The real cost of a decision method is the amount of human attention required to resist the shortcuts the method was invented to prevent.
For decades, researchers in forecasting, intelligence analysis, engineering, and management have built procedures for making uncertainty visible. The Delphi method, developed at RAND, gathers anonymous judgments from experts, returns the group’s aggregate view, and repeats the process so participants can revise their answers without rank or personality dominating the room.[3] the CIA’s Analysis of Competing Hypotheses (ACH) puts hypotheses across one side of a matrix and evidence down the other, forcing an analyst to test each piece of evidence against every live explanation rather than collect support for a favorite.[4] Failure Mode, Effects, and Criticality Analysis systematically identifies potential failure modes, traces their effects, and classifies their criticality.[5]
These methods share a hidden demand: procedural completeness, iteration, and resistance to premature stopping. Delphi needs repeated rounds. ACH needs a credible set of alternatives and a full evidence-by-hypothesis comparison. FMECA needs the team to keep asking what could fail after the obvious answers are gone. Analytic Hierarchy Process requires pairwise judgments for (n) criteria before checking whether those judgments are mutually consistent.[6]
The arithmetic is merciless. Ten criteria mean 45 comparisons. Twenty mean 190. Add alternatives, stakeholders, new evidence, or another iteration, and the “framework” becomes a staffing plan.
So we adapt. Four Delphi rounds become two. The hypothesis set stops when the meeting ends. The FMEA reflects what the people in the room happen to remember. The assumptions exercise identifies the assumptions polite enough to write on a whiteboard. The method survives in name while its protection against omission is quietly removed.
“Expensive” means a lot more than money
The bill for rigorous deliberation gets paid in people, time, discipline, and status - not just money.
Staffing is not just the cost of analysts and researchers. A serious process may require subject-matter experts, facilitators, data gatherers, reviewers, decision-makers, and someone to maintain the record. Then someone has to recruit them, brief them, align their terminology, coordinate their calendars, manage disagreements, track versions, chase missing inputs, and keep the process moving. Before the first serious conclusion appears, an organization may already have created a small management problem simply to operate the method.
Rigor that arrives too late is as useless as no rigor at all. Some methods are slow by design: evidence has to be collected before it can be assessed, participants must complete one round before the next begins, and hypotheses, assumptions, and contradictions all have to cycle back through the process before a conclusion is reached. Even an organization that can afford the people, sustain the discipline, and accept the conclusions may find the method simply takes longer than the decision allows. By the time the analysis is finished, the window to act may have closed or the evidence gone stale.
Discipline is its own cost, even without pressure. These methods are rigorous because they refuse the stopping points that feel natural: another hypothesis after the obvious ones are listed, another failure mode after the team feels finished, another round after everyone believes the answer is clear. Sustaining that voluntarily is arduous enough. Add a real deadline, incomplete evidence, and tired participants, and it rarely survives contact with the deadline.
Then there is ego and social pressure. A good method may reveal that a leader’s favored strategy rests on a vulnerable assumption, that the strongest evidence supports a rival explanation, or that the most confident expert is inconsistent. But raising an alternative can look like disloyalty, revising a position like weakness, and delivering an unwelcome answer like failure. A leader can approve a rigorous process while quietly expecting validation, and a junior participant may see the flaw but decide that naming it is professionally unwise. This cost rarely appears in the project plan, yet it can determine the result: the financial cost of rigor can be approved in a budget; the ego cost has to be paid in the room.
More hidden costs stack on top of these: pulling scarce experts away from other work, transferring context to everyone involved, and building the psychological safety genuine dissent requires. These costs compound - more participants mean more coordination, more rigor means more time, and higher stakes mean more politics.
Many prominent methods come from governments, militaries, large engineering programs, and well-funded institutions because those organizations can sometimes absorb these costs. Most people and teams cannot. The methods are not conceptually beyond their reach, their operating requirements are.
What if rigor just became cheap?
AI matters here not because it can make decisions for us, but because it can perform much of the procedural labor that made disciplined deliberation unaffordable.
A language model can hold a long argument and its evidence in context. It can generate a first-pass hypothesis set, classify hundreds of candidate assumptions, populate a comparison matrix, search for missing counter-evidence, and return structured output to the next step in a workflow. Multiple calls can examine the same claim separately under different instructions. Tool access can bring in current, dated sources rather than leaving the exercise trapped inside the model’s training data.
This is part of what the Computational Deliberation Lab exists to investigate. The aim is not to announce that AI has solved deliberation or to force a model into the role of decision-maker. It is to take established methods apart, test which steps models can perform reliably, measure where they improve coverage and where they merely produce more plausible-looking text, and design workflows that make their failures visible. Where does AI genuinely save human labor? Where does it introduce correlated errors? Which judgments must remain human? What provenance, adversarial testing, and review are required before an output deserves trust?
The goal is to find ways to make AI work for us while being exact about where it does not. The limitation map matters as much as the capability map. A useful “deliberation calculator” must not only perform the method, it must reveal when its output is fragile, when it needs different evidence or independent scrutiny, and when the question must be handed back to a person.
None of this makes a model an expert panel. A generated persona is not a cardiologist, an intelligence analyst, or the person who will live with the consequences. Models might invent facts, collapse distinct viewpoints into the same statistical center, and might make a neat table look more certain than its inputs justify. But the choice is not between perfect human deliberation and synthetic imitation. In ordinary life, the alternative is usually no Delphi process, no competing-hypothesis matrix, no systematic assumption check, and no red team at all. The decision gets whatever analysis fits between the next two meetings.
That is the opening - and a proposition to test, not a result to assume. AI may be able to draft a broad, structured first pass cheaply enough for a human to inspect. That only holds if the workflow exposes provenance, tests coverage, and invites challenge. It can preserve the matrix while people debate the judgments inside it. It can repeat the procedure when new evidence arrives instead of preserving an obsolete conclusion because the workshop was expensive. It can make scrutiny abundant while leaving authority with the person or institution responsible for the choice.
The calculator does not remove the need for judgment - it changes what judgment gets to look at.
From methods to machines
The opportunity is to turn established deliberation procedures into computational services without pretending that computation settles the decision.
I want to be able to bring a consequential question to a machine and ask for more than an answer. Map the plausible hypotheses. Show me which evidence discriminates among them. Surface the plan’s load-bearing assumptions. Enumerate the failures the current team is least likely to remember. Build the strongest objection. Tell me where the conclusion changes if one judgment changes.
The blueprints already exist. RAND researchers, intelligence analysts, reliability engineers, philosophers of argument, and decision scientists spent decades designing them. Their work tells us where intuition fails and what a disciplined process should force into view. What they did not have was a reasoning layer with a much lower marginal cost for drafting and procedural repetition than equivalent human labor - one that could execute dozens of narrow steps, preserve their outputs, and run the sequence again.
That does not mean every methodology should be automated as written. Some steps depend on genuine expertise. Some require representative participation rather than simulated voices. Some contain value judgments that should remain visibly human. Automating a method can also automate its blind spots - and hide them behind a polished interface.
The next phase is empirical: determine which parts of these methods AI models can perform reliably. Can they supply genuine cognitive diversity, reason at the required depth, gather trustworthy evidence, make the many small judgments expert analysis depends on, and combine independent outputs without collapsing them into false consensus? The opportunity is real only to the extent that methods designed for human teams survive that transfer - and we can identify exactly where they do not.
Sources
- “Assumption Based Planning, by James A. Dewar,” ProjectNet book review, pp. 22–23: https://cdn.ymaws.com/www.projectmanagement.org.za/resource/resmgr/bookreviews/janfeb05.pdf↩︎
- James A. Dewar, Assumption-Based Planning: A Tool for Reducing Avoidable Surprises (RAND/Cambridge University Press, 2002): https://www.rand.org/pubs/commercial_books/CB399.html↩︎
- RAND, Methodological Guidance for Conducting and Critically Appraising Delphi Panels (2024), section 2: https://www.rand.org/content/dam/rand/pubs/tools/TLA3000/TLA3082-1/RAND_TLA3082-1.pdf↩︎
- Richards J. Heuer Jr., Psychology of Intelligence Analysis, chapter 8 (CIA Center for the Study of Intelligence): https://www.cia.gov/resources/csi/static/9a5f1162fd0932c29bfed1c030edf4ae/Pyschology-of-Intelligence-Analysis.pdf↩︎
- U.S. Department of Defense, MIL-STD-1629A: Procedures for Performing a Failure Mode, Effects and Criticality Analysis (1980), sections 4.1–4.5: https://extapps.ksc.nasa.gov/Reliability/Documents/milstd1629_FMEA.pdf↩︎
- Thomas L. Saaty, “A Scaling Method for Priorities in Hierarchical Structures,” Journal of Mathematical Psychology 15 (1977), 234–281: https://doi.org/10.1016/0022-2496(77)90033-5↩︎