When Orchestration Moves Inside the Service

When Orchestration Moves Inside the Service

Posted
Sep 22, 2026 12:35 PM
Type
Essay

OpenAI says it used around 10,000 concurrent agents to produce its resolution of the Navier-Stokes problem.

Ten thousand.

Most of the discussion I saw focused on the mathematics. Is the proof correct? How important is it? What does it mean for mathematicians? What happened around the controversy?

But that is not what I got the chills from.

Underneath the proof was an 88-hour operation involving a frontier model, roughly 10,000 agents, millions of interactions, competing lines of attack, dynamic allocation of compute, and another 17 hours of formalization and verification.

This was not one brilliant model call.

It was a massive industrial-scale deliberation machine built on top of a frontier model.

I saw very little discussion about that. And I think it may be the more important signal.

People are staring at the theorem. OpenAI is showing us how intelligence will be delivered.

The proof is the output. The machine is the story.

OpenAI is not merely investing in smarter models. It is investing in systems that organize frontier models into temporary problem-solving institutions.

Navier-Stokes was not the first clue.

The published prompt for OpenAI's claimed proof of the Cycle Double Cover Conjecture gave the root system access to as many as 64 concurrent agents. More importantly, it did not prescribe a fixed workflow. It told the orchestrator to "use multiagent v2 aggressively and dynamically."

Begin with genuinely different approaches. Keep most agents unaware of the currently favored route so they do not converge too early. Reopen blocked approaches only when a genuinely new mechanism appears. Launch new rounds. Redirect effort. Preserve independence. Verify.

Then the scale jumped from 64 agents to roughly 10,000.

These are different problems and different systems, but the engineering pattern is difficult to miss. When OpenAI applies frontier capability to problems that resist a single reasoning trajectory, it reaches for massive, dynamic multi-agent orchestration.

The proof is the artifact everyone sees.

The orchestration is the machine that produced it.

Training the model is only one lever

When each new frontier model is extraordinarily expensive to train, better orchestration is the obvious way to extract more capability from the model you already have.

The normal story of AI progress is a story about weights: more data, larger training runs, better architectures, a new model generation.

That remains true. But it is only one path.

A lab can also improve the system between model generations by learning how to spend inference better. How many branches should this problem receive? Which approaches should remain isolated? When should the orchestrator challenge convergence? Which agents need tools? Which findings deserve independent verification? When is another million tokens worth buying?

Ten thousand agents are not cheap. But changing the runtime organization around a model is faster and more flexible than training the next frontier model from scratch. It turns capability into a function of two things: the intelligence of the model and the quality of the machine assembled around it.

This reframed my own experiment. One frontier Fable pass found more distinct planted flaws than twenty Opus calls pooled. The details and caveats are in the experiment addendum. My first reaction was that orchestration might be losing its value.

Not so fast.

OpenAI's behavior suggests the opposite. Orchestration is becoming so valuable that the labs will not leave it as a craft practiced manually by users. They will absorb it into the service, train models to do it, and improve it alongside the models themselves.

Prompting and orchestration are not becoming obsolete because they do not work.

They are becoming obsolete as human-facing disciplines because the service will do them for us.

Inside the model is only half the story

There are two very different ways for orchestration to move "inside": into the model's reasoning, and into the service that surrounds the model.

The paper Reasoning Models Generate Societies of Thought is relevant to the first path. Its authors found that reasoning-optimized models produce traces with more questioning, perspective shifts, conflict, reconciliation, and apparent diversity of expertise than instruction-tuned models. In one experiment, steering a conversational feature increased verification and backtracking while roughly doubling accuracy on a bounded reasoning task.

That suggests the model may already simulate something like multiple perspectives inside one reasoning trajectory. One call can contain an internal argument.

But this is not the same as the service-level shift.

At the service level, a root model receives a task and constructs an external organization around itself. It decides whether to launch two agents or two thousand. It writes their prompts. It partitions the evidence. It chooses which agents can see one another's work. It notices convergence and sends new agents elsewhere. It creates tools, runs code, commissions simulations, allocates more compute to promising routes, closes dead ends, and assembles the answer.

The user does not design that topology. The model does, ad hoc, for the problem in front of it.

This is not orchestration disappearing into the weights. It is orchestration becoming a learned behavior of the product.

The labs can train the orchestrator

The next low-hanging fruit may be training models not only to solve problems, but to design the temporary institutions that solve them.

Consider what the labs now possess after large multi-agent runs. They have records of which decompositions worked. Which agents found rare ideas. Which groups converged too early. Which handoffs destroyed useful information. Which verification steps caught errors. Which additional calls changed the answer, and which only burned tokens.

That is training data.

A root model can be rewarded not only for the final answer, but for the orchestration that produced it. Did it know when to branch? Did it create genuinely different searches or merely ten copies of the same one? Did it preserve a dissenting result? Did it allocate compute where uncertainty was highest? Did it stop when another round had no expected value? Did an independent verifier reproduce the critical claim?

Over time, the model does not need a human to prescribe the harness. It learns a policy for building one.

This creates a faster improvement loop alongside frontier training. The base model gets stronger when a new generation arrives. The service gets stronger between generations by learning how to spend inference better. Each improvement in the model also improves the orchestrator, which can then coordinate more capable sub-agents, which produces better trajectories for training the next orchestrator.

The loop compounds.

Prompt engineering becomes machine language

The human prompt shrinks as the machine-generated prompting behind it expands.

Today, extracting serious work from a frontier model can require pages of instructions. Define the methodology. Specify the roles. Demand adversarial passes. Protect independence. Add verification. Tell the model when to use tools, when to branch, and when to stop.

That is a temporary interface.

No mainstream product will require an executive to know that a strategy question needs four isolated scenario agents, a source auditor, a red team, a negative-space pass, and a synthesis procedure that preserves minority findings. The executive will ask one question. The service will infer the rest.

Prompting does not cease to exist. Models prompt other models. Orchestration does not cease to exist. Models orchestrate other models. Both become implementation details, like memory management or query optimization: still essential, increasingly invisible.

This is why I now think the current market for elaborate prompts, skills, and hand-built orchestrations has a short half-life. They matter today because the capability is present but the interface has not caught up. The labs will close that gap.

Robert Martin's harness is the small-scale version of the same story. He built an explicit system around agents. The agents improved. The explicit system became overhead. Carry that curve forward, and the logical endpoint is not a better universal harness built by a person.

It is a model that builds the right temporary harness for every task, then throws it away.

What still has to remain outside?

Once orchestration becomes invisible, the durable external layer is no longer capability. It is assurance.

A self-orchestrating service can search more broadly than most hand-built systems. It can also hide more. We may see the final answer without knowing which alternatives were seriously explored, whether the agents were meaningfully independent, what evidence was excluded, or which dissenting result disappeared during synthesis.

External structure still matters where it creates guarantees the service cannot simply claim about itself: provenance, reproducibility, protected dissent, model-family diversity, information boundaries, independent verification, and human responsibility for the verdict.

That changes the job of computational deliberation. The goal is not to build a cleverer swarm than the frontier labs. They are already showing what happens when enormous compute meets dynamic orchestration.

The goal is to build the constitutional layer around the swarm.

Prompting is moving inside the service. Orchestration is moving inside the service. Soon, the harness itself may be generated for a single question and vanish when the answer is complete.

So the open question is no longer who can build the best harness.

It is this: when the harness builds itself, what must we still require it to prove?