This closes a three-part series on running a swarm of AI agents. Part one argued that the brief is an interface and deserves a validator. Part two catalogued the ways a crowd fails while looking like it succeeded. Both of those are defense. This post covers why I keep doing it anyway.
The value shows up when several agents work the same question without any way to talk to each other. Each one takes a different facet. Each one works alone. If they come back and their findings point at the same underlying conclusion, that agreement means something, because nothing coordinated it. Six agents studying six different systems, each separately landing on the same principle, is worth far more than one agent asserting it. It is the mirror image of adversarial review. When you ask several critics to refute a claim and none of them can, your confidence should go up. When several blind investigators converge on the same answer, the same thing is happening. It is not proof. The agents share a lineage, and shared lineage means shared blind spots, so a crowd can absolutely converge on the same wrong answer. But it is a kind of evidence one agent alone cannot give you, and the shared-blind-spot problem is exactly why the last check in this post exists.
But there is a trap sitting right next to that signal, and it took me a while to see it.
The agents converge on the idea. They diverge on the words. Six agents will arrive at the same concept and give it six different names. One calls it a “transport layer,” another an “envelope,” a third a “wire format,” and they are all pointing at the same thing. If you do the lazy version of synthesis, collect the six summaries into one document and call it done, the result reads like six experts disagreeing. It is the opposite. They agree completely and describe it in six dialects.
So the synthesis step, the part where I read everything and write the one document that actually gets used, is the real work, not cleanup. The job is to pick one set of terms and translate everyone into it, and, more importantly, to figure out whether two agents are disagreeing about names or disagreeing about substance. Disagreements about names get merged. Disagreements about substance get kept as open questions, because averaging two genuinely different findings just manufactures a confident answer that is wrong in a brand new way.
And then one last check, the one I trust most. Once the crowd has converged and the reconciled answer is written up, I hand it to a different model entirely and ask it to challenge the conclusion. Not a seventh copy of the same model reading the same brief, that just repeats the same blind spots. A different model, from a different lineage, with the explicit job of attacking the result. More than once that challenge has taken a verdict the whole fleet agreed on, “this is clearly the right approach,” and narrowed it into something more honest, “this is right under these conditions and breaks outside them.” A good challenge does not only catch wrong answers. It puts walls around right ones.
The thread
Step back and the three posts make one argument.
The brief is a contract, so you build it like an interface and validate it before you spend a cent. The failures pass for success, so you assume the confident output is wrong until a check says otherwise, and you make the pipeline itself do the doubting instead of relying on your own vigilance. And trust is not something a swarm hands you, it is something you engineer, out of independent corroboration, careful reconciliation, and a challenger built by someone else.
Getting answers out of a swarm is the easy part. The entire job, the thing that separates a useful crowd of agents from an expensive way to generate plausible nonsense, is knowing which answers to believe. The skill is not running the swarm. Anybody can launch six agents. The skill lives in two places: checking what goes in, and doubting what comes out. That is the operator’s whole job, and you cannot automate it.