[Project] 9. pi-multi-agent — Trading Cheap Model Clusters for Premium Output

The Core Proposition

Can 3-4 fast, cheap models working as a team outperform 1 expensive, slow model?

The answer is yes — and it’s not just intuition. A growing body of research from 2024–2025 backs this up. pi-multi-agent is the engineering realization of this idea: break a big task into pieces, assign each to an agent with a distinct perspective, let them debate, collide, and cross-review. The collective output consistently exceeds what any single model produces alone.

Research Background: Why Multi-Agent Works

Google: Chain-of-Agents Framework

Google Research presented the Chain-of-Agents (CoA) framework at NeurIPS 2024. Worker agents process chunks of long context sequentially, passing aggregated information forward. The result: +10% over single-model baselines on QA, summarization, and code completion, with advantages widening to ~100% when input exceeds 400K tokens. Critically, CoA is training-free — the performance gain comes purely from the collaboration structure.

Clinical Medicine: Multi-Agent at Scale

A 2025 PubMed paper tested clinical workloads (retrieval, extraction, dosing calculations). Single-agent accuracy collapsed from 73.1% to 16.6% as batch size increased from 5 to 80. The multi-agent system held at 90.6% → 65.3% — still degraded, but with an enormous absolute advantage. Multi-agent also consumed 65× fewer tokens and kept latency growth under control.

Code Optimization: Small Models Beat Large Models

NeurIPS 2025’s “Lessons Learned” framework had multiple small LLMs form a team with a knowledge-sharing mechanism — successes and failures are deposited into an experience bank, which other agents query and reuse. A team of small models outperformed a single large model on code optimization tasks.

A Counterpoint: Strong Single-Agent Prompts May Suffice

An ACL 2024 paper offers an important caveat: with sufficient few-shot demonstrations, a well-prompted single agent approaches multi-agent discussion performance on many reasoning tasks. Multi-agent advantage is most pronounced when demonstrations are absent, tasks demand diverse skills, or contexts are very long.

This “but” matters — multi-agent isn’t a silver bullet. The scenario has to be right: large goals, multiple dimensions, long contexts, and the need for cross-validation.

pi-multi-agent’s Design Philosophy

Based on this research, pi-multi-agent centers on one question: how do you make multiple cheap models work as a team, producing output that rivals or exceeds a premium model?

Architecture Overview

Figure 1 — pi-multi-agent architecture: Orchestrator + Agent Cluster + Dashboard

Two Packages

Dashboard has zero intrusion into core — core is completely unaware of dashboard’s existence. Want a different UI? Uninstall the dashboard package.

Design Tradeoffs: Key Decisions We Bled For

The Orchestrator Cannot Read, Write, or Execute Bash

This is the single most important tradeoff, arrived at after repeated painful trial and error.

Imagine the orchestrator has read access. It reads some code, thinks “let me just fix this real quick,” and calls write. Then it wants to verify — calls bash to run tests. Tests fail — read again, write again, run again. Three cycles, and the context balloons from 2K to 20K tokens. The orchestrator’s own “brain” explodes first.

This lesson was paid for with real token costs during pi-orchestrator development. The conclusion: the orchestrator only schedules, never touches code. Reading, writing, and running commands are all sub-agent responsibilities. Each sub-agent operates in its own isolated session — contexts never cross-contaminate.

Two direct benefits:

• Orchestrator context stays tiny (a few hundred to 1-2K tokens). Scheduling decisions are fast and cheap.
• Sub-agent context growth has a ceiling — compact, clear, and stop provide precise control.

Token-Saving Patterns Inherited from pi-orchestrator

pi-multi-agent preserves several critical token-saving mechanisms from pi-orchestrator:

• Async dispatch (the only mode): Every agent_command returns a handle immediately. The orchestrator never blocks. Sub-agents run in the background, and results arrive as summaries injected into the next turn’s systemPrompt. No polling, no waiting.
• Summary delivery, not full output: Completed tasks deliver only a summary (task ID + status + first 500 characters). The orchestrator decides whether to call agent_logs for full details. This prevents “a coder finished and its entire output just nuked the orchestrator’s context.”
• Blackboard forwarding: Agents share data through a blackboard. agent_forward transmits only key/value pairs — O(1) transfer cost, not O(output size).

Sub-Agents Never Talk Directly to Each Other

The first reaction to this design is usually: “Wait, every message goes through the LLM orchestrator? That sounds expensive.”

Flip it around. What if sub-agents notified each other directly? Agent-A finishes a task and pushes its full output (potentially thousands of tokens) to Agent-B. Agent-B is mid-task, and its context explodes from the interruption. If 5 agents broadcast to each other — O(N²) token explosion.

Routing everything through the orchestrator produces three layers of benefit:

Layer 1: The orchestrator acts as a pre-filter. It reads the sub-agent’s summary, then decides: “Does this result need to be forwarded? To whom? What exactly?” Most intermediate results get “noted, no action needed.” Only genuinely valuable results are agent_forward-ed to the blackboard. Net token consumption is actually lower.

Layer 2: The AI naturally “pauses to ask you.” When the orchestrator receives a sub-agent’s return, it often pauses with a question: “coder-1 finished the database design — want to review before we proceed?” This is the LLM’s natural confirmation-seeking tendency. Without new triggers, this pause is a dead end — the user wouldn’t see it until they next open pi. But because another agent returns shortly after, the new return triggers a fresh turn. The orchestrator gets “woken up,” having both paused to show intermediate state AND auto-continued. A natural async rhythm emerges.

Layer 3: Orchestration itself produces intelligence. When the LLM reads sub-agent summaries and decides next actions, it’s performing meta-level reasoning. Sub-agents focus on “how to do it”; the orchestrator focuses on “who does what, in what order, should we switch personnel.” The scheduling decision itself requires global understanding — and that understanding process is an additional source of intelligence.

The Cluster Pattern’s Sweet Spot

This pattern is not for simple Q&A. You don’t need three agents in a meeting to answer “what’s 1+1”. The cluster pattern shines when:

A concrete, large-scope goal is set, and the cluster thinks and processes it together. For example:

• “Refactor this module, ensuring architectural compliance”
• “Security-audit this codebase against OWASP Top 10”
• “Design an API, then implement, test, and document it”

The flow: Architecture Committee (2 analyzers debate the approach) → Engineering Team (2 coders implement in parallel + cross-review) → QA Team (testing + architecture compliance → sign-off) → Merge → Cleanup.

Dispatching agents with conflicting personas is where this pattern delivers its core value. You can give three analyzers radically different stances:

• “You’re an aggressive refactoring advocate — split everything that can be split”
• “You’re a conservative — don’t refactor unless absolutely necessary”
• “You’re a performance absolutist — ignore architecture, only look at benchmark numbers”

They debate in the blackboard. The orchestrator synthesizes all three perspectives for a decision. Three cheap models, each attacking from a different angle, collectively produce answers that regularly beat a single expensive model — because there are no blind spots.

Repository

github.com/GOODDAYDAY/pi-multi-agent

References

• Chain of Agents: Large Language Models Collaborating on Long-Context Tasks (Google Research, NeurIPS 2024)
• Two Heads Are Better Than One: Test-time Scaling of Multi-agent Collaborative Reasoning (NeurIPS 2025)
• Scaling LLM-based Multi-Agent Collaboration (MacNet, arXiv 2024)
• Orchestrated Multi Agents Sustain Accuracy Under Clinical-Scale Workloads (PubMed, 2025)
• Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key? (ACL 2024)
• Lessons Learned: A Multi-Agent Framework for Code LLMs (NeurIPS 2025)