# [Project] 9. pi-multi-agent — Trading Cheap Model Clusters for Premium Output ## The Core Proposition **Can 3-4 fast, cheap models working as a team outperform 1 expensive, slow model?** The answer is yes — and it's not just intuition. A growing body of research from 2024–2025 backs this up. pi-multi-agent is the engineering realization of this idea: break a big task into pieces, assign each to an agent with a distinct perspective, let them debate, collide, and cross-review. The collective output consistently exceeds what any single model produces alone. ## Research Background: Why Multi-Agent Works ### Google: Chain-of-Agents Framework Google Research presented the Chain-of-Agents (CoA) framework at NeurIPS 2024. Worker agents process chunks of long context sequentially, passing aggregated information forward. The result: **+10% over single-model baselines** on QA, summarization, and code completion, with advantages widening to **~100% when input exceeds 400K tokens**. Critically, CoA is training-free — the performance gain comes purely from the collaboration structure. ### Clinical Medicine: Multi-Agent at Scale A 2025 PubMed paper tested clinical workloads (retrieval, extraction, dosing calculations). Single-agent accuracy collapsed from 73.1% to 16.6% as batch size increased from 5 to 80. The multi-agent system held at 90.6% → 65.3% — still degraded, but with an enormous absolute advantage. Multi-agent also consumed **65× fewer tokens** and kept latency growth under control. ### Code Optimization: Small Models Beat Large Models NeurIPS 2025's "Lessons Learned" framework had multiple small LLMs form a team with a knowledge-sharing mechanism — successes and failures are deposited into an experience bank, which other agents query and reuse. **A team of small models outperformed a single large model** on code optimization tasks. ### A Counterpoint: Strong Single-Agent Prompts May Suffice An ACL 2024 paper offers an important caveat: with sufficient few-shot demonstrations, a well-prompted single agent approaches multi-agent discussion performance on many reasoning tasks. Multi-agent advantage is most pronounced when **demonstrations are absent, tasks demand diverse skills, or contexts are very long**. This "but" matters — multi-agent isn't a silver bullet. The scenario has to be right: **large goals, multiple dimensions, long contexts, and the need for cross-validation**. ## pi-multi-agent's Design Philosophy Based on this research, pi-multi-agent centers on one question: **how do you make multiple cheap models work as a team, producing output that rivals or exceeds a premium model?** ### Architecture Overview pi-multi-agent architecture **Figure 1 — pi-multi-agent architecture: Orchestrator + Agent Cluster + Dashboard** ### Two Packages | Package | Role | |---|---| | `pi-multi-agent-core` | Orchestration engine — 17 tools, agent lifecycle, event persistence, workflow templates | | `pi-multi-agent-dashboard` | Dashboard — real-time Web monitoring + TUI overlay, depends on core solely through public-api | Dashboard has zero intrusion into core — core is completely unaware of dashboard's existence. Want a different UI? Uninstall the dashboard package. ## Design Tradeoffs: Key Decisions We Bled For ### The Orchestrator Cannot Read, Write, or Execute Bash This is the single most important tradeoff, arrived at after repeated painful trial and error. Imagine the orchestrator has `read` access. It reads some code, thinks "let me just fix this real quick," and calls `write`. Then it wants to verify — calls `bash` to run tests. Tests fail — read again, write again, run again. **Three cycles, and the context balloons from 2K to 20K tokens. The orchestrator's own "brain" explodes first.** This lesson was paid for with real token costs during pi-orchestrator development. The conclusion: the orchestrator **only schedules, never touches code**. Reading, writing, and running commands are all sub-agent responsibilities. Each sub-agent operates in its own isolated session — contexts never cross-contaminate. Two direct benefits: - **Orchestrator context stays tiny** (a few hundred to 1-2K tokens). Scheduling decisions are fast and cheap. - **Sub-agent context growth has a ceiling** — `compact`, `clear`, and `stop` provide precise control. ### Token-Saving Patterns Inherited from pi-orchestrator pi-multi-agent preserves several critical token-saving mechanisms from pi-orchestrator: - **Async dispatch (the only mode)**: Every `agent_command` returns a handle immediately. The orchestrator never blocks. Sub-agents run in the background, and results arrive as summaries injected into the next turn's systemPrompt. No polling, no waiting. - **Summary delivery, not full output**: Completed tasks deliver only a summary (task ID + status + first 500 characters). The orchestrator decides whether to call `agent_logs` for full details. This prevents "a coder finished and its entire output just nuked the orchestrator's context." - **Blackboard forwarding**: Agents share data through a blackboard. `agent_forward` transmits only key/value pairs — O(1) transfer cost, not O(output size). ### Sub-Agents Never Talk Directly to Each Other The first reaction to this design is usually: "Wait, every message goes through the LLM orchestrator? That sounds expensive." Flip it around. What if sub-agents notified each other directly? Agent-A finishes a task and pushes its full output (potentially thousands of tokens) to Agent-B. Agent-B is mid-task, and its context explodes from the interruption. If 5 agents broadcast to each other — **O(N²) token explosion.** Routing everything through the orchestrator produces three layers of benefit: **Layer 1: The orchestrator acts as a pre-filter.** It reads the sub-agent's summary, then decides: "Does this result need to be forwarded? To whom? What exactly?" Most intermediate results get "noted, no action needed." Only genuinely valuable results are `agent_forward`-ed to the blackboard. **Net token consumption is actually lower.** **Layer 2: The AI naturally "pauses to ask you."** When the orchestrator receives a sub-agent's return, it often pauses with a question: "coder-1 finished the database design — want to review before we proceed?" This is the LLM's natural confirmation-seeking tendency. Without new triggers, this pause is a dead end — the user wouldn't see it until they next open pi. But because another agent returns shortly after, **the new return triggers a fresh turn**. The orchestrator gets "woken up," having both paused to show intermediate state AND auto-continued. A natural async rhythm emerges. **Layer 3: Orchestration itself produces intelligence.** When the LLM reads sub-agent summaries and decides next actions, it's performing meta-level reasoning. Sub-agents focus on "how to do it"; the orchestrator focuses on "who does what, in what order, should we switch personnel." The scheduling decision itself requires global understanding — and that understanding process is an additional source of intelligence. ### The Cluster Pattern's Sweet Spot This pattern is **not for simple Q&A**. You don't need three agents in a meeting to answer "what's 1+1". The cluster pattern shines when: **A concrete, large-scope goal is set, and the cluster thinks and processes it together.** For example: - "Refactor this module, ensuring architectural compliance" - "Security-audit this codebase against OWASP Top 10" - "Design an API, then implement, test, and document it" The flow: Architecture Committee (2 analyzers debate the approach) → Engineering Team (2 coders implement in parallel + cross-review) → QA Team (testing + architecture compliance → sign-off) → Merge → Cleanup. **Dispatching agents with conflicting personas is where this pattern delivers its core value.** You can give three analyzers radically different stances: - "You're an aggressive refactoring advocate — split everything that can be split" - "You're a conservative — don't refactor unless absolutely necessary" - "You're a performance absolutist — ignore architecture, only look at benchmark numbers" They debate in the blackboard. The orchestrator synthesizes all three perspectives for a decision. **Three cheap models, each attacking from a different angle, collectively produce answers that regularly beat a single expensive model — because there are no blind spots.** ## Repository [github.com/GOODDAYDAY/pi-multi-agent](https://github.com/GOODDAYDAY/pi-multi-agent) ## References - Chain of Agents: Large Language Models Collaborating on Long-Context Tasks (Google Research, NeurIPS 2024) - Two Heads Are Better Than One: Test-time Scaling of Multi-agent Collaborative Reasoning (NeurIPS 2025) - Scaling LLM-based Multi-Agent Collaboration (MacNet, arXiv 2024) - Orchestrated Multi Agents Sustain Accuracy Under Clinical-Scale Workloads (PubMed, 2025) - Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key? (ACL 2024) - Lessons Learned: A Multi-Agent Framework for Code LLMs (NeurIPS 2025)