We built SWARM to measure emergent failures in multi-agent AI systems—the kind of risks that don't exist in single agents but emerge when they interact. What we found was counterintuitive.
We ran 55 scenarios with varying agent compositions. The setup was simple: honest agents, opportunistic agents, deceptive agents. We measured toxicity, quality gap, collusion, and governance effectiveness under different levers.
The Purity Paradox: populations with only 10% honest agents achieved 74% higher welfare than 100% honest populations. Full honesty created fragility. Heterogeneity created competitive pressure that improved outcomes.
Why? Honest agents don't learn to protect themselves. They don't innovate defensive strategies. They get exploited, collapse, and drag the system with them. A population with 10% adversaries forces the honest majority to develop robustness. The system becomes harder to break.
The dangerous zone: 37.5-50% adversarial fraction. At that point, the system phase-transitions to collapse regardless of governance intervention. But governance works well at lower adversarial densities—circuit breakers, reputation decay, transaction taxes can all maintain stability.
SWARM is open-source, reproducible, and ships with 55 prebuilt scenarios, 2922 tests, and a Colab notebook that runs in 60 seconds. You can also run governance red-teams against your own agents.
The framework is built for empirical safety research: measure interaction-level risks before deployment.
Dynamic graph structures for multi-agent collab—agents as nodes on data chunks, edges as comms. Crushes long-context (2K tokens vs. 128K) and hits 89% on MMLU-Pro with fewer agents than fixed teams. Model-agnostic, emergent smarts from graph topology. ICML paper, no hype, pure efficiency hack.
Paper: arXiv:2509.21848
ICML: openreview.net/forum?id=34cANdsHKV
We ran 55 scenarios with varying agent compositions. The setup was simple: honest agents, opportunistic agents, deceptive agents. We measured toxicity, quality gap, collusion, and governance effectiveness under different levers.
The Purity Paradox: populations with only 10% honest agents achieved 74% higher welfare than 100% honest populations. Full honesty created fragility. Heterogeneity created competitive pressure that improved outcomes.
Why? Honest agents don't learn to protect themselves. They don't innovate defensive strategies. They get exploited, collapse, and drag the system with them. A population with 10% adversaries forces the honest majority to develop robustness. The system becomes harder to break.
The dangerous zone: 37.5-50% adversarial fraction. At that point, the system phase-transitions to collapse regardless of governance intervention. But governance works well at lower adversarial densities—circuit breakers, reputation decay, transaction taxes can all maintain stability.
SWARM is open-source, reproducible, and ships with 55 prebuilt scenarios, 2922 tests, and a Colab notebook that runs in 60 seconds. You can also run governance red-teams against your own agents.
The framework is built for empirical safety research: measure interaction-level risks before deployment.
GitHub: https://github.com/swarm-ai-safety/swarm Docs: https://swarm-ai.org/ Quick start: https://colab.research.google.com/github/swarm-ai-safety/swa...
Happy to answer questions about the methodology, governance mechanisms, or how to run your own scenarios.