Multi-Agent System Testing
Description
Multi-agent system testing examines systems in which several AI agents act on shared tasks or resources, to find failures that no single agent shows on its own. Testers run the agents together in simulated or scripted scenarios, often with injected faults such as failed sensors, delayed messages or conflicting goals, and record the failures that appear. Checks cover whether the task is completed, whether coordination breaks down (for example through deadlock, collisions or one agent's error spreading to others), and whether stated safety rules are ever broken. For language-model agents, benchmarks such as MultiAgentBench score task completion and the quality of collaboration or competition. For small or abstract models, a model checker such as MCMAS can search every reachable state for a violation of a rule written in formal logic and, for many kinds of rule, return an example run that breaks it.
Example Use Cases
Safety
A logistics firm models its warehouse robots' movement rules in a model checker and asks whether two robots can ever occupy the same aisle cell or wait on each other forever. Any counterexample the checker returns is a concrete sequence of events that engineers replay in simulation before the control software goes live.
A trading firm's risk team runs all of its cooperating trading agents together in a market simulator replaying a high-volatility day. It checks whether their combined orders breach position limits or form patterns that compliance would treat as manipulation, and whether one agent's failure spreads to the others.
Reliability
A city traffic authority testing coordinated intersection controllers injects sensor failures and conflicting local objectives in a simulator. It checks whether emergency vehicles still get priority and traffic flow stays within agreed limits, and decides from the results whether the system can go to a pilot corridor.
A software company building a team of language-model agents to fix bugs (a planner, a coder and a reviewer) runs the team on a benchmark of shared coding tasks under different coordination layouts, such as a chain or a star. It compares task completion and coordination scores before choosing a layout for production.
Limitations
- The number of possible interactions grows explosively with the number of agents, so tests sample a tiny part of the space. A clean run shows that the scenarios tried did not fail, not that none will, and the gap widens as agents are added.
- Model checking needs a formal model of each agent and its environment, and the answer is only as good as that model. Large state spaces limit how many agents it can handle, and agents that learn and change behaviour are hard to capture, so it is usually combined with simulation testing.
- Benchmark scores for communication and planning come from a language model acting as judge on a coarse five-point scale, so they inherit its biases. A high coordination score does not show that a safety property holds.
- Simulators simplify real conditions such as communication delays, sensor noise and physical or social dynamics, and realistic simulation is costly to build and needs domain modelling expertise. Failures that occur only in the real environment will not appear.
- Tests and model checks find only the failures the testers specify. A hazard that no one wrote a safety rule or scenario for is not tested, and rules written in temporal logic need specialist skill to state correctly.
Resources
Research Papers
MCMAS: an open-source model checker for the verification of multi-agent systems
We present MCMAS, a model checker for the verification of multi-agent systems. MCMAS supports efficient symbolic techniques for the verification of multi-agent systems against specifications representing temporal, epistemic and strategic properties. We present the underlying semantics of the specification language supported and the algorithms implemented in MCMAS, including its fairness and counterexample generation features. We provide a detailed description of the implementation. We illustrate its use by discussing a number of examples and evaluate its performance by comparing it against other model checkers for multi-agent systems on a common case study.
MultiAgentBench : Evaluating the Collaboration and Competition of LLM agents
Large Language Models (LLMs) have shown remarkable capabilities as autonomous agents, yet existing benchmarks either focus on single-agent tasks or are confined to narrow domains, failing to capture the dynamics of multi-agent coordination and competition. In this paper, we introduce MultiAgentBench, a comprehensive benchmark designed to evaluate LLM-based multi-agent systems across diverse, interactive scenarios. Our framework measures not only task completion but also the quality of collaboration and competition using novel, milestone-based key performance indicators. Moreover, we evaluate various coordination protocols (including star, chain, tree, and graph topologies) and innovative strategies such as group discussion and cognitive planning. Notably, gpt-4o-mini reaches the average highest task score, graph structure performs the best among coordination protocols in the research scenario, and cognitive planning improves milestone achievement rates by 3%. Code and datasets are public available at https://github.com/MultiagentBench/MARBLE.