AGENTIC-SYSTEMSApril 27, 20261 min read8 views

Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents

A new study using the Superminds Test framework reveals that even with millions of AI agents, collective intelligence fails to emerge due to sparse and shallow interactions.

Computer Science > Artificial Intelligence

Title:Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents

View PDF HTML (experimental)Abstract:Collective intelligence refers to the ability of a group to achieve outcomes beyond what any individual member can accomplish alone. As large language model agents scale to populations of millions, a key question arises: Does collective intelligence emerge spontaneously from scale? We present the first empirical evaluation of this question in a large-scale autonomous agent society. Studying MoltBook, a platform hosting over two million agents, we introduce Superminds Test, a hierarchical framework that probes society-level intelligence using controlled Probing Agents across three tiers: joint reasoning, information synthesis, and basic interaction. Our experiments reveal a stark absence of collective intelligence. The society fails to outperform individual frontier models on complex reasoning tasks, rarely synthesizes distributed information, and often fails even trivial coordination tasks. Platform-wide analysis further shows that interactions remain shallow, with threads rarely extending beyond a single reply and most responses being generic or off-topic. These results suggest that collective intelligence does not emerge from scale alone. Instead, the dominant limitation of current agent societies is extremely sparse and shallow interaction, which prevents agents from exchanging information and building on each other's outputs.

Current browse context:

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

A Critical Analysis of Trustworthy AI Tools, Mark Frameworks, and the Implementation Chasms

A critical analysis of OECD data reveals significant gaps in operationalizing Trustworthy AI (TAI) frameworks, showing that current efforts are fragmented and heavily focused on post-development stages while neglecting early design and sustainability.

agentic-systems

Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve ARC-AGI-3?

A new study evaluates whether coding agents need executable world models, simplification, and verification to solve the challenging ARC-AGI-3 benchmark, revealing that a complete verification treatment combined with advanced GPT models can fully solve the public dataset.

agentic-systems

Precise but Uncoupled: Reviewer Precision Does Not Guarantee Critique Uptake in Multi-Agent Math Reasoning

A new study reveals that in multi-agent AI systems, a reviewer's ability to detect errors is empirically decoupled from the system's ability to adopt those critiques. Surprisingly, broadcast-style peer discussion outperforms hierarchical pipelines in complex math reasoning, despite the latter having more precise reviewers.

agentic-systems

AnovaX: A Local, Multi-Agent Voice Assistant with LLM Planning, Typed Executors, and Adaptive Recovery

AnovaX is a local-first desktop voice assistant that uses an LLM planner and a multi-agent orchestrator to execute complex tasks directly on the user's machine. It features adaptive recovery, specialized agent classes, and a remote control interface, demonstrating powerful local automation with minimal code.

agentic-systems

Cura 1T: Specialized Model for Agentic Healthcare

Researchers have introduced Cura 1T, a specialized large language model for healthcare trained through a human-gated self-evolution loop. The model excels at complex tasks ranging from patient consultation and multimodal clinical reasoning to utilizing electronic health records.

agentic-systems

GraphDx: A Cost-Aware Knowledge-Enhanced Multi-Agent Framework for Sequential Diagnosis

Researchers have introduced GraphDx, a novel multi-agent framework that balances diagnostic accuracy and resource costs in clinical settings. By combining knowledge graphs with LLMs, GraphDx improves diagnostic success rates up to 93% while cutting testing costs by up to 54%.