Orchestra-o1: Omnimodal Agent Orchestration

Orchestra-o1 is a breakthrough omnimodal agent orchestration framework designed to coordinate diverse inputs like text, image, audio, and video. Powered by the new DA-GRPO reinforcement learning approach, Orchestra-o1-8B achieves state-of-the-art performance among open-source omnimodal agents.

Computer Science > Artificial Intelligence

Title:Orchestra-o1: Omnimodal Agent Orchestration

View PDF HTML (experimental)Abstract:The recent success of agent swarms has shifted the paradigm of large language model (LLM)-based agents from single-agent workflows to multi-agent systems, highlighting the importance of agent orchestration for task decomposition and collaboration. However, existing orchestration frameworks are limited to a narrow set of modalities and struggle to generalize to more complex settings where heterogeneous modalities coexist and interact. This limitation becomes particularly pronounced in omnimodal scenarios, where tasks require the unified understanding and coordination of diverse inputs such as text, image, audio, and video. In this work, we propose Orchestra-o1, an omnimodal agent orchestration framework designed to support efficient agent collaboration across multiple modalities. Orchestra-o1 introduces a unified orchestration mechanism that enables modality-aware task decomposition, online sub-agent specialization, and parallel sub-task execution. This scalable design allows agent systems to effectively tackle complex real-world tasks involving heterogeneous information sources, surpassing the second-best approach by 10.3% accuracy on the OmniGAIA benchmark. Furthermore, we introduce decision-aligned group relative policy optimization (DA-GRPO), an efficient agentic reinforcement learning approach for training Orchestra-o1-8B, which also achieves state-of-the-art performance against all existing open-source omnimodal agents.

Source: arXiv cs.AI Recent

More in this category

NOW LET US Related – UP-NRPA: User Portrait based Nested Rollout Policy Adaptation for Planning with Large Language Models in Goal-oriented Dialogue Systems

agentic-systems

UP-NRPA: User Portrait based Nested Rollout Policy Adaptation for Planning with Large Language Models in Goal-oriented Dialogue Systems

Researchers have proposed UP-NRPA, a novel online framework that enables Large Language Models to dynamically adapt dialogue strategies based on real-time user portraits. The framework achieves a 100% success rate in multiple dialogue tasks and significantly improves negotiation outcomes without requiring offline reinforcement learning.

agentic-systems

History of the Muddy Children Puzzle

A recent study traces the two-century history of the "Muddy Children Puzzle", a classic problem that inspired the development of epistemic logic in AI. The paper also introduces unique variations and a novel self-referential puzzle.

agentic-systems

A Deep Reinforcement Learning (DRL)-Based Transformer Method for Solving the Open Shop Scheduling Problem

Researchers have developed a novel Transformer-based method integrated with Deep Reinforcement Learning (DRL) to solve the Open Shop Scheduling Problem (OSSP). This approach offers a highly scalable, learning-based alternative to traditional dispatching rules, generalizing effectively from small to large-scale industrial scenarios without retraining.

agentic-systems

TrajGenAgent: A Hierarchical LLM Agent for Human Mobility Trajectory Generation

Researchers have proposed TrajGenAgent, a hierarchical LLM-agent framework that generates realistic human mobility trajectories without model fine-tuning, addressing privacy and cost constraints in urban planning and epidemic control.

agentic-systems

From AGI to ASI

Over the last decade, building human-level artificial general intelligence has moved from far-fetched speculation to being a concrete next-decade target. This report investigates the transition from human-level AGI to artificial general superintelligence (ASI).

agentic-systems

Definitional alignment before capability alignment: a Design-Science framework for adjudicating claims about AGI

A new research paper proposes DAF-AGI, a design-science framework to resolve conflicting claims about the arrival of Artificial General Intelligence (AGI) by prioritizing definitional alignment over capability alignment.

EXPLORE TOPICS