AGENTIC-SYSTEMSJune 1, 20261 min read7 views

BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs

A new study introduces BilliardPhys-Bench, a benchmark evaluating the physical reasoning and visual dynamics of multimodal LLMs. Testing on GPT, Claude, Gemini, and Qwen reveals that top AI models still struggle with complex physical movements and exhibit a consistent 'stasis bias'.

Computer Science > Artificial Intelligence

Title:BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs

View PDF HTML (experimental)Abstract:Current multimodal models handle static image recognition well, but intuitive physical reasoning remains a weakness. Predicting how objects will move and interact from a single image is still difficult for these systems. We present BilliardPhys-Bench, a benchmark for physical reasoning in synthetic billiards environments. Its procedural engine generates randomized scenarios with friction and elastic collisions. The benchmark tests three abilities: (1) predicting ball-to-ball collisions, (2) reasoning about wall bounces, and (3) estimating final ball positions after motion stops. We evaluate recent MLLMs from the GPT, Claude, Gemini, and Qwen families. Performance drops as simulation time increases and scene geometry grows more complex. We also observe a consistent failure mode we call "stasis bias": when the correct physical outcome is harder to infer, models tend to predict no interaction. These findings show where current MLLMs break down on visual dynamics and point toward the need for better physical inductive biases in multimodal architectures.

Current browse context:

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

HG-RAG: Hierarchy-Guided Retrieval-Augmented Generation for Structured Knowledge Graphs

HG-RAG (Hierarchy-Guided Retrieval-Augmented Generation) is a novel framework designed to overcome the limitations of traditional flat RAG systems by traversing hierarchical knowledge graphs, significantly improving multi-hop reasoning and reducing hallucinations in LLMs.

agentic-systems

Enhancing Small Language Models Reasoning through Knowledge Graph Grounding

Researchers have proposed a neuro-symbolic framework that enhances the reasoning capabilities of Small Language Models (SLMs) like Gemma 3 and Llama 3.2 using Knowledge Graphs, achieving up to a 2x performance boost.

agentic-systems

Orchestrating Power Grid Studies with Multi-Agent AI and MCP Servers

This paper explores how Agentic AI and Model Context Protocol (MCP) can support power-grid studies by integrating LLMs with numerical simulation tools under human supervision.

agentic-systems

Human AI Construction of Bayesian Networks for Operational Decision Support -- A Virtual Survey Approach

Researchers propose a new methodology using Large Language Models and AI agents to construct Bayesian Belief Networks, bridging the gap between expert opinion and data-driven learning. The framework is demonstrated through a healthcare decision-making model, revealing key insights into customer behavior.

agentic-systems

Interpretable Language Model for Closed-Loop Type 1 Diabetes Control

Researchers have developed LLM-T1D, a novel approach combining reinforcement learning and large language models to automate insulin delivery for Type 1 Diabetes. The system not only outperforms traditional methods but also explains its decisions in plain language, addressing the 'black-box' challenge in medical AI.

agentic-systems

RegNetAgents: A Multi-Agent Framework for Cross-Network Regulatory Driver Identification in Cancer Genomics

Researchers have introduced RegNetAgents, an AI-oriented multi-agent framework designed to identify regulatory driver candidates across heterogeneous gene networks in cancer genomics. By integrating bulk tumor and single-cell data, the system offers a powerful tool for discovering therapeutic targets in breast and colorectal cancers.