AGENTIC-SYSTEMSJuly 20, 20261 min read

SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction

Researchers have introduced SeerGuard, a proactive safety framework that prevents mobile GUI agents from executing risky actions by predicting consequences beforehand using a Safety-Augmented World Model (SAWM).

Computer Science > Artificial Intelligence

Title:SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction

View PDF HTML (experimental)Abstract:Mobile graphical user interface (GUI) agents have demonstrated remarkable capabilities in automating complex tasks, yet they introduce critical safety risks where a single erroneous action can lead to irreversible consequences. Existing safety mechanisms are primarily reactive, lacking the ability to assess risks before execution. In this paper, we introduce SeerGuard, a consequence-aware safety framework designed to mitigate these risks through pre-execution instruction-level screening and action-level risk assessment. Specifically, the action-level assessment analyzes agent-proposed actions within current GUI states, anticipating likely outcomes to identify risks before they are executed. To enable these capabilities, we construct a unified safety-augmented world model (SAWM) via multi-task learning, integrating semantic next-state prediction with safety risk assessment. Extensive experiments demonstrate that SeerGuard generalizes effectively across diverse mobile GUI agents. On Qwen3-VL-8B-Instruct, it increases the safety-utility score from $0.191$ to $0.596$ at $\omega=0.8$ and reduces the risk-cost score from $0.347$ to $0.130$ at $\alpha=0.8$. Further analyses on our SAWM validate the effectiveness of the instruction-level screening, alongside the capability of action risk assessment and next-state prediction.

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

S1-Omni: A Unified Multimodal Reasoning Model for Scientific Understanding, Prediction, and Generation

Researchers have introduced S1-Omni, a unified multimodal reasoning model designed for AI for Science (AI4S). Outperforming advanced models like GPT-5.5 and Gemini-3.1-Pro on various benchmarks, S1-Omni addresses the fragmentation in modeling complex scientific data.

agentic-systems

NeurOWL: An LLM-Based Neural-symbolic Framework for Incomplete OWL Ontology Reasoning

Researchers have introduced NeurOWL, a novel LLM-based neuro-symbolic framework designed to address reasoning challenges in incomplete OWL ontologies. By unifying subsumption verification and ontology abduction, NeurOWL demonstrates robust performance across multiple domains.

agentic-systems

ToolVerse: Unlocking Massive Environments and Long-Horizon Tasks for Agentic Reinforcement Learning

Researchers introduce ToolVerse, a comprehensive framework designed to scale up agentic reinforcement learning environments and enable LLM agents to perform complex, long-horizon reasoning tasks using thousands of real-world tools.

NOW LET US Related – Neuro-Symbolic AI for LEED compliance: Document-Centric Benchmarking, Deterministic Numeric Checking, and When Multimodal Hurts

agentic-systems

Neuro-Symbolic AI for LEED compliance: Document-Centric Benchmarking, Deterministic Numeric Checking, and When Multimodal Hurts

A new study introduces a local neuro-symbolic AI pipeline to automate LEED v4.1 green building certification screening. The findings reveal that the small 4-billion-parameter Gemma 3 model outperforms larger models, while incorporating multimodal drawing images consistently degrades performance.

NOW LET US Related – MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion

agentic-systems

MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion

Researchers have proposed MGDT, a novel framework for Multimodal Knowledge Graph Completion (MKGC) that utilizes an align-then-diffuse paradigm. By integrating a frozen MLLM and a Relation-Adaptive Mixture-of-Experts module, MGDT significantly outperforms existing baselines.

agentic-systems

DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings

Researchers have introduced DrawingVQA, the first benchmark designed to evaluate multimodal large language models (MLLMs) on real-world construction drawings. Evaluation results reveal a significant performance gap between state-of-the-art AI models and human experts when dealing with these complex engineering documents.