AGENTIC-SYSTEMSMay 21, 20261 min read10 views

Progressive Autonomy as Preference Learning: A Formalization of Trust Calibration for Agentic Tool Use

A new study formalizes trust calibration for AI agents using tools as a preference-learning problem. By leveraging Preferential Bayesian Optimization, the system dynamically classifies actions into allow, block, or ask regions based on human risk tolerance.

Computer Science > Artificial Intelligence

Title:Progressive Autonomy as Preference Learning: A Formalization of Trust Calibration for Agentic Tool Use

View PDF HTML (experimental)Abstract:We formalize trust calibration for agentic tool use (deciding when an automated agent's proposed action may execute autonomously versus require human approval) as a preference-learning problem. A policy gateway maintains a Gaussian-process posterior over a latent human risk-tolerance function, observed through a probit likelihood on binary approve/deny feedback, and escalates to the human exactly where the approval outcome is most uncertain. We show this is structurally an instance of Preferential Bayesian Optimization, inheriting its inference machinery (approximate Gaussian-process classification) and its sample-efficiency argument (uncertainty-targeted querying), while differing in objective: classifying an action space into allow/block/ask regions rather than optimizing a design.

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

RegNetAgents: A Multi-Agent Framework for Cross-Network Regulatory Driver Identification in Cancer Genomics

Researchers have introduced RegNetAgents, an AI-oriented multi-agent framework designed to identify regulatory driver candidates across heterogeneous gene networks in cancer genomics. By integrating bulk tumor and single-cell data, the system offers a powerful tool for discovering therapeutic targets in breast and colorectal cancers.

agentic-systems

IMEX Interaction-Based Model Explanation

The IMEX (Interaction-Based Model Explanation) framework offers a novel approach to explainable AI by identifying key features and complex, higher-order interactions within black-box predictive models.

agentic-systems

DialogueVPR: Towards Conversational Visual Place Recognition

Researchers have introduced DialogueVPR, a novel paradigm that casts visual place recognition as an interactive, dialogue-driven reasoning process. By coupling a cross-modal retriever with an intelligent questioner called DQ-pilot, this framework effectively resolves ambiguous real-world descriptions and significantly outperforms traditional static retrieval methods.

agentic-systems

Intelligent Three Level Learning Architecture for Autonomous UAV Swarms in Search and Rescue

Researchers have proposed a novel three-level hierarchical learning architecture for autonomous UAV swarms in search and rescue operations, mimicking biological reflexes, skills, and reasoning.

agentic-systems

ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capability

Researchers introduce ToolAnchor, a novel framework that helps LLM agents seamlessly adapt to new tools by injecting counterfactual anchor contexts, effectively overcoming the challenge of behavioral inertia.

agentic-systems

When a Verified World Model Still Loses: Play-Adequacy vs Prediction-Accuracy in LLM-Synthesized Code World Models

A new study reveals that LLM-synthesized Code World Models can achieve near-perfect prediction accuracy yet fail systematically in actual play, exposing a critical 'verified-vs-correct' gap in planning-oriented AI.