AGENTIC-SYSTEMSMay 27, 20261 min read10 views

JobBench: Aligning Agent Work With Human Will

Researchers have introduced JobBench, a new benchmark that evaluates AI agents based on tasks humans actually want to delegate rather than purely economic replacement value.

Computer Science > Artificial Intelligence

Title:JobBench: Aligning Agent Work With Human Will

View PDF HTML (experimental)Abstract:Current benchmarks for occupational AI agents are scoped primarily by economic values, telling a replacement story. We introduce JobBench, which evaluates AI agents on the workflows that experts identify as high-priority for delegation, empowering humans based on their needs instead of replacing them with GDP value. JobBench covers 130 agentic tasks across 35 occupations. Each task is packaged as a workspace of heterogeneous reference files, requiring the agent to reason through the cluttered information streams of real professional work. Outputs are graded by a fact-anchored chain of rubrics, averaging 35.6 binary criteria per task. We evaluate 36 models; the strongest, Claude Opus~4.7 under Claude Code, reaches only 45.9 %. We hope JobBench shifts the community's target labour-market effect from replacement to enhancement: building agents that do what humans actually want delegated, not only what is most economically valuable.

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

RegNetAgents: A Multi-Agent Framework for Cross-Network Regulatory Driver Identification in Cancer Genomics

Researchers have introduced RegNetAgents, an AI-oriented multi-agent framework designed to identify regulatory driver candidates across heterogeneous gene networks in cancer genomics. By integrating bulk tumor and single-cell data, the system offers a powerful tool for discovering therapeutic targets in breast and colorectal cancers.

agentic-systems

IMEX Interaction-Based Model Explanation

The IMEX (Interaction-Based Model Explanation) framework offers a novel approach to explainable AI by identifying key features and complex, higher-order interactions within black-box predictive models.

agentic-systems

DialogueVPR: Towards Conversational Visual Place Recognition

Researchers have introduced DialogueVPR, a novel paradigm that casts visual place recognition as an interactive, dialogue-driven reasoning process. By coupling a cross-modal retriever with an intelligent questioner called DQ-pilot, this framework effectively resolves ambiguous real-world descriptions and significantly outperforms traditional static retrieval methods.

agentic-systems

Intelligent Three Level Learning Architecture for Autonomous UAV Swarms in Search and Rescue

Researchers have proposed a novel three-level hierarchical learning architecture for autonomous UAV swarms in search and rescue operations, mimicking biological reflexes, skills, and reasoning.

agentic-systems

ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capability

Researchers introduce ToolAnchor, a novel framework that helps LLM agents seamlessly adapt to new tools by injecting counterfactual anchor contexts, effectively overcoming the challenge of behavioral inertia.

agentic-systems

When a Verified World Model Still Loses: Play-Adequacy vs Prediction-Accuracy in LLM-Synthesized Code World Models

A new study reveals that LLM-synthesized Code World Models can achieve near-perfect prediction accuracy yet fail systematically in actual play, exposing a critical 'verified-vs-correct' gap in planning-oriented AI.