AGENTIC-SYSTEMSJuly 16, 20261 min read1 views

LAPO: Leave-One-Turn Attribution for Self-Generated Process Rewards in Multi-Turn Search Reasoning

Researchers have proposed LAPO, a self-generated process-supervision method that improves multi-turn search reasoning in reinforcement learning. By evaluating individual search turns through a leave-one-turn attribution mechanism, LAPO significantly boosts accuracy without requiring external reward models or LLM judges.

Computer Science > Artificial Intelligence

Title:LAPO: Leave-One-Turn Attribution for Self-Generated Process Rewards in Multi-Turn Search Reasoning

View PDF HTML (experimental)Abstract:Reinforcement learning for multi-turn search reasoning typically relies on terminal outcome rewards, which cannot distinguish useful, redundant, and harmful intermediate interactions. We propose LAPO, a self-generated process-supervision method based on backward leave-one-turn attribution. For each search turn, LAPO replaces the turn and its retrieval observation with a fixed [DELETE] placeholder and measures the resulting change in the current policy's mean log-likelihood of the gold answer. This Answer-Likelihood Gain estimates the turn's contribution while preserving all downstream interactions, allowing early evidence to be evaluated in the complete reasoning context. LAPO further applies sign-consistency gating, retaining only normalized process advantages whose directions agree with their raw attribution scores. The method requires no additional reward model, teacher, verifier, or LLM-as-a-Judge. Across seven knowledge-intensive question-answering datasets with local retrieval, LAPO achieves an average exact-match score of 0.326, outperforming the strongest step-reward baseline, IGPO, by 0.053. Ablations show complementary benefits from backward attribution and sign-consistency gating, demonstrating that policy-derived retrospective attribution can provide effective process supervision for multi-turn search agents.

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

Multi-Agent Collaborative Reasoning with Tool-Augmented Evidence for Urban Region Profiling

Researchers have introduced UrbanAgent, a novel agentic framework that revolutionizes urban region profiling. By leveraging multi-agent collaborative reasoning and tool-augmented external knowledge retrieval, UrbanAgent outperforms traditional models in estimating population, GDP, and carbon emissions.

agentic-systems

AI advice suppresses people's willingness to say "I don't know", even when the advice is wrong and accuracy is incentivized

A new study reveals that access to AI assistants nearly eliminates people's willingness to say "I don't know," even when the AI's advice is incorrect. Despite a significant drop in accuracy, users' confidence nearly doubled, suggesting AI alters our metacognitive threshold for decision-making.

agentic-systems

CayleyR: Solving the TopSpin puzzle via cycle intersection

Researchers have introduced cayleyR, an R package designed to solve complex permutation puzzles like TopSpin. By combining Cayley graph theory, bidirectional search, and Vulkan GPU acceleration, this tool opens new avenues for combinatorial optimization.

agentic-systems

Probabilistic Extension of Neuro-Symbolic AGI Robots based on Belnap's Typed Intensional FOL

Researchers have expanded the cognitive power of neuro-symbolic AGI robots by integrating probability computation based on Belnap's logic and Nilsson's probability structure, enabling real-time decision-making under uncertainty.

agentic-systems

OriginBlame: Record- and Token-Level Data Provenance for AI Training Datasets

Researchers have introduced OriginBlame, a record- and token-level data provenance system for AI training datasets. It precisely identifies data to be removed upon author request, eliminating catastrophic over-deletion and improving machine unlearning efficiency by 42%.

agentic-systems

AI-Native Insurance for Agentic AI: Pricing, Underwriting, and End-to-End Automation

The rise of Agentic AI, capable of autonomous decision-making and environmental interaction, introduces unprecedented risk and liability challenges. A new research paper proposes an AI-native mathematical framework for underwriting, pricing, and end-to-end automation of insurance contracts tailored for autonomous AI deployments.