Making Failure Safe: A Constrained, Verifiable Agent Framework for Open-Web Data Collection

Researchers have proposed a new constrained, verifiable agent framework that shifts LLM output from free-form code to typed JSON configurations, addressing common web scraping errors. This approach minimizes operational costs by using zero LLM tokens during execution while ensuring high reusability.

Computer Science > Artificial Intelligence

Title:Making Failure Safe: A Constrained, Verifiable Agent Framework for Open-Web Data Collection

View PDF HTML (experimental)Abstract:LLMs and agents can generate web scrapers from natural-language requirements, but direct generation remains unreliable because of dependency errors, broken selectors, schema mismatches, and heterogeneous page structures. We propose a constrained, verifiable agent framework that shifts LLM output from free-form code to typed JSON collector configurations, combining a six-type collector taxonomy, template and utility-function constraints, static Airflow DAG execution, rule-based quality checking, and structured feedback correction. Experiments on 138 tasks show that the taxonomy supports description-based requirement typing, while confirming that stable instantiation requires completing source, field, and execution constraints beyond the initial description. On 80 independently source-verified tasks, the framework runs with zero execution-stage LLM tokens and the lowest average wall-clock time, trading moderate one-shot quality for a reusable, deterministic, and verifiable execution path suited to repeated scheduled collection. These results position the framework as a reusable, low-cost, and verifiable execution path for repeated open-web data collection.

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

Mnemosyne: Agentic Transaction Processing for Validating and Repairing AI-generated Workflows

Researchers introduce Mnemosyne, an open-source runtime utilizing Agentic Transaction Processing (ATP) to validate and repair AI-generated workflows, ensuring system correctness and safety against untrusted proposals from Large Language Models (LLMs).

agentic-systems

Constructive Alignment: Governing Preference Dynamics in Human-AI Interaction

Researchers propose 'Constructive Alignment', a new paradigm that reframes AI alignment as managing evolving human preference trajectories rather than satisfying static desires.

agentic-systems

HARC: Coupling Harmfulness and Refusal Directions for Robust Safety Alignment

Researchers have introduced HARC, a new fine-tuning method that enhances the safety of Large Language Models (LLMs). By coupling "harmfulness" and "refusal" directions within the model's internal representations, HARC effectively prevents jailbreak attacks without degrading general performance.

agentic-systems

Solution space path planning for supporting en-route air traffic control

Researchers have developed a novel solution-space path-planning algorithm designed to support en-route air traffic controllers by aligning with human decision logic. The algorithm achieves conflict-free path generation in just 3.69 milliseconds, significantly improving computational efficiency and operational safety.

agentic-systems

AGI Maze as a Benchmark Framework for World-Modeling Agents

A new research paper introduces AGI Maze, a benchmark framework designed to evaluate how AI agents build and manipulate internal world models. Initial evaluations show that even powerful LLMs struggle to solve simple mazes that humans can easily navigate.

agentic-systems

AI Native Games: A Survey and Roadmap

A new research paper defines 'AI-native games' where generative AI is core to the gameplay loop, analyzing 53 projects to map out a development roadmap for this emerging sector.

EXPLORE TOPICS