AGENTIC-SYSTEMSJuly 24, 20261 min read

PlanE: Meta Planning of Data, Tuning, and Inference for Extractive-based LLMs

Researchers introduce PlanE, a novel framework featuring a Data-Tuning-Inference (DTI) planner designed to optimize data, tuning, and inference processes for extractive-based LLMs while reducing annotation costs.

Computer Science > Artificial Intelligence

Title:PlanE: Meta Planning of Data, Tuning, and Inference for Extractive-based LLMs

Abstract: Enhancing the task-specific capabilities of Large Language Models (LLMs) primarily requires substantial instruction-tuning datasets. However, the sheer volume of such data imposes a considerable annotation cost, and a lack of optimization methods for tailoring LLMs to specific tasks. To address the above issues, we propose a Planning framework for constructing Extractive-based LLMs called PlanE, which includes data decomposition, instruction tuning, and prompt inference. Additionally, we introduce a Data-Tuning-Inference (DTI) planner, aimed at selecting the optimal base-LLM and its DTI combinations for specific datasets to improve construction efficiency. The experimental results demonstrate the effectiveness of our PlanE from two views: (1) across different datasets using the same base-LLM, and (2) on the same dataset using different base-LLMs. Furthermore, we validate the generalizability of the proposed DTI planner under different optimization objectives.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents

InferenceBench is a new benchmark evaluating AI agents on open-ended LLM inference optimization tasks under realistic constraints. While agents outperform default setups, qualitative analysis reveals key limitations in their strategic exploration capabilities.

agentic-systems

PersonaTrail: Benchmarking Personalized Web Agents through Browsing Trails

Researchers have introduced PersonaTrail, a novel benchmark evaluating personalized web agents through realistic browsing histories. Alongside it, the PACMem memory framework structures raw browsing data to significantly boost agent performance in inferring user preferences.

agentic-systems

Beyond Liars' Bench: The Impact of Lie Typology, Depth, and Sparsity on Deception Detection in LLMs

A new study reveals that deception detection in LLMs heavily depends on lie typology and training data structure, highlighting why current detection probes struggle in out-of-domain scenarios.

agentic-systems

Benchmarking Large Language Models on Multi-Sensor Physical Hazard Assessment

A new benchmark evaluates five leading large language models on multi-sensor physical hazard assessments, revealing a critical vulnerability when evaluating combined sensor risks below individual thresholds. Despite near-perfect accuracy on single-sensor breaches, all models failed to issue precautionary signals for multi-sensor hazards.

agentic-systems

DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding

Researchers have introduced DC-Leap, a training-free framework that accelerates inference in Diffusion Large Language Models (dLLMs) by up to 105x when combined with KV-Cache while maintaining generation quality.

agentic-systems

JAXBench: Benchmarking Autonomous TPU Kernel Optimization

Researchers introduce JAXBench, a TPU-native benchmark suite designed to evaluate AI-generated kernel optimization. Testing shows that providing TPU-specific documentation enables Gemini 3 Flash to achieve up to a 1.36x speedup over the default XLA compiler.