AGENTIC-SYSTEMSJuly 24, 20261 min read

Beyond Liars' Bench: The Impact of Lie Typology, Depth, and Sparsity on Deception Detection in LLMs

A new study reveals that deception detection in LLMs heavily depends on lie typology and training data structure, highlighting why current detection probes struggle in out-of-domain scenarios.

Computer Science > Artificial Intelligence

Title:Beyond Liars' Bench: The Impact of Lie Typology, Depth, and Sparsity on Deception Detection in LLMs

View PDF HTML (experimental)Abstract:Training probes to detect deceptive outputs from large language models is still an open problem. Recent work has demonstrated that detection probes fail especially in out-of-domain scenarios -- training on one type of lie does not transfer well to deception scenarios involving other types of lies. In this work, we conduct a systematic study on how various factors impact detection performance: representation depth, probe expressivity, sparse feature representations, and the lie typology of the training data. To this end, we augment standard benchmark training data with a supplementary dataset containing diverse types of deception, including fabrication, omission, and exaggeration examples. Analyzing these factors across seven probe types, our experimental results show that the optimal representation depth is highly dataset-dependent, more expressive probes provide only selective gains over linear baselines, and sparse autoencoder features perform similarly to dense hidden states. Ultimately, we demonstrate that the choice of training data and lie typology substantially changes detectability, highlighting that deception detection is a highly representation-dependent problem.

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents

InferenceBench is a new benchmark evaluating AI agents on open-ended LLM inference optimization tasks under realistic constraints. While agents outperform default setups, qualitative analysis reveals key limitations in their strategic exploration capabilities.

agentic-systems

PersonaTrail: Benchmarking Personalized Web Agents through Browsing Trails

Researchers have introduced PersonaTrail, a novel benchmark evaluating personalized web agents through realistic browsing histories. Alongside it, the PACMem memory framework structures raw browsing data to significantly boost agent performance in inferring user preferences.

agentic-systems

Benchmarking Large Language Models on Multi-Sensor Physical Hazard Assessment

A new benchmark evaluates five leading large language models on multi-sensor physical hazard assessments, revealing a critical vulnerability when evaluating combined sensor risks below individual thresholds. Despite near-perfect accuracy on single-sensor breaches, all models failed to issue precautionary signals for multi-sensor hazards.

agentic-systems

DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding

Researchers have introduced DC-Leap, a training-free framework that accelerates inference in Diffusion Large Language Models (dLLMs) by up to 105x when combined with KV-Cache while maintaining generation quality.

agentic-systems

PlanE: Meta Planning of Data, Tuning, and Inference for Extractive-based LLMs

Researchers introduce PlanE, a novel framework featuring a Data-Tuning-Inference (DTI) planner designed to optimize data, tuning, and inference processes for extractive-based LLMs while reducing annotation costs.

agentic-systems

JAXBench: Benchmarking Autonomous TPU Kernel Optimization

Researchers introduce JAXBench, a TPU-native benchmark suite designed to evaluate AI-generated kernel optimization. Testing shows that providing TPU-specific documentation enables Gemini 3 Flash to achieve up to a 1.36x speedup over the default XLA compiler.