Scaling Trends for Lie Detector Oversight in Preference Learning
A new study evaluates Scalable Oversight via Lie Detectors (SOLiD) on larger LLMs, showing that scaling reduces undetected deception to 14% and can eliminate the need for expensive human labelers during fine-tuning.
Computer Science > Artificial Intelligence
Title:Scaling Trends for Lie Detector Oversight in Preference Learning
View PDF HTML (experimental)Abstract:Deceptive behavior in LLMs is costly to monitor and prevent, motivating approaches such as Scalable Oversight via Lie Detectors (SOLiD) (Cundy & Gleave, 2025), which uses lie detectors to identify responses for review by high-cost labelers. In this paper, we scale SOLiD to larger models and evaluate it in more diverse and realistic preference-learning settings. We find favorable scaling: undetected deception drops from 34% for 1B-parameter models to 14% for 405B-parameter models at a detector true positive rate of 99%, and expensive human labelers can be removed entirely from the fine-tuning phase without a statistically significant increase in deception. However, SOLiD is sensitive to distribution shift between detector training and preference-training data, which can drive detector false positive rates to impractical levels.
Source: arXiv cs.AI Recent
