Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models
This paper analyzes how finite numerical precision in floating-point representations leads to unpredictability in LLMs through rounding error propagation, identifying an 'avalanche effect' and three distinct behavioral regimes.
Computer Science > Artificial Intelligence
Title: Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models
As Large Language Models (LLMs) are increasingly integrated into agentic workflows, their unpredictability stemming from numerical instability has emerged as a critical reliability issue. While recent studies have demonstrated the significant downstream effects of these instabilities, the root causes and underlying mechanisms remain poorly understood. In this paper, we present a rigorous analysis of how unpredictability is rooted in the finite numerical precision of floating-point representations, tracking how rounding errors propagate, amplify, or dissipate through Transformer computation layers.
Specifically, we identify a chaotic "avalanche effect" in the early layers, where minor perturbations trigger binary outcomes: either rapid amplification or complete attenuation. Beyond specific error instances, we demonstrate that LLMs exhibit universal, scale-dependent chaotic behaviors characterized by three distinct regimes:
- A stable regime, where perturbations fall below an input-dependent threshold and vanish, resulting in constant outputs;
- A chaotic regime, where rounding errors dominate and drive output divergence; and
- A signal-dominated regime, where true input variations override numerical noise.
We validate these findings extensively across multiple datasets and model architectures.
Source: arXiv cs.AI Recent
