$500 GPU outperforms Claude Sonnet on coding benchmarks

Adaptive Test-time Learning and Autonomous Specialization

A.T.L.A.S achieves 74.6% LiveCodeBench pass@1-v(k=3) with a frozen 14B model on a single consumer GPU -- up from 36-41% in V2 -- through constraint-driven generation and self-verified iterative refinement. The premise: wrap a frozen smaller model in intelligent infrastructure -- structured generation, energy-based verification, self-verified repair -- and it can compete with frontier API models at a fraction of the cost. No fine-tuning, no API calls, no cloud. Fully self-hosted -- no data leaves the machine, no API keys required, no usage metering. One GPU, one box.

Hardware: RTX 5060 Ti 16GB | Model: Qwen3-14B-Q4_K_M (frozen)

| Benchmark | Score | Tasks | Method | |---|---|---|---| | LiveCodeBench v5 | 74.6% pass@1-v(k=3)* | 599 | V3 pipeline: PlanSearch + self-verified PR-CoT repair | | GPQA Diamond | 47.0% | 198 | k=5, multiple-choice knowledge reasoning | | SciCode | 14.7% | 341 | k=1, cross-domain scientific coding |

*pass@k-v(k=3) = one solution submitted per task, but generated via best-of-3 candidates + Lens selection + iterative repair on failures.

V3 ablation breakdown

| Condition | Configuration | Pass Rate | Delta | |---|---|---|---| | A | Baseline (no V3) | 54.9% | -- | | B | +Phase 1 (PlanSearch + BudgetForcing + DivSampling) | 67.3% | +12.4pp | | C | +Phase 1+2 (Lens routing) | 67.3% | +0.0pp | | D | +Phase 1+3 (self-verified refinement) | 74.6% | +7.3pp |

Comparison with Frontier Models

| System | LCB pass@1 | Est. cost/task | Notes | |---|---|---|---| | DeepSeek V3.2 Reasoning | 86.2% | ~$0.002 | API, single-shot | | GPT-5 (high) | 84.6% | ~$0.043 | API, single-shot | | ATLAS V3 (pass@1-v(k=3)) | 74.6% | ~$0.004 | Local electricity only | | Claude 4.5 Sonnet | 71.4% | ~$0.066 | API, single-shot | | Claude 4 Sonnet | 65.5% | ~$0.066 | API, single-shot |

Methodology & Architecture

ATLAS trades latency for cost and privacy. The pipeline uses a patched llama-server providing speculative decoding (~100 tok/s). The Geometric Lens selects the best candidate, and failed tasks enter Phase 3 for iterative repair via PR-CoT using model-generated test cases. Real tests are only used for final scoring. This entire stack runs locally on K3s, ensuring no data ever leaves the local environment.