CC-Canary: Detect early signs of regressions in Claude Code
CC-Canary is a new set of Agent Skills for Claude Code that detects performance drift and regressions by analyzing local session logs. It provides detailed forensic reports on metrics like cost, reasoning depth, and tool usage without requiring network access or telemetry.
Drift detection for Claude Code, packaged as two installable Agent Skills. Reads the JSONL session logs Claude Code already writes to ~/.claude/projects/, detects whether the model has been drifting on your own work, and produces a shareable forensic report.
No network, no account, no telemetry, no background daemon. Runs on the data already on your disk.
Status: 0.x / pre-alpha — output format and metric set may change.
| Skill | Invocation | Output |
|---|---|---|
| cc-canary | /cc-canary [window] | forensic markdown writeup (./cc-canary-<date>.md) — paste-ready for GitHub issues or gists |
| cc-canary-html | /cc-canary-html [window] | same report as a dark-theme HTML dashboard (./cc-canary-<date>.html), auto-opens in your browser |
Window defaults to 60d. Accepts 7d / 14d / 30d / 60d / 90d / 180d.
Each report includes:
- Verdict — HOLDING / SUSPECTED REGRESSION / CONFIRMED REGRESSION / INCONCLUSIVE
- Headline metrics table (pre vs post, with 🟢/🟡/🔴 band verdicts)
- Weekly trend bars — cost (USD), read:edit ratio, reasoning loops, tokens/turn
- Cross-version comparison — same user, different model versions, controlling for task mix
- Auto-detected inflection date — composite health-score break
- Findings with model-side / user-side / ambiguous classification
- Appendices — hour-of-day thinking depth, word-frequency shift, and more
npx skills add delta-hq/cc-canary
Then from any Claude Code session:
/cc-canary 60d
/cc-canary-html 30d
Requirements: python3 ≥ 3.8 on your PATH. macOS / Linux / WSL for the cc-canary-html auto-open step.
How it works:
- Scan. A bundled Python script walks
~/.claude/projects/**/*.jsonl. - Dedupe. Assistant messages are deduped on
(message.id, requestId). - Aggregate. Per-session metrics: tool-mix, read:edit ratio, reasoning-loop phrases, self-admitted errors, token usage, cost.
- Detect inflection. Composite health score per day; finds the date with the most significant change.
- Pre-render & Fill. Script writes a skeleton; Claude fills in the narrative analysis and saves the final file.
Privacy: Fully local. Zero network calls. User-prompt content is truncated and redacted for sensitive paths/emails before being processed.
Source: Hacker News
