We replaced RAG with a virtual filesystem for our AI documentation assistant

RAG is great, until it isn't. Our assistant could only retrieve chunks of text that matched a query. If the answer lived across multiple pages, or the user needed exact syntax that didn't land in a top-K result, it was stuck. We wanted it to explore docs the way you'd explore a codebase. Agents are converging on filesystems as their primary interface because grep, cat, ls, and find are all an agent needs. The obvious way to do this is to just give the agent a real filesystem. Most harnesses solve this by spinning up an isolated sandbox and cloning the repo. Our p90 session creation time was ~46 seconds. Beyond latency, dedicated micro-VMs introduced a serious infrastructure bill: north of $70,000 a year. We built ChromaFs: a virtual filesystem that intercepts UNIX commands and translates them into queries against that same database. Session creation dropped from ~46 seconds to ~100 milliseconds, and since ChromaFs reuses infrastructure we already pay for, the marginal per-conversation compute cost is zero. ChromaFs is built on just-bash by Vercel Labs. We store the entire file tree as a gzipped JSON document. On init, the server fetches and decompresses this document into in-memory structures. Once built, ls, cd, and find resolve in local memory with no network calls. We intercept just-bash’s grep, parse the flags, and translate them into a Chroma query. Chroma acts as a coarse filter, and we bulkPrefetch matching chunks into a Redis cache for fine filter in-memory execution. ChromaFs powers the documentation assistant for hundreds of thousands of users across 30,000+ conversations a day.