Beyond rate limits: scaling access to Codex and Sora

In the past year, both Codex and Sora have seen rapid adoption, with usage quickly pushing beyond what we originally expected. We’ve seen a consistent pattern: users dive in, find real value, and then run into rate limits.

Rate limits can help smooth demand and ensure fair access; however, when users are getting value, hitting a hard stop can be frustrating. We wanted a way for users to keep going, while protecting system performance and user trust in our approach.

To solve this, we built a real‑time access engine that counts usage. One of the layers in that engine is the ability to purchase credits. When users exceed their rate limits, credits let them keep using our products by spending down their credit balance.

Underneath this is a complex system that fuses limits, real‑time usage tracking, and credit balances in a single access model. This post covers why scaling Codex and Sora required rethinking access control, how a provably correct, real-time system blends rate limits and credits per request, and how that foundation now unlocks additional access for both products.

Zooming out, traditional access models tend to force a choice:

Rate limits can be helpful at first, but leave users with a bad experience when they run out: “come back later”. Usage‑based billing is flexible, but leaves users paying from the first token—not ideal for supporting early exploration.

For Codex and Sora, neither was sufficient on its own. If we simply raised rate limits, we’d lose important demand-smoothing and fairness controls and run out of capacity to serve everyone. If we relied entirely on asynchronous usage billing, we’d introduce lag, overages, or reconciliation issues—exactly the kinds of problems users notice when they’re most engaged.

What we needed instead was a single hybrid system combining real-time limits with pay-as-you-go access. This system had to: enforce rate limits until they’re reached; seamlessly transition to credits within the same request; make that decision in real time; and be rigorously accurate and auditable when tracking credit consumption.

One of the key conceptual shifts we made was modeling access as a decision waterfall. Instead of asking “is this allowed?”, we ask “how much is allowed, and from where?” When counting usage, the system goes through a sequence where rate limits, free tiers, credits, promotions, and enterprise entitlements are all just layers in the same decision stack. From a user’s perspective, they don’t “switch systems”—they just keep using Codex and Sora.

We evaluated third‑party usage billing and metering platforms to handle credit consumption. They’re well‑suited for invoicing and reporting, but didn’t meet two critical requirements: low latency and transparency. When a user hits a limit and has credits available, the system must know immediately. We also needed to offer transparency into every outcome: why a request was allowed or blocked, how much usage it consumed, and which limits or balances were applied.

To power this, we built a distributed usage and balance system designed specifically for synchronous access decisions. At a high level, the system tracks per‑user usage, maintains rate‑limit windows, maintains real‑time credit balances, and debits balances idempotently through a streaming async processor.

One of the key design principles is that we must be able to prove that our billing is correct. We have three separate datasets that tie together: Product usage events, Monetization events, and Balance updates. Separating what occurred, any associated charges, and what we debited lets us independently audit, replay, and reconcile every layer. Every event carries a stable idempotency key to prevent double-charging. We choose provable correctness and user trust over strict enforcement.

All of this rigor supports one objective: to make access simple and safe. When people are creating or coding, they shouldn’t have to wonder whether a request will go through. By making usage, billing, and balances provably correct, we give users a system that doesn’t distract from their experience. That’s what lets us replace hard stops with continuous access.