Intelligence is Free, Now What? <br> Data Systems for, of, and by Agents

... government of the people, by the people, for the people ...

— Abraham Lincoln, Gettysburg Address (1863)

The cost of AI is dropping rapidly. GPT-4-class capabilities cost roughly $30 per million tokens in early 2023; today the same runs under $1, and some providers are pushing costs below $0.10. Across benchmarks, inference prices have fallen between 9x and 900x per year, with a median decline near 50x. Even frontier models are getting dramatically cheaper each generation, with open-source models following closely behind. And crucially, even if "Nobel-Prize-winning genius-level" intelligence isn't here yet, the intelligence that suffices for the vast majority of knowledge work is here today, and getting cheaper by the month. At this rate, we are soon entering the era of virtually free intelligence — the kind that is more than enough for everyday knowledge work.

Disclosure: This post is a perspective led by Aditya G. Parameswaran — an Associate Professor of EECS and co-director of the EPIC Data Lab at UC Berkeley — together with his collaborators. It is part landscape survey and part perspective, and several of the research directions discussed below (including agentic speculation, structured memory, and synthesizing custom data systems from scratch) draw on the authors' own ongoing work.

So, what does this new era of near-free intelligence mean for data systems? We believe three new challenges — and opportunities — stem from near-zero inference costs:

Data Systems For Agents. Agents will soon become the dominant workload for data systems — with swarms of agents spun up in response to each end-user request. Given differences in characteristics between agents and humans — or applications acting on their behalf —

how should we redesign data systems for such agentic users?

Data Systems Of Agents. As agents start taking on the bulk of knowledge work, a new substrate is needed for thousands of agents to manage state over long-running tasks, coordinate and reach consensus, and deal with failures.

What do data systems that reliably and efficiently run and manage agent swarms look like?

Data Systems By Agents. Agents are rapidly becoming capable of synthesizing entire data systems in one go — meaning we can rebuild custom systems for each new workload. Verifying that such systems match intended behavior is a challenge.

What does it take to let agents synthesize data systems we can actually trust?

Data Systems For, Of, and By Agents *

Next, we will discuss each in more detail, followed by discussing the intertwined future of data systems and agents, especially as the three challenges intersect.

Data Systems For Agents

An agent querying a database doesn't behave like a person or a BI tool. It performs what we call agentic speculation: a high-volume, heterogeneous stream of work spanning schema introspection, columnar exploration, partial and then full query formulation. With multiple agents each exploring portions of the hypothesis space, each user request could amount to 1000s of individual SQL queries. Now, users can issue "high-level" data tasks, e.g., root-cause analysis — e.g., "why did coffee sales in Berkeley drop this year" — or exploratory cohort analysis — e.g., "which user segments are most likely to churn next quarter" — each involving a combinatorial space of potential joins, aggregations, and filter combinations.

Data Systems Redesigned to More Effectively Support Agentic Speculation *

The requests from these agents have various opportunities for optimization. For instance, on a text-to-SQL benchmark with multiple agents attempting each task, only 10-20% of the sub-plans are distinct. Thus, 80-90% of sub-queries perform duplicate work. The same experiments show task success rates significantly increasing with more agentic attempts — so the redundancy is actually helpful. But from the data system perspective it's wasted work.

An agent-first data system can exploit such properties to help agents make progress faster. It can reuse results across overlapping sub-plans, drawing on ideas from decades-old literature on multi-query optimization and shared scans. Or the data system can try to satisfice, returning approximate answers that are good enough for agents to make progress, leveraging work from the AQP literature — or streaming the results of the final or intermediate operators to help agents decide if seeing the rest is necessary or helpful.

Another opportunity here is to rethink the query interface entirely: instead of agents issuing a single SQL query at a time, they could instead issue a batch of queries, each with its own approximation requirements. Since enumerating an exponential search space (as in the root cause or cohort analysis examples above) isn't a good use of agentic reasoning ability, perhaps data systems should support higher-level primitives rather than requiring agents to list each SQL query explicitly. One idea here is to draw on DBT-style Jinja macros to provide looping-based primitives for agents to interact with data systems.

A Caffeinated Army of Agents Ready to Tirelessly Complete Your Data Tasks *

A final opportunity here is to stop thinking of data systems as passive executors of queries; data systems could be proactive, as they possess more grounding in data and system characteristics that agents may lack a priori — they could steer agents in different directions, provide results for related queries, and also provide performance-level feedback (e.g., instead of executing an expensive query, the system could first provide the agent a latency estimate). The reason we can do this now as opposed to the past is that an agent can accept any form of textual feedback and isn't expecting a strict SQL query result. In fact, the data system could also prepare both materialized and virtual views for an agent in advance, provided to the agent as part of context, as this may be cheaper or more effective than having an agent author or use them.

Data Systems Of Agents

Previously, we focused on how agents interact with data systems. Now, we consider everything else agents need to keep working: where they live, how they remember, how they coordinate with each other, and how they deal with failures of each other. This agentic substrate is separate from the inference stack powering raw intelligence. However, the inference stack itself is being abstracted away through APIs (e.g., from OpenAI or Anthropic), or, for open-weight models, through serving frameworks that hide low-level details. So far, the agentic substrate has been managed through harnesses like Claude Code and Codex, coupled with various mechanisms to store and retrieve memory.

First, on the memory front, the current wisdom is that files are all you need; agents write to unstructured markdown (MD) files, which can then be searched using grep, or via embedding-based retrieval. In fact, many argue that the solution to continual learning is having agents consume a lot (e.g., an entire codebase, slack, company wikis, ...) and then write their learnings into MD files, which are then retrieved selectively on demand. Indeed, file systems, bash scripting, and MD files are and will still be important for agents. However, at scale, when agents are doing the vast majority of knowledge work, this approach will no longer be effective.

Given limited context windows, retrieving all MD file fragments that may be relevant and stuffing it into the context will break down at some point. Even if context windows continue to grow, there are latency benefits to not put all information into context — and in many cases, e.g., when knowledge work involves interacting with large databases or code bases, it will be infeasible to serialize all relevant data into context.

Data Systems As A Substrate for Multi-Agent Swarms *

One could use a knowledge graph representation, but knowledge graphs suffer from the same limitations as unstructured MD-based memory due to inherent limitations in scaling and retrieving data accurately at scale.