Andy Pavlo: What Happens When Billions of AI Agents Hit Your Database

Summary · 6 min read instead of 1 h 17 m

Andy Pavlo is a database professor at Carnegie Mellon University whose courses, posted free online, taught a generation of engineers and, he says, the AI models too. He runs the database of databases site and led self-driving database research, including the Peloton project. This summer he joined ClickHouse as VP of Database Research to found ClickHouse Labs.

Pavlo argues that agents mostly reuse existing database ideas: vector search became a standard index, and the controls needed to stop agents deleting production data already exist. The biggest change he expects is volume, with agents running around the clock and possibly driving 10 to 100x more queries, though he says it is unclear whether their query patterns differ from human ones. He reports that LLMs can now write whole database systems, that over 60% of open-source databases have commits from agents, and that LLMs get about 85% of the way on tuning in 15 minutes. He also explains why he joined ClickHouse, why analytics companies are adding Postgres, and why he is skeptical of graph databases and cautious about GPU databases.

Minute by minute

1. 1:15

   Vector databases were just an index. Pavlo says many vector databases existed since around 2016 and were well placed when chatbots took off, because approximate similarity search handles fuzzy natural-language queries. But the moat was small since "it's just an index": within a year nearly every database vendor added a vector index. Vector search still improves agents, he says, but you no longer need a specialized database that only does that.

2. 4:04

   Agents mostly branch databases for development. Turck cites a Neon stat that agent-created databases went from 30% to 80% when Databricks acquired Neon. Pavlo says the metric was mostly about branches: copy-on-write snapshots of a production database that let coding agents, or tests run by PRs in GitHub Actions, try schema and code changes without touching live data. Agents also create databases from scratch when asked to build an application.

3. 7:15

   Guardrails for agents already exist. Pavlo points to a widely publicized case where a Replit agent deleted database data, and says such stories now come up every month. He compares agents to toddlers who need a gate on the stairs: set permissions so they cannot drop production tables, rather than approving every step by hand. He says agents behave like humans apart from running 24/7, and enterprise commercial databases already have the protection mechanisms needed.

4. 10:43

   Agents are the fourth era of volume. Pavlo describes three earlier jumps in database traffic: workplace adoption in the 1970s and 80s, the internet in the 1990s and 2000s, and mobile phones. Agents are the fourth, since each person may run hundreds of them around the clock doing mixed reads and writes. He expects a 10 to 100x increase in queries and says ClickHouse is seeing this, but it is unclear whether agent queries are more or less complex than human ones.

5. 14:59

   Everything is a database, including memory. Asked whether files or databases should hold agent memory, Pavlo says a file system is a database. Agents take human-readable text as context, so the source matters less than the guarantees: a system like SQLite handles safe concurrent changes and commits for you. He says everything should go in a database system, and shared state across many agents is one advantage a file system lacks.

6. 18:18

   Models default to recommending Postgres. Pavlo says models usually recommend Postgres for transactional apps, DuckDB or SQLite for embedded uses, and RocksDB for key-value needs. Two database companies asked him how to get agents to recommend them over Postgres; he told them to talk to model builders, calling it like SEO. He adds that models often cite his own open-source course materials back to him, even when he excludes Carnegie Mellon.

7. 23:02

   Text-to-SQL works with a semantic layer. Pavlo says a large bank told him its text-to-SQL tool started at about 60% accuracy and reached 99.5% after years and millions of dollars building a richer semantic and context layer. He doubts academic benchmarks like Spider because models are trained on them. He calls MCP essentially a REST interface: inefficient text over the network but good enough, with the same security checks as SQL arriving through JDBC or ODBC.

8. 27:42

   Agents can now build entire databases. Pavlo says a year ago agents could not finish his Carnegie Mellon database course projects, but after Anthropic's Opus 4 they could do nearly all assignments with little prompting, helped by open-source course code. With enough tokens and guidance, he says, agents can vibe code a whole database system. His dbdb.io tracking of nightly GitHub commits shows over 60% of open-source database systems have commits from agents.

9. 30:31

   LLMs changed self-driving database tuning. Pavlo recounts a decade of work on auto-tuning indexes and knobs, including the Peloton system, where the main obstacle was getting enough production training data. LLMs trained on blogs, docs and tuning guides changed this: they get about 85% of the way in 15 minutes, versus models that took hours to train. He notes many users ran default cloud configurations believing Amazon tuned them, which LLMs can quickly fix.

10. 35:33

    Database knowledge still matters for students. Pavlo says understanding database internals helps anyone debug slow queries, though writing a system from scratch alone may fade and the traditional DBA role will be relegated, shifting work to data modeling. He put his Carnegie Mellon courses online free partly to compete with MIT, Stanford and Berkeley. He hired a DJ for lectures in 2019 and made a final exam question on the Wu-Tang Clan members worth full marks.

11. 42:00

    Why he built a lab inside ClickHouse. Pavlo says US university research funding is tight and agents raise questions about how many PhD students are needed. Other companies offered engineering roles, but ClickHouse asked him to do research knowing it may not pan out, which he calls rare for a company that is not yet public. Two months in, he is turning unpublished ClickHouse work into papers and testing ideas on real production workloads, including its Postgres offering.

12. 45:45

    ClickHouse seemed too good in 2016. Pavlo assumed ClickHouse was fake when it appeared in 2016 because no open-source OLAP engine had columnar storage, vectorized execution, compression and compaction together. He says those ideas trace to Snowflake and VectorWise from CWI and are now table stakes, but ClickHouse kept improving performance. He sees customers using it for agent telemetry, financial markets and fraud detection, where fast ingest and millisecond queries matter.

13. 52:14

    Analytics firms should add Postgres. Pavlo says Databricks buying Neon, Snowflake buying Crunchy and ClickHouse launching managed Postgres reflect a sensible path: start with analytics, then add transactional offerings. Hybrid systems have long struggled because operational and analytics teams each want the best tool, and for transactions that is now Postgres. He thinks Databricks, and possibly ClickHouse, could overcome this by putting a Postgres-compatible front end in place.

14. 58:09

    Verdicts on vector, graph and GPU databases. Pavlo says vector databases will survive by becoming general systems of record or Elasticsearch-style add-ons, naming TurboPuffer as one doing well. He calls graph databases a niche and a bad idea, noting SQL now supports property graph queries, and has a Hacker News bet that graph will not overtake relational by 2030. On GPU databases, he notes NVIDIA bought Voltron and HeavyDB but says GPUs are costly and the economics are unproven.

15. 1:08:25

    The relational model will not be replaced. Pavlo says the database market only looks stagnant next to AI's pace, which he likens to a cheetah on cocaine in a Ferrari. He argues the relational model is like arithmetic and will not be thrown out for agents, though SQL may become like assembly as text-to-SQL improves. He says core analytics engines are now similar, so what matters is everything around them, such as developer experience and ingestion, which Snowflake did well.

Published Oct 8, 2026.