Why AI Agents Will Take a Decade to Deliver: Andrej Karpathy on Hype, Reality, and What's Missing
Some conversations cut through the noise. This one does exactly that. Dwarkesh Patel sits down with Andrej Karpathy—former Tesla AI director, OpenAI founding member, and one of the sharpest minds in machine learning—for a reality check on where AI actually stands. What follows is a bracingly honest tour through everything the industry gets wrong about timelines, a deep dive into why reinforcement learning is "terrible" (his word), and a vision for education that might matter more than AGI itself. If you've been drowning in AI hype, this is the antidote.
The Core Idea:
AI will transform everything, but the gap between impressive demos and actual products is wider than the industry wants to admit—and that gap will take a decade, not a year, to close.
They kick off by unpacking why Andrej called this "the decade of agents" instead of "the year of agents."
It's a direct pushback against the breathless timelines coming out of AI labs. Andrej explains that current agents—even the impressive ones like Claude and Codex—just don't work reliably enough. They lack continual learning, multimodal fluency, and basic cognitive capabilities that would make you actually want to hire them as an intern. The analogy is simple: you wouldn't delegate real work to these systems today because they'd fail in ways that matter. What will it take to close that gap? A decade of unglamorous, incremental progress across data quality, algorithms, hardware, and system design. Not one breakthrough. Not six months of RL fine-tuning. A decade. And Andrej has the pattern-matching from 15 years in AI to back that up—he's watched predictions fail, watched demos fizzle, watched the industry overpromise and underdeliver on loop.
One of the most memorable lines comes early: "We're not building animals. We're building ghosts."
Andrej draws a sharp distinction between biological intelligence—evolved over millions of years, encoded in DNA, shaped by survival—and LLMs, which are trained by imitating the internet. Animals come with massive amounts of hardwired behavior. A zebra runs minutes after birth. That's not reinforcement learning; that's evolution doing the heavy lifting. LLMs, by contrast, are "ethereal spirit entities"—fully digital, mimicking human patterns, but starting from a fundamentally different place. This matters because people keep reaching for biological analogies (brains, neurons, learning) that don't actually map. Andrej isn't saying we shouldn't study biology for inspiration, but he's skeptical of using it as a blueprint. What we're building is something new, something that has to be understood on its own terms. And that means we can't just assume the path to AGI will look like evolution 2.0.
Andrej has a blunt take on reinforcement learning: "It's terrible. It just so happens that everything we had before was much worse."
He walks through the core problem with RL as it's practiced today. You sample hundreds of rollouts, check which ones hit the right answer, and then you up-weight every single token in the successful trajectories. But here's the issue: just because a trajectory reached the correct answer doesn't mean every step along the way was smart. You might have wandered down dead ends, made lucky guesses, or stumbled into the solution. RL treats all of it as gospel. Andrej calls this "sucking supervision through a straw"—you're doing all this computational work, getting a single binary signal at the end (right or wrong), and then broadcasting that signal backward across an entire minute-long reasoning trace. It's noisy, inefficient, and deeply unlike how humans actually learn. A human would review their work, identify what went wrong, generate better examples, and iterate. LLMs don't do that. Not yet, anyway.
There's a fascinating detour into model collapse and why synthetic data is harder than it looks.
Andrej explains that when you sample from an LLM, the outputs are "silently collapsed"—they occupy a tiny slice of the possible space. Ask ChatGPT to tell you a joke, and you'll get the same three jokes over and over. That's not because it only knows three jokes. It's because the model's distribution is narrow and biased. If you try to train on that synthetic data, you amplify the collapse. The model gets worse, not better. And here's the kicker: humans collapse too. As you age, you overfit to your experiences, revisit the same thoughts, lose entropy. Children are un-collapsed—they say shocking, unpredictable things because they haven't narrowed down yet. Andrej sees this as a fundamental challenge: how do you maintain diversity in a system that's constantly reinforcing its own outputs? One intriguing thread he mentions: maybe dreaming evolved to prevent this kind of overfitting, by putting you in weird, out-of-distribution scenarios that keep your brain flexible. It's speculative, but it's the kind of thing that makes you think.
The self-driving analogy hits hard.
Andrej spent five years leading autopilot at Tesla, and he uses that experience to ground the conversation. Self-driving demos have existed since the 1980s. Waymo gave him a flawless ride in 2014. And yet, a decade later, we still don't have self-driving at scale. Why? Because demos are easy. Products are hard. Every additional "nine" in reliability—going from 90% to 99%, 99% to 99.9%—takes the same amount of work. And you need a lot of nines before you can remove the safety driver. Andrej sees the same pattern playing out with AI agents. The demos are impressive. The vibe coding is fun. But when you try to deploy these systems in production, where the cost of failure is high, you hit a wall. And that wall doesn't come down with one more training run. It comes down with years of iteration, edge case discovery, infrastructure buildout, and cultural adaptation. He's not pessimistic—he thinks it's tractable—but he's allergic to the hype cycle that pretends it's already solved.
There's a striking moment where Andrej admits coding agents weren't that helpful for building Nanochat.
Nanochat is his recent project—8,000 lines of code that walk you through the full pipeline of building a ChatGPT clone. It's clean, pedagogical, and technically rigorous. And he barely used AI agents to build it. Why? Because the models kept misunderstanding his custom implementations, kept trying to impose boilerplate patterns, kept suggesting deprecated APIs. They couldn't grasp that he wasn't writing "normal" code—he was writing teaching code, where every choice is deliberate and every line matters. Autocomplete? Incredibly useful. Agent-driven vibe coding? Not so much. The lesson here is subtle but important: these tools are asymmetrically good at different tasks. They excel at boilerplate, at translating between languages you don't know well, at filling in the obvious next steps. But for novel, intellectually demanding work that hasn't been done before? They're still not there. And that gap matters a lot if you're trying to forecast whether AI will recursively self-improve its way to superintelligence.
Andrej pushes back hard on the idea of an intelligence explosion.
Not because he thinks AI won't get better—he does—but because he sees it as continuous with everything that's already happened. "We're in an intelligence explosion already," he says. "We have been for decades." GDP growth has been exponential for centuries. Computers, the internet, compilers, search engines—it's all been a gradual process of automating cognitive work. AI is just the next layer. He can't find AI in the GDP curve, just like you can't find the iPhone or the personal computer as discrete jumps. Everything diffuses slowly, gets averaged into the same exponential, and keeps compounding. So when people talk about AGI causing a sudden phase transition—20% growth instead of 2%—Andrej is skeptical. He thinks we'll hit AGI (by the "economically valuable tasks" definition) and it'll still feel like business as usual. Not because it won't matter, but because it'll matter in the same way everything else has: gradually, unevenly, with tons of messy integration challenges and societal lag. It's a sobering take, but one rooted in historical pattern-matching.
The conversation takes a turn toward education, and you can feel the energy shift.
Andrej's new project, Eureka, is about building "the Starfleet Academy"—an elite, up-to-date institution for technical knowledge in the age of AI. But he's not just slapping an LLM on top of online courses and calling it a day. He's after something harder: the experience of a perfect human tutor. He describes learning Korean with a one-on-one tutor who instantly understood where he was, served him exactly the right challenges, and made him feel like he was the only constraint—not the material, not the explanation, not the pacing. That's the bar. And current AI can't hit it. Not even close. But Andrej thinks it's tractable, and when it works, it'll unlock something profound: a world where learning anything is as easy as going to the gym. People will learn five languages because why not? They'll understand undergrad-level math, biology, physics—because the barriers will be gone. And maybe, just maybe, that's how humanity stays empowered in a world where AI does most of the work. Not by competing with machines, but by becoming unrecognizably smarter than we are today.
One insight stands out: Andrej thinks the "cognitive core" of intelligence might be as small as a billion parameters.
Current frontier models are in the trillion-parameter range, but he suspects most of that is memory—hazy recollections of internet documents—not cognition. The core algorithms for reasoning, planning, problem-solving? They might be dramatically smaller. This ties into his physics background, where the instinct is always to find the first-order term, the essential component, the spherical cow that captures 80% of the system. He wants to strip away the knowledge and isolate the intelligence—the thing that lets you learn, adapt, and generalize. And he thinks that thing is smaller than people assume. In ten years, he says, you might have billion-parameter models that think like humans but look everything up in real-time. That would flip the entire paradigm: less memory, more cognition, more like an agent than a database. Whether the industry actually builds in that direction is an open question, but the vision is compelling.
The episode winds down with a meditation on humanity's role in an AI-saturated future.
Andrej's big fear isn't that AI gets too smart. It's that humanity gets sidelined—that we end up in a Wall-E or Idiocracy scenario where we're just... along for the ride. He's betting on something he calls "the timelessness of human nature": the idea that people will always want to learn, create, and push themselves, the same way they've always valued physical fitness even though we don't need muscles to move heavy objects anymore. Education becomes the gym of the mind. You go to school like you go to the gym—not because you have to, but because it feels good, because it's what humans do. It's a hopeful vision, but it requires work. It requires building systems that make learning feel rewarding, not punishing. And it requires rejecting the lazy version of the future where AI does everything and we just... watch. Andrej doesn't want that. And this whole project—Eureka, Nanochat, the pedagogy—feels like his way of fighting for a different outcome.
Final beat: Andrej has one of the best teaching philosophies I've heard in a while.
He talks about how physicists think—building models, finding first-order terms, understanding what matters and what's noise. And he applies that same instinct to education. When he built Micrograd (100 lines of Python that explain backpropagation), he wasn't just simplifying. He was distilling—finding the irreducible core and serving it up clean. "Everything else is just efficiency," he says. That's the trick: separate the essence from the optimization. Show the pain before you show the solution. Never present an answer before letting the student guess. And always, always make them build it themselves—no copy-paste, just reference. It's the Feynman principle: if you can't build it, you don't understand it. And if you can't explain it, you definitely don't understand it. That ethos runs through everything Andrej does, and it's why his tutorials land when so many others don't. He's not just teaching concepts. He's teaching how to think.