Update — August 3, 2026: Karpathy’s original X post crossed ~2.8M views. The body below now quotes his full framing: leaving the pelican-SVG eval era, the stamina argument for hyper-custom worlds, spectator/NPC participation, and the multimodal gameplay audit gap.
Andrej Karpathy’s latest Opus 5 experiment is a better AI test than another pelican-on-a-bicycle SVG. He said it directly: we are “starting to leave the territory” of tiny visual coding tests. As a generalization, he gave Opus 5 the opening paragraph of The Lord of the Rings, a 1M-token budget (~$10), and asked for a procedural Three.js render of the story.
According to that August 2, 2026 post, Opus went off for about two hours and wrote approximately 5,500 lines of code that procedurally rendered the story. The resulting 93-second video is, in his words, “kind of janky but fun.” The mindboggling part is the mechanical work itself: the LLM had to place and orchestrate polygon assets in (x, y, z) coordinates and write code that animates it all — and that it does anything at all.
The important lesson is not that an AI built a finished Tolkien game for $10. It did not. The public artifact is a video, not a repository or playable release. The lesson is that custom visual software is becoming cheaper to generate faster than agents are becoming capable of watching, playing, and critiquing it.
TL;DR — what Karpathy’s Opus 5 demo shows
| Question | Direct answer |
|---|---|
| Why this eval? | Generalizes past “SVG of a pelican on a bicycle” |
| Input? | First paragraph of The Lord of the Rings + Three.js render request |
| Model? | Claude Opus 5, according to Karpathy |
| Budget? | 1M tokens, described as roughly $10—not an itemized actual-cost receipt |
| Runtime and output? | About two hours and approximately 5,500 lines of code |
| What shipped publicly? | A 93-second X video (~2.8M views on the post); no public repo or playable build linked |
| Did Opus make the audio? | No. Karpathy said he manually used ElevenLabs for the voice |
| Economic claim? | Work nobody would hand-author becomes “sure, why not, it’s ~free” because LLMs have stamina |
| Product vision? | Hyper-custom worlds — drop in as spectator NPC, a character, or similar; “ephemeral GTA of X” |
| Main weakness? | Cannot efficiently/natively perceive video or play the game; slow screenshots → jank |
Watch Karpathy’s original Opus 5 video
What Karpathy actually said
His August 2 post (paraphrased only where needed for length; key claims preserved):
We're starting to leave the territory where you'd test an LLM by e.g. "create an svg of pelican on a bicycle". As one idea to generalize it, I was interested what Opus 5 would do if I gave it the first paragraph of the Lord of the Rings, a 1M token budget (~$10) and asked for three js render of it. Opus went off for ~2 hours and wrote 5500 lines of code that (procedurally) rendered the story. It's kind of janky but fun. But it's a bit mindboggling that the LLM has to place and orchestrate various polygon assets in (x,y,z) coordinates and write code that animates it all, and that it even does anything at all.
On demand and economics:
I also like this kind of examples because no one in their right mind would ever spend the time to write something this custom but LLMs have all the stamina and patience in the world, so it's an example where we go from "no one would ever do this" to "sure, why not, it's ~free". There might be a lot more. But I'm excited about creating hyper custom worlds that you can imagine dropping players into, e.g. here to participate in the LoTR story as a spectator NPC, or one of the characters, or etc. Something like an ephemeral GTA of X on demand.
On the audit failure mode:
Last thought is that the domain of worlds/games exposes a weakness in LLMs: they can't easily audit their work because they aren't able to efficiently and natively perceive videos or play games within them. Here, Opus 5 had to very slowly and painstakingly take screenshots at different points, and it messed up a few times and created a bunch of jank. An example of raw capability (multimodal, gameplay) that I think is still quite lacking.
Those three paragraphs are the whole thesis: harder eval → stamina economics → perception gap.
What was generated—and what was not
The distinction between artifact, workflow, and speculation matters here.
Reported as generated by Opus 5
- roughly 5,500 lines of code;
- a procedural Three.js scene derived from the supplied literary paragraph;
- polygonal assets positioned and orchestrated in
(x, y, z)space; - animation and scene orchestration sufficient to produce the shared video;
- repeated, slow screenshot-based attempts to inspect and improve the result — with admitted mess-ups and leftover jank.
Added or handled outside the model’s reported work
Karpathy clarified in a reply that the audio came from ElevenLabs. He said an LLM could call the API, but he performed that step manually because he cared about the chosen voice. The video should therefore not be described as an entirely autonomous text-to-finished-film pipeline.
Not demonstrated publicly
- a downloadable or open-source codebase;
- a live playable role as spectator NPC or story character (Karpathy named that as the exciting next step);
- dynamic generation of new worlds while someone plays;
- a general-purpose learned world model;
- production-quality animation, collision, interaction, or narrative continuity;
- a reproducible token and cost trace;
- efficient native video perception or in-world gameplay for self-audit.
Karpathy’s phrase “an ephemeral GTA of X on demand” describes the direction he finds exciting, not the current artifact. This demo is closer to an automatically authored browser diorama than a full open-world game.
That boundary resembles other Opus 5 experiments tracked by explainx.ai. The Homeworld-style space RTS had a video and detailed token receipt but no public repository. The Rocket League-style clone offered stronger reproducibility because people could inspect the code and play the build.
Why this is a better evaluation than a tiny visual prompt
Small visual coding tests remain useful. An SVG task can reveal spatial reasoning, instruction following, and whether a model understands browser graphics. It is also easy to overfit to familiar examples and easy for humans to judge from one frame.
Karpathy’s experiment expands the evaluation along several dimensions:
| Dimension | Simple SVG task | Procedural story world |
|---|---|---|
| Time horizon | Minutes | Roughly two hours |
| Code scale | Tens or hundreds of lines | About 5,500 lines |
| Ambiguity | One object and composition | Characters, environment, sequence, mood, camera, timing |
| State | Mostly static | Animated events over time |
| Assets | A few shapes | Many reusable procedural objects |
| Verification | Inspect one image | Inspect many moments and transitions |
| Failure surface | Shape or styling error | Geometry, camera, continuity, occlusion, motion, pacing, performance |
This is not automatically a fair model benchmark. Karpathy did not publish the full prompt, harness configuration, tool list, retry count, token trace, source code, or grading rubric. The post is an informative public experiment, not a controlled comparison across models.
Still, it tests something that standard coding benchmarks often miss: can an agent sustain hundreds of interdependent creative and technical decisions long enough to produce one coherent artifact? Anthropic’s Opus 5 announcement emphasizes long-running work, stronger visual outputs, and more careful iteration. Karpathy’s result provides a concrete, imperfect example of that claimed capability regime.
The economic shift: from “not worth making” to “why not?”
Karpathy’s most important argument is economic — and he frames it as stamina. A human developer could build a custom animated interpretation of one paragraph. “No one in their right mind would ever spend the time to write something this custom,” he wrote, “but LLMs have all the stamina and patience in the world.” The shift is from nobody would do this to sure, why not, it’s ~free.
Traditional software economics rewards reuse. Teams build products for many users because design, engineering, art, testing, and maintenance are expensive. A bespoke interactive scene for one reader, one lesson, one birthday, or one tabletop campaign is usually not worth commissioning.
Long-running agents lower the creation threshold. If a rough custom world costs a few dollars and a few hours of machine time, demand can appear where no market existed before — including the interactive futures Karpathy sketches: drop into the story as a spectator NPC, as one of the characters, or something adjacent:
- a child’s story rendered as a temporary explorable scene;
- a teacher turning one historical passage into a spatial lesson;
- a game master generating a location for one evening’s campaign;
- a novelist walking through a draft setting to find continuity problems;
- a fan making a private visualization of a favorite passage;
- a product designer generating a disposable simulation for one decision.
This is the world-building version of software for one: artifacts become worthwhile even when their audience is a single person and their useful life is short.
“Roughly free” still needs qualification. A token budget excludes the user’s setup time, model subscription, tool infrastructure, manual review, audio generation, and the cost of failed attempts. Copyright also matters. A private experiment based on a protected fictional world is not permission to distribute or sell a commercial adaptation.
The paradox: generation outruns perception
Karpathy’s post names the most interesting weakness directly: worlds and games expose a raw multimodal/gameplay gap. Opus 5 could write thousands of lines that generate a moving 3D world, but it could not efficiently and natively perceive video or play inside what it built.
The model had to “very slowly and painstakingly take screenshots at different points,” mess up a few times, and leave “a bunch of jank.” That is a slow and lossy feedback loop:
write code
-> render scene
-> choose one timestamp
-> capture screenshot
-> inspect still image
-> infer what happened between frames
-> patch code
-> repeat
A screenshot can reveal a misplaced mesh, bad color, or broken camera angle. It cannot reliably show whether an animation stutters, a character teleports between frames, the pacing drags, audio loses sync, or an interactive control feels wrong.
Games add another layer. Visual correctness is not playability. An agent must act in the environment, observe how state changes, remember its objective, explore edge cases, and translate “this feels wrong” into a code-level hypothesis.
The Opus 5 Rocket League demo is relevant because its builder showed the model controlling the game to verify mechanics. That closes more of the loop than screenshots alone, but even computer-use input is slow compared with reading structured game state or processing video natively.
How to build a better visual-world agent harness
The next improvement may come as much from the harness as the model. A stronger loop would combine several feedback channels.
1. Structured scene inspection
Expose the scene graph, object coordinates, camera state, animation timeline, collision events, and frame timing as machine-readable data. The agent should not infer every problem from pixels.
2. Video understanding
Record short clips after milestones and let a video-capable evaluator inspect motion, continuity, pacing, and temporal artifacts. Sampled stills should supplement—not replace—the clip.
3. Scripted playthroughs
Define camera paths and player actions that reproduce important sequences. A visual regression should follow the same route after every major change.
4. Independent critics
Separate the builder from the evaluator. One agent implements; another receives the brief and finished capture without the builder’s rationale. This reduces the tendency to excuse its own output.
5. Quantitative gates
Track frame rate, loading time, draw calls, memory, animation completion, collision failures, and input latency. A beautiful single frame can hide a scene running at ten frames per second.
This is harness engineering, not prompt decoration. The model needs the right observation and verification surfaces around it.
A more useful private eval for AI-generated worlds
Instead of asking several models for one static image, give each the same story paragraph and fixed budget. Score the finished artifact across eight categories:
- Semantic fidelity: does the scene represent the important entities and relationships in the text?
- Spatial coherence: are scale, placement, occlusion, and camera paths sensible?
- Temporal continuity: do movements and transitions remain coherent across the full video?
- Interaction: can a user navigate or affect the scene without breaking it?
- Performance: does it sustain the target frame rate on fixed hardware?
- Self-correction: how many visible failures does the agent detect before a human points them out?
- Cost and time: what were actual tokens, model rates, retries, and wall-clock duration?
- Reproducibility: are prompt, code, assets, harness, and grader available?
Run each model more than once. A single creative run can be unusually inspired or unusually broken. Publish failed runs alongside the best one.
The top Opus 5 game prompts already show how different prompt structures produce different evaluation loops. Karpathy’s experiment adds a useful constraint: derive an entire visual sequence from prose rather than copying a familiar game’s feature list.
What builders should take away
First, raise the scale of your model evaluations. A toy UI can test syntax but miss the endurance, state management, and self-correction that determine whether an agent can finish ambitious work.
Second, measure the audit loop separately from generation. A model that creates an impressive scene but misses obvious temporal failures needs better perception and instrumentation, not only a larger coding budget.
Third, distinguish cheap generation from production readiness. The output can be valuable as a sketch, prototype, lesson, or personal artifact even when it remains too janky to ship.
Finally, do not treat copyrighted fictional worlds as free production assets. The general capability—turning prose into explorable procedural scenes—is more important than the specific franchise used in the demo.
Bottom line
Karpathy’s Opus 5 world is compelling because it sits between a benchmark and a product. It is too complex to dismiss as a tiny coding trick and too janky, closed, and unverified to call a finished game.
Its clearest signal is economic: long-horizon agents can now attempt hyper-specific visual artifacts that no rational team would previously budget. Its clearest limitation is epistemic: the agent still struggles to see the thing it made as efficiently as it can generate it.
When agents gain fast native video perception, structured world state, and reliable playtesting, “ephemeral GTA of anything” moves from a metaphor toward an engineering roadmap.
Related on explainx.ai
- WorldClaw: agentic 3D open worlds on Claude Opus 4.8 — Tencent's mesh-based take on prompt-to-world
- Top 10 Claude Opus 5 game prompts
- Opus 5 built a Homeworld-style space RTS
- Opus 5 built and played a Rocket League clone
- Claude of Duty: procedural Three.js FPS
- img2threejs: procedural 3D as editable code
- Anatomy Atelier: 3D human anatomy built with Codex and TripoAI
- Software for one: personal apps built by agents
- Context, prompts, loops, and harness engineering
Sources: Andrej Karpathy’s original post and video (~2.8M views as of the August 3, 2026 refresh) · Anthropic’s Opus 5 announcement · Three.js
Claims about runtime, code length, token budget, stamina economics, interactive vision, workflow, and audio reflect Karpathy’s public post and replies on August 2, 2026, with view-count and quote refresh on August 3, 2026. No source repository, full prompt, itemized bill, or playable build was linked in the supplied thread.