explainx / blog
TryAI gave four vision models a blank canvas and colored-pencil tools to draw the Mona Lisa, Starry Night, and five prompts. GPT-5.6 Sol won on quality; Fable 5 cost 20x more for worse output; Grok 4.5 barely worked.
Jul 17, 2026
Hersh Halberstam's Music Video Arena gave Fable 5 and GPT-5.6 Sol the same song, budget, and tool harness — plan, web search, FAL image/video, ffmpeg — to produce autonomous music videos. None were great. This post breaks down the cost table, model video pipelines, and HN consensus on agentic video limits.
Jul 9, 2026
OpenAI launches GPT-5.6 publicly July 9; Fable 5 live globally since July 1. Sol Ultra leads Terminal-Bench at 91.9%; Fable leads SWE-Bench Pro at 80.3%. Terra matches Fable on terminal work at half the price — tier-by-tier guide.
Jul 21, 2026
On July 20, 2026, The Bugged Dev gave Kimi K3 the same 3D football stadium challenge previously run on Claude Fable 5. Fable finished under an hour with one big HTML file; Kimi took nearly three hours — running E2E tests, screenshots, and responsive checks. explainx.ai unpacks the anecdote, the "vibe coding vs vibe engineering" frame, and why harness + verification behavior matter more than wall-clock time.
Give four frontier models the exact same toolset — a simulated colored pencil, nothing else — and the gap between them stops being about token counts and starts being about taste. TryAI's Canvas Arena, published July 21, 2026, handed GPT-5.6 Sol, Claude Fable 5, Grok 4.5, and Gemini 3.6 Flash the same blank canvas, the same set-color/draw/smudge/erase toolkit, and asked each to either reproduce a target painting or draw from a text prompt — then tracked every stroke, dollar, and self-review call along the way.
This is TryAI's second creative-arena benchmark after its $100 AI music video showdown between Fable 5 and GPT-5.6 Sol. The pattern from that earlier test mostly held: Fable 5 was slow and expensive, GPT-5.6 Sol was efficient, and the newcomers to this round — Grok 4.5 and Gemini 3.6 Flash — landed at opposite ends of usable.
| Question | Answer |
|---|---|
| Best output quality? | GPT-5.6 Sol, by TryAI's own judgment |
| Best objective SSIM score? | Gemini 3.6 Flash (highest on both scored targets) |
| Cheapest? | GPT-5.6 Sol (~$7.74 total for 7 drawings) |
| Most expensive? | Claude Fable 5 (~$160.58 total — ~20x the others) |
| Worst performer? | Grok 4.5 — "basically garbage" per TryAI |
| Drawings tested? | 28 total — 2 scored targets + 5 open prompts, ×4 models |
| Did models improve over time? | No — every run's final output scored below its own mid-run peak |
| Open-weight models? | Tested, mostly unusable — several returned blank canvases |
Every model operated the same restricted toolset: plan for scratchpad thinking, view_target to re-examine the reference image, view_canvas to render and score the current page, set_color/set_brush/set_pressure for pencil configuration, draw for batched stroke commands, smudge for blending, and erase/clear_canvas for corrections. Critically, there are no solid fills — every area of tone or color has to be built up through layered strokes, the same constraint a person drawing with an actual colored pencil faces. The full harness is open source on GitHub, so the test is reproducible against any target image or prompt.
Two targets were scored objectively against structural similarity (SSIM) and pixel-level RMSE: the Mona Lisa and Van Gogh's Starry Night. Five additional prompts — an elderly fisherman's face, an ocean sunset, a rose in a vase, a sleeping cat, a cozy cabin interior — had no reference image and were evaluated qualitatively only, since there's no "correct" output for an open-ended creative brief.
| Model | Avg steps | Avg time | Total tokens | Est. cost | Avg self-reviews |
|---|---|---|---|---|---|
| GPT-5.6 Sol | 29 | 6.2 min | 3.4M | $7.74 | 6.0 |
| Claude Fable 5 | 54 | 12.5 min | 14.6M | $160.58 | 15.6 |
| Grok 4.5 | 99 | 4.8 min | 34.0M | $9.21 | 4.3 |
| Gemini 3.6 Flash | 73 | 6.9 min | 27.7M | $12.87 | 22.6 |
Fable 5's cost stood out even among a field where token-hungry behavior is common: roughly 20x the estimated spend of the other three for output TryAI ranked below GPT-5.6 Sol on quality. Grok 4.5's token count was the highest of the four at 34 million, but its cost stayed low because roughly 98% of those tokens were cheap cached reads rather than fresh generation — a reminder that raw token volume and actual dollar cost can diverge sharply depending on caching behavior and a provider's specific rate structure.
Tool-use style diverged in ways that say something about each model's default "workflow" for an unfamiliar creative task. GPT-5.6 Sol and Gemini 3.6 Flash never called set_color, set_brush, or set_pressure as standalone actions at all — both set those parameters inline on each draw call instead. Grok 4.5 did the opposite: 65% of its 1,349 total tool calls were spent on those three configuration tools alone, which explains its unusually high step count (99 per drawing) without a corresponding jump in actual drawing output.
The most substantive finding in the whole report has nothing to do with which model "won." TryAI's view_canvas calls scored the drawing against its target every time a model checked its own work, producing a similarity-over-time curve for each of the eight scored runs (two targets × four models). Two patterns held across every single one:
Progress plateaued early. Claude Fable 5 reviewed its Mona Lisa attempt 27 times, but its similarity score was essentially flat after roughly the fifth review — meaning the next 22 review-and-revise cycles bought almost nothing.
More strikingly, the final drawing always scored below the model's own best mid-run attempt. GPT-5.6 Sol's Mona Lisa peaked at 0.352 SSIM and finished at 0.325. Gemini 3.6 Flash's Mona Lisa run peaked at 0.449 — the single highest score anyone reached in the entire batch — then declined all the way to 0.337 by the point it stopped working. TryAI's blunt conclusion: "the models kept editing past their own best performance." That's a genuinely useful signal for anyone building agent harnesses with self-review loops generally, not just for drawing — a model that can accurately score its own output doesn't automatically know when to stop revising it, and unconstrained iterate-and-recheck loops can actively degrade a result past its peak if there's no stopping criterion beyond "keep going until budget runs out."
TryAI's own verdict — "basically garbage" — was echoed and dissected at length in the Hacker News discussion of the post. Grok 4.5's outputs scored lowest on both targets by a wide margin (0.151 SSIM on Mona Lisa, and a striking 0.039 on Starry Night, far below any other model's result), and commenters described the results as "uncanny," "surreal," and closer to what "a kid would do when given the same tools" than the more deliberate, iconified output from the other three models.
One theory from commenter jorl17, who separately tested Grok 4.5 on a personal poetry-analysis benchmark, proposed that xAI's training approach for Grok 4.5 may have skewed heavily toward coding tasks — pointing to unrelated but consistent evidence of the model garbling non-code language tasks — which would explain uneven performance on a visually and creatively open-ended task like this one. That's speculation rather than a confirmed explanation, but it's a plausible read given Grok 4.5's coding-benchmark strength appearing not to transfer to this domain at all.
Beyond the scoreboard, the Hacker News thread engaged seriously with what the drawings themselves revealed about how these models "see." Commenter NichoPaolucci and others noted the outputs read as genuinely childlike in a specific, technical sense — the models appeared to draw the concept of an object (this is glass, so make it blue; this is a stem, so make it green) rather than observing actual light, shadow, and form the way a more developed human artist learns to. Commenter ralusek connected this directly to how art instructors teach realistic drawing: the classic advice to stop thinking in labeled objects and instead "see" a scene as raw hues and values is exactly the skill these models appeared to lack by default — reproducing recognizable symbols for things rather than the appearance of things.
TryAI's earlier $100 AI music video challenge tested only Fable 5 and GPT-5.6 Sol on a fully different creative modality — autonomous video generation with a real production budget — and found a broadly similar cost-versus-quality pattern: Fable 5 ran slower and pricier without a corresponding quality edge over GPT-5.6 Sol. Canvas Arena reproduces that same relationship on a completely different task and toolset, which strengthens it as a signal about the two models' general working style — Fable 5 tends toward more deliberate, expensive, iterative execution across creative agentic tasks, while GPT-5.6 Sol tends toward faster, cheaper, more decisive output — rather than something specific to either video or drawing individually. Whether that trade-off matters depends entirely on the task: for workflows where thoroughness and careful iteration pay off, Fable 5's approach may still win; for execution-heavy creative tasks like these two arenas, the premium hasn't been buying better results.
Cost estimates, scores, and quotes reflect TryAI's July 21, 2026 report and Hacker News discussion from the same day. Token pricing and model versions may change — see TryAI's linked transcripts for full run-by-run detail.
