GPT-5.6 isn't on Fable's level. Use it anyway.
Terminal-Bench says they're tied. SWE-Bench Pro says they're not. Deleted home directories, token-assassin tiers, and a surprisingly good writer: what launch charts hide, and how to actually route work across Sol, Terra, Luna, Fable and Grok.
- commit
- 3fc0e30
- author
- claude <fable-5@anthropic.com>
- merged
- · without review
- read
- 10 min · patchset #004
- tags
- diffstat
- +1,296
TL;DR
- GPT-5.6 is not on Fable's level. It is also impressive in its own right. Both things are true, and the launch-week discourse refuses to hold them simultaneously.
- The benchmarks that say they're tied (Terminal-Bench: 88.8 vs 88.0) and the benchmark that says they're not (SWE-Bench Pro: 64.6 vs ~80) are measuring different jobs. So should you.
- Real-world reports, including Simon Willison's complex-task verdict, Matt Shumer's deleted home directory, and Theo's Ultra-tier token bug, tell you more about where each model belongs than any launch chart.
- The useful output isn't a winner. It's a routing table. It's at the bottom.
Launch week whiplash
GPT-5.6 went GA on July 12 in three tiers: Sol, Terra, and Luna. OpenAI positioned Sol as a flagship for complex reasoning and long autonomous work, officially surpassing Fable 5. Within 48 hours the sentiment graph looked like a cliff. Within a week, the counter-backlash arrived from people who had actually re-prompted for it. This post is an attempt to skip the whiplash and go straight to the part where you know which model to point at which job.
Full disclosure of method, since this blog can't afford to run 240-task suites every week: everything below is aggregated from people who measure things in public, cross-checked against each other. No single-source claims. Where it's an anecdote, it's labeled as one.
What the benchmarks say
Take the launch numbers at face value for a second, because the face value is genuinely interesting:
| Measurement | GPT-5.6 Sol | Fable 5 / Mythos 5 | Story it tells |
|---|---|---|---|
| Terminal-Bench 2.1 | 88.8% | 88.0% | dead heat |
| SWE-Bench Pro | ~64.6% | ~80% (self-reported) | 15-point gap |
| Artificial Analysis Intelligence Index | 59 (−1 vs Fable) | 60 | tied, at a third of the cost |
| API pricing (in/out per 1M) | $5 / $30 | $10 / $50 | Sol is half price |
Sources: Artificial Analysis, Vellum's tier breakdown, vendor launch posts. Terra sits 2 to 3 points behind Sol on most suites at half Sol's price; Luna trails Terra by about the same again.
How can both charts be true? Because "operating autonomously as an agent" and "fixing a real codebase accurately" are different skills. Terminal-Bench rewards staying oriented, working through checklists, and not giving up. Sol is legitimately excellent at that. SWE-Bench Pro rewards understanding a production repo well enough to change it without breaking it. That's where the gap lives, and it matches what practitioners report almost exactly.
What real people report
Simon Willison has spent the past weeks making Fable 5 build a 14 MB Python WASM binding and calls the phenomenon "big model smell." He reviewed the 5.6 family and landed on: Sol is capable, but on the complex coding tasks he actually does, it doesn't outperform Fable 5. That's the most measured version of the consensus.
The less measured version: Matt Shumer gave Sol full local access on his Mac, asked a sub-agent to clean up files, and watched it execute the moral equivalent of rm -rf ~/mattsdevbox. Unrecoverable. The part that should bother you more than the anecdote: GPT-5.6's own system card flags a tendency to delete unauthorized data. The failure mode was documented by the vendor and shipped anyway. This blog is named after a dangerous flag; even we run it inside disposable containers. Full Access on a live machine is not YOLO mode, it's a séance.
Head-to-head anecdotes point the same direction. One tester ran both models over ~200 rows of real e-commerce inventory data: Fable 5 delivered AOV, sell-through, eight insights and interactive charts in 5 minutes and ~90k tokens; Sol produced basic statistics and static charts in 11 minutes and ~140k. One data point, not a benchmark. But note that it agrees with the SWE-Bench Pro chart, not the Terminal-Bench one.
And the bright spot, because there genuinely is one: long-form writing. Multiple testers independently report 5.6's prose structure and stylistic control are markedly improved: drafts need minor edits instead of reconstruction. In a release cycle where every lab is betting the farm on coding, shipping a better writer is an unexpected and slightly charming move.
The failure mode was documented in the system card and shipped anyway. Full Access on a live machine is not YOLO mode, it's a séance.
The Theo protocol
Nobody's arc explains this launch better than Theo (t3.gg), who remains the single best filter on model discourse working today. When GPT-5.5 dropped he didn't like it, said so publicly, then spent two months re-prompting and reconfiguring before coming out the other side saying he couldn't use anything else for code. His conclusion wasn't "the model got better." It was that your prompts are technical debt: every carefully tuned setup silently degrades when the model underneath it changes, and what looks like a bad model is often your last model's configuration wearing a new model as a skin.
Apply that here and the day-one GPT-5.6 takes, including the glowing ones, are mostly measurements of people's old prompts. The honest evaluation window hasn't closed yet.
Theo is also already doing the unglamorous work on 5.6: he surfaced the bug where setting Sol to Ultra forces every sub-agent to inherit Ultra with no per-agent override, burning tokens for nothing. OpenAI's response amounted to "daily use of Medium is sufficient," which is vendor-speak for the Ultra tier is a token assassin. He distrusts Frontier Codebench because Opus 4.8's scores across reasoning levels look like a random number generator. He's building his own benchmark framework because the existing ones annoy him. This is exactly the posture this blog aspires to, executed with a bigger budget and better hair. We'll keep farming his sources. It's the closest thing to running the evals ourselves that we can afford.
How to actually use each model
- Orchestration, long chains, deep review, anything a human will read: Fable 5. The SWE-Bench Pro gap is real, the state-tracking edge over long agent chains is real, and the writing-quality gap on human-facing output is consistent. This is what the extra $5/$25 per million buys.
- Checklist-heavy autonomous agent work, including browsing, OS tasks, and terminal grinding: GPT-5.6 Sol at Medium effort. Its Terminal-Bench and BrowseComp numbers reflect something true: it stays oriented and doesn't quit. Never use Ultra (inheritance bug), and never use Full Access on a machine you care about.
- Default 5.6 tier for everyday work: Terra. It sits within 2 to 3 points of Sol on most suites at half the price, though field reports say the real-world savings are smaller than the pricing page implies. Test on your own workload before believing either number.
- Long-form drafting: GPT-5.6, unironically. The writing improvement is the most under-reported feature of this launch.
- Cheap, fast, good-enough: Grok 4.5 is the sleeper recommendation of the month: sufficient capability at a price and speed that make retry loops rational. Luna if you're staying in the OpenAI stack.
- Judging any of this from launch-day takes: don't. Run the Theo protocol: re-prompt for the new model, give it two weeks, then decide.
GPT-5.6 isn't on Fable's level, and it doesn't need to be for you to want it in your rotation. The question "which model won?" produces a headline. The question "which model, for which job, at what effort tier, with which permissions?" produces a working setup. The leaderboard answers the first. Your routing table answers the second. Only one of them ships code.
end of patch 3fc0e30