Claude Opus 5.5 shipped with nine benchmarks. Anthropic hedged them itself

Claude Opus 5.5 landed on 22 September 2026, and Anthropic published a table of nine benchmarks. Opus 5.5 leads on seven. On the other two, AutomationBench and Terminal-Bench-Science, GPT-6 Astra is ahead.

Directly under that table, in Anthropic's own words, sits the line that "benchmark margins have become a less reliable guide to real-world differences". A few lines further down: "In our own use, the gap between Opus 5.5 and Claude Fable 5.1 is narrower than these scores suggest."

That's a vendor publishing its own scoreboard and, in the same breath, asking you not to settle the question on it.

Which leaves the question of what you should settle it on. The answer is on the same page, in the twelve customer quotes Anthropic chose to print. Not one of them is a benchmark score.

Claude Opus 5.5 has landed, and the price came down

Start with the price, because it's the only part of the announcement that's a number you can check yourself.

Claude Opus 5.5 costs $4 per million input tokens and $20 per million output tokens. Opus 5 cost $5 and $25. That's 20 percent off both. A five-minute cache write drops from $6.25 to $5, and a one-hour write from $10 to $8. Through the Batch API it's half as much, $2 and $10.

The line that moves furthest is the one fewest people watch. Cache reads fall from $0.50 to $0.20 per million tokens. That's 60 percent, three times as far as any of the other four lines travel. If you run an agent that reads the same set of documents over and over, that's the line that decides your bill, and it isn't in any headline.

The rest of the specifications: a one-million-token context window, output up to 128,000, and a knowledge cutoff of June 2026 against Opus 5's May. Anthropic says it generates output more than 30 percent faster than Opus 5. There's also a fast mode in research preview on the Claude API, up to 2.5 times faster again at double the price, $8 and $40.

Anthropic's own summary reads: "Our tests show that at default settings it will cost 40% less than Opus 5 on typical workloads." The two qualifiers in that sentence aren't decoration. "At default settings" matters because the default setting changed in the same release: the effort parameter now sits at medium, where on Opus 5 it sat at high. So part of that 40 percent is the dial being turned down. Anthropic isn't hiding this. They write it on the migration page and tell you to set effort explicitly and re-run your own sweep.

And "on typical workloads" means Anthropic's typical workloads, not yours. How much of that 40 percent shows up on your own bill depends on what you actually run.

One more thing that matters if you're working under European rules. Opus 5.5 can be used with zero data retention, and the wording has to be exact here: "Like previous Opus models, Opus 5.5 is available with zero data retention." So the retention rules haven't moved. What's moved is what you get inside them.

Claude Fable 5.1 requires 30-day retention and isn't available under zero data retention unless Anthropic expressly authorises it. If your data processing agreement shuts Fable 5.1 out, and Opus 5.5 performs at Fable 5.1's level on most work by Anthropic's own account, then there's more capability inside the boundary you already drew. Get that confirmed by your supplier rather than assuming it.

Claude Sonnet 5.5 and Claude Haiku 5.5 are coming "in coming weeks". Hold on to that sentence. It comes back.

The table Anthropic hedged itself

Nine rows. Terminal-Bench 4.0: 66.4 percent against Opus 5's 52.3 and Fable 5.1's 55.8. FrontierCode v1.1: 54.4 against 48.0. CursorBench 4.0: 57.8 against 46.6. GDPval-AA v2.1: 1,846 Elo against 1,708. Humanity's Last Exam with tools: 67.7 against 63.6. OSWorld 2.0: 81.8 against 74.0. Chartography: 89.0 against 83.4.

Two rows go the other way. On AutomationBench, GPT-6 Astra scores 41.4 percent against Opus 5.5's 40.0. On Terminal-Bench-Science 0.1 the gap is wider: 64.6 against 58.7.

These are Anthropic's figures about Anthropic's own model, and they should be read that way. But notice what Anthropic does alongside them, because that's the unusual part. They print the margin and then play it down themselves.

Read the sentence again: the margins have become "a less reliable guide to real-world differences". Nobody is saying the numbers are wrong. The claim is about resolution. Two models sitting at 66.4 and 55.8 do sit measurably apart. The question is whether they sit apart on your work, and there Anthropic says no: in their own use the distance is smaller than the table suggests.

When a vendor puts that caveat on its own launch page, something has happened that nobody said out loud. The measuring became your job, and no notice went out saying the work had moved.

It's also where a typical AI implementation goes wrong, and rarely because of the model. Nobody ever made a choice that could be checked afterwards. Someone read a table, picked the name at the top, and moved on.

Count what you pay for

So what did the people who actually measured do?

Anthropic prints twelve customer quotes on the launch page. Go through them looking for a benchmark score. There isn't one. Here are six of them, and every one counts something the company could count itself:

Quantium, verbatim: "A complex coding task that previously took 38 prompts over four days came in at 11 prompts over three hours." Two numbers, no benchmark. How many times somebody typed at the model, and how long it took.

Box: "Claude Opus 5.5 used a third of the tokens Opus 5 did, and its answers were 40% less verbose."

Rogo: "Claude Opus 5.5 beat Opus 5 at high effort on our BigFinance Bench with about 60% fewer output tokens."

Kiro: "Claude Opus 5.5 solved more than Opus 5 while making about 40% fewer calls."

GitHub: "Claude Opus 5.5 used among the fewest tokens and steps we measured."

Deloitte: "Claude Opus 5.5 caught 72% of known bugs in our code reviews to Opus 5's 56% at high effort."

Prompts. Hours. Tokens. Calls. Steps. Bugs caught. That's the currency, and it wasn't chosen out of modesty. It was chosen because it's the only unit these companies can count themselves, and the only one that says anything about their own work.

Look at the Quantium number once more. Thirty-eight prompts over four days became eleven prompts over three hours. No scale, no percentage improvement. A task that used to eat four working days now finishes before lunch. Anyone in a 40-person company can see that difference without understanding a single benchmark.

And here's what makes it more than an observation: you can count exactly the same things. Number of prompts, how long it took, number of times you sent it back. It needs no developer, no API access and no account beyond the one you already have. That's why these are the numbers customers can quote in public, and it's why they're the right numbers for you.

Your AI strategy has to outlive the model

Here's that sentence again. Sonnet 5.5 and Haiku 5.5 are a few weeks out.

Now look at what happened to Opus 5. It shipped on 24 July 2026. Today, 22 September, its page at Anthropic is headed Legacy, and its status reads "Active (legacy)". Anthropic's own definition of Legacy is a model that will no longer receive updates and may be deprecated later. The page also says: "Although Claude Opus 5 is still available, you should consider migrating to Claude Opus 5.5 for improved performance."

Opus 5 isn't deprecated and it isn't switched off. It runs, it's supported, and its retirement date is no earlier than 24 July 2027. But as the newest Opus model, it lasted sixty days.

Sixty days is shorter than a procurement cycle at most companies. It's shorter than a pilot. It's a good deal shorter than the kind of vendor comparison where three models go head to head and six weeks get set aside to work out which one wins.

That's where the usual approach breaks. A comparison that takes six weeks delivers its answer after its own subject has changed. You end up with a thorough document about a lineup that no longer exists, and because the work was expensive, the conclusion stands for a year.

The same is true of the instructions you've written yourself. If you have a CLAUDE.md file or a set of skills, they were written for whichever model was newest at the time. The command /claude-api prompt-audit in Claude Code holds your own prompts, skills and instruction files up against a target model you name, Opus 5.5 for instance. It changes nothing on its own: you get a report with a file and line number for every place written for an older model, and a proposed diff. That tidies up what you've written. The other half is measuring your own work.

The only measurement that's worth anything now is the one you can run again in an afternoon. The rigour stays. The subject changes: away from which model is best, towards what your own ten tasks cost in prompts and hours right now, so you have something to hold the next release up against.

An AI strategy that names a model lasts until the next launch. An AI strategy that names a measurement lasts across them.

Ten tasks, three numbers, one afternoon

Here's the sheet. That's the whole method, and it takes an afternoon.

Pick ten tasks you actually ran last month. Not ten you can imagine. Ten you can find again: a proposal that got written, a dataset that got cleaned up, an email thread that got summarised, a contract that got read through.

Write three numbers next to each one:

1. How many prompts it took before you were done and could use the answer.
2. How long it took from starting to finishing.
3. How many times you sent it back because the first answer wasn't good enough.

Put today's date on the sheet. That's it.

Those three numbers were chosen because you can fill them in whatever you're running. You don't need to own the integration. You don't need to see which model sits under the tool, or what effort level it's running at. Prompts, time and send-backs can all be counted from the outside, by the person sitting there waiting for an answer. It's the only measurement in this whole argument that needs access to nothing.

Here's what a filled row looks like, borrowing Quantium's task: 38 prompts, four days, an unknown number of send-backs, dated before the upgrade. The same task after: 11 prompts, three hours. That's the entire analysis. There isn't a model in it.

When Sonnet 5.5 lands in a few weeks, you take the sheet out and run the same ten tasks again. That's one more afternoon. The difference between the two sheets is the only comparison that's about your work.

The hard part isn't counting. The hard part is knowing whether the numbers you get are any good. Eleven prompts in three hours says nothing on its own, and you've got nothing to hold it against except the vendor's own page. As an AI consultant in Copenhagen, that's the part I do for companies: set the measurement up so it lands on the work that actually costs something, and read the result with you, so a number turns into a decision. It's also what separates a real AI audit from a tour of the tooling, and it's the first thing I put in place on a Claude implementation.

If you'd rather have somebody watching this continuously instead of having the same argument at every launch, an external AI lead on a retainer is the shape that fits. And if you want to see what this kind of measurement did to a real bill, I wrote about it when a model upgrade made the work 36 percent more expensive.

One last thing, and it's what wasn't anywhere in the material. Every number in the release compares Opus 5.5 to Opus 5, to Fable 5.1 or to GPT-6 Astra. Not one compares it to what you're running today, because nobody at Anthropic knows what that is. Their page is missing nothing. That measurement was never going to be anybody's job but yours.

Sources

• Anthropic: Introducing Claude Opus 5.5
• What's new in Claude Opus 5.5: the breaking changes
• The Claude Opus 5.5 model page: pricing and dates
• The Claude Opus 5 model page, now headed Legacy
• Anthropic's model deprecations page
• Claude Platform release notes, 22 September 2026
• Anthropic's pricing page
• Brinvik: a better model, a 36% bigger bill

How this article was made. Claude read Anthropic's own pages, re-fetched every quote word for word, and built every file with nobody present. Gate 1 killed the first angle, because a Brinvik article from 1 September had already written it. A verbatim source check killed the next one: Anthropic says zero data retention on Opus 5.5 is as it was on previous Opus models, so there was no change to write about. Kim wrote the rules, the voice and the checklists in advance, and they were enforced throughout.