Claude Opus 5 vs. GPT-5.5 vs. Gemini 3.1 Pro: How to Choose When the Benchmarks Stopped Matching

The Comparison Table You Have Been Shown Is Probably Wrong

Anthropic shipped Claude Opus 5 on 24 July 2026. Within hours the comparison posts started, and almost all of them lead with a benchmark table putting three models side by side on the same axis.

Most of those tables do not survive checking. Across the three current flagships, exactly one benchmark is reported by more than one vendor: SWE-bench Verified, where Anthropic reports 96.0% for Opus 5 and Google reports 80.6% for Gemini 3.1 Pro. OpenAI published no SWE-bench Verified score for GPT-5.5 at all. Everything else in circulation is either single-vendor, or two numbers from benchmarks that only look like the same test.

This article gives you the figures that are genuinely comparable, flags the ones that are not, and sets out a selection method that does not depend on a leaderboard at all.

The Three Current Flagships, Verified

Every figure below comes from the vendor's own model documentation, checked on 24 July 2026. We excluded models you cannot buy, which is a filter worth applying to any comparison you read.

Three observations matter more here than any benchmark. Output ceilings differ by a factor of two, and if your product generates long documents or large code diffs in a single call, Gemini's 65,536-token cap forces chunking and merge logic into your codebase. Gemini's pricing has a step function that doubles input cost above 200K tokens, which long-context workloads hit routinely. And the knowledge cutoffs sit eight months apart, which shows up as wrong answers about recent frameworks and library versions unless you plan retrieval deliberately.

Key takeaway: the verifiable specifications already separate these models clearly enough to make a decision, without reference to a single contested benchmark score.

Why the Benchmark Scoreboard Stopped Working

This section changes how you read every other comparison article on this topic, so it is worth being specific about what each lab actually published.

Only one row has numbers in two columns. On SWE-bench Verified, Opus 5's 96.0% against Gemini 3.1 Pro's 80.6% is a real, like-for-like comparison and the single most useful published data point across these three models. Every other row is single-vendor, which means a table showing all three scoring on the same benchmark is combining different sources, different harnesses, and sometimes different benchmark versions.

Look closely at the ARC-AGI rows, because this is where most comparison content goes wrong. Google reported ARC-AGI-2 and Anthropic reported ARC-AGI 3. Those are different benchmarks, not different runs of the same one, and putting 77.1% next to an Opus 5 ARC-AGI 3 result produces a ranking that means nothing.

There is a second trap worth knowing about, because it caught us. Anthropic's launch announcement contains almost no absolute numbers. It reports relationships: Opus 5 more than doubles Opus 4.8 on Frontier-Bench v0.1, and at maximum effort lands within 0.5% of Claude Fable 5's peak CursorBench 3.2 score at half the cost. The absolute scores, including SWE-bench Verified, are in the 193-page System Card published alongside the model. If you check only the announcement page, you will wrongly conclude the numbers were never published.

The benchmarks themselves are also under scrutiny. In July 2026 OpenAI published an audit finding that roughly 30% of SWE-bench Pro tasks are fundamentally flawed, which means a third of a widely cited coding benchmark was called into question by one of the labs being measured by it.

Key takeaway: treat any cross-vendor benchmark table published in 2026 as a claim to verify, not a source. Check whether both numbers came from the same benchmark, at the same version, run by the same party.

The Cost Math That Actually Applies to Your Workload

Benchmarks are contested. Invoices are not. Take a mid-sized product feature handling 50,000 requests per month at roughly 8,000 input and 1,500 output tokens per request, which comes to 400M input and 75M output tokens monthly.

Gemini comes out 56% cheaper than Opus 5 on this shape of workload, before caching. Now push average input context to 250,000 tokens, which is normal for document analysis or reasoning across a large codebase.

The advantage narrows from 56% to 21% because the long-context tier kicked in. The ranking held, but most of the margin that justified the switch evaporated. Caching moves the picture again: GPT-5.5 discounts cached input to $0.50 per million, which on a large stable prompt prefix can turn the most expensive option into the least.

Key takeaway: model cost is a function of your traffic shape, not a property of the model. A comparison table cannot tell you which is cheapest for you, but twenty minutes with your own token counts can. Our guide to reducing Claude API token costs covers the settings that change this number regardless of which model you land on.

Availability and Lock-In: The Question Nobody Asks

Pricing and limits are the easy part. The harder question is what happens in eighteen months.

Claude Opus 5 has the widest deployment surface of the three, running on the Claude API, AWS Bedrock, Claude Platform on AWS, Google Cloud, and Microsoft Foundry. If your infrastructure is already committed to AWS or Azure, you can consume it without adding a vendor relationship or moving data across a new boundary. For regulated clients that single fact often settles the question before pricing comes up, which is why it features early in most of our enterprise Claude API integration engagements. Gemini 3.1 Pro is Google-native, and GPT-5.5 runs on OpenAI's API and Azure.

There is a second-order risk worth naming. Several of the most-discussed models in this category are gated, unshipped, or already superseded. Gemini 3 was current in February and is now behind. GPT-5 has been superseded more than once. Opus 5 replaces Opus 4.8, which replaced 4.7, which replaced 4.6, all within roughly six months.

Any architecture that hard-codes a single model ID will need revisiting within two quarters. We now treat the model layer as swappable by default, with the provider behind an internal interface and evaluations that run against whatever sits behind it. That is a modest amount of extra work at the start and it has repeatedly saved clients a rewrite. If you are already on one provider and reconsidering, our OpenAI to Claude migration work exists precisely because so many teams skipped that abstraction the first time.

Key takeaway: choose deliberately, then insulate the choice. The cost of an abstraction layer is small next to the cost of a migration you did not plan for.

How We Actually Choose

Since the scoreboard is unusable, here is the sequence we run instead. It takes about a day.

1. Filter by hard constraints first. Output ceiling, context window, cloud availability, data residency, and compliance posture. This usually eliminates at least one option before any evaluation work begins, and it is the cheapest step in the process.

2. Build a 30-task evaluation from your own product. Not SWE-bench. Thirty real inputs from your actual use case, with outputs you can grade. This is the highest-value hour in the entire exercise and the only benchmark that describes your workload. Vendors cannot publish it, which is exactly why it is worth having.

3. Run every surviving candidate at matched settings. Opus 5 exposes an effort parameter that defaults to high on the Claude API. Comparing a high-effort Claude run against a default-configuration competitor is not a fair test and will flatter Anthropic.

4. Price the winner against your real traffic shape, using the method above with your own token counts and cached-prefix rates.

5. Re-run the evaluation quarterly. Given the release cadence above, an annual review is too slow.

Steps two and three are where teams get the most value and invest the least. If the decision in front of you is narrower than this — which coding assistant your engineers should standardise on rather than which foundation model your product should call — our Claude vs. Codex comparison covers that question directly. For teams that have already chosen and need the integration built properly, Claude implementation covers the work that follows.

So Which One Should You Pick

The honest answer is that the published numbers settle less than they appear to. One benchmark is genuinely shared across two of these three vendors, several widely quoted pairings are not comparable at all, the shared benchmarks are being audited by the labs themselves, and the highest-scoring models are frequently ones you cannot buy.

What remains verifiable is enough to decide with. Gemini 3.1 Pro is materially cheaper until your context crosses 200,000 tokens, and it caps output at half what the others do. Claude Opus 5 has the widest cloud availability and the most recent knowledge cutoff, priced identically to the model it replaces. GPT-5.5 sits between them, with aggressive input caching that rewards stable prompt prefixes.

All three are closed-weight APIs, which is the shared constraint worth noticing. If data residency, self-hosting, or control over the weights is part of your decision, the comparison changes shape entirely, and our Kimi K3 vs Claude Opus 5 piece covers that axis against the strongest open-weight option currently available.

Your next step is step two above: pull 30 real inputs from your product and grade three models against them. That artifact will outlast every model in this article. If you would rather not build the evaluation harness yourself, we do this as part of technical discovery through our AI development practice, and you keep the harness either way.