China's open-weight models caught up on agentic work

For the past two years, the safe assumption for serious agentic work — multi-step reasoning, tool calls, hundreds of pages of documents — was that only the frontier models could do it. Open-weight alternatives were something you benchmarked out of curiosity and then quietly set aside.

That assumption just expired. We ran the same production tender analysis twice — once on Claude, once on Zhipu’s open-weight GLM-5.3 — and both runs reached the same recommendation, flagged the same three most serious problems, and landed within 17% of each other on cost.

Tender analysis is just the example here. The finding generalises to any automated workflow — the same kind of work as the use cases we’ve written about: AML and compliance checks, document generation, building condition reports, employee helpdesks, ERP automation. If an open-weight model can carry a 94-requirement procurement analysis end to end, the “open models can’t do real work” objection is gone — and that changes how you should think about model choice.

What changed in three months

In May, we benchmarked four models on a synthetic tender. GLM-5.2 finished last on reliability: it completed one run out of six before failing on malformed JSON. That was the state of the art for open-weight models on long automated pipelines — impressive on paper, unable to finish the job.

In August we re-ran the comparison on a live telecommunications procurement: a real vendor’s bid against a real tender with 94 checkable requirements, 40 of them mandatory. No planted defects, no synthetic setup. One run used Claude (Opus for the main reasoning agent, Sonnet for verification). The other used GLM-5.3 for both stages, accessed through OpenRouter.

Both runs independently recommended the same thing — submit, with improvements — and both found the same three critical issues:

• An incomplete price table that pushed variable cost outside the fixed contract sum
• A certificate declared by the bidder but never actually attached
• A large share of mandatory requirements the vendor admitted would not be ready by the submission date

Three months from “can’t finish the job” to “agrees with the frontier model on everything that matters” is a remarkable pace.

Cheap per token is not cheap per job

Here is the part the price sheet won’t tell you. On list prices, GLM-5.3 looks roughly 3.6× cheaper than Claude on input tokens and up to 5.7× cheaper on output. If you budgeted from the price sheet, you would expect around 70% savings.

The invoices said otherwise. The Claude run cost $6.64. The GLM-5.3 run cost $5.52 — a saving of 17%, not 70%.

The gap has a simple explanation: the cheaper model worked harder for the same result. GLM-5.3 burned 9.42 million tokens across 223 model calls, against Claude’s 5.49 million across 147 calls — 1.7× more tokens for the same job. Its reasoning traces alone consumed 295,000 thinking tokens against Claude’s 17,000. And Claude’s prompt caching (a tenth of the input price on cache reads) clawed back most of the remaining difference.

This matches what we see across the workloads running on the Mitigate AI Platform: per-token price is a marketing number; cost per completed job is the real one. It’s exactly why the platform tracks spend per workspace — so you can compare providers on what a finished task actually costs you, not on what the rate card promises.

Where the frontier still earns its price

The two runs were not identical, and the differences are instructive.

• Claude was more complete. It surfaced 12 findings to GLM’s 11, including two issues GLM missed entirely: service-level commitments that only started after a delivery milestone, and delivery dates quietly conditioned on the buyer responding within five working days.
• GLM was more precise with numbers. It counted the mandatory requirements exactly — 27 of 40 met, 12 partial, 15 not yet built — and quoted exact contract sums and deadlines where Claude summarised.
• Reliability still favours the frontier. GLM-5.3 emitted one 98-kilobyte unterminated string, silently dropping a single requirement check. In May that same failure mode killed five runs out of six; now it cost one item out of 94. Improving fast, but not yet boring — and for production agents, boring is the goal.

The public benchmarks tell the same story. Independent reporting puts GLM-5.3 slightly ahead of Claude Opus 4.8 on Code Bench (31.4% against 29.5%) and within a point of Claude Mythos 5 on CyberGym vulnerability discovery — while Claude Fable 5 still leads Code Bench at 39.5%. The frontier is still ahead; “far behind” is no longer an accurate description of the gap.

So the honest summary is not “open models won.” It’s that the gap has narrowed from category difference to engineering trade-off — completeness and reliability on one side, price and precision on the other.

The question is no longer capability

For many of our clients — especially in the public sector and regulated industries — the first question about any AI system is not “how good is it?” but “where does my data go?”

That is where open-weight models genuinely change the conversation. When the weights are downloadable, the question shifts from do I trust this vendor’s API? to which infrastructure do I want this running on? — your own hardware, a vetted European provider, or a routing service. That is a procurement decision, not a leap of faith.

Two honest caveats, both worth stating plainly:

• GLM-5.3’s weights were not yet public at the time of the analysis. Zhipu announced the model on August 14 with a two-week delay on the weight release, citing safety evaluation — its predecessor GLM-5.2 is openly available under an MIT licence.
• OpenRouter is a routing service, not a data-residency solution. Running an open-weight model through an API gateway gives you model choice and price competition; it does not by itself keep your documents on European soil. If residency is your requirement, that takes a deliberate hosting decision.

How this works on the Mitigate AI Platform

We built the platform on the assumption that no single model provider would stay the right answer for long. This month is proving that assumption right faster than we expected.

• Provider choice per workspace. The platform runs on OpenAI, Anthropic, Google Gemini, Vertex AI, Amazon Bedrock, Mistral, DeepSeek, xAI — and OpenRouter, which is the door to the whole open-weight ecosystem, GLM included. Each workspace carries its own provider keys and model selection.
• Switch without starting over. Your knowledge base, connectors, and agent configuration stay put when you change models. Trying an open-weight model on your actual workload is a settings change, not a migration project.
• Different models for different jobs. Chat and image generation can each use a different model, and reasoning depth is a per-workspace dial — so you can put a frontier model on the high-stakes analysis and a cheaper one on the routine volume.
• Cost tracking per workspace. The numbers above are one data point from one workload. Yours will differ — and the only way to know is to measure cost per job on your own documents, which the platform reports out of the box.

We use this ourselves: our own workspaces don’t all run on the same provider, because the right answer differs by workload. That is the position we’d recommend to anyone — not loyalty to a logo, but a setup where switching is cheap and the invoice, not the price sheet, makes the decision.

What to do with this

If you evaluated open-weight models a year ago and wrote them off, that conclusion has expired. If your organisation handles documents that cannot leave your infrastructure, the analysis you were told required a frontier API may no longer require one. And if you’re already running automated workflows on a frontier model, it is worth an afternoon to run one of them against an open-weight model and read the invoice.

We can set that comparison up on your own documents, on your own workloads, with the results in a cost report you can take to your board. Get in touch — we’ll show you what your cost per job actually is.

Sources

• Claude API pricing (Anthropic). Source of the Opus 4.8 ($5 / $25 per million tokens) and Sonnet 4.6 ($3 / $15) rates, and of the cache-read and cache-write multipliers used to compute the run cost.
• GLM-5.3 — API pricing and providers (OpenRouter). Source of the $1.40 input / $4.40 output / $0.26 cache-read rates and the 1M-token context window.
• Pricing — Z.AI API platform (Z.AI Developer Documentation). Z.ai’s own list price for GLM-5.3: $1.40 per million input tokens, $0.26 cached input, $4.40 output — the same rates OpenRouter charges.
• GLM-5.3: Frontier Coding with Emergent Cyber Capabilities (Z.ai, 14 August 2026). The primary announcement. Source of the launch date, the commitment to “release the weights in two weeks after launch, once safety evaluation and hardening are complete”, and the statement that “as we scaled post-training, cyber capability developed faster than we expected”.
• zai-org/GLM-5.2 (Hugging Face). Source of GLM-5.2’s MIT licence, 753-billion-parameter count, and June 2026 release.
• New Released — model release notes (Z.AI Developer Documentation). Source of the 18 August 2026 API-availability entry for GLM-5.3 and of Z.ai’s own claim that the model matches Claude Mythos 5 on white-box code review and vulnerability discovery.
• GLM 5.3: benchmarks, pricing and the held-back weights (Fello AI, August 2026). Source of the reported Code Bench and CyberGym figures against Claude Opus 4.8, Claude Mythos 5, and Claude Fable 5.
• Mitigate AI Platform (Mitigate AI). The platform this analysis ran on — provider choice per workspace, extended reasoning settings, and per-workspace cost tracking.
• Use cases (Mitigate AI). The automated workflows referenced in the introduction: AML and compliance checks, document generation, building condition reports, employee helpdesks, and ERP automation.
• Workspaces (Mitigate AI Platform documentation). Source for workspace-level provider keys — OpenAI, Anthropic, Google Gemini, DeepSeek, Mistral, OpenRouter — and per-workspace model selection.
• Organization settings — models (Mitigate AI Platform documentation). The full provider table shown in the screenshot above, including OpenRouter chat and transcription support.