GLM-5.3 Flash open weights tested
Z.ai’s multimodal MoE model, with 320 billion parameters and 18 billion active, tested through OpenRouter on the core suite and CCC Language Audit.
CymraegBench v0.1 result
| Measure | Score |
|---|---|
| Overall score | 74.38 |
| Welsh reasoning | 76.21 |
| Practical Welsh | 72.56 |
| Legislation translation | 58.41 BLEU |
GLM-5.3 Flash placed tenth on the extended leaderboard, between Kimi K3 and GLM-5.3. It completed all 2,053 cases without refusal, invalid output or infrastructure errors.
What does the profile show?
Strengths
Welsh ARC-Easy 96, Welsh COPA 95, ARC-Easy mini 94, Welsh PIQA 87 and Welsh TruthfulQA 86.
Practical Welsh
Registers 81, ToxiGen 77, obscenities 72, vocabulary and yes–no 70; the category score was 72.56.
Specific gaps
CEFR levels 28, bilingual place names 43 and Welsh MGSM 59 were the weakest core tests.
CCC Language Audit
The model achieved a beta average of 84.96 across the seven subsets. It scored 96.67 on English instruction following, 93.75 on terminology, 93.33 on Welsh instructions and 90.30 on summarisation and quotation. Correction and explanation was the weakest subset at 66.95: correction 71.88 and explanation 43.75.
How was the model tested?
The model was tested as z-ai/glm-5.3-flash through OpenRouter, with temperature 0, random seed 1, reasoning_effort=low and up to 100 cases per core suite. OpenRouter was asked to exclude reasoning from the response, and only the final answer content was scored. Each response had a 32,768-token limit; all 2,053 requests ended with stop.
Download the results
- Score summary (CSV)
- Scores for every suite (CSV)
- CCC Language Audit dimensions (CSV)
- Test details (JSON)
Model: zai-org/GLM-5.3-Flash · API service: OpenRouter · MIT licence · test cymraegbench-glm-5-3-flash-openrouter-v0.1 · 27 August 2026.