Evaluations → GLM-5.3

GLM-5.3 open weights tested

Z.ai’s open-weight reasoning model, post-trained from the same base model as GLM-5.2. It was tested through OpenRouter on the core suite and CCC Language Audit.

Four evaluators studying abstract models in a transparent testing arena

CymraegBench v0.1 result

Measure	Score
Overall score	73.38
Welsh reasoning	77.31
Practical Welsh	69.44
Legislation translation	58.05 BLEU

GLM-5.3 placed eleventh on the extended leaderboard, between GLM-5.3 Flash and Qwen3.8 27B. It completed all 2,053 cases without refusal, invalid output or final infrastructure errors.

What does the profile show?

Strengths

Welsh COPA 94, Welsh ARC-Easy 91, Welsh MGSM and ARC-Easy mini 88, Welsh PIQA 87 and Welsh ARC-Challenge 85.

Practical Welsh

ToxiGen 83, MMLU-Lite 81 and registers 78 were the strongest practical results; the category score was 69.44.

Specific gaps

CEFR levels 33, grammar 49 and bilingual place names 51 were the weakest core tests.

CCC Language Audit

The model achieved a beta average of 82.94 across the seven subsets. It scored 95.83 on English instruction following, 93.33 on summarisation and quotation and 89.17 on Welsh instructions. Correction and explanation was the weakest subset at 67.58: correction 70.31 and explanation 44.53.

How was the model tested?

The model was tested as z-ai/glm-5.3 through OpenRouter, with temperature 0, random seed 1, reasoning_effort=low and up to 100 cases per core suite. OpenRouter was asked to exclude reasoning from the response, and only the final answer content was scored. A 32,768-token limit was used on the first attempt; 17 requests returning empty or incomplete answers were repeated with the limit raised to 65,536 tokens. All then returned complete answers. Recorded successful calls cost $0.7541; the key’s total daily usage after the test was below $0.90.

Download the results

Model: zai-org/GLM-5.3 · API service: OpenRouter · GLM-5.3 licence · test cymraegbench-glm-5-3-openrouter-v0.1 · 28 August 2026.