Qwen3.8 2.4T-A95B open weights tested
Qwen’s large MoE model, with 2.4 trillion parameters and 95 billion active, tested through OpenRouter on the core suite and CCC Language Audit.
CymraegBench v0.1 result
| Measure | Score |
|---|---|
| Overall score | 81.09 |
| Welsh reasoning | 86.28 |
| Practical Welsh | 75.89 |
| Legislation translation | 60.84 BLEU |
Qwen3.8 2.4T-A95B placed sixth on the extended leaderboard, between Claude Opus 4.8 and Qwen3.8 Flash. It is the second-highest open-weight model in the table, after Muse Spark 1.2. It completed all 2,053 cases without refusal, invalid output or infrastructure errors.
What does the profile show?
Strong reasoning
Welsh ARC-Easy 98, Welsh COPA 98, Welsh MGSM 97, ARC-Easy mini 96 and Welsh ARC-Challenge 93.
Practical Welsh
Welsh ToxiGen 90, registers 85, vocabulary 76 and yes–no 74; the category score was 75.89.
Specific gaps
CEFR levels 32, bilingual place names 51 and grammar 54 were the weakest core tests.
CCC Language Audit
The model achieved a beta average of 89.99 across the seven subsets, the sixth-highest result. It scored 100 on Welsh and English instruction following, 90.63 on references and terminology, 89.91 on summarisation and quotation, and 84.38 on idiom. Correction and explanation was the weakest subset at 74.38: correction 79.69 and explanation 56.25.
How was the model tested?
The model was tested as qwen/qwen3.8-2.4t-a95b through OpenRouter, with temperature 0, random seed 1, seven concurrent requests and up to 100 cases per core suite. The setting was reasoning_effort=low; reasoning was kept separate and only the final answer was scored. Each response had a 32,768-token limit to avoid cutting off reasoning before completion. All 2,053 requests completed after one request was repeated following a transient API error.
Download the results
- Score summary (CSV)
- Scores for every suite (CSV)
- CCC Language Audit dimensions (CSV)
- Test details (JSON)
Model: Qwen/Qwen3.8-2.4T-A95B · API service: OpenRouter · test cymraegbench-qwen3-8-2p4t-a95b-openrouter-v0.1 · 22 August 2026.