Qwen3.8 Flash open weights tested
Qwen’s multimodal reasoning model tested through OpenRouter on the core suite and CCC Language Audit.
CymraegBench v0.1 result
| Measure | Score |
|---|---|
| Overall score | 79.81 |
| Welsh reasoning | 84.06 |
| Practical Welsh | 75.56 |
| Legislation translation | 58.79 BLEU |
Qwen3.8 Flash placed seventh on the extended leaderboard, between Qwen3.8 2.4T-A95B and Claude Fable 5. It received a score on all 2,053 cases: 2,052 complete answers and one invalid output scoring zero. There were no refusals or unresolved infrastructure errors.
What does the profile show?
Strong reasoning
Welsh ARC-Easy and COPA 98, Welsh MGSM 97, ARC-Challenge and ARC-Easy mini 94, and Welsh MMLU-lite 90.
Practical Welsh
Welsh ToxiGen 94, registers 87, grammar 78, vocabulary 75 and yes–no 70; the category score was 75.56.
Specific gaps
Bilingual place names and CEFR levels, both at 33, were the weakest core tests.
CCC Language Audit
The model achieved a beta average of 91.92 across the seven subsets, the fourth-highest result. It scored 100 on Welsh and English instruction following, 95.18 on summarisation and quotation, 93.75 on idiom and 89.22 on references. On correction and explanation, it scored 89.06 for correction and 58.59 for explanation.
How was the model tested?
The model was tested as qwen/qwen3.8-flash through OpenRouter, with temperature 0, random seed 1 and up to 100 cases per core suite. Reasoning was enabled by default; it was kept separate and only the final answer content was scored. A 32,768-token limit was generally used for each response. One CCC response reached that limit, and then the maximum of 131,072 tokens without producing a final answer; it was recorded as invalid with a zero score under the protocol.
Download the results
- Score summary (CSV)
- Scores for every suite (CSV)
- CCC Language Audit dimensions (CSV)
- Test details (JSON)
Model: Qwen/Qwen3.8-Flash-Next (the open-weight model underlying the Flash service) · API service: OpenRouter · test cymraegbench-qwen3-8-flash-openrouter-v0.1 · 27 August 2026.