Testing, measuring and developing Welsh AI
Evaluations → MiMo V2.6 Pro

MiMo V2.6 Pro open weights tested

Eighth on the extended CymraegBench leaderboard with an overall score of 80.73.

Four evaluators studying abstract models in a transparent testing arena

The facts

DeveloperXiaomi
API IDmimo-v2.6-pro
Test serviceLocal OpenAI-compatible endpoint
Reported context262,144 tokens
Test cost$0.00 (local service)

CymraegBench result

Overall scoreWelsh reasoningPractical WelshTranslation BLEU
80.7383.7977.6761.98

The same sample was used as for the other leading models: up to 100 cases from each of the 19 core suites, with random seed 1. The 232 CCC cases do not contribute to the overall score.

MiMo V2.6 Pro is eighth on the extended leaderboard: 0.36 points behind Qwen3.8 2.4T-A95B and 0.92 points ahead of Qwen3.8 Flash. It completed 2,036 of the 2,053 cases; the remaining 17 invalid outputs scored zero.

Note on long outputs: 55 requests failed to produce a final answer within the initial 2,048-token limit. Only those requests were rerun with higher limits. Thirty-eight were recovered, including eight at 16,384 tokens. Seventeen remained invalid: 11 correction-and-explanation cases, four place names, and one case each in ARC Challenge and ARC Easy. There were no unresolved infrastructure errors.

CCC Language Audit

SubsetScore
Correction and explanation70.70
Idiom81.25
Terminology78.13
Summarisation and quotation87.89
References74.69
Welsh prompt100.00
English prompt100.00
Average across the seven subsets84.66

On correction and explanation, MiMo scored 75.00 for correction and 42.19 for explanation. Eleven of the 64 cases were invalid outputs counted as zero. It completed the rest of the CCC audit.

See the full leaderboard, CCC Language Audit, or download the summary, scores by suite, CCC dimensions and the metadata.