Evaluations → DeepSeek V4.1 Flash

DeepSeek V4.1 Flash open weights tested

DeepSeek’s open-weight model, run locally on the core suite and CCC Language Audit.

Four evaluators studying abstract models in a transparent testing arena

CymraegBench v0.1 result

Measure	Score
Overall score	69.45
Welsh reasoning	76.23
Practical Welsh	62.67
Legislation translation	69.03 BLEU

DeepSeek V4.1 Flash ranks 13 on the extended leaderboard, between Qwen3.8 27B and GLM-5.2. It completed all 2,053 cases on the first attempt, without refusal, invalid output, retry or infrastructure error.

What does the profile show?

Stronger reasoning

It scored 97 on Welsh ARC-Easy, 91 on ARC-Challenge, 89 on COPA and 86 on PIQA. Its reasoning score of 76.23 is 13.56 points higher than its practical score.

Strong translation

Its BLEU of 69.03 is the seventh highest in the full table, and higher than every model below rank 8.

Clear improvement

Compared with DeepSeek V4-Flash-0731, its overall score rose 7.82 points, reasoning 13.09, practical Welsh 2.56, BLEU 4.08 and the CCC average 6.56.

CCC Language Audit

The model achieved a beta average of 82.06 across the seven subsets, the twelfth-highest result. It was strong on summarisation and quotation (96.25) and idiom (93.75), and scored 90.00 with an English prompt compared with 76.67 with a Welsh prompt. Within the correction test, it scored 79.69 for correction but 48.44 for explanation, a gap of 31.25 points.

How was the model tested?

The model was tested as deepseek-v41-flash locally through vLLM on eight GPUs with tensor parallelism. Temperature 0, random seed 1, seven concurrent calls and up to 100 cases per core suite were used. The model’s thinking mode was disabled. The full run took around 9 minutes 41 seconds.

The test used 270,864 input tokens and 23,761 output tokens. As it used our own local service, no API cost is recorded.

Download the results

Model: DeepSeek V4.1 Flash · service: local vLLM · test cymraegbench-deepseek-v41-flash-local-v0.1 · 10 September 2026.