Testing, measuring and developing Welsh AI
Evaluations → Claude Opus 5.5

Claude Opus 5.5 commercial tested

Second on the extended CymraegBench leaderboard with an overall score of 87.55.

Four evaluators studying abstract models in a transparent testing arena

The facts

DeveloperAnthropic
Released22 September 2026
API IDclaude-opus-5-5
Price$4 / $20 per million tokens (input/output)
Context1 million tokens; up to 128,000 output tokens
LicenceProprietary (API)

Model sources: Anthropic announcement and Claude Opus 5.5 specification.

CymraegBench result

Overall scoreWelsh reasoningPractical WelshTranslation BLEU
87.5589.5585.5672.52

The same sample was used as for the other leading models, selected deterministically to return the same cases each time: up to 100 cases from each of the 19 core suites, with random seed 1. The settings were effort=low and a minimum output allowance of 512 tokens; four responses that reached the limit were rerun with 2,048 tokens. The 232 CCC cases do not contribute to the overall score.

Opus 5.5 is second on the extended leaderboard: 2.58 points behind GPT-6 Astra and 2.39 points ahead of Opus 5. It completed all 2,053 cases without a refusal or invalid output.

Scoring note: on the binary Grammar and ToxiGen tests, a clear label of Y or N at the start of the answer is accepted even if followed by an explanation. A database copy was retained before rescoring. The audit also revealed one answer ending in Y when the target was N ; this was not counted as correct. That is why the published result is 87.55, not the initial estimate of 87.61.

CCC Language Audit

SubsetScore
Correction and explanation96.48
Idiom100.00
Terminology100.00
Summarisation and quotation94.46
References86.41
Welsh prompt100.00
English prompt100.00
Average across the seven subsets96.76

Opus 5.5 scored full marks for correction, category and rule, and 64.84 for explanation. On the references test it scored 90.63 for selecting references, 73.44 for answer content, and 100 for not inventing a source.

See the full leaderboard, CCC Language Audit, or download the summary, scores by suite, CCC dimensions and the metadata.