GPT-6.1 Sol commercial tested
Second on the extended CymraegBench leaderboard with an overall score of 89.05.
The facts
| Developer | OpenAI |
|---|---|
| API ID | gpt-6.1-sol |
| Standard pricing | $2 / $10 per million tokens (input/output) |
| Context | 1,050,000 tokens; up to 128,000 output tokens |
| Full run cost | $1.17 |
CymraegBench result
| Overall score | Welsh reasoning | Practical Welsh | Translation BLEU |
|---|---|---|---|
| 89.05 | 88.11 | 90.00 | 72.16 |
The same sample was used as for the other leading models: up to 100 cases from each of the 19 core suites, with seed 1 and reasoning_effort=low. The model completed all 2,053 cases without refusal, invalid output or infrastructure errors.
GPT-6.1 Sol is second: 1.08 points behind GPT-6 Astra and 1.50 points ahead of Claude Opus 5.5.
CCC Language Audit
| Subset | Score |
|---|---|
| Correction and explanation | 92.42 |
| Idiom | 96.88 |
| Terminology | 96.88 |
| Summarisation and quotation | 92.68 |
| References | 89.69 |
| Welsh prompt | 64.17 |
| English prompt | 56.67 |
| Average across the seven subsets | 84.20 |
GPT-6.1 Sol scored 96.88 for correction and 57.81 for explanation. On the references test, it scored 100 for references and not inventing a source, but 65.63 for answer content.
How was the model tested?
The model was tested through OpenAI’s Responses API with seven concurrent calls. A ceiling of 32,768 tokens allowed hidden reasoning to finish, and only the final answer was scored. The run used 248,950 input tokens and 67,686 output tokens; 40,698 output tokens were hidden reasoning and 26,988 were visible answers.
Download the results
- Score summary (CSV)
- Scores for every suite (CSV)
- CCC Language Audit dimensions (CSV)
- Test details (JSON)
Model: GPT-6.1 Sol · test cymraegbench-gpt-6-1-sol-v0.1 · 15 October 2026.