«Distaw'r lli, dyst yw'r lleuad»
A lone fisherman · cynghanedd groes
Evaluations → GPT-5.6 Sol
GPT-5.6 Sol commercial tested
An overall score of 82.94 and translation BLEU of 71.93 in CymraegBench v0.1.
The facts
| Developer | OpenAI |
|---|---|
| Released | Limited preview on 26 June 2026; public release on 9 July 2026 |
| The family | Three tiers: Sol (the most capable model), Terra (middle), Luna (fast and inexpensive) |
| Architecture | Undisclosed; external sources mention a 1.5-million-token context window, but this has not been officially confirmed |
| Licence | Proprietary (API only) |
Welsh: no data from OpenAI
We searched the GPT-5.6 system card, OpenAI’s safety hub and developer documentation, but there is no benchmark, claim or figure for Welsh. The GPT-5 system card (arXiv:2601.03267) evaluated multilingual MMLU in 13 or 14 languages, from Arabic to Yoruba, but Welsh was not among them. OpenAI has not published figures for Welsh text tokenisation efficiency either.
That does not mean Sol cannot use Welsh. The Coleg Cymraeg report found that earlier OpenAI models were among the most useful tools for correcting and structuring academic Welsh, although their idiom was often unnatural. The problem is the absence of public measurement by the developer. The Welsh Government has a partnership with OpenAI to share Welsh data, but no benchmarking results from it have yet been published.
CymraegBench v0.1 result
| Overall score | Welsh reasoning | Practical Welsh | Translation BLEU |
|---|---|---|---|
| 82.94 | 84.32 | 81.56 | 71.93 |
Sol is fourth among the twelve leading models, 0.07 points behind Muse Spark 1.2, 2.22 points behind Claude Opus 5, and 7.19 behind GPT-6 Astra, without refusing any of the 1,821 requests. See the full leaderboard and the open results.
Cynghanedd: High/Flex result
Sol was tested on Bangor AI’s cynghanedd protocol with High reasoning on the Flex tier. Here are four examples from the 40 themes held out from training:
| Valid, 7 syllables (raw) | Appropriate + valid + original | Unique lines | Proportion of requested lines generated |
|---|---|---|---|
| 23.0% | 16.0% | 100% | 42.5% |
«Marchnad y dre: lle i'r llu»
The town market · cynghanedd sain
«Ar draeth, heb don, mae'n llonydd»
A boat on the beach · cynghanedd lusg
«Trwm yw llaw y distawrwydd»
A closed quarry · cynghanedd lusg
The corrected checker accepted 46 of the 200 lines (23.0%). Of these, 43 were clean and free of unknown words; Claude Opus 4.8, acting as an independent semantic judge, judged 32 relevant and appropriate to their themes, giving a full score of 16.0%. All 85 returned lines were unique, and no exact match for the 32 was found in the project’s known public corpus. Since OpenAI’s training corpus is unavailable for inspection, that is the limit of the originality test. The model generated five lines each for 17 of the 40 themes; the other 23 produced no scorable lines and received zero. The 85 lines and judgements have been published so the totals can be checked again.
What comes next
Sol’s row has been rescored and the underlying data is now available. The next step is a human literary review of the highest-scoring lines and rescoring the other models’ raw outputs with the same checker version. Until then, the comparative figures are provisional.