Bangor AI research
Bangor AI's open research into improving language models' Welsh output, and checking and evaluating the results.
Bangor AI contributes to this work. The project repository is BangorAI/cynghanedd on GitHub; the cynghanedd checker's code and development work are published there.
Bangor AI's research programme considers a challenging question: can a machine compose correct cynghanedd? Welsh strict-metre poetry is a useful test of machine capability in Welsh because it demands detailed phonology, natural syntax and meaning simultaneously. Formal aspects of each line can be assessed against rules codified since the publication of Cerdd Dafod by John Morris-Jones.
How the system works
This is the idea we are testing:
Once the phonology is known, rules can determine whether cynghanedd is valid. Generating it is a search problem. The rules engine therefore judges, while language models propose lines.
Bangor AI's work has developed step by step. First came a phonological engine, combining Bangor's pronunciation dictionary with letter-to-sound rules. Next came a rules-based cynghanedd checker, tested and strengthened on a corpus of real poetry. The next step was a generation cycle in which a model proposes a line and the checker judges it. From that grew the improvement cycle: the model generates many lines, the checker labels them, and the labelled examples train the next model. The team is now also conducting cluster experiments to fine-tune larger models on Welsh.
The rules engine
A cynghanedd checker that follows the same rules every time: how it pronounces, validates and responds to shortcuts discovered by models.
The improvement cycle
How a 9B model went from 18% to 65% valid lines on one graphics card—and four ways it exploited weaknesses in the measures.
From laboratory to benchmark
Bangor AI's research measures—valid-at-7, novel and clean, and thematically relevant—now underpin AIsteddfod's evaluation pages.
Being open about the work
The work is still developing. One experiment produced a cywydd without formal errors according to the checker, though human literary judgement remains necessary; other experiments, such as fine-tuning on an eight-machine cluster, remain incomplete. Bangor AI publishes problems alongside results to help explain AI's capabilities and limitations in Welsh. Bangor AI's tools—above all the checker—help us assess model-generated work.