About
An independent site observing, testing and recording the progress of AI in Welsh.
The idea in brief
Artificial intelligence models are learning to use Welsh — in text and speech — but public evidence of their ability is scattered. AIsteddfod keeps a consistent record of their progress by publishing tests, results and open research.
Why look beyond a single score?
A model may score reasonably well on a Welsh multiple-choice test yet fail to produce a valid englyn or a story that reads naturally. Form, meaning, naturalness and understanding of context need to be considered alongside scores. The Coleg Cymraeg report and our research into cynghanedd help explain those differences.
Our principles
- Transparency. Testing methods are published alongside results so others can inspect and repeat them. We aim to provide a source for every number and clearly indicate when no measurement is available.
- Respect for tradition. We use Eisteddfod forms as a starting point, recognising that human creativity and experience give them meaning. This is a test of technology, not a parody.
- Consistent judging. Where rules are explicit — in cynghanedd, metre and rhyme — a rules-based engine judges first. Our Bangor AI research has shown that models can find unexpected shortcuts.
- Honesty about the current position. We want to record what models can do today, without exaggerating their ability or dismissing their progress. The systems we have tested so far have not produced a worthy awdl. Recording that provides a starting point for measuring progress.
Who is behind this?
Bangor AI’s research programme investigates strict-metre poetry and language models — from the cynghanedd rules engine to fine-tuning experiments on local clusters. The checker’s project repository is BangorAI/cynghanedd. AIsteddfod does not represent the National Eisteddfod or the Coleg Cymraeg Cenedlaethol, and has no commercial affiliation with any AI lab.
Contact
Send suggestions, corrections — especially language corrections — or ideas for new tests to aisteddfod@bangor.ai.