6 November 2026

Small speech, big data: a week of building Welsh AI

A 106 MB speech model, five BangorAI data releases, four CymraegBench evaluations and the community-led UK LLM initiative expand the tools available to build and test Welsh AI.

It is tempting to measure progress in Welsh AI by model size. This week's evidence offers a better method: measure what works, where it fails, what data it uses and exactly what others can reproduce.

Robert Breda released three small Moonshine-based Welsh speech-recognition models. The most interesting, moonshine-base-welsh-compact, has 53.2 million parameters and is a 106 MB file. BangorAI published four new datasets and one major new version on 4–5 November. At the same time, AIsteddfod published four new CymraegBench runs: a BritLLM base model, its instruction-tuned Machynlleth derivative, and two Gemma models adapted for Welsh.

Another community initiative is also taking shape. Ash Hart, creator of TensorFold, is coordinating “UK LLM” around a shared test, a council pilot and early results; several contributors are working on a small Welsh model. There is no public Welsh checkpoint yet, so this is work in progress rather than a model launch.

There is no single “winner” here. The small speech model is impressive on one podcast set but weak on spontaneous conversation. The new data is large but carries quality flags that must be used. The largest text model leads overall, but not on every practical Welsh task. That is the story: an open artefact, its licence and its test profile matter more than a large number on a model card.

106 MB of Welsh speech recognition

The Welsh ASR project starts with about 175 hours of audio: Common Voice, Lleisiau Arfor, English FLEURS, Bangor transcription-bank podcasts and CLlLC local-authority meetings. The base and tiny Moonshine models were published on 2 November, followed by the compact version on 3 November.

The compact model was made by averaging the final ten checkpoints, reducing the token vocabulary from 32,768 to 12,735, and disabling five decoder attention heads associated with repetition. According to the project page, the final change affected about 0.18% of the weights and required no retraining. The weights are MIT-licensed, while the Listenr training and deployment toolkit is available under MPL-2.0.

Compared with the author's 1.5-billion-parameter Bangor Whisper model, Moonshine Compact has 29 times fewer parameters and is about 58 times smaller on disk. The project estimates 4.3 rather than 217 GFLOP for each second of audio. That is not a speed test on a named phone, but it is enough to make local deployment a realistic question.

On the Bangor podcast test, the model card reports a word error rate of 32.7%, compared with 34.7% for the Whisper model. But on Common Voice, the compact model scored 25.8% and the comparator 10.3%. The author warns that some Common Voice test material may have appeared in Whisper training, so the latter comparison is not entirely clean.

The most important number is 76.8%: the error rate on held-out spontaneous Siarad conversation. Changing the attention heads reduced repetition failure—from about 87% to about 77% on this test—but the result remains far too high for dependable general dictation.

The model also shows the cost of compression. English FLEURS worsened from 16.6% on the base model to 21.0% on the compact version after vocabulary pruning. That could matter in naturally code-switched Welsh conversation.

This is therefore a genuinely useful release: open weights, open tools, several test sets, and an honest correction when Lleisiau Arfor test sentences overlapped training prompts. But it remains the author's own evaluation. The next step is an independent test on real devices, using spontaneous speech, varied accents and language switching.

BangorAI: five new layers of data

BangorAI's release is not one anonymous heap of text. The five datasets fill different gaps, and each retains provenance and quality flags:

The quality details matter as much as the sizes. The translation memory marks 16,361 pairs that appear in at least one unproofread memory, as well as identical-side and one-to-many pairs. The government-pages dataset provides alignment quality, flags untranslated English, and retains the source URL, date and public body. Version 2 of the announcements corpus offers 177,167 rows that pass the card's recommended clean-data filter.

The historical material is more experimental. The book OCR was produced by vision models and checked with a second model and samples, not corrected page by page by people. The dataset marks 160 Drych pages and 36 Enwogion pages for review. In the proverbs set, 165 pairs need checking because numbering on facing pages did not align exactly.

This is the right kind of openness: not hiding the difficult rows, but giving builders the information needed to filter them or use them as a test. The datasets offer formal government language, professional translation, historical print and culturally specific idioms. They do not replace natural conversation, regional varieties or contemporary literature.

From a raw base to a Welsh model

The four new CymraegBench runs show a different kind of journey.

The zero score for BritLLM 3B does not mean the model contains no Welsh. It is a base model, not one tuned to follow instructions. It often continued the prompt or entered a loop, producing 1,167 invalid answers out of 2,053. A probability-based multiple-choice test asks a different question; this run measures how usable the raw artefact is through an instruction interface.

Machynlleth 3B Instruct descends from the same family after further Celtic and Scots text and 41,025 instruction conversations. Its invalid rate fell to 1.8%, while translation BLEU rose from 14.94 to 41.03. But the overall score is 16.44, with zero on several subsets. Its CC BY-NC 4.0 licence also restricts commercial use.

The two EryriLabs models are much stronger under the same harness. Gemma 4 E4B Cymraeg v4 scored 49.70 and Gemma 4 12B Cymraeg v6 scored 52.70. Neither produced an invalid answer or refusal.

But size alone does not explain the difference. The E4B model averaged 70.36 across the practical CCC Welsh tasks, slightly above the 12B model's 69.70. Their profiles differed: 12B was stronger on Welsh instructions, idioms and terminology; E4B was stronger on summarisation, English instructions and source-grounded content.

Do not confuse three questions

The complete picture separates three things that are often conflated:

1. Base-model capability. A model may contain Welsh knowledge without knowing how to answer the test.
2. Instruction tuning. This can turn continued text into a useful answer, but it does not guarantee reasoning or correct Welsh.
3. Welsh specialisation. Continued training and Welsh instruction data can improve the profile, but individual tasks must be tested and training artefacts retained.

The third point matters for reproducibility. EryriLabs publishes quantised weights and adapters, but the 12B card says the merged BF16 model and two continued-pretraining adapters are unavailable. The card also warns about loops and English leaking into long answers. CymraegBench provides useful independent evidence; it does not reconstruct the full training pipeline.

UK LLM: building a model as a community

Two similar names need to be separated. BritLLM is a UCL-led research project that created the Caernarfon 3B base model and BritEval benchmark. “UK LLM” is a new community initiative coordinated by Ash Hart, creator of the open-source TensorFold inference engine.

In a public post on 1 November, Hart said people were volunteering their time to build an LLM together, with a council pilot, a shared test set and first results already under way. Another post directs contributors to a projects and working list in the TensorFold community's UK LLM section. Donating a graphics card to a student contributor is a concrete example of volunteer participation.

Within this broader work, several people are working on a small Welsh model. That is promising, particularly now that BangorAI data and Welsh benchmarks are available. But no public checkpoint, model card, repository, data recipe, licence or Welsh score was available to verify. It should not yet be called a new model.

The next public step is straightforward: a short project card naming the base model and contributors, stating which data is being used, and publishing a locked Welsh test before and after training. That would turn community energy into an experiment others can inspect and repeat.

Two lessons from other languages

Two new papers provide a useful check on the Welsh interpretation.

First, a Tajik report on Qwen3 shows that lower text loss did not always produce better understanding. Small LoRA runs reduced Tajik loss but did not improve reading comprehension at 1.7B or 8B, and the score fell at 4B. Only the heavier 8B recipe—about 375 million continued-training tokens followed by instruction tuning—produced a clear 11.4-point gain. For Welsh, the warning is simple: fluency or a loss statistic is not enough; understanding must be tested.

Second, NaijaFacts crosses the language of a question with the country of the fact. On average, models answered about half of English questions concerning the United States or United Kingdom, but only 20% of questions about Nigerian statistics. Asking the Nigerian questions in Yoruba reduced the score by another nine points. This separates two failures: not understanding the language and not knowing the place.

Wales should do the same. A model can write fluent Welsh while lacking current knowledge of Welsh institutions, law or demography. A local bilingual benchmark built from authoritative sources would test the two dimensions separately.

New work on Igala dialect translation, Hausa medical ASR and Tetun medical questions adds the same message: break averages down by dialect, compare domain adaptation against a general baseline, and use human experts to verify translated data.

What should happen next?

Five small steps are now worth more than a grand launch:

1. run Moonshine Compact on a locked set of natural Welsh conversation and a named phone or laptop, publishing latency, memory, battery use and error types;
2. independently sample the BangorAI datasets, measure overlap, and split by document and date before training;
3. publish a project card and baseline for the small Welsh model within UK LLM;
4. retest base models with a probability method alongside the instruction test, so knowledge and compliance are not conflated;
5. build a Welsh NaijaFacts: dated local facts from public sources, with the same questions in Welsh and English.

This week shows real progress. A Welsh speech model can fit in 106 MB. A new layer of traceable data is available to build on. A new community offers people, tools and compute. Instruction tuning and Welsh specialisation can improve a model substantially. But all four elements need to come together: licensed data, a public recipe, an open checkpoint and a test that reveals failure as well as success.

Timing note: the scan was completed and amended at 10:51 GMT on 6 November 2026. Material published later that day will be considered in the next scan.

Sources and further reading

• Welsh ASR: Moonshine models, the compact model card and Listenr
• Welsh Government Translation Memory 1, Llyw.Cymru Pages 1 and Welsh Government Announcements 2
• NLW Books OCR 1 and Welsh Proverbs
• BritLLM Caernarfon 3B and Machynlleth 3B Instruct
• Gemma 4 E4B Cymraeg v4 and Gemma 4 12B Cymraeg v6
• Ash Hart's UK LLM update, the community work-list invitation and TensorFold
• Tajik Language Modeling with Qwen3 and NaijaFacts
• Dialect Matters: English–Igala MT, Hausa medical ASR and Labadain-MedQA