13 November 2026

Small models, growing data: new momentum for Welsh AI

Techiaith, Ash Hart's community initiative, Llandudno, EryriLabs' Gemma 4 models and eight new BangorAI datasets show how an ecosystem for small Welsh models is taking shape.

This week's story is not one model. Several parts of a small-model Welsh ecosystem are moving at once: people are organising, open models are appearing in more than one family, common tests are comparing them, and public Welsh and Celtic data is growing quickly.

Techiaith is a core part of that picture. It is not a new Week 46 launch, but the long-term infrastructure connecting speech corpora, small models, developer tools and local deployment. The new momentum is more credible because it is growing alongside practical work that already puts Welsh technology on laptops and into real workflows.

It begins with an ordinary resource: one computer with an RTX 4090 graphics card. On it, Jonathan Poole built two new Welsh conversational models, Llandudno-4B and Llandudno-9B. Their weights are public under Apache-2.0, the recipe is reasonably detailed, and AIsteddfod has independently rerun the tests.

They are not the first Welsh models, nor the strongest on every test. EryriLabs' Gemma 4 E4B Cymraeg v4 and Gemma 4 12B Cymraeg v6 already offer another open route: Gemma 4 foundations, Welsh specialisation, local-use formats and detailed evaluation evidence. They were covered in last week's article; they are not new launches this week.

The new development is that Llandudno now makes it possible to compare four open Welsh models across the same family of tests. That is more useful than naming one “winner”: one leads on reasoning, another on translation, and the two smaller models change places depending on the kind of practical Welsh being measured.

Grasmere-E4B, an earlier experiment in the same series, was published on 7 November. Llandudno followed by 10 November, with both repositories updated the next day. It is more accurate to treat Grasmere as the public step that led to the Llandudno pair than to count three separate launches.

Not one project, but momentum

The early indicator was Ash Hart's community “UK LLM” initiative. In a post on 1 November, Hart described volunteers building together, a common test set, a council pilot and initial results. A follow-up post directed contributors to the community work list, and several people were working on a small Welsh model.

There is not yet a public Welsh checkpoint from that initiative to test. There is also no evidence that Llandudno or EryriLabs' models are official products of it. Its importance is as a leading indicator: people were beginning to coordinate compute, data and tests at the same time as independent artefacts were becoming public.

The picture now looks like this:

• Techiaith provides corpora, speech and translation models, developer tools and routes to local deployment;
• Llandudno provides two Welsh-first Qwen3.5 models trained on one computer;
• EryriLabs provides two Gemma 4 routes, including GGUF versions for local use;
• AIsteddfod puts all four through the same family of tests;
• BangorAI has added eight more datasets across Welsh, Irish, Scottish Gaelic and Scots.

This is not a central programme. But it is beginning to look like an ecosystem.

Techiaith: the infrastructure that makes momentum useful

The Techiaith blog, part of Wales's national language-technologies portal, shows why the small-model story does not begin with the latest checkpoint. It connects products, developer resources, open translation memories, speech recognition and synthesis, data and APIs. That is ecosystem infrastructure, not one model release.

On 6 October, Techiaith published a guide to running small language models locally with LM Studio. It showed how a laptop with about 16 GB of RAM could run Mistral-7B-Cymraeg-Welsh-v2 locally using a Q4_K_M GGUF build of roughly 4.4 GB. It also said that a GGUF version of the Language Technologies Unit's 1B Welsh model was in the pipeline. This is not a new Week 46 model; its value is the deployment path: open weights, a practical format, a local interface and privacy without a cloud API.

On 29 October, Techiaith went further with Trawsgrifiwr Byw, a free Windows, Mac and Linux app that can run offline without sending audio away from the machine. It combines a 120 MB Kaldi speech-recognition model trained on about 159 hours of spoken Welsh with a 21 MB Marian translation model trained on 11.8 million Welsh–English sentence pairs. On a 500-sentence spoken test, Techiaith reports that a linguistic normaliser between the models increased translation BLEU from 32.6 to 44.7. This is a developer-reported result, not an independent evaluation.

The same post states the limits clearly: this is sentence-by-sentence translation, not simultaneous interpreting; spoken Welsh, dialect and code-switching expose the gap between natural-speech data and formal written data; and the tool should not replace an interpreter in medical, legal or rights-affecting settings. That transparency is part of the momentum too: a useful small model, task-specific data, declared limitations and qualified people still in the loop.

Techiaith's article on AI sovereignty supplies the broader frame: sovereignty need not mean building everything alone, but retaining the ability to govern compute, data and oversight while collaborating and localising. That is the connection between Techiaith's long-term work and the newer models, datasets and tests.

All three posts fall outside the 7–13 November window and are not presented as new launches. They are included as essential context: Llandudno and EryriLabs expand the choice of checkpoints, BangorAI expands the raw material, and Techiaith shows how small models become useful tools.

Eight new datasets: the raw material

BangorAI published five Welsh datasets late on 6 November, after the previous week's report and article had closed. Three more—for Irish, Scottish Gaelic and Scots—followed on 11 November. This is a genuine new development, not a repeat of the five BangorAI releases from 4–5 November covered last week.

The largest is Llyw.Cymru Documents: 12,578 PDF or Word document pairs and just over three million sentence pairs that pass the recommended filter. The process removed about 7,400 pairs whose document language did not match and 230 documents carrying third-party copyright. Welsh Legislation uses official XML identifiers to pair provisions rather than relying only on statistical similarity.

The three Celtic datasets are smaller but matter for a different reason. They use provenance structures, alignment scores and filters similar to the Welsh collections. That makes an experiment with one small Celtic model—or adapters sharing the same pipeline—much easier to audit. Much of the material remains machine-aligned: builders should use the quality flags rather than pouring every row into training.

What was built?

According to the Llandudno-9B model card, the work began with Qwen3.5 9B and 4B. Continued pretraining used 68.4 million tokens, followed by 43,402 conversations for instruction tuning. The mixture included Welsh, Irish, Scottish Gaelic, Scots and English; bilingual Senedd and Welsh Government material; Wikipedia; grammar and mathematics examples; and synthetic data for register, slang and dialect.

The card says the Llandudno release took about two days on the same 4090. It also records the use of Claude Code in data engineering, training and evaluation. That is part of the method, not a detail to hide: if AI tools help shape training data or test code, the record should say so.

The weights are Apache-2.0, but that does not make every part of the training data “open” in the same way. The card lists several licences and warns that some instruction targets sourced from BBC Cymru Fyw, Golwg and S4C remain copyrighted, even though they arrived through a crawl collection. The transparency is useful; it also limits what can safely be redistributed.

Four models, four profiles

The independent Llandudno results are very close to the developer's scores: 55.64 rather than 55.36 for 4B, and 58.31 rather than 58.07 for 9B. This is good evidence that the broad picture holds in another runtime. It is not a full reproduction: the models were not retrained, so the test confirms the behaviour of the published weights, not the provenance of every example or the full pipeline.

The profile is more interesting than the ranking. The 9B model leads overall and clearly on reasoning. But in the developer's specific “Practical Welsh” group, 4B was slightly higher. On AIsteddfod's wider CCC suite, 9B was stronger: a mean of 77.14 versus 67.36. The labels sound similar, but the task collections are not the same. That is why the test methodology matters more than comparing adjacent numbers.

EryriLabs changes the interpretation. Its 12B model still leads on translation, with BLEU 52.30 versus 49.67 for Llandudno-9B. Although Llandudno-4B is higher overall and on the Practical Welsh group, EryriLabs E4B is higher on the CCC audit: 70.36 versus 67.36. “Four billion” or “twelve billion” is not a capability profile.

The artefacts also differ. EryriLabs E4B provides merged BF16 weights, training adapters and GGUF versions for local use in roughly 5–9 GB. The 12B repository provides merged weights and the SFT adapter, but links to the continued-pretraining adapter rather than republishing it. Llandudno offers a compact one-GPU recipe and permissively licensed weights, while its card discloses some crawl-derived targets that remain copyrighted. Openness is more than the weight licence.

The developer's dialect test sets an important limit. Only 8% of 4B answers and 12% of 9B answers mirrored the user's dialect. A model may recognise northern or southern Welsh and then answer in standard Welsh. This is not merely a failure to “speak Welsh”; it is the difference between understanding variation and respecting a speaker's language choice.

A closed model at the top—with a methodological asterisk

Anthropic released Claude Fable 5.2 on 9 November. It scored 89.45 in AIsteddfod's Welsh evaluation: 88.34 on reasoning, 90.56 on practical Welsh and BLEU 74.64.

The score changed when the binary-answer extractor was corrected. Eighty-five answers already gave a clear Y or N at the beginning or end of the response; the old rule did not always accept them. After rescoring, the overall result rose from 84.73 to 89.45.

The correction is reasonable and documented. But older runs have not all been rescored with the same rule. The result shows that Fable 5.2 is very strong in Welsh, but the exact table gap should not be treated as though every model faced an identical version of the harness. It is also an API model: its weights cannot be downloaded, its training inspected or its behaviour adapted like Llandudno.

A wider lesson: tokens are a language cost

Two new tokenisation preprints offer a practical way to examine cost and equality. The Indic Token Tax tested nine tokenisers across 25 Indian languages. For the same content, GPT-4o used 1.57 times as many tokens for Hindi as for English, 5.01 times for Odia and 13.7 times for Santali in Ol Chiki script. No tokeniser placed more than 35% of its boundaries on morpheme boundaries.

That can make a user pay more, fill the context window sooner and give the model more processing steps for the same meaning. Welsh was not measured, so the numbers cannot simply be transferred. The test can: run the same corpus in Welsh and English, publish the token ratio, and add actual price and latency.

Havon proposes a multiword tokeniser. At the same vocabulary size, version 4 used fewer tokens than GPT-4o in 161 of 204 languages, with a median ratio of 0.74 across the 25 Indian languages. But this is a compression test, not a full trained model. A shorter route can be cheaper without making the model more accurate.

Do not improve one language by losing another

Capability Drift in Multilingual Post-Training shows another risk. One post-trained branch gained 36.07 points on MATH but lost 19.32 F1 points on XQuAD-Indic. A more language-aware recipe recovered Indian-language tasks, though not every script test, and gave up some competitive coding and mathematics.

For Welsh, the lesson is direct: run a Welsh regression set after every general post-training stage. “The large score increased” is not enough if Welsh instruction following, correct orthography or knowledge of Welsh institutions got worse.

A Central Asian embedding model shows the same trade-off on another task. After adapting EmbeddingGemma 300M with 279,904 generated query–passage pairs, recall@1 reached 75.2–77.7 across Tajik, Kazakh, Uzbek and Kyrgyz. English fell by 4.2 points and Russian by 3.0. A single small model for the Celtic family could be a valuable experiment, but it should retain English examples and measure the loss.

Finally, Fikiria, a Swahili reasoning model, reports 79.2 on AfriMGSM versus 68 for GPT-4o under the same strict prompt. Yet in a small preference study conducted by the co-authors, GPT-4o's answers were preferred. Automated score, fluent language and speaker preference are three different questions.

These papers are preprints, not final verdicts. Their value here is in offering tests that can be brought to Wales: token tax, regression after post-training, and speaker judgement alongside automatic metrics.

Two other items to record accurately

On 10 November, Nation.Cymru reported on a five-lesson bilingual AI teaching pack. The original page describes material on bias, hallucinations, safety, misinformation and critical thinking for progression step 3. It is education through Welsh, not a new model or dataset.

The record from an earlier week also needs correcting. Golwg 360 published an interview with Gruffudd Prys on 28 October, “Welsh should not have to depend on the charity of companies like Apple”. The automated scan could not reach it at the time; the page was opened directly later. It is not news from this week's window.

Prys argues that public bodies and schools should use their purchasing power and language duties to demand Welsh support rather than wait for the goodwill of large companies. Product comparisons and data figures in the article are attributed to him and were not independently re-audited here. The procurement principle is testable: specify a Welsh requirement, a test set, privacy evidence and a right to inspect results before signing a contract.

What should happen next?

Together, Llandudno and EryriLabs make five experiments realistic:

1. a locked human evaluation of fluency, factuality, northern and southern Welsh, and language switching;
2. a Welsh–English token-tax table across open and commercial models;
3. an ablation that clearly separates licensed data from copyrighted crawl-derived targets;
4. a Welsh regression set run after every training stage;
5. a small domain adapter—health, law or public service—with the general model retested too.

The momentum is now visible in five layers: Techiaith's speech, data and local-deployment infrastructure; Ash Hart's community initiative; Llandudno and EryriLabs models; comparative testing; and BangorAI data stretching from Welsh place names to Irish legislation and Scottish Gaelic. These are not one programme, and none solves Welsh AI alone. Together, however, they reduce the cost of the next experiment—and make its result easier to verify.

Timing note: the scan was completed during the morning of 13 November 2026. Material published later that day will be considered in the next scan.

Sources and further reading

• Llandudno-4B, Llandudno-9B and Grasmere-E4B
• EryriLabs Gemma 4 E4B Cymraeg v4, Gemma 4 12B Cymraeg v6 and the Week 45 comparison
• Llandudno-4B evaluation, Llandudno-9B and Claude Fable 5.2
• EryriLabs E4B evaluation and 12B
• Ash Hart's UK LLM update and the community work list
• Techiaith blog, LM Studio guide, Trawsgrifiwr Byw and the AI sovereignty article
• Data Cymru Names and Terms, GOV.UK Welsh, Welsh Legislation, Welsh Public Sector and Llyw.Cymru Documents
• Acts of the Oireachtas, Scottish Government – Gaelic and Scots Government – Scots
• The Indic Token Tax, Havon and Capability Drift
• Central Asian EmbeddingGemma, Fikiria and Nepali–English ASR
• Nation.Cymru and the bilingual AI education pack
• Golwg 360 interview with Gruffudd Prys, 28 October
• Earlier Kolibri article and sovereign comparison