9 October 2026
Welsh punctuation on your device: an open model, a focused test
Two new open models can restore punctuation to Welsh text without changing the words. They are small enough to run on a local computer or in a browser—but a test on written text is not the same as a test on real-world Welsh speech.
Putting the marks back
Horizon Labs released two models on 3 October: a 307-million-parameter base version and a small 140-million version. The base model metadata and small model metadata record the creation date. Both use Apache-2.0, with Safetensors weights, ONNX files and code to run them. The ONNX versions can run on a CPU or in a browser.
The task is narrow and clear. After each word, the model chooses a full stop, comma, question mark, colon, dash or no mark. It does not change words or try to restore capitalisation. It can therefore sit after a speech recognition system: the first system produces the words, and this model tries to make the transcript easier to read and edit.
On the Welsh portion of FLORES-200, the base model card reports F1 of 0.849 with capitalisation preserved, and 0.784 after lowercasing the text. The small model scores 0.844 and 0.756. The difference between them is small on capitalised text, but larger on lowercase input, which is closer to many speech systems' output.
That is a genuine Welsh result, not just a ‘multilingual’ label. The artifacts can also be downloaded: the base model's quantised ONNX file is around 641 MB, compared with 268 MB for the small model. A local test could therefore measure quality, memory and speed together.
What the score does not prove
The cerdyn model says training used around 119 million words in 1,168,626 text windows across 88 languages. It gives no separate Welsh total. The text comes from the web, and FLORES contains professional translations of news-, travel- and wiki-like material.
This is not live conversation. It does not contain everything that makes speech difficult: fillers, sentence restarts, pauses, dialect, noise, incomplete words or switching between Welsh and English. It is also the publisher's own result; there has not yet been an independent test by Welsh editors or listeners.
The task's limits matter. Only five types of punctuation are restored. Exclamation marks, quotation marks, brackets and semicolons are not part of the output, and the cards say colons and dashes are weaker than the most common marks. The conclusion is therefore not ‘Welsh punctuation solved’. The fair conclusion is that an open, measurable component is now ready to test on real Welsh transcripts.
A new institutional boundary
The Arts Council of Wales also published its approach to generative AI . The page gives no visible publication date, but was indexed as new material during the week and its server metadata changed on 8 October.
The Council says people and human creativity should remain at the heart of its work. Its priorities include confidentiality, intellectual property, a fair and transparent process, climate justice, linguistic equality and Welsh-language inclusion. It also acknowledges that not every AI tool supports Welsh fully or accurately, so output needs checking.
One boundary is particularly clear: AI will not be used to decide or evaluate funding applications. Applicants currently do not have to declare AI use, although that may change. The policy sets a useful principle, but does not yet specify models, Welsh tests, audit records or how someone could challenge a failure. Those are the operational details to watch next.
The token is part of the test
new paper on the Latent Core Tokenizer, submitted on 8 October, includes direct Welsh results. A tokeniser decides how text is split into pieces before a model processes it. Poor segmentation can make a language computationally more expensive and hide word structure.
In the paper's Welsh table, LCT records a fertility of 1.854 tokens per word, morphological F1 of 0.672 and 3.440 bytes per token. The corresponding figures for standard BPE are 1.889, 0.668 and 3.334. The difference is modest, but moves all three measures in the right direction: fewer pieces, slightly better morphological boundaries, and more text per token.
Across four multilingual benchmarks, the method improves the aggregate score by 1.48 points over BPE, 1.83 over Unigram and 2.00 over BPE designed for linguistic equality. But the downstream table contains no Welsh task. The paper is a preprint under review, with one pre-training seed. It provides evidence about how Welsh is segmented and compressed, not proof that a model writes better Welsh.
This adds evidence to a question raised in our 3 October article about Kolibri-1 and a potential model for Wales: vocabulary design is part of the decision, not a minor detail at the end. There was no substantial change to the official Kolibri-1 card after 3 October, so there is no reason to repeat its launch or the cost comparison this week.
Native tests and code-switching
Two other preprints offer methods that could transfer to Welsh.
sk-bench combines 30 datasets in 33 variants for Slovak. Model rankings on native and translated tests were very similar, but the relationship between human-written and model-generated questions was much weaker. Continued Slovak pre-training of Qwen3-14B reduced its overall score by 13.9 points; a small instruction set recovered around three quarters of the loss. That warns us not to measure ‘more Welsh’ alone: instruction-following must be retested after language adaptation.
a study of phonological interference shows that speech models can lose sounds when language changes within an utterance. Across several language pairs, 32–79% of recognition ability was lost for phonemes not shared between the two languages. Providing language information for each word or span, rather than one label for the whole utterance, removed much of the interference. The paper has no Welsh test, but the method suits a Welsh–English question: what happens to distinctive Welsh sounds near a language switch?
The a study of offline AI modules adds the hardware dimension. Three quantised language models ran on a Raspberry Pi 5 at 6.1–7.5 tokens per second, while Amharic and Oromo results showed that a larger speech model is not necessarily worth the extra delay on a small device. A Welsh test should therefore publish quality, speed, memory, power and licensing together.
The next test
This week's news can become a simple experiment. Take a rights-cleared sample of natural Welsh speech: different accents, some noise, some code-switching and sentences not read from a script. Transcribe it with a speech system, run both punctuation models, and ask Welsh editors to measure not only F1 but how much work is saved.
The test should name the exact version, device, runtime and memory; keep lowercase output separate; and analyse each punctuation type. It should also record when the model makes text more confusing, not just when it adds a correct mark.
That is the thread connecting the week. An open model gives us something concrete to test. The Arts Council's policy explains why checking Welsh is an institutional responsibility. Research on tokenisation, native tests and code-switching explains what a superficial test might miss. Supporting Welsh is not a label: it means a specific artifact, a Welsh test, a known licence and clearly stated limits.