3 October 2026
From Kolibri to Wales: how much would a sovereign Welsh AI model cost?
Kolibri-1 puts concrete numbers on training a sovereign model. A Welsh model of the same size and ambition would cost around £2.5m–£3.5m for training compute, and around £10m for a practical programme including data, people, testing and deployment.
A sovereign model, with open evidence
Aleph Alpha released Kolibri-1 on 3 October, German Unity Day. It is a mixture of experts model: it has 78.1 billion parameters in total, but only 3.46 billion are active per token. There are 384 experts, with six plus one shared expert active. The model focuses on German and English, offers up to one million tokens of context, and its weights are on Hugging Face under Apache 2.0.
The main run used 768 B200 for 21 days to process 20 trillion tokens. This was followed by 3.44 trillion mid-training tokens and 200 billion for long context. The technical report gives a net total of 377,000 GPU-hours for the main run, and records 38 interruptions recovered automatically.
What would an equivalent Welsh model look like?
The fairest comparison is a Welsh–English MoE model matching Kolibri in size and compute budget:
| Feature | Kolibri-1 | Welsh reference |
|---|---|---|
| Architecture | Sparse MoE | Equivalent sparse MoE |
| Total parameters | 78.1B | around 78B |
| Active parameters per token | 3.46B | around 3.5B |
| Experts | 384; six plus one shared expert active | the same order of magnitude |
| Base-model training | 23.64T tokens | around 20T–24T tokens |
| Main-run context | 16k | 16k |
| Long-context adaptation | 256k | up to 256k |
| Main languages | German–English | Welsh–English |
‘Equivalent’ here means the same order of size, compute and ambition. It is not a promise of equal quality: data composition, training decisions and Welsh evaluation would be crucial.
More Welsh in every token
One of Kolibri’s most interesting advantages is its tokeniser. Aleph Alpha developed UniBPE for German word structure, particularly compound words. On the company’s German web test, Kolibri encodes 4.90 bytes per token, compared with 4.35 for GPT-5; a higher number means better compression. That amounts to around 11% fewer tokens for the same amount of text on that test. The original comparison table also shows Kolibri maintaining English efficiency.
Welsh has its own needs. The digraphs ch, dd, ff, ng, ll, ph, rh and th yn are individual letters in the Welsh alphabet despite containing two characters. A dedicated Welsh–English tokeniser could learn efficient units for these digraphs, mutations, word forms and common Welsh patterns, instead of inheriting vocabulary priorities designed mainly for English or hundreds of languages.
The advantage should not be assumed: a general tokeniser may already combine many of these patterns. The programme should publish a representative Welsh test — formal, spoken, dialectal, bilingual and public-service text — comparing bytes and words per token, sequence lengths, speed and quality. If the benefit were similar to Kolibri’s, every context window would hold more Welsh and training and inference would be cheaper.
Around £2.5m–£3.5m in compute
Aleph Alpha did not publish a price. Applying public B200 rental prices to the 377,000 hours puts the main run at roughly $2.52m–$3.34m, or around £1.90m–£2.53m. The range comes from public prices from Lambda, Nebius, CoreWeave and Lambda clusters, using the 2 October exchange rate.
The later base-model stages add 18.2% to the token count, at longer contexts that may reduce throughput. Another 16,900 equivalent B300-hours are recorded for the final reinforcement-learning run — around £121,000 at Nebius’s public B300 price — but there is no combined total for all supervised fine-tuning runs.
Rounding this evidence, £2.5m–£3.5m is a reasonable envelope for renting processors to train a Welsh model at Kolibri’s scale. This estimates rental value; it is neither Aleph Alpha’s bill nor a quotation for Wales.
The main run took 21 days. Time for every other stage has not been published, so four to six weeks for the main compute campaign is a planning inference, not a reported fact. The surrounding data, research and deployment work would take much longer.
Why a practical programme is closer to £10m
One clean main run is not a sovereign programme. Sufficient resources are needed to learn, repeat and fail safely. An illustrative planning envelope might look like this:
| Workstream | Illustrative allowance |
|---|---|
| Production training, experiments, failures and post-training | £4.00m |
| Rights, collection, cleaning and curation of Welsh data, and tokeniser work | £1.75m |
| A multidisciplinary team of engineers, researchers and Welsh-language specialists | £2.00m |
| Independent evaluation, safety, cultural alignment and governance | £0.75m |
| Storage, deployment, inference, programme operations and contingency | £1.50m |
| Total | £10.00m |
This is neither a quotation nor a Government proposal. It illustrates why sustainable national capability would cost substantially more than the GPUs for a single run.
Compute is not the greatest shortage
Aleph Alpha had a unique pool of 2.4 trillion German tokens, presenting around 4.3 trillion during pre-training. Its pipeline processed over 200 trillion raw tokens to select the final 20 trillion. No open, authentic, licensed Welsh pool exists at that scale.
A Welsh programme would need to decide openly on the mixture: authentic Welsh, English and perhaps other languages; repetition levels; public and cultural text with clear rights; synthetic or paraphrased data; code; and reasoning and tool-use data. It would need a tokeniser respecting Welsh morphology, dialect and code-switching data, and tests written by Welsh speakers.
The hardware is therefore within reach. The real sovereignty question is whether Wales would control the data, decisions, evaluation, infrastructure and ability to maintain the model after training ends.