Jupiter-N-120B: a large model trained on Welsh, available to everyone

Locai Labs and UCL have openly released a model, a Welsh post-training method and the associated datasets.

Four people sharing ideas in a colourful newsroom

Yesterday George Drayson of Locai Labs and UCL submitted the Jupiter-N technical report to arXiv, and the model weights are already available on Hugging Face. The model is based on NVIDIA Nemotron-3-Super-120B-A12B, a hybrid LatentMoE model (Mamba-2, MoE and attention; 120 billion parameters, a one-million-token context, and switchable reasoning). It was post-trained through LoRA (r=16, α=32) for three purposes: strengthening its agentic ability, aligning it with Britain’s cultural context, and improving its Welsh.

The Welsh results

Developers publish few Welsh benchmark results. Jupiter’s report adds two measures: an increase of 18 points on Welsh ARC-Easy and 5.25 points on Welsh MMLU-Lite compared with the base model. The data comes from Bangor University’s parallel corpora — Senedd proceedings and UK legislation — alongside machine-translated instruction data.

The datasets have also been published. They include nemotron-chat-welsh, the Assembly and Senedd record (cofnodycynulliad_en_cy), bilingual legislation, subtitles and Welsh content from Wikimedia. Further details are on Locai Labs’ page and in Drayson’s blog post on collecting translation data.

‘Forget-Me-Not’ — retaining existing abilities while learning new ones

A relevant element of the report is its anti-forgetting framework. It replays synthetic data generated by the original base model to retain core capabilities such as coding and reasoning while adding Welsh. Its agentic score, for example, improves by 9.1 points on Terminal Bench 2. The authors describe the approach as ‘a reproducible template for sovereign post-training’: an approach nations and language communities could study and adapt.

From a September promise to an April release

In September 2025, Bangor, UCL, NVIDIA and Nscale announced the UK-LLM project to post-train Llama Nemotron Super 49B and Nemotron Nano 9B on Welsh data. Bangor linguists were to check translations, including mutations. Although it was an exciting announcement, the project has not yet produced public weights or an operational API service.

Jupiter-N-120B offers an open starting point at 120B scale for Welsh post-training research, under the Nemotron Open Model License. Bangor AI is studying it as an option for poetry experiments and wider language research.

Sources: arXiv 2604.17429; HuggingFace locailabs; Bangor/NVIDIA announcements, September 2025. Full model profile in the evaluations section.