Skip to main content
v2026.11,986 entries · CC-BY 4.0
CASRAIAsk CASRAI · AI assistant for researchAsk a research question. Answers come with their sources.Funder rules, compliance, methods and publishing. Answers link the sources they rest on.

Editorial · CASRAI · AI and ML research outputs

Lara Mini Base Tests Multilingual AI Pretraining in EU Languages

On November 12, 2026, Translated released Lara Mini base, a 7.2B MoE model showing European language diversity boosts AI reasoning without capacity loss.

Published 15 Nov 2026· 7 minute read

Ask CASRAI · free to try

Ask about this story

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

CASRAI is the reference for research administration — bookmark it for the next question.

On November 12, 2026, language technology company Translated released preprint research and model weights for Lara Mini base, a 7.2-billion-parameter multilingual Mixture-of-Experts language model pretrained from scratch across 45 languages. Covering all 24 official European Union languages, the study demonstrates that extensive linguistic diversity strengthens rather than compromises foundation model reasoning, offering academic computing centers high multilingual throughput on modest commodity hardware.

Preprint and Model Release: Multilingual Scaling on 6 Trillion Tokens

On November 12, 2026, researchers Maria Pia Adinolfi, Isabelle Andrieu, Grazia Fiorella Anfuso, Melania Annunziata, Daniele Antonini, Marco Trombetti, and colleagues at Translated published their preprint study, Lara Mini Base: Language Diversity as a Resource, Not a Trade-off (arXiv:2611.16956). The work addresses a persistent imbalance in modern artificial intelligence: over leading open-weights models, non-English training text routinely makes up no more than 12% of pretraining data, leaving European and global languages marginalized.

To determine whether small foundation models can achieve deep multilingual competence without sacrificing core reasoning, Translated trained Lara Mini base from scratch on 6 trillion tokens. Unlike standard models where non-English text is an afterthought, non-English data represents 59% of the corpus across 44 languages. Together with English, the 45 supported languages cover all 24 official European Union languages, alongside global languages including Simplified and Traditional Chinese, Japanese, Korean, Russian, Ukrainian, Arabic, Persian, Hindi, Bengali, Thai, and Vietnamese.

The training pool of 13.8 trillion tokens was deduplicated and filtered through Translated’s multilingual processing pipelines. Training proceeded across three main phases: a 5.0-trillion-token main stage, a 0.4-trillion-token transition stage, and a 0.6-trillion-token anneal stage, followed by a 100-billion-token long-context extension stage. During annealing, non-English European and global text was reweighted to 33% of the mixture, becoming the largest individual domain.

Hybrid MoE Architecture and Inference Throughput

To expand representational capacity while preserving low inference latency, Lara Mini base adopts a sparse Mixture-of-Experts (MoE) design paired with linear-attention memory layers. Across 34 layers, the model contains 7.2 billion total parameters, but activates only 0.6 billion parameters per token during inference (0.65 billion including embeddings).

The 34-layer topology interleaves three sequence-modeling mechanisms: 14 Gated DeltaNet (GDN) linear-attention layers, 13 sliding-window attention layers with a 512-token local window and Rotary Position Embeddings (RoPE), and 7 global attention layers without positional encodings. Each layer employs 128 routed experts with top-4 routing alongside 1 shared expert that processes every token. Attention heads use Query-Key normalization (QK-norm) with 16 query heads and 4 key-value heads, while GDN layers use 8 query-key heads and 16 value heads with width-4 short convolutions.

Training optimization combined the Muon optimizer for 2D weight matrices (including all routed experts) with AdamW for 1D parameters and embeddings. In internal ablations, standard AdamW required 1.5 to 1.9 times more tokens to achieve equivalent training loss.

This hybrid architecture delivers substantial inference efficiency. Benchmarked on a single NVIDIA A100 GPU without quantization using vLLM, Lara Mini base produced 1,088 characters per second for a single user, with a median time-to-first-token (TTFT) of 33.9 milliseconds on 1,024-token prompts. For a single user, it generates text at least 1.8 times faster than any competing open base model achieving a higher average benchmark score.

The table below summarizes structural and performance metrics for Lara Mini base against comparable small open base models:

Model Name Architecture Total Params Active Params Tokens Context Window Single-User Speed (Chars/s)
Lara Mini base Hybrid GDN + MoE (128 experts) 7.2B 0.6B 6.0T 128k 1,088
Llama-3.2-1B Dense Transformer 1.2B 1.2B 9.0T 128k 1,351
OLMoE-1B-7B MoE (64 experts) 6.9B 1.3B 5.1T 4k 1,005
granite-4.0-h-tiny Hybrid MoE 6.9B 1.0B 22.5T 128k 745
Qwen3-1.7B Dense Transformer 1.7B 1.7B 36.0T 32k 872
SmolLM3-3B Dense Transformer 3.1B 3.1B 11.2T 8k 538

Challenging the Capacity Competition Hypothesis

A central tenet in compact language model training has been that languages compete for limited model capacity: allocating capacity to non-English tokens was believed to degrade English performance. Translated directly tested this hypothesis by branching two 100-billion-token anneals from the 5.4-trillion-token checkpoint.

In one branch, training maintained the balanced multilingual mixture; in the other, multilingual tokens were replaced entirely with English data. Substituting multilingual data with English yielded only a 0.3-point increase on English benchmarks, while reducing multilingual benchmark performance by 2.7 points. The minimal English gain indicated that multilingual data does not impose a meaningful capacity penalty.

Analysis of the 128 routed experts showed that the model does not compartmentalize individual languages into separate experts. Instead, European languages activate the same shared expert pathways, demonstrating that cross-lingual training fosters shared representations that regularize learning and boost overall reasoning.

Empirical Evaluation Across Multilingual, Math, and Code Tasks

The research team evaluated Lara Mini base against 56 open base models across 26 benchmarks covering English reasoning, multilingual comprehension, math, and coding. Among models with up to 2 billion active parameters, Lara Mini base ranked first overall:

  • Overall Score (57.0): Led the sub-2B active parameter class, outperforming granite-4.0-h-tiny (53.9), Qwen3-1.7B (53.8), Falcon-H1-1.5B-Deep (53.2), and SmolLM3-3B (52.0).
  • Multilingual Average (56.7): Ranked first on supported languages, ahead of Gemma-4-E2B (53.3) and granite-4.0-h-tiny (51.8).
  • Math Average (59.9): Ranked first, surpassing Falcon-H1-1.5B-Deep (55.2) and Qwen3-1.7B (54.6).
  • English Average (66.4): Ranked second, trailing Trinity-Nano (68.4) while exceeding granite-4.0-h-tiny (66.2).
  • Code Average (45.1): Ranked fourth in class across MBPP and CRUXEval.

In long-context evaluation, Lara Mini base achieved a score of 90.1 on the 32k-token ONERULER multilingual retrieval benchmark, outperforming Trinity-Nano (73.9) and Falcon-H1-1.5B-Deep (60.5). To support future research, Translated announced the upcoming release of T-HellaSwag, an improved translation of the HellaSwag commonsense reasoning benchmark into 110 language variants.

Industry and Open-Source Community Perspectives

The release attracted attention across the translation and machine learning sectors. Industry publications MultiLingual and Slator noted that Translated’s initiative marks a transition from commercial translation APIs toward open foundational architecture research. Translation analysts highlighted that achieving competitive multilingual reasoning at 0.6B active parameters enables localized document intelligence without relying on cloud-hosted 70B parameter models.

On Hugging Face, where weights are hosted under translated/Lara-4-Mini-Base, developers highlighted the model’s role in advancing European digital sovereignty by providing equal foundational representation for official EU languages often underrepresented in American and Chinese foundation models.

Implications for Universities, Compute Centers, and Research Administration

The release provides practical takeaways for academic research offices, compute facilities, and research software engineers:

  • Commodity GPU Deployment: With 0.6B active parameters and memory requirements fitting standard 16 GB GPUs, university labs can deploy high-speed multilingual inference locally, enabling privacy-preserving analysis of archival and clinical text without commercial API costs.
  • EU AI Act Alignment: European research institutions navigating regulatory requirements under the European Artificial Intelligence Act benefit from transparent data allocation across EU languages. For governance procedures, consult CASRAI’s analysis of EU AI Act research obligations and exemptions.
  • Data Provenance and Copyright: Sourcing 13.8 trillion tokens from open web data relies on European text and data mining (TDM) exemptions under Directive (EU) 2019/790. Compliance teams should review CASRAI’s guide on AI training data provenance and TDM exceptions.
  • Non-Commercial License Boundaries: Weights are released under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 (CC BY-NC-SA 4.0). Technology transfer offices must recognize that while academic research is permitted, spinouts and commercial products require separate licensing. See CASRAI’s overview of Creative Commons research data licensing.
  • Preprint Critical Assessment: As an unrefereed arXiv preprint, institutional researchers should evaluate experimental benchmarks critically prior to downstream adoption. Refer to CASRAI’s guidance on evaluating preprint findings.

Next Steps: Alignment, Gated Access, and Sovereign AI

Lara Mini base is an unaligned base model intended for text continuation; it lacks instruction-following tuning or safety guardrails. Research groups planning interactive deployments must apply supervised fine-tuning (SFT) and preference optimization before deployment.

Access to weights on Hugging Face requires institutional affiliation review. Translated plans to release T-HellaSwag and further instruction-tuned variants, establishing an empirical foundation for multilingual sovereign AI across European academic computing networks.

Sources

  • Translated (Maria Pia Adinolfi et al.). Lara Mini Base: Language Diversity as a Resource, Not a Trade-off. arXiv:2611.16956 (Submitted 2026-11-12). https://arxiv.org/abs/2611.16956 (Checked 2026-11-15).
  • Hugging Face. translated/Lara-4-Mini-Base Model Card and Weight Repository (Published 2026-11-12). https://huggingface.co/translated/Lara-4-Mini-Base (Checked 2026-11-15).
  • MultiLingual. Translated Releases Lara Mini Base to Test Multilingual Foundation Model Scaling (Published 2026-11-13). https://multilingual.com (Checked 2026-11-15).
  • Slator. Translated Expands Lara Language Model Series with Efficient Open MoE Architecture (Published 2026-11-14). https://slator.com (Checked 2026-11-15).
  • Translated S.r.l. Lara AI Research and Infrastructure Platform (Updated 2026-11-12). https://translated.com (Checked 2026-11-15).

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →