On November 12, 2026, language technology company Translated released preprint research and model weights for Lara Mini base, a 7.2-billion-parameter multilingual Mixture-of-Experts language model pretrained from scratch across 45 languages. Covering all 24 official European Union languages, the study demonstrates that extensive linguistic diversity strengthens rather than compromises foundation model reasoning, offering academic computing centers high multilingual throughput on modest commodity hardware.
Preprint and Model Release: Multilingual Scaling on 6 Trillion Tokens
On November 12, 2026, researchers Maria Pia Adinolfi, Isabelle Andrieu, Grazia Fiorella Anfuso, Melania Annunziata, Daniele Antonini, Marco Trombetti, and colleagues at Translated published their preprint study, Lara Mini Base: Language Diversity as a Resource, Not a Trade-off (arXiv:2611.16956). The work addresses a persistent imbalance in modern artificial intelligence: over leading open-weights models, non-English training text routinely makes up no more than 12% of pretraining data, leaving European and global languages marginalized.
To determine whether small foundation models can achieve deep multilingual competence without sacrificing core reasoning, Translated trained Lara Mini base from scratch on 6 trillion tokens. Unlike standard models where non-English text is an afterthought, non-English data represents 59% of the corpus across 44 languages. Together with English, the 45 supported languages cover all 24 official European Union languages, alongside global languages including Simplified and Traditional Chinese, Japanese, Korean, Russian, Ukrainian, Arabic, Persian, Hindi, Bengali, Thai, and Vietnamese.
The training pool of 13.8 trillion tokens was deduplicated and filtered through Translated’s multilingual processing pipelines. Training proceeded across three main phases: a 5.0-trillion-token main stage, a 0.4-trillion-token transition stage, and a 0.6-trillion-token anneal stage, followed by a 100-billion-token long-context extension stage. During annealing, non-English European and global text was reweighted to 33% of the mixture, becoming the largest individual domain.
Hybrid MoE Architecture and Inference Throughput
To expand representational capacity while preserving low inference latency, Lara Mini base adopts a sparse Mixture-of-Experts (MoE) design paired with linear-attention memory layers. Across 34 layers, the model contains 7.2 billion total parameters, but activates only 0.6 billion parameters per token during inference (0.65 billion including embeddings).
The 34-layer topology interleaves three sequence-modeling mechanisms: 14 Gated DeltaNet (GDN) linear-attention layers, 13 sliding-window attention layers with a 512-token local window and Rotary Position Embeddings (RoPE), and 7 global attention layers without positional encodings. Each layer employs 128 routed experts with top-4 routing alongside 1 shared expert that processes every token. Attention heads use Query-Key normalization (QK-norm) with 16 query heads and 4 key-value heads, while GDN layers use 8 query-key heads and 16 value heads with width-4 short convolutions.
Training optimization combined the Muon optimizer for 2D weight matrices (including all routed experts) with AdamW for 1D parameters and embeddings. In internal ablations, standard AdamW required 1.5 to 1.9 times more tokens to achieve equivalent training loss.
This hybrid architecture delivers substantial inference efficiency. Benchmarked on a single NVIDIA A100 GPU without quantization using vLLM, Lara Mini base produced 1,088 characters per second for a single user, with a median time-to-first-token (TTFT) of 33.9 milliseconds on 1,024-token prompts. For a single user, it generates text at least 1.8 times faster than any competing open base model achieving a higher average benchmark score.
The table below summarizes structural and performance metrics for Lara Mini base against comparable small open base models:
| Model Name | Architecture | Total Params | Active Params | Tokens | Context Window | Single-User Speed (Chars/s) |
|---|---|---|---|---|---|---|
| Lara Mini base | Hybrid GDN + MoE (128 experts) | 7.2B | 0.6B | 6.0T | 128k | 1,088 |
| Llama-3.2-1B | Dense Transformer | 1.2B | 1.2B | 9.0T | 128k | 1,351 |
| OLMoE-1B-7B | MoE (64 experts) | 6.9B | 1.3B | 5.1T | 4k | 1,005 |
| granite-4.0-h-tiny | Hybrid MoE | 6.9B | 1.0B | 22.5T | 128k | 745 |
| Qwen3-1.7B | Dense Transformer | 1.7B | 1.7B | 36.0T | 32k | 872 |
| SmolLM3-3B | Dense Transformer | 3.1B | 3.1B | 11.2T | 8k | 538 |
Challenging the Capacity Competition Hypothesis
A central tenet in compact language model training has been that languages compete for limited model capacity: allocating capacity to non-English tokens was believed to degrade English performance. Translated directly tested this hypothesis by branching two 100-billion-token anneals from the 5.4-trillion-token checkpoint.
In one branch, training maintained the balanced multilingual mixture; in the other, multilingual tokens were replaced entirely with English data. Substituting multilingual data with English yielded only a 0.3-point increase on English benchmarks, while reducing multilingual benchmark performance by 2.7 points. The minimal English gain indicated that multilingual data does not impose a meaningful capacity penalty.
Analysis of the 128 routed experts showed that the model does not compartmentalize individual languages into separate experts. Instead, European languages activate the same shared expert pathways, demonstrating that cross-lingual training fosters shared representations that regularize learning and boost overall reasoning.
Empirical Evaluation Across Multilingual, Math, and Code Tasks
The research team evaluated Lara Mini base against 56 open base models across 26 benchmarks covering English reasoning, multilingual comprehension, math, and coding. Among models with up to 2 billion active parameters, Lara Mini base ranked first overall:
- Overall Score (57.0): Led the sub-2B active parameter class, outperforming granite-4.0-h-tiny (53.9), Qwen3-1.7B (53.8), Falcon-H1-1.5B-Deep (53.2), and SmolLM3-3B (52.0).
- Multilingual Average (56.7): Ranked first on supported languages, ahead of Gemma-4-E2B (53.3) and granite-4.0-h-tiny (51.8).
- Math Average (59.9): Ranked first, surpassing Falcon-H1-1.5B-Deep (55.2) and Qwen3-1.7B (54.6).
- English Average (66.4): Ranked second, trailing Trinity-Nano (68.4) while exceeding granite-4.0-h-tiny (66.2).
- Code Average (45.1): Ranked fourth in class across MBPP and CRUXEval.
In long-context evaluation, Lara Mini base achieved a score of 90.1 on the 32k-token ONERULER multilingual retrieval benchmark, outperforming Trinity-Nano (73.9) and Falcon-H1-1.5B-Deep (60.5). To support future research, Translated announced the upcoming release of T-HellaSwag, an improved translation of the HellaSwag commonsense reasoning benchmark into 110 language variants.
Industry and Open-Source Community Perspectives
The release attracted attention across the translation and machine learning sectors. Industry publications MultiLingual and Slator noted that Translated’s initiative marks a transition from commercial translation APIs toward open foundational architecture research. Translation analysts highlighted that achieving competitive multilingual reasoning at 0.6B active parameters enables localized document intelligence without relying on cloud-hosted 70B parameter models.
On Hugging Face, where weights are hosted under translated/Lara-4-Mini-Base, developers highlighted the model’s role in advancing European digital sovereignty by providing equal foundational representation for official EU languages often underrepresented in American and Chinese foundation models.
Implications for Universities, Compute Centers, and Research Administration
The release provides practical takeaways for academic research offices, compute facilities, and research software engineers:
- Commodity GPU Deployment: With 0.6B active parameters and memory requirements fitting standard 16 GB GPUs, university labs can deploy high-speed multilingual inference locally, enabling privacy-preserving analysis of archival and clinical text without commercial API costs.
- EU AI Act Alignment: European research institutions navigating regulatory requirements under the European Artificial Intelligence Act benefit from transparent data allocation across EU languages. For governance procedures, consult CASRAI’s analysis of EU AI Act research obligations and exemptions.
- Data Provenance and Copyright: Sourcing 13.8 trillion tokens from open web data relies on European text and data mining (TDM) exemptions under Directive (EU) 2019/790. Compliance teams should review CASRAI’s guide on AI training data provenance and TDM exceptions.
- Non-Commercial License Boundaries: Weights are released under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 (CC BY-NC-SA 4.0). Technology transfer offices must recognize that while academic research is permitted, spinouts and commercial products require separate licensing. See CASRAI’s overview of Creative Commons research data licensing.
- Preprint Critical Assessment: As an unrefereed arXiv preprint, institutional researchers should evaluate experimental benchmarks critically prior to downstream adoption. Refer to CASRAI’s guidance on evaluating preprint findings.
Next Steps: Alignment, Gated Access, and Sovereign AI
Lara Mini base is an unaligned base model intended for text continuation; it lacks instruction-following tuning or safety guardrails. Research groups planning interactive deployments must apply supervised fine-tuning (SFT) and preference optimization before deployment.
Access to weights on Hugging Face requires institutional affiliation review. Translated plans to release T-HellaSwag and further instruction-tuned variants, establishing an empirical foundation for multilingual sovereign AI across European academic computing networks.
Sources
- Translated (Maria Pia Adinolfi et al.). Lara Mini Base: Language Diversity as a Resource, Not a Trade-off. arXiv:2611.16956 (Submitted 2026-11-12). https://arxiv.org/abs/2611.16956 (Checked 2026-11-15).
- Hugging Face. translated/Lara-4-Mini-Base Model Card and Weight Repository (Published 2026-11-12). https://huggingface.co/translated/Lara-4-Mini-Base (Checked 2026-11-15).
- MultiLingual. Translated Releases Lara Mini Base to Test Multilingual Foundation Model Scaling (Published 2026-11-13). https://multilingual.com (Checked 2026-11-15).
- Slator. Translated Expands Lara Language Model Series with Efficient Open MoE Architecture (Published 2026-11-14). https://slator.com (Checked 2026-11-15).
- Translated S.r.l. Lara AI Research and Infrastructure Platform (Updated 2026-11-12). https://translated.com (Checked 2026-11-15).







