Skip to main content
v2026.11,986 entries · CC-BY 4.0

Editorial · CASRAI · AI and ML research outputs

TNG reports modular domain experts for MoE models

TNG reports modular Rust and Haskell experts for a frozen MoE model, with post-training and serving questions still open.

Published 15 Nov 2026· 2 minute read

Ask CASRAI · free to try

Ask about this story

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

CASRAI is the reference for research administration — bookmark it for the next question.

On November 10, 2026, TNG Technology Consulting reported adding independently trained Rust and Haskell experts to frozen Qwen3.5-35B-A3B-Base weights. Its experiments examine modular additions to a mixture-of-experts model, with only the new experts and routing parameters trained.

Reported gains and evaluation scope

TNG reports Rust pass@1 rising from 0.62 to 0.73 and Haskell from 0.32 to 0.51. Combining experts without retraining scored 0.70 and 0.50, respectively. Its general suite covered ARC Easy/Challenge, BoolQ, HellaSwag, MMLU, PIQA and WinoGrande: aggregate scores were 0.80 for the base, 0.78 with Rust and 0.80 with Haskell.

These author-reported results do not establish zero regressions across tasks. A frozen base can still produce different outputs when routing changes. The aggregate scores also make the evaluation boundary important: a claim about those tests cannot establish preservation of every research workflow.

How this differs from other modular methods

The distinction matters when comparing the report with Amazon’s HOPE expert-pruning work. The following comparison separates what each method changes, using the authors’ descriptions.

Method What changes
TNG domain experts Adds separately trained experts and routing parameters to a frozen base.
EMO Encourages modular expert groups during pretraining through document-level expert pools.
LoRA Learns low-rank adaptation matrices while freezing pretrained weights.
HOPE Removes existing experts using pairwise interaction information.

The EMO preprint focuses on making expert subsets independently usable and composable through pretraining design. The original LoRA paper already freezes pretrained weights, so that property alone does not distinguish a new adaptation method. HOPE addresses compression and expert cooperation when selecting removals. These methods therefore answer different experimental questions.

What it means for university research

TNG leaves post-training retention, larger expert collections, inference efficiency and load balancing unresolved. Its experiments concern continued pretraining in two coding domains.

For university AI laboratories, the report offers a hypothesis to test: whether separately developed domain components can remain useful together. For research offices allocating computing resources, the relevant distinction is between a promising adaptation experiment and demonstrated operational savings. Independent evaluations on institutional tasks, alongside memory and throughput measurements, would be needed to support the latter.

A useful model card for a future institutional evaluation would distinguish the tested checkpoint, adaptation stage and benchmark outcomes from untested capabilities. Scientific-domain transfer and campus deployment benefits should remain separate evaluation questions.

Sources

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →