On November 10, 2026, TNG Technology Consulting reported adding independently trained Rust and Haskell experts to frozen Qwen3.5-35B-A3B-Base weights. Its experiments examine modular additions to a mixture-of-experts model, with only the new experts and routing parameters trained.
Reported gains and evaluation scope
TNG reports Rust pass@1 rising from 0.62 to 0.73 and Haskell from 0.32 to 0.51. Combining experts without retraining scored 0.70 and 0.50, respectively. Its general suite covered ARC Easy/Challenge, BoolQ, HellaSwag, MMLU, PIQA and WinoGrande: aggregate scores were 0.80 for the base, 0.78 with Rust and 0.80 with Haskell.
These author-reported results do not establish zero regressions across tasks. A frozen base can still produce different outputs when routing changes. The aggregate scores also make the evaluation boundary important: a claim about those tests cannot establish preservation of every research workflow.
How this differs from other modular methods
The distinction matters when comparing the report with Amazon’s HOPE expert-pruning work. The following comparison separates what each method changes, using the authors’ descriptions.
| Method | What changes |
|---|---|
| TNG domain experts | Adds separately trained experts and routing parameters to a frozen base. |
| EMO | Encourages modular expert groups during pretraining through document-level expert pools. |
| LoRA | Learns low-rank adaptation matrices while freezing pretrained weights. |
| HOPE | Removes existing experts using pairwise interaction information. |
The EMO preprint focuses on making expert subsets independently usable and composable through pretraining design. The original LoRA paper already freezes pretrained weights, so that property alone does not distinguish a new adaptation method. HOPE addresses compression and expert cooperation when selecting removals. These methods therefore answer different experimental questions.
What it means for university research
TNG leaves post-training retention, larger expert collections, inference efficiency and load balancing unresolved. Its experiments concern continued pretraining in two coding domains.
For university AI laboratories, the report offers a hypothesis to test: whether separately developed domain components can remain useful together. For research offices allocating computing resources, the relevant distinction is between a promising adaptation experiment and demonstrated operational savings. Independent evaluations on institutional tasks, alongside memory and throughput measurements, would be needed to support the latter.
A useful model card for a future institutional evaluation would distinguish the tested checkpoint, adaptation stage and benchmark outcomes from untested capabilities. Scientific-domain transfer and campus deployment benefits should remain separate evaluation questions.
Sources
- TNG Technology Consulting, Bring Your Own Expert! Extending MoE Models with New Domain Expertise. Checked 2026-11-15.
- Wang, Bhagia and Min, EMO: Pretraining Mixture of Experts for Emergent Modularity. Checked 2026-11-15.
- Hu and colleagues, LoRA: Low-Rank Adaptation of Large Language Models. Checked 2026-11-15.
- Amazon, Your Experts Have Partners. There’s HOPE for Keeping Them Together. Checked 2026-11-15.







