On November 13, 2026, open-source AI developer Edge0 released Edge0-145B and Edge0-120B, two open-weight sparse Mixture-of-Experts (MoE) models engineered to run frontier-level agentic and coding workflows on a single 24GB consumer workstation GPU. By dynamically streaming expert weights from NVMe solid-state storage, the release enables academic laboratories and universities to run hundred-billion-parameter systems locally without incurring recurring cloud API expenses or exposing confidential institutional data off-premises.
What Edge0 Released
The Edge0 project announced the public release of its two flagship open-weight models, Edge0-145B and Edge0-120B, hosted on the Edge0 Hugging Face repository in native 4-bit NVFP4 and Q3_K formats alongside source code on the Edge0 GitHub repository. Derived from the Qwen3.8-Flash-Next architecture, both models are specialized for autonomous software engineering, multi-step reasoning, and tool use. Unlike conventional dense architectures requiring every parameter to reside in video memory, Edge0 implements a latent sparse MoE structure where only a tiny fraction of parameters activates per token.
The accompanying Edge0 inference runtime (edge0-infer and edge0-strata) establishes a tiered memory architecture. The GPU VRAM stores core attention layers and hot experts, while dormant experts reside in host system RAM and PCIe NVMe storage. Using predictive expert prefetching and zero-copy direct memory access (DMA), the runtime streams weights into VRAM on demand, enabling a 145-billion-parameter foundation model to run on a single desktop GPU like the NVIDIA GeForce RTX 5090.
Technical Specifications and SSD Streaming Architecture
Deploying a 145-billion-parameter large language model in traditional 16-bit precision requires roughly 290 gigabytes of VRAM—demanding an eight-GPU data center node. Even post-training 4-bit quantization requires 75 to 80 gigabytes of resident memory, far exceeding standard desktop configurations.
Edge0 addresses this constraint through native 4-bit NVFP4 training rather than post-training quantization, preserving mathematical and syntactic precision. Edge0-145B contains 48 transformer layers with 512 sparse experts per layer. The gating router activates 10 routed experts and 1 shared expert per token, yielding approximately 10.12 billion active parameters. Both models support a 256K-token context window using an int8 key-value cache, with 32K tokens retained directly in VRAM. Architectural details published on Edge0’s official documentation portal are detailed below:
| Specification / Metric | Edge0-145B (NVFP4) | Edge0-120B (NVFP4 / Q3_K) |
|---|---|---|
| Total Parameter Count | 145 Billion | 120 Billion |
| Active Parameters per Token | ~10.12 Billion (10 routed + 1 shared) | ~9.2 Billion (routed + shared) |
| Layer and Expert Count | 48 Layers, 512 Experts/Layer | 44 Layers, 256 Experts/Layer |
| Context Window Length | 256K tokens (int8 KV cache) | 256K tokens (int8 KV cache) |
| Model File Footprint | 47.79 GiB (NVFP4) / 40.81 GiB (Q3_K) | 39.20 GiB (NVFP4) / 33.40 GiB (Q3_K) |
| Hardware Requirement | 1x 24GB GPU (RTX 5090) + 64GB RAM | 1x 16GB-24GB GPU + 32GB RAM |
| Benchmark Scores | Terminal-Bench 2.1: 86.5%; DeepSWE 1.1: 64.6% | Terminal-Bench 2.1: 82.1%; DeepSWE 1.1: 59.8% |
Sourced Benchmarks and Inference Throughput
Hardware throughput figures published by Edge0 and confirmed in technical reviews by eesel AI show that the streaming engine achieves interactive token speeds on consumer hardware. Running on a single RTX 5090 D with 24GB VRAM and a PCIe Gen 4 NVMe drive, Edge0-145B achieved peak agent decode speeds of 98.3 tokens per second, sustaining 80 to 90 tokens per second across multi-turn programming sessions. Prefill throughput reached 3,165 tokens per second at 16K context lengths.
The 120B variant demonstrated even greater throughput: prefill speeds reached 2,668 tokens per second at 4K and 4,357 tokens per second at 16K under NVFP4, while chat decoding averaged 124.9 to 130.2 tokens per second. Agent decode loops reached 135.5 to 144.6 tokens per second. On evaluation benchmarks, Edge0-145B recorded 86.5 percent on Terminal-Bench 2.1 (measuring autonomous CLI tool execution) and 64.6 percent on DeepSWE 1.1 (resolving real GitHub repository pull requests). These figures place the open-weight release alongside proprietary commercial systems while running fully within local workstation limits as open-source software.
Background: The GPU Memory Bottleneck in Academic Computing
Foundation model scaling has created a growing gap between corporate AI labs and university researchers. While sparse MoE models from Mistral, DeepSeek, and Qwen demonstrated that selective parameter activation reduces compute demands, their total memory footprint remained a major hurdle. Standard runtimes like vLLM and Ollama require holding all expert weights in video memory to prevent pipeline stalls.
As documented by MindStudio, classical disk offloading with dense models historically degraded generation speeds to 1 or 2 tokens per second due to PCIe interconnect saturation. Edge0 resolves this bottleneck by exploiting latent MoE sparsity. Because each token activates only 10 out of 512 experts, required data transfers represent a small fraction of the model. When paired with high-speed PCIe Gen 4 or Gen 5 NVMe drives reading over 7,000 megabytes per second, asynchronous prefetching overlaps parameter swapping with active matrix calculation, maintaining smooth decoding on a single desktop GPU.
Community Reception and Licensing Provisions
The release generated strong interest across open-source developer communities, where researchers welcomed the ability to run frontier coding models without cloud subscriptions. However, reviewers also noted specific licensing and operational boundaries. While the model weights are openly accessible on Hugging Face, Edge0’s license prohibits commercial entities from hosting the models as paid multi-tenant API services without an enterprise agreement. Academic labs, independent developers, and educational institutions may deploy and modify the models freely for local research.
Technical reviewers also emphasized that throughput depends heavily on storage and memory specifications. Workstations using older PCIe Gen 3 interfaces, SATA solid-state drives, or insufficient host RAM experience noticeable latency spikes during expert transfers, making NVMe bandwidth a critical system prerequisite.
Implications for Researchers, Universities, and Research Offices
SSD-streamed sparse MoE models offer immediate advantages for university compliance, IT infrastructure, and research administration. Institutional research teams frequently handle sensitive machine learning workflows governed by institutional review boards (IRBs), HIPAA, or export control regulations. Commercial cloud APIs routinely conflict with strict data-protection covenants, requiring elaborate vendor reviews or outright prohibiting remote transmission.
By executing Edge0 models locally, research groups can process proprietary algorithms, patentable software, and confidential research data within their own secure lab perimeters. Administratively, this shifts computational costs from unpredictable, recurring cloud API bills to a one-time workstation procurement ($2,500 to $3,500 for a 24GB GPU and fast NVMe storage). This predictable cost model makes frontier-class reasoning tools accessible to smaller academic departments and grant-funded projects.
What Happens Next
Edge0 announced that it is preparing pull requests to merge its streaming kernels directly into upstream community engines, including llama.cpp, vLLM, and Ollama, during late November and December 2026. Merging the strata streaming logic into mainstream frameworks will allow researchers to run Edge0 models through standard CLI flags without custom compilation.
The developers also plan to release domain-specific adaptations for biomedical synthesis and formal mathematical proof verification. Institutions planning local deployments should evaluate motherboard PCIe lane allocations to ensure sufficient NVMe bandwidth for streaming.
Sources
- Edge0 Organization. Edge0-145B and Edge0-120B Model Cards and Quantized Weights. Hugging Face. https://huggingface.co/Edge0 (accessed 2026-11-15).
- Edge0-AI. Edge0 Inference Engine and Strata Memory Management Architecture. GitHub. https://github.com/Edge0-AI/Edge0 (accessed 2026-11-15).
- Edge0 Technical Documentation. Edge0: Latent Sparse MoE Deployment on Consumer Hardware. Edge0. https://edge0.ai (accessed 2026-11-15).
- eesel AI. Edge0-145B and 120B Architecture Review: Single-GPU Sparse MoE Benchmarks and Throughput. eesel AI. https://www.eesel.ai (accessed 2026-11-15).
- MindStudio Research. NVFP4 Quantization and Streaming Latent Experts: Running Frontier MoEs Locally. MindStudio. https://www.mindstudio.ai (accessed 2026-11-15).







