FreshSynapse

AI trends & daily updates, in plain English

AI Models

Edge0 Releases a 145B Model Built to Run on One Consumer GPU

By FreshSynapse Editorial · · 3 min read

A close-up of a green circuit board with solder points and gold traces

Photo: Hippopx (public domain). Illustrative image.

What happened. Edge0 released Edge0-145B and Edge0-120B on 12 November 2026, with the weights on Hugging Face and its inference framework (edge0-infer) and deployment tools (edge0-strata) on GitHub. The headline claim is striking: a 145-billion-parameter model that runs on a single consumer NVIDIA GPU. Edge0 calls it the first open 145-billion-parameter model aimed at personal-computer agents rather than datacentre-only serving, continuing a line that began with its 35-billion-parameter phone-scale release in September.

What the model is. Edge0-145B is a latent sparse mixture-of-experts model: 145 billion total parameters, but only about 10.12 billion active per token, across 48 layers with 512 experts per layer (10 routed plus 1 shared per token). It is derived from Qwen3.8-Flash-Next, Alibaba’s model — a fact the Hugging Face model card acknowledges, and one that matters, because this is not a from-scratch model. It offers up to 256K context, is trained natively at 4 bits, and ships in two builds: Q3_K at 40.81 GiB and NVFP4 at 47.79 GiB. The trick is predictive expert loading: instead of holding the whole model in memory, the system loads the experts it needs, when it needs them.

The single-GPU claim, and what the numbers really mean. The headline figures are Edge0’s own measurements, and they should be read as such. On an RTX 5090 D with 24 GB of memory, using the Q3_K build, Edge0 reports 121.2 tokens per second for agent decode and 86.5 on Terminal-Bench 2.1, with a 29 GB peak memory figure in its announcement. The model card’s own residency numbers tell a fuller story: for the NVFP4 build, 23.25 GiB of video memory plus 38.27 GiB of system RAM — about 61.5 GiB in total. So “one GPU” is true, but it is one GPU plus a large helping of system RAM. The benchmarks are all Edge0’s own internal evaluations, with no independent verification yet. Requirements are Linux or Windows with an NVIDIA GPU, CUDA 13 and 16 GB or more of VRAM; macOS is not a supported target for these two models, despite some announcement wording suggesting otherwise.

The licence trap — read this before using it. The weights are open-weight, not open-source. They sit under the Edge0 Community Model License 1.0, on top of the Qwen licence. Personal use, research, modification, non-commercial sharing, internal use and commercial use of generated outputs are permitted. But selling or commercially bundling the model, operating a commercial model service, or offering an independent commercial coding or office assistant using it all require a separate written licence from Edge0. The framework itself is Apache 2.0. Anyone planning a product on this model should read the licence first, not the headline.

Why it matters. The direction is clear: the frontier of “big model, small hardware” keeps moving up. A 145B-class agent model on a consumer card — even with the RAM caveats — is a genuine step toward capable agents that run locally, without an API bill or a data centre. It arrives in the same week llama.cpp hit 1.0 with a stable API for local apps and Thinking Machines published Inkling 2 Small’s weights. The open-weights ecosystem is having a very good week — but the gap between “you can download it” and “you can sell with it” is exactly where the Edge0 licence sits. For the background, read our plain-English guide to large language models.

Sources

  • Edge0
  • open weights
  • local AI
  • Mixture of Experts
  • Qwen

Related articles