FreshSynapse

AI trends & daily updates, in plain English

AI Models

llama.cpp Hits 1.0: the Engine Behind Local AI Finally Promises a Stable API

By FreshSynapse Editorial · · 3 min read

A laptop screen showing HTML and CSS code, with the keyboard in the foreground

Photo: Hippopx (public domain). Illustrative image.

What happened. llama.cpp — the C/C++ inference engine behind Ollama, LM Studio and a long tail of apps that run language models on ordinary hardware — released version 1.0.0 on 13 November 2026, together with ggml 1.0.0, the tensor library underneath it. The release was announced on Hugging Face’s blog, where the maintainers (Georgi Gerganov and colleagues) now work after joining Hugging Face in February 2026. The projects stay open source under the MIT licence.

The news is the contract, not a feature. For three and a half years, llama.cpp used raw commit numbers as version strings — builds like “12360” that told developers nothing about compatibility. Version 1.0 adopts semantic versioning with explicit rules: a MAJOR bump means a breaking change to the public C API in the ./include directory; MINOR covers backward-compatible features, backend updates, model support and API additions; PATCH is reserved for bug fixes. As the team put it, “every new integration means more developers depending on our library interfaces and therefore having a flexible but stable API is paramount.”

What 1.0 actually changes. The release uses the major-version milestone to clean house: it removes 35 deprecated functions, plus the llama_set_warmup setting and the defragmentation-threshold option, and promotes the NextN speculative-decoding API into the main llama.h header. It adds FP8 (E4M3) KV caches, a host-memory cache for mixture-of-experts models, and support for three new text architectures — including Kimi-K3.1-Pro. It also hardens llama-server against server-side request forgery, CORS and DNS-rebinding attacks, and adds a new /v1/audio/speech endpoint and a redesigned web UI. That means 1.0 is not a drop-in upgrade for every app: code calling any of the 35 removed functions will break at compile time.

Why it matters. llama.cpp is the quiet infrastructure of local AI. It supports more than 150 model-architecture variants, and its self-hosted test runners run on hardware supplied by NVIDIA, AMD, Qualcomm, Arm, IBM and Intel. A stable API promise means the apps built on it — desktop chat apps, coding tools, on-device assistants — can update without fearing a silent break. That matters more as local models get bigger: a 145-billion-parameter model now claims to run on a single consumer GPU, and Thinking Machines’ Inkling 2 Small weights are now downloadable too. If you want the background on what these engines actually run, start with our plain-English guide to large language models.

Sources

  • llama.cpp
  • local AI
  • open source
  • Hugging Face
  • LLM inference

Related articles