Skip to content

Releases

Changelog

Notable changes to kLLM, release by release. The Unreleased section lists changes on the main branch that are not yet part of a tagged release.

  1. Unreleased

    Main branch

    Added

    • Model management commands: kllm pull, kllm run (interactive or one-shot chat against a background engine server), kllm list (ls), kllm ps, kllm stop, kllm rm, kllm show, kllm check (re-hash a stored model against its store record and, for a catalog model, its published manifest, or check an https:// gateway's attestation binding), kllm create (seal a pulled model into the local store), kllm key new|list|import|rm, kllm doctor, kllm request, kllm search, kllm logs and kllm completion. Models, keys, server records and logs live under KLLM_HOME (default ~/.kllm); the bundled library can be overridden with KLLM_LIBRARY. kllm pull checks every file of a catalog model against its published kllm-manifest.json, and kllm key import accepts the model keys issued for catalog models. Existing commands are unchanged.
    • Website (web/): product, model library, download, security, research, enterprise and model key application pages, built as static files for GitHub Pages or any CDN. The model catalog is web/data/models.json; the bundled CLI library (src/kllm/models/library.json) is kept consistent with it.
    • Key-application service (services/access/) and its back office (web/apps/admin): applications for model keys (one application can cover several catalog models) with email verification, review by administrators, an audit log and delivery of the issued key files through one-time, expiring links.
    • Documentation for the new commands, the local model store, keys, model keys and the official catalog.
    • Repository infrastructure: issue forms (including a model request wishlist), a pull request template, Dependabot, documentation, website and service workflows, SECURITY.md, CODE_OF_CONDUCT.md and SUPPORT.md.
    • --transform perm: signed permutations of the residual stream. Every sealed weight is a bit-exact rearrangement of an original one.
    • MiniMax H3 (FL2VA): the text encoder, DiT, latent interfaces and tokenizers are sealed together; kllm serve runs it with sglang serve --model-variant fl2va.
    • Model families: Qwen2/2.5 and Llama 3 (attention biases, no Q/K norms, RoPE scaling), Qwen3-MoE (expert permutation and per-expert MLP transforms), and FP8 checkpoints with block-wise or per-tensor scales under perm.
    • --engine vllm: serve sealed models with vLLM as well as SGLang.
    • kllm unlock: decrypt a sealed model's tokenizer into memory and print its path, for launchers such as Dynamo and llm-d.
    • kllm serve --int8: INT8 attention (Triton) and on-the-fly W8A8 linear layers for the MiniMax H3 DiT, 25% less time per denoising step on an A40.
    • Client-keyed mode, without a confidential VM: kllm seal --client also writes a -cloud directory with the weights only; kllm serve --client serves it without a key (SGLang or vLLM with --skip-tokenizer-init, or a latent server for Z-Image), and kllm client proxy / kllm client generate tokenize, detokenize and decode on the user's side. Z-Image results are encrypted to a one-off key of each request. Scope and limits in docs/topics/client-mode.rst; benchmarks/client.sh and benchmarks/zimage_client.py measure it.
    • The catalog (tools/catalog/): popular models sealed ahead of time, each under its own key derived from an offline master key. build.py downloads each base at a pinned revision, seals it and adds the license files, a model card and a manifest; issue_key.py writes one model's key for a user. kllm verify seals the public base again under the key and compares the result tensor by tensor; kllm cloud writes the keyless copy of a client-keyed model.
    • Locked weights: kllm seal also encrypts every weight in place (AES-CTR within the finite values of its format, --no-lock to skip). The files keep their layout and load in any engine, but compute nothing useful and cannot be matched to a public base without the key. kllm serve --key unlocks them into a 0700 cache; kllm cloud --key unlocks the keyless copy on the user's side; kllm verify compares them unlocked.
    • benchmarks/: one command per README table; examples/ with a quickstart, Dynamo and Kubernetes setups; a Dockerfile.

    Changed

    • The keyed code (kllm-trust) is now part of the open package as kllm.keyed (pip install kllm[seal]). Key release inside confidential VMs and the attesting gateway moved to kLLM Enterprise.
    • The schemes are reduced to L1 (static transforms of the weights plus a permuted vocabulary).

    Removed

    • The static model hub (hub/). The website in web/ replaces it; its catalog moved to web/data/models.json.
  2. 0.1.0

    Release

    First version: recipes for SGLang, kllm seal / kllm serve --key for Qwen3 under orth, and the Z-Image demo.