Releases
Changelog
Notable changes to kLLM, release by release. The Unreleased section lists changes on the main branch that are not yet part of a tagged release.
Unreleased
Main branchAdded
- Model management commands:
kllm pull,kllm run(interactive or one-shot chat against a background engine server),kllm list(ls),kllm ps,kllm stop,kllm rm,kllm show,kllm check(re-hash a stored model against its store record and, for a catalog model, its published manifest, or check anhttps://gateway's attestation binding),kllm create(seal a pulled model into the local store),kllm key new|list|import|rm,kllm doctor,kllm request,kllm search,kllm logsandkllm completion. Models, keys, server records and logs live underKLLM_HOME(default~/.kllm); the bundled library can be overridden withKLLM_LIBRARY.kllm pullchecks every file of a catalog model against its publishedkllm-manifest.json, andkllm key importaccepts the model keys issued for catalog models. Existing commands are unchanged. - Website (
web/): product, model library, download, security, research, enterprise and model key application pages, built as static files for GitHub Pages or any CDN. The model catalog isweb/data/models.json; the bundled CLI library (src/kllm/models/library.json) is kept consistent with it. - Key-application service (
services/access/) and its back office (web/apps/admin): applications for model keys (one application can cover several catalog models) with email verification, review by administrators, an audit log and delivery of the issued key files through one-time, expiring links. - Documentation for the new commands, the local model store, keys, model keys and the official catalog.
- Repository infrastructure: issue forms (including a model request wishlist), a pull request
template, Dependabot, documentation, website and service workflows,
SECURITY.md,CODE_OF_CONDUCT.mdandSUPPORT.md. --transform perm: signed permutations of the residual stream. Every sealed weight is a bit-exact rearrangement of an original one.- MiniMax H3 (FL2VA): the text encoder, DiT, latent interfaces and tokenizers are sealed
together;
kllm serveruns it withsglang serve --model-variant fl2va. - Model families: Qwen2/2.5 and Llama 3 (attention biases, no Q/K norms, RoPE scaling),
Qwen3-MoE (expert permutation and per-expert MLP transforms), and FP8 checkpoints with
block-wise or per-tensor scales under
perm. --engine vllm: serve sealed models with vLLM as well as SGLang.kllm unlock: decrypt a sealed model's tokenizer into memory and print its path, for launchers such as Dynamo and llm-d.kllm serve --int8: INT8 attention (Triton) and on-the-fly W8A8 linear layers for the MiniMax H3 DiT, 25% less time per denoising step on an A40.- Client-keyed mode, without a confidential VM:
kllm seal --clientalso writes a-clouddirectory with the weights only;kllm serve --clientserves it without a key (SGLang or vLLM with--skip-tokenizer-init, or a latent server for Z-Image), andkllm client proxy/kllm client generatetokenize, detokenize and decode on the user's side. Z-Image results are encrypted to a one-off key of each request. Scope and limits indocs/topics/client-mode.rst;benchmarks/client.shandbenchmarks/zimage_client.pymeasure it. - The catalog (
tools/catalog/): popular models sealed ahead of time, each under its own key derived from an offline master key.build.pydownloads each base at a pinned revision, seals it and adds the license files, a model card and a manifest;issue_key.pywrites one model's key for a user.kllm verifyseals the public base again under the key and compares the result tensor by tensor;kllm cloudwrites the keyless copy of a client-keyed model. - Locked weights:
kllm sealalso encrypts every weight in place (AES-CTR within the finite values of its format,--no-lockto skip). The files keep their layout and load in any engine, but compute nothing useful and cannot be matched to a public base without the key.kllm serve --keyunlocks them into a 0700 cache;kllm cloud --keyunlocks the keyless copy on the user's side;kllm verifycompares them unlocked. benchmarks/: one command per README table;examples/with a quickstart, Dynamo and Kubernetes setups; a Dockerfile.
Changed
- The keyed code (
kllm-trust) is now part of the open package askllm.keyed(pip install kllm[seal]). Key release inside confidential VMs and the attesting gateway moved to kLLM Enterprise. - The schemes are reduced to L1 (static transforms of the weights plus a permuted vocabulary).
Removed
- The static model hub (
hub/). The website inweb/replaces it; its catalog moved toweb/data/models.json.
- Model management commands:
0.1.0
ReleaseFirst version: recipes for SGLang,
kllm seal/kllm serve --keyfor Qwen3 underorth, and the Z-Image demo.