Llama 3.1

# Download the library tag into the local store; a manifest records each file's hash
$ kllm pull llama3.1:8b
# Start SGLang or vLLM in the background on 127.0.0.1 and open a chat
$ kllm run llama3.1:8b

Requests to the running server

# The local proxy listens on 127.0.0.1:8001 and tokenizes on your machine
curl http://127.0.0.1:8001/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "Llama-3.1-8B-Instruct-sealed", "messages": [{"role": "user", "content": "Hello"}]}'

• Each repository carries kllm-manifest.json: the base and its revision, the key fingerprint and key version, the kLLM version and the SHA-256 digest of every file.
• Built with Llama. Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.

Check a published model against its public base

With the model key, kllm verify seals the public base again under the same key and transform in a temporary directory and compares the result tensor by tensor, unlocked, together with the tokenizer, every other file and the manifest.

$ hf download ANetResearch/Llama-3.1-8B-Instruct-kllm --local-dir Llama-3.1-8B-Instruct-kllm
$ kllm verify Llama-3.1-8B-Instruct-kllm --base meta-llama/Llama-3.1-8B-Instruct --revision 0e9e39f249a16976918f6564b8830bc894c89659 --key Llama-3.1-8B-Instruct-kllm.key

Apply for model keysThe catalog in the documentation

Gated repositories require accepting the license on Hugging Face and setting HF_TOKEN before kllm pull.

• To an operator who reads GPU memory, user text reduces to a substitution cipher over tokens: 23.8% of tokens were decoded by unigram frequency alone on Qwen3-8B.
• The sealed weights derive from a public checkpoint. Locked files cannot be matched to it without the key; once unlocked to run, a holder of that checkpoint recovers the transforms and 98.5% of the vocabulary permutation (measured on Qwen3-8B), for unmodified public models and fine-tunes alike.
• Client-keyed mode keeps the key and the tokenizer on the user's machine. The copy that the cloud serves is unlocked on the user's side (kllm cloud --key), because the cloud computes with the weights. The cloud server (kllm serve --client) binds 0.0.0.0 by default and has no authentication; bind it to 127.0.0.1 or place it behind a gateway.
• Unless sealed with --no-lock, the weights on disk are locked: encrypted in place with AES-256-CTR under a key derived from the model key. kllm serve --key unlocks them into ~/.cache/kllm/unlocked (KLLM_UNLOCKED_DIR, mode 0700) before the engine starts, so whoever runs the model holds the unlocked weights. Each tensor's largest binary exponent is stored in the clear.
• Every approved holder of an official model's key holds the same model key, and the team derives it from an offline master key. A model key is not a confidentiality boundary between its holders; for a key known to no one else, seal the model under a private key.
• Outside a confidential VM, kllm serve --key decrypts the tokenizer into host /dev/shm and offers no protection from the host. It is a development mode.

Security modelMeasured limits in the documentation