Skip to content

Serve and verify

Optional. Tensorseed can launch and vouch for an OpenAI compatible inference engine that you install separately (llama.cpp or vLLM). It serves only a copy that verified against the signed catalog, and can sign every response so clients can check which exact weights answered. Pulling, verifying, and seeding never need an engine.

Selling inference?

Pull and verify a model, serve it, sign responses, publish a signed claim about the weights, and list the endpoint.

Read the seller guide

Running a marketplace or gateway?

Trust catalogs, check a seller’s claim, verify signed responses, and run logprob spot challenges.

Read the integration guide

How it fits together

  1. A seller pulls a model and verifies it against a catalog the buyer can trust.
  2. tensorseed serve-inference starts the engine on that verified copy only.
  3. The seller publishes a signed claim naming the exact model, revision, and file hashes behind the endpoint.
  4. With signing on, every answer carries an Ed25519 signature from the claim’s response key.
  5. A marketplace checks the claim against the signed catalog, verifies responses, and runs rare logprob challenges.

A claim is a statement the seller makes and can be held to, not proof of what runs. Signed responses make a swap attributable to the seller’s key, and challenges make it detectable at a low, tunable rate.

Commands

tensorseed serve-inference serves a verified copy through an OpenAI compatible engine, and tensorseed sign-responses puts a signing proxy in front of one. Tensorseed Desktop does not serve inference yet; use the CLI.

Install an engine

The engine is not part of Tensorseed. Engines are large, need builds that match your GPU, and update often, so they come from their own projects.

  • llama.cpp (llama-server) serves GGUF models. Install it with winget install llama.cpp on Windows, brew install llama.cpp on macOS or Linux, your distribution’s package, or an archive from the llama.cpp releases page.
  • vLLM (vllm) serves safetensors models. Install it with pip install vllm in a Python virtual environment. It needs Linux and a supported GPU (on Windows, use WSL 2).

Tensorseed finds the engine on PATH, or at inference.llama_server or inference.vllm in the config:

[inference]
llama_server = "/opt/llama.cpp/bin/llama-server"   # "" = llama-server on PATH
vllm = "~/vllm-env/bin/vllm"                      # "" = vllm on PATH

Without an engine, serve-inference stops with “llama.cpp is not installed” (or “vLLM is not installed”), the install options, and a link to the inference guide.