Local HTTP inference service for Qwen3 GGUF models, built on the odin-infer library. Exposes an Ollama-compatible API so other apps can call it over HTTP.

• Odin compiler
• Clone odin-infer as a sibling directory:

Claw/
├── odin-infer/          # library
├── odin-infer-mac/      # CLI
└── odin-infer-server/   # this repo

./build.sh

./odin-infer-server Qwen3-0.6B-Q4_K_M.gguf --port 11435 -g 1

Default bind: 127.0.0.1:11435 (port chosen to avoid conflict with Ollama on 11434).

curl http://127.0.0.1:11435/api/generate -d '{
  "model": "qwen3",
  "prompt": "Why is the sky blue?",
  "stream": false
}'

curl http://127.0.0.1:11435/api/chat -d '{
  "model": "qwen3",
  "messages": [{"role": "user", "content": "Hello!"}],
  "stream": false
}'

Set "stream": true for newline-delimited JSON (application/x-ndjson).

• Single-model server: one GGUF loaded at startup.
• Requests are handled sequentially (one inference at a time).
• Qwen3 chat template is applied automatically for /api/chat.
• Thinking mode is disabled by default (matches CLI -k 0).

• odin-infer — inference library
• odin-infer-mac — interactive CLI