LLama2.Odin is a straightforward port of Karpathy's LLama2.c written in the Odin programming language.

odin build . -o:speed

./llama2odin <checkpoint> [options]

• -t <float> - temperature in [0,inf], default 1.0
• -p <float> - p value in top-p (nucleus) sampling in [0,1] default 0.9
• -s <int> - random seed, default time(NULL)
• -n <int> - number of steps to run for, default 256. 0 = max_seq_len
• -i <string> - input prompt
• -z <string> - optional path to custom tokenizer
• -m <string> - mode: generate|chat, default: generate
• -y <string> - (optional) system prompt in chat mode

# Generate text with default settings
./llama2odin stories15M.bin

# Generate with custom prompt and 256 steps
./llama2odin stories15M.bin -n 256 -i "Once upon a time"

# Chat mode
./llama2odin stories15M.bin -m chat