Local model guide · updated July 2026 · 10 min read

How to Run Llama 4 Locally

Llama 4 is Meta's mixture-of-experts generation, natively multimodal and built for very long context. Scout is the one most people can run, since 17B active parameters keep it fast and a 4-bit quant fits a single large card or a well-specced Mac; Maverick is the multi-GPU sibling. This guide sizes the hardware, handles the gated download, and runs both across llama.cpp and vLLM.

Can you run it?

Both activate only 17B parameters per token, so they run at that speed while needing memory for their full size. Scout is the realistic single-machine target; Maverick is a data-center model.

Getting access

Accept the license on the Hugging Face model page, then log in so downloads are authorized:

pip install huggingface_hub
hf auth login
hf download meta-llama/Llama-4-Scout-17B-16E-Instruct --local-dir Llama-4-Scout

Community GGUF conversions also exist under other orgs if you prefer to pull a quantized build straight into llama.cpp.

Scout with llama.cpp

Build llama.cpp (steps here), then serve Scout. On a 24GB card, offload the experts to system RAM:

./llama.cpp/llama-server \
  -hf unsloth/Llama-4-Scout-17B-16E-Instruct-GGUF:Q4_K_M \
  --jinja \
  -ngl 99 \
  -ot ".ffn_.*_exps.=CPU" \
  -c 32768 \
  --host 127.0.0.1 --port 8080

On an 80GB GPU or a 64GB-plus Mac you can drop the -ot offload and keep everything in fast memory.

Maverick with vLLM

vllm serve meta-llama/Llama-4-Maverick-17B-128E-Instruct \
  --tensor-parallel-size 8 \
  --max-model-len 131072 \
  --enable-auto-tool-choice \
  --tool-call-parser llama4_pythonic

On a Mac

pip install mlx-lm
mlx_lm.server --model mlx-community/Llama-4-Scout-17B-16E-Instruct-4bit --port 8080

A 64GB or larger unified-memory Mac holds Scout at 4-bit; LM Studio provides the same through a desktop app.

Tool calling

Enable tools with the Llama 4 parser. In vLLM use --tool-call-parser llama4_pythonic with --enable-auto-tool-choice. In llama.cpp pass --jinja for the model's template. The endpoint is OpenAI-compatible for any agent.

Very long context

Scout advertises an extremely long context window, but the practical limit is memory, not the model. The KV cache grows with every token you keep, so a multi-million-token window would need enormous memory. Set -c or --max-model-len to the context you actually use, quantize the KV cache with --cache-type-k q8_0 when it is tight, and only reach toward the maximum when you have measured the cache fits.

Troubleshooting

• Download refused. You have not accepted the license or logged in. Accept on the model page and run hf auth login.
• Out of memory. Offload more experts with -ot, drop a quant tier, lower context, or add GPUs for Maverick.
• Context blows up memory. The KV cache, not the weights, is the cost. Lower -c and quantize the cache.
• Tool calls not parsed. Use llama4_pythonic in vLLM, or --jinja in llama.cpp.

FAQ

Can Scout run on one GPU? At 4-bit its weights are around 58 GB, so an 80GB GPU or a 64GB-plus Mac, or a 24GB card with expert offload to RAM.

Is it open source? Open-weight but under Meta's Community License, which is not OSI-approved and is gated on Hugging Face. Read the terms for commercial use.

Tool calling? Yes, with the llama4_pythonic parser in vLLM or --jinja in llama.cpp.