Skip to content
Local AI Inference Recipes

Run Models Optimally On Your Local NVIDIA AI System

RTX Spark

GB

7 models available on this hardware at 128 GB.

Recipe details

Execution
Native
Backend version
b11229
Peak GPU Memory
36 GB
Speculative Decoding
DFlash
Concurrency
1
Served model
unsloth/Qwen3.8-27B-GGUF
Port
9931
Notes
Projector: mmproj-F16.gguf.
Recipe ID
qwen/qwen3.8-27b/rtx-spark/windows/llama-cpp/36gb/q4-k-m/vision-dflash

Selected recipe

Recommended

RTX Spark · Windows · llama.cpp · Q4_K_M · vision-dflash

Experimental recipe — more optimizations upcoming.

Launch DFlash · n=7

Run in PowerShell with llama-server.exe on PATH. llama-server downloads and caches the named target and draft files from Hugging Face. The omitted load-mode flag retains default mmap behavior. Sampling — Target backend sampling: OFF; speculative draft backend sampling: ON; --temp=1.0; --top-p=0.95; --top-k=20; --min-p=0.0; --presence-penalty=0.0; --repeat-penalty=1.0

llama-server.exe -hf "unsloth/Qwen3.8-27B-GGUF" `
  -hff "Qwen3.8-27B-UD-Q4_K_M.gguf" `
  --mmproj-url "https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/4ca720788d1e01f1bff70c033e0d0028fd02e502/mmproj-F16.gguf" `
  --host 127.0.0.1 `
  --port 9931 `
  --batch-size 4096 `
  --ubatch-size 512 `
  --ctx-size 262144 `
  --parallel 1 `
  -ngl 999 `
  --spec-type draft-dflash `
  --spec-draft-n-max 7 `
  -hfrd "incoai/Qwen3.8-27B-DFlash2-GGUF" `
  --spec-draft-model "Qwen3.8-27B-DFlash2-Q4_K_M.gguf" `
  --spec-draft-backend-sampling `
  --temp 1.0 `
  --top-p 0.95 `
  --top-k 20 `
  --min-p 0.0 `
  --presence-penalty 0.0 `
  --repeat-penalty 1.0

Measured performance

Not measured for this exact recipe.