Skip to content
Local AI Inference Recipes

Run Models Optimally On Your Local NVIDIA AI System

RTX Spark

GB

11 models available on this hardware at 128 GB.

Recipe details

Execution
Native
Backend version
b11617
Backend min version
b11370
Peak GPU Memory
78 GB
Speculative Decoding
MTP
Concurrency
1
Served model
unsloth/Qwen3.8-Flash-Next-GGUF
Port
9931
Notes
Target shards: 1, 2, 3. MTP head: mtp-Qwen3.8-Flash-Next-Q8_0.gguf. Projector: mmproj-BF16.gguf.
Recipe ID
qwen/qwen3.8-flash-next/rtx-spark/windows/llama-cpp/78gb/iq4-xs/vision-mtp

Selected recipe

OptimizedRecommended

RTX Spark · Windows · llama.cpp · IQ4_XS · vision-mtp

Launch MTP · n=3

Run in PowerShell from the extracted b11617 directory. llama-server downloads the specified target shards, Q8_0 MTP head and BF16 projector using its normal cache. Sampling — --temp=1.0; --top-p=0.95; --top-k=20; --min-p=0.0; --presence-penalty=0.0; --repeat-penalty=1.0

.\llama-server.exe -hf "unsloth/Qwen3.8-Flash-Next-GGUF" `
  -hff "UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf" `
  --mmproj-url "https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF/resolve/38bb39ee97821de2c9009abb7e93950eec396e66/mmproj-BF16.gguf" `
  --host 127.0.0.1 `
  --port 9931 `
  --batch-size 2048 `
  --ubatch-size 512 `
  --ctx-size 262144 `
  --parallel 1 `
  -fa on `
  -lzm on `
  --temp 1.0 `
  --top-p 0.95 `
  --top-k 20 `
  --min-p 0.0 `
  --presence-penalty 0.0 `
  --repeat-penalty 1.0 `
  --spec-type draft-mtp `
  --spec-draft-n-max 3 `
  -hfrd "ggml-org/Qwen3.8-Flash-Next-GGUF" `
  --spec-draft-model "mtp-Qwen3.8-Flash-Next-Q8_0.gguf"

Measured performance

Optimized

Measured 2026-10-14 · NVIDIA RTX Spark N1X

E2E Completion Time
1,118.16 s
AA-AgentPerf-local
Decode Throughput
35.37 tok/s
SPEED-Bench Qualitative Coding
1× NVIDIA RTX Spark N1XWindows ARM64Driver 617.06CUDA 13.4llama.cpp b11617