Skip to content
Local AI Inference Recipes

Run Models Optimally On Your Local NVIDIA AI System

RTX Spark

GB

9 models available on this hardware at 128 GB.

Recipe details

Execution
Native
Backend version
b11617
Backend min version
b11370
Peak GPU Memory
78 GB
Speculative Decoding
MTP
Concurrency
1
Served model
unsloth/Qwen3.8-Flash-Next-GGUF
Port
9931
Notes
Target shards: 1, 2, 3. MTP head: mtp-Qwen3.8-Flash-Next-Q8_0.gguf. Projector: mmproj-BF16.gguf.
Recipe ID
qwen/qwen3.8-flash-next/rtx-spark/windows/llama-cpp/78gb/iq4-xs/vision-mtp

Selected recipe

OptimizedRecommended

RTX Spark · Windows · llama.cpp · IQ4_XS · vision-mtp

Launch MTP · n=3

Run in PowerShell from the extracted b11617 directory. llama-server downloads the specified target shards, Q8_0 MTP head and BF16 projector using its normal cache. Sampling — --temp=1.0; --top-p=0.95; --top-k=20; --min-p=0.0; --presence-penalty=0.0; --repeat-penalty=1.0

.\llama-server.exe -hf "unsloth/Qwen3.8-Flash-Next-GGUF" `
  -hff "UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf" `
  --mmproj-url "https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF/resolve/38bb39ee97821de2c9009abb7e93950eec396e66/mmproj-BF16.gguf" `
  --host 127.0.0.1 `
  --port 9931 `
  --batch-size 2048 `
  --ubatch-size 512 `
  --ctx-size 262144 `
  --parallel 1 `
  -fa on `
  -lzm on `
  --temp 1.0 `
  --top-p 0.95 `
  --top-k 20 `
  --min-p 0.0 `
  --presence-penalty 0.0 `
  --repeat-penalty 1.0 `
  --spec-type draft-mtp `
  --spec-draft-n-max 3 `
  -hfrd "ggml-org/Qwen3.8-Flash-Next-GGUF" `
  --spec-draft-model "mtp-Qwen3.8-Flash-Next-Q8_0.gguf"

Measured performance

Optimized

Measured 2026-10-14 · NVIDIA RTX Spark N1X

E2E Completion Time
1,118.16 s
AA-AgentPerf-local
Decode Throughput
35.37 tok/s
SPEED-Bench Qualitative Coding
1× NVIDIA RTX Spark N1XWindows ARM64Driver 617.06CUDA 13.4llama.cpp b11617