Skip to content
Local AI Inference Recipes

Run Models Optimally On Your Local NVIDIA AI System

RTX Spark

GB

12 models available on this hardware at 128 GB.

Recipe details

Execution
Native
Backend version
b11901
Backend min version
b11370
Peak GPU Memory
78 GB
Speculative Decoding
MTP
Concurrency
1
Served model
unsloth/Qwen3.8-Flash-Next-GGUF
Port
9931
Notes
Target: four-shard UD-IQ4_XS at revision 084c631ac221016019d6e0a863afe92d90edd3b0. MTP head: mtp-Qwen3.8-Flash-Next-Q8_0.gguf. Projector: mmproj-BF16.gguf. Performance shown was measured on the earlier three-shard upload (revision 38bb39ee97821de2c9009abb7e93950eec396e66); this four-shard target has not been benchmarked or revalidated yet.
Recipe ID
qwen/qwen3.8-flash-next/rtx-spark/windows/llama-cpp/78gb/iq4-xs/vision-mtp

Selected recipe

Recommended

RTX Spark · Windows · llama.cpp · IQ4_XS · vision-mtp

Experimental recipe — more optimizations upcoming.

Launch MTP · n=3

Run in PowerShell from the extracted b11901 directory. llama-server downloads the specified target shards, Q8_0 MTP head and BF16 projector using its normal cache. Sampling — --temp=1.0; --top-p=0.95; --top-k=20; --min-p=0.0; --presence-penalty=0.0; --repeat-penalty=1.0

.\llama-server.exe -hf "unsloth/Qwen3.8-Flash-Next-GGUF" `
  -hff "UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00004.gguf" `
  --mmproj-url "https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF/resolve/38bb39ee97821de2c9009abb7e93950eec396e66/mmproj-BF16.gguf" `
  --host 127.0.0.1 `
  --port 9931 `
  --batch-size 2048 `
  --ubatch-size 512 `
  --ctx-size 262144 `
  --parallel 1 `
  -fa on `
  -lzm on `
  --temp 1.0 `
  --top-p 0.95 `
  --top-k 20 `
  --min-p 0.0 `
  --presence-penalty 0.0 `
  --repeat-penalty 1.0 `
  --spec-type draft-mtp `
  --spec-draft-n-max 3 `
  -hfrd "ggml-org/Qwen3.8-Flash-Next-GGUF" `
  --spec-draft-model "mtp-Qwen3.8-Flash-Next-Q8_0.gguf"

Measured performance

Optimized

Measured 2026-10-27 · NVIDIA RTX Spark N1X

E2E Completion Time
1,029.10 s
AA-AgentPerf-local
Decode Throughput
36.58 tok/s
SPEED-Bench Qualitative Coding
1× NVIDIA RTX Spark N1XWindows ARM64Driver 620.91CUDA 13.4llama.cpp b11901AA-AgentPerf-local 0.3.4