Write the workload once.
Run it anywhere.
Unchanged. Tasks, GPUs, health checks.
Workstation, Slurm or Kubernetes: swap one file.
Porting one job means
rewriting all of this.
Every fix, three times
A timeout bumped on Slurm stays wrong on Kubernetes until someone remembers.
Copies drift apart
Three scripts evolve separately; the "same" job quietly stops being the same.
Numbers stop comparing
Different launch glue on each platform: benchmark results are no longer like for like.
6 concerns × 3 platforms = 18 pieces of glue to write, debug and keep in sync.
Two files. Only one of them changes.
Those six concerns are written once. Only how to launch changes per platform.
- commands and start order
- GPUs per task
- readiness and failure checks
- cleanup, logs, results: sflow does these
How this platform launches it.
One per platform, sharedThe workload names roles,
never platforms.
- name: server
operator: server
script:
- set -x
- >
vllm serve ${{ artifacts.LOCAL_MODEL_PATH.path }}
--served-model-name ${{ variables.SERVED_MODEL_NAME }}
--host 0.0.0.0 --port 8000
--tensor-parallel-size ${{ variables.SERVER_GPUS }}
--trust-remote-code
resources:
gpus:
count: ${{ variables.SERVER_GPUS }}
operator: serverA role. The backend decides what launches it.
gpus: countHow many GPUs, never which node or device.
no srun · no docker · no podNothing in this file ties it to a platform.
You write the role.
sflow writes the launch.
operator: serverand from each backend file sflow generates the native launch command:Workstation
- name: server
type: docker_run
image: ${{ variables.SERVER_IMAGE }}
docker run --rm \
--gpus "device=0,1,2,3" -v … \
nvcr.io/…/vllm-runtime:1.0.0 …Slurm
- name: server
type: srun
container_image: ${{ variables.SERVER_IMAGE }}
container_writable: true
mpi: pmix
ntasks_per_node: 1
srun --nodes 1 --nodelist cluster-node0 \
--ntasks-per-node 1 --mpi pmix \
--container-image nvcr.io/…/vllm-runtime:1.0.0 …Kubernetes
- name: server
type: k8s
image: ${{ variables.SERVER_IMAGE }}
kubectl apply -f server.k8s.yaml
# Pod · nvidia.com/gpu: 4
# pinned to the reserved nodeThe same file. Three platforms.
never edited
runs on
changes
sflow run -f backends/docker.yaml -f workloads/vllm_serve.yaml -f benchmark.yaml
sflow run -f backends/slurm.yaml -f workloads/vllm_serve.yaml -f benchmark.yaml
sflow run -f backends/k8s.yaml -f workloads/vllm_serve.yaml -f benchmark.yaml
Add --dry-run to any of them: the whole plan, without touching a cluster.
What you gain.
Debug small, submit big
Prove it on a workstation with Docker, then submit the identical workload to the cluster.
Platform glue, written once
One person writes the Slurm or Kubernetes backend file. Everyone else reuses it.
Fair platform comparisons
Same workload, same benchmark, different backend: the comparison is like for like.
Scale by numbers, not rewrites
More nodes, GPUs or replicas is a value change. Multi-node disaggregated LLM serving ships as ordinary recipes.
Try it in a minute.
$ uv tool install nv-sflow $ sflow sample self_contained/local/hello_world \ -o hello.yaml $ sflow run -f hello.yaml
“Set up sflow for me by followinghttps://raw.githubusercontent.com/NVIDIA/nv-sflow/main/docs-site/agents-src/setup.mdstep by step, then tell me what you installed and where.”