Skip to main content
Version: main

Quick Reference

All sflow.yaml config fields at a glance. The Required column indicates mandatory fields.

For detailed explanations and examples, see Configuration.

Root-Level​

FieldRequiredTypeDefaultDescription
versionYesstring—Schema version. Must be "0.1".
variablesdict / list—Global variables available to expressions and task env.
artifactsdict / list—Named resources referenced by URI.
backendsdict / list—Compute backends (local, slurm, docker, kubernetes).
operatorsdict / list—Task execution operators (bash, srun, docker_run, ssh, python, k8s, k8s_mpi).
storagedict / list—Post-execution upload targets (S3, …) referenced by task uploads / upload_all (see Storage).
workflowYesobject—Workflow definition containing name and tasks.

Variables​

YAML path: variables.<name>

FieldRequiredTypeDefaultDescription
valueYesany—Variable value (int, float, bool, string, or list).
descriptionstringnullHuman-readable description.
domainlistnullAllowed values; enables replica variable sweeps. value must be in domain if set.
typestring"string"Type hint (string, integer, etc.).

Artifacts​

YAML path: artifacts.<name>

FieldRequiredTypeDefaultDescription
uriYesstring—Resource URI with scheme (fs://, file://, http://, s3://).
descriptionstringnullHuman-readable description.
contentstringnullInline file content. Only valid with file:// URI.

Storage​

YAML path: storage.<name> — post-execution upload targets referenced by task uploads / upload_all. Credentials come from the provider's default chain (e.g. boto3 for S3) — never inline secrets in YAML. See Uploads.

FieldRequiredTypeDefaultDescription
nameYesstring—Target name (referenced by uploads.target / upload_all.target).
typeYesstring—Target type. Currently s3.

S3 target (type: s3) adds bucket (Yes, string / expr) plus optional region, prefix, endpoint_url (S3-compatible stores such as MinIO / Ceph RGW), storage_class, and addressing_style (auto / virtual / path).

Backends — Common Fields​

YAML path: backends.<name>

FieldRequiredTypeDefaultDescription
typeYesstring—local, slurm, docker, or kubernetes.
defaultboolfalseMark as the default backend (only one allowed).
gpus_per_nodeint / exprnullGPUs per node for sflow planning / packing. Does not add Slurm GPU allocation flags.
include_nodeslist[string]nullRestrict allocation to these node hostnames (unioned with --include-nodes). Translated per backend: Slurm --nodelist, k8s nodeAffinity In, Docker host filter.
exclude_nodeslist[string]nullExclude these node hostnames (unioned with --exclude-nodes).

Backends — Local​

Additional fields when type: local

FieldRequiredTypeDefaultDescription
nodesint / expr1Number of synthetic local nodes.

Backends — Slurm​

Additional fields when type: slurm

FieldRequiredTypeDefaultDescription
accountYesstring / expr—Slurm account.
partitionYesstring / expr—Slurm partition.
timeYesstring / int / expr—Time limit — HH:MM:SS string or integer minutes.
nodesYesint / expr—Number of nodes.
gpus_per_nodeYesint / expr—GPUs per node for planning. Set to 0 for CPU-only partitions; tasks that request resources.gpus against a zero-capacity backend will be rejected.
extra_argslist[string]nullExtra salloc arguments (e.g. --exclusive, --gpus-per-node=8).
job_namestringnullJob name; defaults to workflow name.

Backends — Kubernetes​

Additional fields when type: kubernetes. The workload image lives on the operator (k8s / k8s_mpi), not the backend. Cluster/context/namespace are selected with sflow run flags (--kubeconfig, --kube-context, --kube-namespace), never in YAML. See Backends.

FieldRequiredTypeDefaultDescription
namespacestring / exprnullNamespace for all pods; must already exist. One namespace per backend. Overridable with --kube-namespace.
nodesint / expr1Number of nodes to reserve.
schedulingdevice_plugin|dradevice_pluginGPU request mode: NVIDIA device plugin (nvidia.com/gpu) or DRA (K8s 1.34+, WIP).
gpu_resource_namestring / exprnvidia.com/gpuDevice-plugin resource name (e.g. MIG nvidia.com/mig-1g.5gb, or another vendor). Ignored under dra.
gpu_product_label_keystring / exprnvidia.com/gpu.productNode label carrying the GPU product string (NVLink-scope auto-detection).
host_networkbooltrueAll pods use host networking (pod IP == node IP). Privileged; disable on CNI-routable clusters.
host_ipcboolfalseShare node IPC namespace + hostPath /dev/shm for cross-pod CUDA IPC over NVLink. Privileged; opt-in.
merge_colocated_gpu_podsbool | auto|on|disableautoMerge co-located single-node GPU tasks into one pod for intra-node NVLink/cuda_ipc.
nvlink_domainauto|node|rack|disableautoAdvisory NVLink-domain scope; set only to correct a wrong auto-detection. Not the MNNVL switch (see compute_domain).
rdmaauto|disable|gke|shared_device_plugin|host_deviceautoRDMA/IB provisioning for GPU pods; auto detects the provider, disable forces sockets.
node_selectormapnullNode-selector labels for all pods. Merge/override with --kube-node-selector.
tolerationslist[map]tolerate nvidia.com/gpuPod tolerations for placeholder + task pods.
image_pull_policystring / exprnullDefault pod pull policy (Always/IfNotPresent/Never); else cluster default.
probe_pod_imagestring / exprcurlimages/curl:latestImage for the in-cluster TCP/HTTP probe pod (mirror for air-gapped).
cpu_per_gpuint / exprnullCPU request per GPU for GPU task pods (requests-only; opt-in).
cpu_requestint / exprnullCPU request for pods without GPUs (requests-only; opt-in).
collect_max_file_sizeint / stringnull (10 MiB)Per-file cap for collecting a task's node-local output back to the driver; 0 disables.
collect_grace_secondsint / expr120Grace period (s) to copy node-local output before pod teardown.
collect_node_local_outputbooltrueMaster switch for the node-local output collect. Left on, the DAG waits on a sentinel the pod writes to its log, so a task's output can be lost to log-delivery lag; false runs tasks with no sflow collect machinery in the pod — completion then depends only on pod status, probes and the merge-pod marker, and outputs must reach the driver via a shared filesystem, uploads: or a PVC.
gib_installer_namespacestring / exprkube-systemNamespace of the GKE gIB (nccl-rdma-installer) DaemonSet.
extra_argslist[string]nullExtra backend arguments (reserved).
draobjectnullDRA GPU-allocation options (see below); used when scheduling: dra.
compute_domainobjectnullMulti-Node NVLink (IMEX ComputeDomain) options (see below).
volumeslist[object]nullPVC / emptyDir volumes mounted into every task pod (see below).
reservationobjectnullNode-reservation tuning (see below).

gpus_per_node (see Common Fields) is derived from node capacity when unset and validated against real GPU capacity at pre-flight; set 0 for CPU-only. PVCs referenced under volumes: must already exist in the namespace — sflow run --kube-skip-pvc drops PVC-backed volumes for debugging.

Kubernetes — volumes[]​

YAML path: backends.<name>.volumes[]. Set exactly one of claim or empty_dir.

FieldRequiredTypeDefaultDescription
nameYesstring—Pod volume name (DNS-1123).
mount_pathYesstring / expr—Absolute mount path inside each task pod.
claimone ofstring / exprnullExisting PVC name. Mutually exclusive with empty_dir.
empty_dirone ofobjectnullEphemeral per-pod scratch volume. Mutually exclusive with claim.
sub_pathstring / exprnullPath within the volume to mount (volumeMount.subPath).
read_onlyboolPVC true / emptyDir falseMount read-only.
ensure_writableboolfalseInject a root initContainer to chmod the subPath. Requires a claim and read_only: false.

volumes[].empty_dir fields: medium (""|Memory, default "" = node disk; Memory = tmpfs), size_limit (string / expr, e.g. 50Gi).

Kubernetes — dra​

YAML path: backends.<name>.dra (used when scheduling: dra; DRA is WIP).

FieldRequiredTypeDefaultDescription
gpu_device_classstring / exprgpu.nvidia.comDeviceClass GPUs are requested from.
device_selectorslist[string]nullCEL expressions narrowing eligible devices (by product/memory).
rdma_device_classstring / exprnullOpt-in GPU↔NIC co-allocation NIC DeviceClass.
rdma_match_attributestring / exprresource.kubernetes.io/pcieRootAttribute the GPU and NIC requests must share (PCIe root).
nvlink_domain_label_keystringnullNode label key identifying the physical NVLink domain (multi-domain placement).

Kubernetes — compute_domain​

YAML path: backends.<name>.compute_domain — Multi-Node NVLink (IMEX). Independent of scheduling.

FieldRequiredTypeDefaultDescription
channelstring / exprnullJoin an existing IMEX channel template; auto = the sole existing domain; off/empty = no claim.
createboolfalseCreate a fresh ComputeDomain CR instead of joining. Privileged; needs computedomains RBAC.

Kubernetes — reservation​

YAML path: backends.<name>.reservation — tuning for the node reservation.

FieldRequiredTypeDefaultDescription
timeoutint / expr600Seconds to wait for every placeholder pod to be scheduled.
handoffauto|create_before_destroy|destroy_before_createautoPlaceholder→task handoff order (auto = destroy-before-create when a GPU ResourceQuota is present).
placeholder_imagestring / exprbash:5Image for reservation placeholder pods (mirror for air-gapped).

Operators — Common Fields​

YAML path: operators.<name>

FieldRequiredTypeDefaultDescription
typeYesstring—Operator type: bash, srun, docker_run, ssh, python, k8s, or k8s_mpi.

Operators — srun​

Additional fields when type: srun

FieldRequiredTypeDefaultDescription
job_idstringnullExisting Slurm job ID.
nodesint / stringnullNode count.
nodelistlist[string][]Node list.
partitionstringnullSlurm partition.
accountstringnullSlurm account.
qosstringnullQOS.
reservationstringnullReservation.
timestringnullTime limit.
constraintstringnullSlurm constraint.
exclusiveboolfalseExclusive node allocation.
chdirstringnullWorking directory.
cpus_per_taskint / stringnullCPUs per task.
gpusstringnullGPU spec (e.g. all, 1, device=0).
gpus_per_taskstringnullGPUs per task.
gresstringnullGeneric resource spec.
memstringnullMemory.
mem_per_cpustringnullMemory per CPU.
ntasksint / stringnullNumber of tasks.
ntasks_per_nodeint / stringnullTasks per node.
exportstring"ALL"Environment export setting.
labelbooltruePrefix output with task label.
unbufferedbooltrueUnbuffered output.
kill_on_bad_exitboolfalseKill job on non-zero task exit.
overlapbooltrueAllow step overlap.
waitint / stringnullWait time.
container_imagestringnullContainer image (Pyxis). Mutually exclusive with container_name.
container_namestringnullExisting container name (Pyxis). Mutually exclusive with container_image.
container_mount_homeboolfalseMount home directory in container.
container_writablebooltrueWritable container filesystem.
container_mountslist[string][]Bind mounts (e.g. "/host:/ctr:rw").
container_workdirstringnullContainer working directory.
container_remap_rootboolfalseRemap root inside container.
mpistringnullMPI type (e.g. pmix, ucx).
extra_argslist[string][]Extra CLI arguments.

Operators — Docker Run​

Additional fields when type: docker_run

FieldRequiredTypeDefaultDescription
imageYesstring—Docker image.
workdirstringnullWorking directory inside container.
mountslist[string][]Bind mounts (e.g. "/host:/ctr:rw").
gpusstringnullGPU spec (e.g. all, device=0).
extra_argslist[string][]Extra docker run arguments.
pass_envsbooltrueForward host environment variables.

Operators — SSH​

Additional fields when type: ssh

FieldRequiredTypeDefaultDescription
hostYesstring—SSH host.
userstringnullSSH user.
portintnullSSH port.
identity_filestringnullPath to identity file.
extra_argslist[string][]Extra SSH arguments.

Operators — Python​

Additional fields when type: python

FieldRequiredTypeDefaultDescription
python_execstring"python"Python executable.
extra_argslist[string][]Extra Python arguments.

Operators — Kubernetes (k8s)​

Additional fields when type: k8s. The workload image lives on the operator (the kubernetes backend has none). See Backends for the complete pod/container passthrough list.

FieldRequiredTypeDefaultDescription
imageYesstring—Container image for the pod.
image_pull_policystringnullAlways / IfNotPresent / Never (else cluster default).
restartstring"Never"Pod restartPolicy.
host_networkboolinherit backendPod IP == node IP (needed by some RDMA paths).
node_selectormapinherit backendConstrain the pod to nodes with these labels.
run_as_rootboolfalseForce runAsUser/Group=0 (root-owned PVCs, MPI/sshd bootstrap).
pass_envsbooltrueCreate + mount the env Secret.
shm_sizestringnullCap for RAM-backed /dev/shm (e.g. 64Gi); unset = node-RAM-bounded tmpfs.
ttyboolfalseAllocate a pseudo-TTY so \r progress bars stream live.
cpuint / stringnullContainer CPU request.
memorystringnullContainer memory request (e.g. 16Gi).
cpu_limitint / stringnullOptional CPU hard cap (unset = requests-only).
memory_limitstringnullOptional memory hard cap (unset = requests-only).
envlist[dict]nullRaw k8s EnvVar entries (only way to use valueFrom).
image_pull_secretslist[string]nullSecret name(s) for pulling from a private registry (e.g. nvcr.io).
service_accountstringnullPod service account (RBAC / cloud workload identity).
working_dirstringnullContainer working directory.
portslist[dict]nullContainer ports entries.
tolerationslist[dict]inherit backendPod tolerations (added to the backend's).
collect_max_file_sizeint/stringinherit backendPer-file cap for collecting node-local output back to the driver; 0 disables.
security_contextmapnullContainer securityContext (deep-merged; caps union-merged).
device_class / device_selectorsstring/listinherit backendDRA overrides (ignored under device_plugin).
container_overrides / pod_overridesmapnullRaw escape hatches, deep-merged last (they win).

Operators — Kubernetes MPI (k8s_mpi)​

Additional fields when type: k8s_mpi — any MPI (mpirun) job, single- or multi-node. Inherits every k8s field above and adds an mpi: block. Write an explicit mpirun -np N ... <workload> and sflow builds the MPI world either way, injecting the SSH keypair, hostfile, and launcher wiring. Use plain k8s for non-MPI, single-pod workloads.

mpi. fieldTypeDefaultDescription
routeauto|operator|podsautoMPIJob CR (Kubeflow MPI Operator) vs plain-pods bootstrap; auto detects the CRD.
slots_per_workerint/stringper-node GPU countSlots advertised per worker.
run_launcher_as_workerbooltrueLauncher pod also runs a rank.
launcher_creation_policyAtStartup|WaitForWorkersReadyWaitForWorkersReadyWhen the launcher starts.
ssh_portint / expr2222Port for the injected sshd.
mpi_implementationOpenMPI|Intel|MPICHOpenMPISelects hostfile/env conventions.
ensure_sshdbooltrueInstall openssh-server if the image lacks sshd.
forward_env_prefixeslist[string][]Extra env-namespace prefixes forwarded to remote ranks.
omp_num_threadsint8Injected OMP_NUM_THREADS per rank; null/0 keeps the image default.
cpu_bindcore|numa|nonecorePer-rank CPU binding, injected only when several ranks share a pod.
cpu_bind_cores_per_rankint8Max cores bound per rank under cpu_bind: core; 0 = uncapped.
worker_setup_timeout_secondsint / string900Per-node worker setup / readiness budget before the launcher starts.
launcher_discovery_timeoutint / string600Max wait for the MPI operator to create the launcher pod (route: operator).

Workflow​

YAML path: workflow

FieldRequiredTypeDefaultDescription
nameYesstring—Workflow name.
tasksYeslist—List of task definitions (must be non-empty).
timeoutstring / intnullNot enforced — accepted and merged, but nothing reads it; a workflow that sets it runs unbounded and sflow run warns. Use the backend's own limit (Slurm --time).
variablesdict / listnullWorkflow-scoped variables (same format as root variables).
upload_allobjectnullZip the whole workflow output dir and upload it to a storage target (see Workflow Upload-All).
monitorobjectnullWorkflow-level hardware monitor (see Monitor).

Tasks​

YAML path: workflow.tasks[]

FieldRequiredTypeDefaultDescription
nameYesstring—Task name (must be unique).
scriptYeslist[string]—Script lines to execute (non-empty).
operatorstring / objectnullOperator name, or inline operator override object.
backendstring / dictnullBackend name, or inline backend override.
depends_onlist[string]nullNames of tasks this task depends on.
required_bylist[string]nullReverse dependency: A required_by: [B] is folded into B depends_on: [A] at load (lets an optional fragment attach to a hub without editing it).
timeoutint / stringnullNot enforced — see Workflow. Bound the task with the backend's own limit instead.
variablesdict / listnullTask-scoped variables.
resourcesobjectnullNode / GPU resource requirements.
replicasobjectnullReplication configuration.
retriesobjectnullRetry configuration.
probesobjectnullReadiness and failure probes.
outputslistnullOutput parsing configuration (legacy MVP).
resultmap / objectnullConsolidated result parsing (regex map, patterns, or file). Writes result.json + workflow results.json.
fail_fastboolbackendPrepend set -e to the script (shell operators) so the task fails on the first failing command. Unset takes the backend default: true on Kubernetes, false on Slurm/local/docker. Set explicitly to override.
uploadslistnullPer-task file uploads to a storage target, fired on COMPLETED (see Task Uploads).
portslistnullService ports the task exposes, feeding ${{ task.<name>.service }} expressions (see Task Ports).
monitorobjectnullPer-task hardware monitor (see Monitor).

Task Resources​

YAML path: workflow.tasks[].resources

FieldRequiredTypeDefaultDescription
nodes.indiceslist[int / expr]nullSpecific node indices (e.g. [0]).
nodes.countint / exprnullNumber of nodes.
nodes.excludeint / list[int] / exprnullNode indices to remove from the placement pool before indices, count, or GPU packing.
nodes.release_afterstringnullWhen node reservations can be reused: workflow_completion, task_ready, or task_completion. sflow reserves nodes only when this is set explicitly; omitted, nodes.indices/count are non-exclusive placement constraints (see note below).
gpus.countOne of count / indicesint / exprnullNumber of GPUs (sets CUDA_VISIBLE_DEVICES). Index-agnostic on its own: the planner packs the task into any contiguous idle GPU run on one node.
gpus.indicesOne of count / indiceslist[int / expr]nullPin the task to specific 0-based, non-negative, unique device ids. Combined with count, count is the total across nodes and indices the per-node slice, so the task fans out over count / len(indices) nodes.
gpus.release_afterstringinferredWhen GPU reservations can be reused: workflow_completion, task_ready, or task_completion.

For nodes, release_after only creates an exclusive node reservation when explicitly set; omitted nodes.indices and nodes.count are placement constraints and may overlap with other planned tasks. For GPUs, omitted release_after is inferred: tasks without readiness probes release GPUs after task completion for downstream dependents, while tasks with readiness probes keep GPUs until workflow completion unless explicitly set to task_ready. task_ready releases after readiness succeeds. task_completion releases after any terminal task status (COMPLETED, FAILED, TIMEOUT, or CANCELLED). Dry-run rehearses these resource lifetimes across the DAG.

Task Replicas​

YAML path: workflow.tasks[].replicas

FieldRequiredTypeDefaultDescription
countint / exprnullNumber of replicas.
policystring / expr"parallel""parallel" or "sequential".
variableslist[string]nullVariable names for sweeps (Cartesian product of domains).

Task Retries​

YAML path: workflow.tasks[].retries

FieldRequiredTypeDefaultDescription
countYesint / expr—Number of retries.
intervalYesint / expr—Delay between retries (seconds).
backoffint / expr1Backoff multiplier.

Task Probes (Readiness / Failure)​

YAML path: workflow.tasks[].probes.readiness or workflow.tasks[].probes.failure

FieldRequiredTypeDefaultDescription
delayint / expr0Initial delay before probing (seconds).
timeoutint / expr1200Max readiness wait time (seconds). Failure probes do not use this as an overall deadline.
each_check_timeoutint / expr30Timeout for a single probe check attempt.
intervalint / expr5Check interval (seconds).
success_thresholdint / expr1Consecutive successes required.
failure_thresholdint / expr3Consecutive failures before failing.

Exactly one probe type must be set per probe:

Probe TypeRequired FieldsOptional FieldsDescription
tcp_portporthost, on_node ("first" / "each")TCP connection check.
http_geturlheadersHTTP GET health check.
http_posturlheaders, bodyHTTP POST health check.
log_watchregex_pattern or match_patternlogger, match_countMatch pattern in task logs. Literal by default; prefix with re: or regex: for regular expressions.

Task Outputs​

YAML path: workflow.tasks[].outputs[]

FieldRequiredTypeDefaultDescription
patternYesstring—Parse pattern (e.g. "TTFT: {ttft:f} ms").
sourcestring"stdout"Log source: stdout or stderr.
metrics.<key>.descriptionstringnullMetric description.
metrics.<key>.typestringnullMetric type.
metrics.<key>.aggregatestringnullAggregation hint.

Task Result​

YAML path: workflow.tasks[].result. Accepts a simple name: regex map, an object with patterns:, or an object with file:. See Results.

patterns and file are mutually exclusive.

FieldRequiredTypeDefaultDescription
patternslistnullAdvanced regex patterns (see below). Mutually exclusive with file.
filestringnullRelative path to a JSON source file ending in .json (e.g. result.json). Normalized into result.json.
sourcestring"log"Source selector. Only log is implemented.

result.patterns[]:

FieldRequiredTypeDefaultDescription
nameYesstring—Result key under values.
regexYesstring—Python regex; prefer one capture group or a named value group.
typestring"auto"auto, string, int, float, bool, or json.
unitstringnullOptional metadata unit (e.g. ms).
aggregatestring"last"first, last, list, count, min, max, avg, or sum.
requiredboolfalseIf true, a missing match marks the result ok: false.
sourcestringinheritedPer-pattern source override; only log is implemented.
groupstring / intvalue then 1Capture group name or index to extract.

A simple name: regex map is shorthand for patterns with type: auto, aggregate: last, required: false, source: log.

To publish a metric literally named file, use patterns:; a top-level file: key is reserved for JSON file-source results.

Task Uploads​

YAML path: workflow.tasks[].uploads[] — copy files to a storage target when the task reaches COMPLETED. See Uploads.

FieldRequiredTypeDefaultDescription
targetYesstring—Name of a storage target.
fromYesstring—Local file or glob to upload.
tostringnullRemote key/prefix; must end with / when from is a glob.
on_errorwarn | failwarnWhat to do if the upload fails.

Workflow Upload-All​

YAML path: workflow.upload_all — zip the entire workflow output dir and upload it.

FieldRequiredTypeDefaultDescription
targetYesstring—Name of a storage target.
tostring${{ workflow.run_id }}.zipRemote key for the uploaded zip.
on_errorwarn | failwarnWhat to do if the upload fails.

Task Ports​

YAML path: workflow.tasks[].ports[] — service ports the task exposes, feeding ${{ task.<name>.service }} expressions.

FieldRequiredTypeDefaultDescription
portYesint / expr—Port the task serves on.
namestringnullOptional port name.

Monitor​

YAML path: workflow.monitor or workflow.tasks[].monitor — hardware sampling during the run. Monitor fields are concrete ints only (not ${{ }}-resolved). See Monitor.

FieldRequiredTypeDefaultDescription
intervalint5000Sampling interval (ms) for built-in scopes without their own.
scopesobjectnull (all)Which scopes to collect — cpu, gpu, memory, disk, network, custom; omit ⇒ all built-ins active.
resourcesobjectnullWhich hardware to target — nodes / gpus (like task resources) or used_by_tasks: [names].
reportobject{enabled: true}Post-run report. format: any of csv, svg, png (png needs the sflow[monitor] extra).
report.enabledbooltrueOn by default whenever monitor: is set. Set false to opt out — worth doing on large fan-outs, since report cost scales with samples × views. Raw samples and the overview are still written.
windowobjectnull{start, end} task-log markers bounding the reported window, so the report covers the benchmark rather than the whole task lifecycle. Task monitors only (rejected on workflow.monitor) and requires report.enabled: true.

Expression Syntax​

Fields marked int / expr or string / expr support ${{ ... }} expressions:

ExpressionExample
Variable${{ variables.MY_VAR }}
Variable domain${{ variables.MY_VAR.domain }}
Backend node IP${{ backends.slurm_cluster.nodes[0].ip_address }}
Artifact path${{ artifacts.model_dir.path }}
Task node IP${{ task.server.nodes[0].ip_address }}

Reserved Environment Variables​

Injected by sflow into task environments​

These are automatically set by sflow and available in every task script.

VariableDescription
SFLOW_WORKSPACE_DIRAbsolute path to the project workspace root.
SFLOW_OUTPUT_DIRGlobal output root directory (default ./sflow_output).
SFLOW_WORKFLOW_OUTPUT_DIROutput directory for the current workflow run (e.g. sflow_output/<run-id>).
SFLOW_TASK_OUTPUT_DIROutput directory for the current task replica (e.g. sflow_output/<run-id>/my_task_0).
SFLOW_TASK_RESULT_FILECanonical per-task result file path (${SFLOW_TASK_OUTPUT_DIR}/result.json). Write JSON here to publish results directly.
SFLOW_WORKFLOW_RESULT_FILEWorkflow-level results index path (${SFLOW_WORKFLOW_OUTPUT_DIR}/results.json).
SFLOW_REPLICA_INDEXZero-based replica index (0, 1, 2, ...).
SFLOW_TASK_ASSIGNED_NODE_NAMESComma-separated hostnames of nodes assigned to this task.
SFLOW_TASK_ASSIGNED_NODE_IPSComma-separated IP addresses of nodes assigned to this task.
SFLOW_BACKEND_JOB_IDBackend allocation/job id when available. For Slurm this mirrors SLURM_JOB_ID / SLURM_JOBID.
SFLOW_BACKEND_NODELISTBackend allocation nodelist when available. For Slurm this mirrors SLURM_JOB_NODELIST / SLURM_NODELIST.
SFLOW_BACKEND_NUM_NODESNumber of nodes in the backend allocation when available. For Slurm this mirrors SLURM_NNODES.
CUDA_VISIBLE_DEVICESComma-separated GPU indices allocated to this task (set when resources.gpus.count is used).

In addition, all resolved variables and artifacts paths are injected as environment variables accessible via ${VAR_NAME} in scripts.

Avoid naming a variable after a reserved SFLOW_* / CUDA_VISIBLE_DEVICES env var above — sflow injects and owns these at launch, so a same-named variable collides and causes undefined behavior. sflow run --dry-run prints a Reserved env collisions section listing any such variables so you can rename them before a real run.

Read by sflow from the host environment​

sflow reads these to detect an existing Slurm allocation and skip salloc. When a Slurm controller provides SLURM_* / SLURMD_* variables, sflow preserves those controller values in task environments even if workflow variables use the same names.

VariableDescription
SLURM_JOB_ID / SLURM_JOBIDCurrent Slurm job ID. Used to detect an existing allocation.
SLURM_JOB_NODELIST / SLURM_NODELISTNode list for the current Slurm allocation.

Provided by Slurm at runtime​

These are set by Slurm (not by sflow) and commonly used in task scripts. The srun operator also maps common Slurm step/rank variables into backend-agnostic aliases.

Slurm variableSFLOW aliasDescription
SLURM_STEP_IDSFLOW_BACKEND_STEP_IDSlurm step id for the current srun step.
SLURMD_NODENAMESFLOW_TASK_NODE_NAMEHostname of the node running the task process.
SLURM_NODEIDSFLOW_TASK_NODE_INDEXNode index within the allocation (useful for NODE_RANK).
SLURM_PROCIDSFLOW_TASK_PROCESS_IDGlobal process/rank id within the step.
SLURM_LOCALIDSFLOW_TASK_LOCAL_PROCESS_IDLocal process/rank id on the node.
SLURM_NTASKSSFLOW_TASK_NUM_PROCESSESNumber of tasks/processes in the step.
SLURM_SUBMIT_DIR—Directory from which the job was submitted.