Heterogeneous Clusters¶

Launch Kit can render one coherent deployment for clusters containing multiple server models, GPU types, or NIC layouts. Discovery preserves each hardware layout as a source group; generation merges only groups that can safely share network resources.

Source Groups¶

Each clusterConfig entry is a source group with its own:

identifier, machineType, and gpuType.
Worker-node list and nvidia.kubernetes-launch-kit.machine selector.
East-west PF inventory, rail assignments, and capabilities.
Storage and third-party RDMA kernel modules.

Fresh discovery derives identifier from the machine/GPU identity, lowercases it, and removes complete NVIDIA segments. For example, DGX-B200-NVIDIA-H200 becomes dgx-b200-h200; the Launch Kit machine node label uses the same value. Common machine segments are shortened in the same pass (ThinkSystem → ts, PowerEdge → pe). Identifiers are bounded to 30 bytes. Long values use balanced machine/GPU prefixes plus a 6-character deterministic hash, with unused prefix space reassigned to the longer component. The separate GPU label still retains its discovered value such as NVIDIA-H200. Use the persisted identifier shown in cluster-config.yaml with --groups.

Inspect the available groups before filtering:

yq '.clusterConfig[] | {
  identifier,
  machineType,
  gpuType,
  workerNodes,
  nodeSelector
}' cluster-config.yaml

Automatic Merging¶

Without a filter, generation buckets source groups by:

GPU type.
Number of east-west rails.

Different machine types can therefore share one generated resource bucket when their GPU type and rail count match. The merged bucket uses the GPU label:

nodeSelector:
  nvidia.kubernetes-launch-kit.gpu: NVIDIA-H200

Groups with different GPU types or east-west rail counts remain separate. A group with an unresolved GPU type is never automatically merged.

North-south PFs do not contribute to the merge rail count.

Select Groups¶

Use --gpu-type to render every source group with a matching GPU type. Matching is case-insensitive:

l8k generate \
  --user-config ./cluster-config.yaml \
  --gpu-type NVIDIA-H200 \
  --save-deployment-files ./deployment

Use --groups for an exact set of source identifiers. Identifier matching is case-sensitive:

l8k generate \
  --user-config ./cluster-config.yaml \
  --groups pe-xe9680-h200,ts-sr680a-v3-h200 \
  --save-deployment-files ./deployment

--groups and --gpu-type are mutually exclusive. Launch Kit reports an error when a requested identifier or GPU type does not match, including the available values.

Example Cluster Shapes¶

Two GPU Types¶

--gpu-type NVIDIA-H200 selects every H200 source group and excludes the H100 group. The generated deployment still has one cluster-wide NicClusterPolicy; bucket-scoped resources target only the selected H200 cohort.

Filter generation to one GPU type

Discovered source groups

Group A DGX-B200 + H100 Filtered out

Group B PowerEdge-XE9680 + H200 Selected by GPU type

Generated scope

NVIDIA-H200 bucket H200 node-selecting policies H200 network and IP pool NicClusterPolicy remains cluster-wide

l8k generate \
  --user-config ./cluster-config.yaml \
  --gpu-type NVIDIA-H200 \
  --fabric ethernet \
  --deployment-type sriov \
  --multirail \
  --save-deployment-files ./deployment-h200

Same GPU, Different Servers¶

Three machine types with the same GPU type and east-west rail count combine into one render bucket. An unfiltered run uses the shared GPU label. A strict-subset --groups rollout keeps bucket-shared resources together but renders flat-selector policies for only the selected sources.

Combine compatible source groups into one render bucket

H200 source groups

Source A DGX-B200 machine label A

Source B ThinkSystem-SR680a-V3 machine label B

Source C PowerEdge-XE9680 machine label C

One compatible bucket

NVIDIA-H200 Same east-west rail count Shared network, resource name, and IP pool GPU selector for the full bucket

# Full H200 bucket
l8k generate \
  --user-config ./cluster-config.yaml \
  --save-deployment-files ./deployment

# Stage two of the three source groups
l8k generate \
  --user-config ./cluster-config.yaml \
  --groups dgx-b200-h200,ts-sr680a-v3-h200 \
  --save-deployment-files ./deployment-stage1

Mixed GPU And Server Matrix¶

In a four-group cluster, the default run renders an H100 bucket and an H200 bucket. A GPU filter selects a row of the matrix; an explicit group list can select a vendor-specific column across GPU types.

Choose a cohort from a mixed cluster

Four discovered source groups

G1 DGX-B200 + H100

G2 ThinkSystem + H100

G3 DGX-B200 + H200

G4 PowerEdge-XE9680 + H200

--gpu-type NVIDIA-H200: G3 + G4 --groups DGX sources: G1 + G3

Resulting buckets

defaultH100: G1 + G2
H200: G3 + G4

--gpu-typeH200: G3 + G4

--groupsH100: G1
H200: G3

Use --gpu-type when all nodes of one GPU type are the deployment unit. Use --groups for a staged rollout, vendor-specific cohort, or CI matrix with an explicit list of source identifiers.

Resource Scope¶

Launch Kit preserves Kubernetes resource ownership while rendering merged groups:

Resource behavior	Examples	Render result
Cluster-wide	`NicClusterPolicy`, Spectrum-X profile `ConfigMap`	One per deployment.
Shared per bucket	Secondary-network CRs, `CIDRPool`, `IPPool`, example workload	One per compatible GPU/rail bucket.
Node-selecting policy	`NicNodePolicy`, `SriovNetworkNodePolicy`, `SpectrumXRailPoolConfig`	One per bucket, or per source group when a strict subset needs flat selectors.
Machine-specific	`NicInterfaceNameTemplate`	One per source group.

When --groups selects only part of an otherwise mergeable bucket, shared resources keep one stable bucket identity while node-selecting policies are rendered for each selected source group. Aggregate workloads and IP pools use a selector expression containing the selected machine-label values.

PCI Layout Differences¶

Two source groups can use the same rail number with different PCI addresses. Launch Kit aggregates non-conflicting addresses per rail. When addresses conflict across rails, it can render a separate NicInterfaceNameTemplate for each source group so policies can use stable interface names instead of ambiguous PCI addresses.

Review generated interface templates whenever a merged bucket spans different machine types:

find deployment/network-operator \
  -name '*nic-interface-name-template*.yaml' \
  -print

NV-IPAM Allocation¶

Subnets are allocated across the final render buckets before templates are written. This keeps generated IP pools disjoint when one cluster produces multiple GPU/rail buckets.

For production, inspect every generated IPPool or CIDRPool and confirm that it does not overlap the cluster pod, service, management, or external network ranges.

East-West And North-South NICs¶

Only east-west PFs produce rails and workload networking resources. North-south PFs remain visible in a mixed source group's inventory but are excluded from generated manifests. A source group containing only north-south PFs is omitted from cluster-config.yaml.

Traffic direction determines whether a PF is rendered

Management / OOB network

BlueField DPU North-south PF Inventory only; not rendered

GPU mesh and cluster nodes

ConnectX NIC or BlueField SuperNIC East-west PF Assigned a rail and rendered

Operational Guidance¶

Re-run discovery after adding a new server model or changing NIC mode.
Keep the Launch Kit machine and GPU labels; generated selectors depend on them.
Prefer --gpu-type when all matching source groups should share one deployment.
Prefer --groups for staged rollouts or machine-specific targeting.
Run l8k deploy --dry-run before applying a newly merged topology.
Complete the rollout with l8k validate and review the node-groups section of the acceptance report.