Heterogeneous Clusters

Launch Kit can render one coherent deployment for clusters containing multiple server models, GPU types, or NIC layouts. Discovery preserves each hardware layout as a source group; generation merges only groups that can safely share network resources.

Source Groups

Each clusterConfig entry is a source group with its own:

• identifier, machineType, and gpuType.
• Worker-node list and nvidia.kubernetes-launch-kit.machine selector.
• East-west PF inventory, rail assignments, and capabilities.
• Storage and third-party RDMA kernel modules.

Fresh discovery derives identifier from the machine/GPU identity, lowercases it, and removes complete NVIDIA segments. For example, DGX-B200-NVIDIA-H200 becomes dgx-b200-h200; the Launch Kit machine node label uses the same value. Common machine segments are shortened in the same pass (ThinkSystem → ts, PowerEdge → pe). Identifiers are bounded to 30 bytes. Long values use balanced machine/GPU prefixes plus a 6-character deterministic hash, with unused prefix space reassigned to the longer component. The separate GPU label still retains its discovered value such as NVIDIA-H200. Use the persisted identifier shown in cluster-config.yaml with --groups.

Inspect the available groups before filtering:

yq '.clusterConfig[] | {
  identifier,
  machineType,
  gpuType,
  workerNodes,
  nodeSelector
}' cluster-config.yaml

Automatic Merging

Without a filter, generation buckets source groups by:

1. GPU type.
2. Number of east-west rails.

Different machine types can therefore share one generated resource bucket when their GPU type and rail count match. The merged bucket uses the GPU label:

nodeSelector:
  nvidia.kubernetes-launch-kit.gpu: NVIDIA-H200

Groups with different GPU types or east-west rail counts remain separate. A group with an unresolved GPU type is never automatically merged.

North-south PFs do not contribute to the merge rail count.

Select Groups

Before selecting a subset for an existing deployment, compare all live cohorts and the stray deletion boundary. A render filter limits output, not ownership of shared or cluster-wide resources. Follow the change procedure when applying a subset to a live cluster.

Use --gpu-type to render every source group with a matching GPU type. Matching is case-insensitive:

l8k generate \
  --user-config ./cluster-config.yaml \
  --gpu-type NVIDIA-H200 \
  --save-deployment-files ./deployment

Use --groups for an exact set of source identifiers. Identifier matching is case-sensitive:

l8k generate \
  --user-config ./cluster-config.yaml \
  --groups pe-xe9680-h200,ts-sr680a-v3-h200 \
  --save-deployment-files ./deployment

--groups and --gpu-type are mutually exclusive. Launch Kit reports an error when a requested identifier or GPU type does not match, including the available values.

A group filter controls rendering, not deployment ownership. A subset bundle can make resources from excluded groups appear as strays during deploy. --overwrite-existing deletes those conflicts even when they lack l8k ownership annotations. Review the complete desired inventory and the deletion boundary before using subset output for a staged rollout.

Example Cluster Shapes

Two GPU Types

--gpu-type NVIDIA-H200 selects every H200 source group and excludes the H100 group. The generated deployment still has one cluster-wide NicClusterPolicy; bucket-scoped resources target only the selected H200 cohort.

Filter generation to one GPU type

Discovered source groups

Group A DGX-B200 + H100 Filtered out

Group B PowerEdge-XE9680 + H200 Selected by GPU type

→

Generated scope

NVIDIA-H200 bucket H200 node-selecting policies H200 network and IP pool NicClusterPolicy remains cluster-wide

l8k generate \
  --user-config ./cluster-config.yaml \
  --gpu-type NVIDIA-H200 \
  --fabric ethernet \
  --deployment-type sriov \
  --multirail \
  --save-deployment-files ./deployment-h200

Same GPU, Different Servers

Three machine types with the same GPU type and east-west rail count combine into one render bucket. An unfiltered run uses the shared GPU label. A strict-subset --groups rollout keeps bucket-shared resources together but renders flat-selector policies for only the selected sources.

Combine compatible source groups into one render bucket

H200 source groups

Source A DGX-B200 machine label A

Source B ThinkSystem-SR680a-V3 machine label B

Source C PowerEdge-XE9680 machine label C

→

One compatible bucket

NVIDIA-H200 Same east-west rail count Shared network, resource name, and IP pool GPU selector for the full bucket

# Full H200 bucket
l8k generate \
  --user-config ./cluster-config.yaml \
  --save-deployment-files ./deployment

# Stage two of the three source groups
l8k generate \
  --user-config ./cluster-config.yaml \
  --groups dgx-b200-h200,ts-sr680a-v3-h200 \
  --save-deployment-files ./deployment-stage1

Mixed GPU And Server Matrix

In a four-group cluster, the default run renders an H100 bucket and an H200 bucket. A GPU filter selects a row of the matrix; an explicit group list can select a vendor-specific column across GPU types.

Choose a cohort from a mixed cluster

Four discovered source groups

G1 DGX-B200 + H100

G2 ThinkSystem + H100

G3 DGX-B200 + H200

G4 PowerEdge-XE9680 + H200

--gpu-type NVIDIA-H200: G3 + G4 --groups DGX sources: G1 + G3

Resulting buckets

defaultH100: G1 + G2
H200: G3 + G4

--gpu-typeH200: G3 + G4

--groupsH100: G1
H200: G3

Use --gpu-type when all nodes of one GPU type are the deployment unit. Use --groups for a staged rollout, vendor-specific cohort, or CI matrix with an explicit list of source identifiers.

Resource Scope

Launch Kit preserves Kubernetes resource ownership while rendering merged groups:

Resource behavior	Examples	Render result
Cluster-wide	NicClusterPolicy, Spectrum-X profile ConfigMap	One per deployment.
Shared per bucket	Secondary-network CRs, CIDRPool, IPPool, example workload	One per compatible GPU/rail bucket.
Node-selecting policy	NicNodePolicy, SriovNetworkNodePolicy, SpectrumXRailPoolConfig	One per bucket, or per source group when a strict subset needs flat selectors.
Machine-specific	NicInterfaceNameTemplate	One per source group.

When --groups selects only part of an otherwise mergeable bucket, shared resources keep one stable bucket identity while node-selecting policies are rendered for each selected source group. Aggregate workloads and IP pools use a selector expression containing the selected machine-label values.

PCI Layout Differences

Two source groups can use the same rail number with different PCI addresses. Launch Kit aggregates non-conflicting addresses per rail. When addresses conflict across rails, it can render a separate NicInterfaceNameTemplate for each source group so policies can use stable interface names instead of ambiguous PCI addresses.

Review generated interface templates whenever a merged bucket spans different machine types:

find deployment/network-operator \
  -name '*nicinterfacenametemplate*.yaml' \
  -print

NV-IPAM Allocation

Subnets are allocated across the final render buckets before templates are written. This keeps generated IP pools disjoint when one cluster produces multiple GPU/rail buckets.

For production, inspect every generated IPPool or CIDRPool and confirm that it does not overlap the cluster pod, service, management, or external network ranges.

East-West And North-South NICs

Only east-west PFs produce rails and workload networking resources. North-south PFs remain visible in a mixed source group's inventory but are excluded from generated manifests. A source group containing only north-south PFs is omitted from cluster-config.yaml.

Traffic direction determines whether a PF is rendered

Management / OOB network

↔

BlueField DPU North-south PF Inventory only; not rendered

GPU mesh and cluster nodes

↔

ConnectX NIC or BlueField SuperNIC East-west PF Assigned a rail and rendered

Operational Guidance

• Re-run discovery after adding a new server model or changing NIC mode.
• Keep the Launch Kit machine and GPU labels; generated selectors depend on them.
• Prefer --gpu-type when all matching source groups should share one deployment.
• Prefer --groups for staged rollouts or machine-specific targeting.
• Run l8k deploy --dry-run before applying a newly merged topology.
• Complete the rollout with l8k validate and review the node-groups section of the acceptance report.