nvalchemi.distributed.ProcessGroupContext#
- class nvalchemi.distributed.ProcessGroupContext(process_group, *, execution_device=None)[source]#
Read-only topology and communication-device details for a supplied group.
The context never creates, destroys, or otherwise owns the process group. Construct it after the caller has initialized the group and confirmed that this process is a member.
- Parameters:
process_group (torch.distributed.ProcessGroup) – An initialized process group that includes this process.
execution_device (torch.device or str, optional) – Communication-device hint. Gloo always uses CPU. NCCL resolves a CUDA hint first, then an initialized
DistributedManagerCUDA device, then the caller’s current CUDA device.
- Raises:
TypeError – If process_group is not a concrete PyTorch process group.
RuntimeError – If distributed is uninitialized, the group is unusable, or NCCL has no usable CUDA device.
ValueError – If the process is not a member or an explicit CUDA index is invalid.
NotImplementedError – If the group’s backend is neither Gloo nor NCCL.
- communication_scope()[source]#
Select NCCL’s resolved device for scoped communication, then restore it.
- Return type:
Iterator[None]
- global_rank(group_rank)[source]#
Translate a group-local rank to its global rank.
- Parameters:
group_rank (int) – Rank within this context’s process group.
- Returns:
The corresponding global rank.
- Return type:
int
- Raises:
TypeError – If group_rank is not a non-boolean integral value.
ValueError – If group_rank is outside this group’s rank range.