nvalchemi.training.distributed.all_gather_objects#
- nvalchemi.training.distributed.all_gather_objects(obj, manager=None)[source]#
Return every rank’s obj, in rank order.
A single process gets
[obj]without a collective. Under a process group the objects travel throughtorch.distributed.all_gather_object(), which pickles them, so the call suits small verdicts and summaries;all_gather_rows()carries tensors.- Parameters:
obj (Any) – This rank’s picklable object.
manager (DistributedManager | None, optional) – Manager whose world size is read when given. Default
Nonereadstorch.distributedor the launcher’s environment.
- Returns:
One entry per rank, this rank’s at its own index.
- Return type:
list[Any]
- Raises:
RuntimeError – If more than one rank is reported but no process group is initialized.