nvalchemi.training.distributed.all_gather_objects#

nvalchemi.training.distributed.all_gather_objects(obj, manager=None)[source]#

Return every rank’s obj, in rank order.

A single process gets [obj] without a collective. Under a process group the objects travel through torch.distributed.all_gather_object(), which pickles them, so the call suits small verdicts and summaries; all_gather_rows() carries tensors.

Parameters:
  • obj (Any) – This rank’s picklable object.

  • manager (DistributedManager | None, optional) – Manager whose world size is read when given. Default None reads torch.distributed or the launcher’s environment.

Returns:

One entry per rank, this rank’s at its own index.

Return type:

list[Any]

Raises:

RuntimeError – If more than one rank is reported but no process group is initialized.