nvalchemi.distributed.collective_error_sync#

nvalchemi.distributed.collective_error_sync(context, *, phase, error_type=RuntimeError)[source]#

Agree ordinary local failures and exchange successful control metadata.

Every process-group member must enter matching calls in the same order. Exactly one object all-gather carries either local failure details or the pickled metadata bytes. The lowest failing group-local rank supplies the shared diagnostic.

Parameters:
  • context (ProcessGroupContext) – Initialized group context shared by this rank.

  • phase (str) – Initial non-empty phase label; the yielded object’s label can change.

  • error_type (type[Exception], default RuntimeError) – Exception class used for the same distributed diagnostic on every rank.

Yields:

CollectivePhase – Mutable phase and metadata state; records is populated only after a successful group exchange.

Raises:
  • TypeError, ValueError – If the context, phase, or configured exception type is invalid.

  • Exception – The configured class if a rank’s body or metadata serialization fails.

Return type:

Iterator[CollectivePhase[Any]]