Multi Device Plugin Resources

class PluginAllReducePathResources

Resource capability owned by one TensorRT plugin AllReduce execution path.

Public Functions

virtual ~PluginAllReducePathResources() noexcept = default

virtual AllReducePathType type() const noexcept = 0

virtual bool registered() const noexcept = 0

inline virtual RuntimeCollectiveResources *runtimeCollectives(

) noexcept

Optional runtime-collective view supplied by communication paths such as NCCL.

class RuntimeCollectiveResources

Non-owning runtime collective capability backed by communicator handles.

Public Functions

virtual ~RuntimeCollectiveResources() noexcept = default

virtual void *communicatorForRank(int32_t rank) const noexcept = 0

virtual bool abortOwnedCommunicators() noexcept = 0

Abort communicators owned by this resource. Externally owned communicators are never aborted implicitly. Returns true when this resource owns the communicators and initiated (or already completed) their abort.

class MultiDevicePluginResources

Non-hot-path owner for plugin communication resources associated with one parallel group.

Subclassed by trt_edgellm::rt::TensorParallelPluginResources

Public Functions

virtual ~MultiDevicePluginResources() noexcept = default

virtual ParallelType type() const noexcept = 0

virtual int32_t size() const noexcept = 0

virtual RuntimeCollectiveResources const *runtimeCollectives(

) const noexcept = 0

virtual bool hasAllReducePath(

AllReducePathType type

) const noexcept = 0

virtual bool abortOwnedRuntimeCollectives() noexcept = 0