TRT Resource Manager

class TRTResourceManager

Owns IO allocations and streams for serialized execution in one caller-owned CUDA context. SafetyRuntime keeps this owner alive until every borrowing graph is destroyed. All operations are externally serialized. Destruction performs no CUDA/DLI calls.

Public Functions

TRTResourceManager(

LifecycleState &lifecycleState,

SafetyErrorState &errorState

) noexcept

~TRTResourceManager() noexcept = default

TRTResourceManager(TRTResourceManager const&) = delete

TRTResourceManager &operator=(TRTResourceManager const&) = delete

SafetyStatus prepareResources(

std::vector<TRTSafetyGraphExecutor*> const &executors,

std::vector<size_t> const &mainStreamIds = {},

std::vector<SafetyIOAliasGroup> const &aliases = {}

) noexcept

Initialization only: allocate and bind every admitted profile once. A CUDA/DLI failure retains partial resources for application containment without rollback.

SafetyStatus deinitialize(

std::vector<TRTSafetyGraphExecutor*> const &executors

) noexcept

Healthy Deinit Preparation only, after every borrowing graph has been destroyed.

std::vector<SafetyProfileResources> const &profiles(

size_t executorIndex

) const noexcept

cudaStream_t mainStream(size_t executorIndex) const noexcept

bool fatal() const noexcept

struct SafetyProfileResources

Stable IO storage for one fixed profile; tensor order matches SafetyProfileMetadata.

Public Members

int64_t profileIndex = {0}

std::vector<nvinfer2::safe::TypedArray> bindings

struct SafetyIOAliasGroup

Explicit exact in-place IO storage for one alias-safe executor; partial overlap is not admitted.

Public Members

size_t executorIndex = {0}

std::vector<std::string> tensors