TRT Safety Graph Executor

class TRTSafetyGraphExecutor

Single-context graph owner; callers serialize every operation and keep its CUDA context current. The shared error state, borrowed bindings, and streams outlive healthy deinitialize(). Destruction performs no CUDA or DLI calls. A fatal failure requires application containment, without cleanup retry.

Public Functions

explicit TRTSafetyGraphExecutor(

SafetyErrorState &errorState

) noexcept

~TRTSafetyGraphExecutor() noexcept = default

TRTSafetyGraphExecutor(TRTSafetyGraphExecutor const&) = delete

TRTSafetyGraphExecutor &operator=(

TRTSafetyGraphExecutor const&

) = delete

TRTSafetyGraphExecutor(TRTSafetyGraphExecutor&&) = delete

TRTSafetyGraphExecutor &operator=(TRTSafetyGraphExecutor&&) = delete

SafetyStatus loadModules() noexcept

Load modules for every application executor before initialize() or other CUDA runtime APIs.

SafetyStatus initialize(SafetyGraphConfig const &config) noexcept

Initialization only: create the graph and query metadata for admitted profiles.

SafetyStatus bindProfile(

int64_t profile,

std::vector<nvinfer2::safe::TypedArray> const &bindings,

cudaStream_t main,

std::vector<cudaStream_t> const &aux

) noexcept

Initialization only: bind each profile once, using one fixed stream configuration for the executor. Tensor buffers and streams are borrowed, and callers supply valid warmup input contents.

SafetyStatus warmup(int64_t profile) noexcept

Execute and synchronize one bound profile during initialization to prepare fixed plugin bindings. Capture also performs one validation replay; caller-supplied inputs must remain valid for that replay.

SafetyStatus seal() noexcept

Enter Operational after every admitted profile is bound and warmed.

SafetyStatus execute(int64_t profile) noexcept

Operational only: execute the selected fixed profile and check host and device errors after completion.

SafetyStatus deinitialize() noexcept

Healthy Deinit Preparation only: destroy the graph before releasing its modules. Immutable profile metadata remains valid until executor destruction.

bool fatal() const noexcept

Includes asynchronously reported recorder failures, even before the next operation observes them.

bool hasGraph() const noexcept

std::vector<SafetyProfileMetadata> const &profiles(

) const noexcept

int32_t requiredAuxStreams() const noexcept

struct SafetyGraphConfig

Borrowed safe engine and matched companion library, with the admitted fixed IO profiles.

Public Members

void const *engine = {nullptr}

int64_t engineBytes = {0}

char const *companionSoPath = {nullptr}

std::vector<int64_t> declaredProfiles

bool enableCudaGraphCapture = {false}

Capture admits GPU IO payloads only; zero-byte metadata does not require storage.

struct SafetyTensorMetadata

Descriptor names and physical layouts remain valid until executor destruction.

Public Members

std::string name

nvinfer2::safe::TensorDescriptor descriptor

struct SafetyProfileMetadata

Tensor order is the graph’s IO index order for this admitted profile.

Public Members

int64_t profileIndex = {0}

std::vector<SafetyTensorMetadata> tensors