TRT Safety Runner

class TRTSafetyRunner

Owns graphs and maps stage/slot pairs to fixed resources in one current CUDA context. SafetyRuntime owns the resource manager and governs lifecycle transitions. The shared runtime error state and recorder outlive its graphs and terminal containment. One active request, externally serialized; destruction performs no CUDA/DLI calls.

Public Functions

TRTSafetyRunner(

LifecycleState &lifecycleState,

SafetyErrorState &errorState,

TRTResourceManager &resources

) noexcept

~TRTSafetyRunner() noexcept = default

TRTSafetyRunner(TRTSafetyRunner const&) = delete

TRTSafetyRunner &operator=(TRTSafetyRunner const&) = delete

SafetyStatus initialize(SafetyRuntimeInitSpec const &spec) noexcept

Initialization only. Validate assignments, query graphs, allocate/bind IO, and publish context metadata. The caller creates the context before this call; modules load before any CUDA runtime API.

SafetyStatus completeContext(

char const *stage,

ResourceSlotId slot,

std::vector<SafetyModelLocalResourceView> const &views

) noexcept

Complete a context once from caller-owned model-local resources, all stable through Deinitialization.

SafetyStatus prepare(

LMStageWarmupCallback warmup,

void *userData = nullptr

) noexcept

Initialization only. Warm every declared stage/profile, then seal all executors for Operational use.

SafetyStatus runLMStage(

LMStageId stage,

ResourceSlotId slot,

LMStageOutputs &outputs,

LMStagePrepareCallback prepare = nullptr,

void *userData = nullptr

) noexcept

Operational catalog dispatch requires one immutable profile for the stage/slot. Preparation updates preallocated payloads on the selected context stream before execution.

SafetyStatus deinitializeGraphs() noexcept

Healthy Deinit Preparation: destroy graphs/modules and stop dispatch, retaining IO/streams.

SafetyStatus deinitializeResources() noexcept

Release IO/streams after graphs and caller-owned model-local/declared resources are destroyed.

std::vector<LMStageContext> const &contexts() const noexcept

bool fatal() const noexcept

struct SafetyRuntimeInitSpec

Public Members

std::vector<SafetyGraphConfig> executors

std::vector<LMStagePlan> stages

std::vector<size_t> executorMainStreamIds = {}

std::vector<SafetyIOAliasGroup> ioAliases = {}