Registry Builder

TensorRegistry trt_edgellm::rt::buildRegistryForLLM(

LLMEngineConfig const &cfg,

std::optional<int32_t> specDecodeBaseOutputHiddenDim = std::nullopt

)

Build a TensorRegistry for the base LLM engine.

Produces a registry with all tensor specs matching the engine’s I/O contract for llmEngineRunner. Config flags control which optional tensor groups (deepstack, Mamba/recurrent state, EAGLE, LoRA) are included.

Parameters:

• cfg – The engine configuration.

• specDecodeBaseOutputHiddenDim – Optional hidden-state output dim for a SpecDecode base engine. When absent, the legacy EAGLE-3 convention is used.

Returns:

A populated TensorRegistry.

TensorRegistry trt_edgellm::rt::buildRegistryForSpecDecodeDraft(

DeploymentConfig const &bundle

)

Build a TensorRegistry for a SpecDecode draft engine.

Produces a registry with all tensor specs matching the engine’s I/O contract for the draft runner. The draft engine always uses plugin-based KV cache and proposal-attention tensors.

Parameters:

bundle – The deployment configuration. bundle.draft and bundle.specConfig must both be set; the draft registry needs the consolidated SpecDecode settings to size cross-engine bindings (e.g. base hidden states fed into the draft).

Returns:

A populated TensorRegistry.