Registry Builder

TensorRegistry trt_edgellm::rt::buildRegistryForLLM(

LLMEngineConfig const &cfg,

std::optional<int32_t> specDecodeBaseOutputHiddenDim = std::nullopt,

)

Build a TensorRegistry for the base LLM engine.

Produces a registry with all tensor specs matching the engine’s I/O contract for llmEngineRunner. Config flags control which optional tensor groups (deepstack, Mamba/recurrent state, EAGLE, LoRA) are included.

Parameters:

• cfg – The engine configuration.

• specDecodeBaseOutputHiddenDim – Optional hidden-state output dim for a SpecDecode base engine. When absent, the legacy EAGLE-3 convention is used.

Returns:

A populated TensorRegistry.

TensorRegistry trt_edgellm::rt::buildRegistryForSpecDecodeDraft(

DeploymentConfig const &bundle,

)

Build a TensorRegistry for a SpecDecode draft engine.

Produces a registry with all tensor specs matching the engine’s I/O contract for the draft runner. The draft engine always uses plugin-based KV cache and proposal-attention tensors.

Parameters:

bundle – The deployment configuration. bundle.draft and bundle.specConfig must both be set; the draft registry needs the consolidated SpecDecode settings to size cross-engine bindings (e.g. base hidden states fed into the draft).

Returns:

A populated TensorRegistry.

TensorRegistry trt_edgellm::rt::buildRegistryForDFlashDraft(

DeploymentConfig const &bundle,

)

Build a TensorRegistry for a DFlash draft engine.

DFlash draft engines are non-autoregressive block drafters: they consume proposal embeddings, accumulated target hidden states, and per-batch delta lengths, and they expose per-layer KV-cache bindings for proposal attention.

TensorRegistry trt_edgellm::rt::buildRegistryForGemma4MTPDraft(

DeploymentConfig const &bundle,

)

Build a TensorRegistry for a Gemma4 MTP assistant draft engine.

Gemma4 assistant engines read the target/base KV cache through past_key_values_* inputs and must not expose present_key_values_*, hidden_states_from_draft, or draft-owned kvcache_start_index.