Compact Capability

struct CompactLayerGap

A layer kind of an engine that cannot run phase 8, and why.

Public Members

CompactLayerKind kind

std::string reason

std::vector<CompactLayerGap> trt_edgellm::rt::compactLayerGaps(

LLMEngineConfig const &config,

int32_t sm

)

The layer kinds of config that cannot execute compact (phase-8) steps on SM sm, in kind order.

Each kind present in the engine is checked with the predicate its plugin applies when it reserves the phase-8 workspace and accepts a phase-8 enqueue (kernels/compactPhase/compactPhaseSupport.h), fed with the layer shapes the engine config records. The reason names the shape, why it has no phase-8 path, and the SMs that have one.

Inputs the config does not record are taken from what the export paths produce: no attention layer is a tree, vision-block or attention-sink layer (speculative and vision-block deployments are refused before this check; sinks are only exported for DSpark drafts), and a KV-shared bounded-SWA-capable layer receives the donor’s current K/V. A sliding-window layer that is not bounded-SWA-capable is checked as a causal one; the kernel predicates accept the two masks alike.

std::string trt_edgellm::rt::compactExecutionGap(

LLMEngineConfig const &config,

int32_t sm

)

Why the layers of config cannot all execute compact (phase-8) steps on SM sm, or empty when they can: every compactLayerGaps() entry, as one sentence naming the SM.