Gemma4 Audio Runner

class Gemma4AudioRunner : public trt_edgellm::rt::MultimodalRunner

Runner for Gemma4 audio encoder.

The encoder is exported at fixed batch=1: the chunked local attention and depthwise convolution are local operators, so cross-clip batching with padding may corrupt outputs. preprocess() walks every audio clip in the ! request batch and invokes the encoder once per clip, writing the encoded ! rows directly into mAudioEmbedding at sequential offsets.

Public Functions

Gemma4AudioRunner(std::string const &engineDir, cudaStream_t stream)

Constructor.

Parameters:

• engineDir – [in] Directory containing the audio encoder engine

• stream – [in] CUDA stream for execution

~Gemma4AudioRunner() noexcept = default

virtual bool preprocess(

rt::LLMGenerationRequest const &request,

std::vector<std::vector<int32_t>> &batchedInputIds,

tokenizer::Tokenizer const *tokenizer,

rt::OptionalOutputTensor mropeCosSinOut,

cudaStream_t stream,

bool imageOnly = false,

) override

Preprocess multimodal input including audio and text.

virtual bool infer(cudaStream_t stream) override

Run inference (no-op, encoding done per-clip in preprocess)

virtual bool validateAndFillConfig(

std::string const &engineDir,

) override

Validate and load configuration from JSON file.

virtual bool allocateBuffer(cudaStream_t stream) override

Allocate buffers for inference.

virtual rt::Tensor &getOutputEmbedding() override

Get audio embeddings from encoder output.

struct Gemma4AudioConfig

Configuration for Gemma4 audio encoder.

Public Members

int32_t melBins = {128}

Number of mel-frequency bins.

int32_t audioFeatureDim = {1536}

Audio output dimension (text_hidden_size after embedder projection)

int32_t subsamplingFactor = {4}

Temporal downsampling factor (2x stride-2 convs)

int32_t audioTokenId = {0}

Audio placeholder token ID.

int32_t beginAudioTokenId = {-1}

boa token ID wrapping each audio span (-1 when absent)

int32_t endAudioTokenId = {-1}

eoa token ID wrapping each audio span (-1 when absent)

int32_t maxAudioClipsPerRequest = {8}

Max audio clips in a single request (for buffer pre-allocation)