Gemma4 Audio Runner

class Gemma4AudioRunner : public trt_edgellm::rt::MultimodalRunner

Runner for Gemma4 audio encoder.

The encoder is exported at fixed batch=1: the chunked local attention and depthwise convolution are local operators, so cross-clip batching with padding may corrupt outputs. preprocess() walks every audio clip in the request batch and invokes the encoder once per clip, writing the encoded rows directly into mAudioEmbedding at sequential offsets.

Public Functions

Gemma4AudioRunner(std::string const &engineDir, cudaStream_t stream)

Constructor.

Parameters:

• engineDir – [in] Directory containing the audio encoder engine

• stream – [in] CUDA stream for execution

~Gemma4AudioRunner() noexcept = default

virtual bool preprocess(

rt::LLMGenerationRequest const &request,

std::vector<std::vector<int32_t>> &batchedInputIds,

tokenizer::Tokenizer const *tokenizer,

rt::OptionalOutputTensor mropeCosSinOut,

cudaStream_t stream,

bool imageOnly = false,

bool skipEncoderWork = false

) override

Preprocess multimodal input including audio and text.

virtual bool infer(cudaStream_t stream) override

Run inference (no-op, encoding done per-clip in preprocess)

virtual bool validateAndFillConfig(

std::string const &engineDir

) override

Validate and load configuration from JSON file.

virtual bool allocateBuffer(cudaStream_t stream) override

Allocate buffers for inference.

virtual rt::Tensor &getOutputEmbedding() override

Get audio embeddings from encoder output.

struct Gemma4AudioConfig

Configuration for Gemma4 audio encoder.

Public Members

int32_t melBins = {128}

Number of mel-frequency bins.

int32_t audioFeatureDim = {1536}

Audio output dimension (text_hidden_size after embedder projection)

int32_t subsamplingFactor = {4}

Temporal downsampling factor (2x stride-2 convs)

int32_t audioTokenId = {0}

Audio placeholder token ID.

int32_t beginAudioTokenId = {-1}

boa token ID wrapping each audio span (-1 when absent)

int32_t endAudioTokenId = {-1}

eoa token ID wrapping each audio span (-1 when absent)

int32_t maxAudioClipsPerRequest = {8}

Max audio clips in a single request (for buffer pre-allocation)