Gemma4 Unified Audio Runner

class Gemma4UnifiedAudioRunner : public trt_edgellm::rt::MultimodalRunner

Frame raw mono PCM into 640-sample tokens and run the lightweight Unified audio projection.

Public Functions

Gemma4UnifiedAudioRunner(

std::string const &engineDir,

cudaStream_t stream,

)

~Gemma4UnifiedAudioRunner() noexcept = default

virtual bool preprocess(

rt::LLMGenerationRequest const &request,

std::vector<std::vector<int32_t>> &batchedInputIds,

tokenizer::Tokenizer const *tokenizer,

rt::OptionalOutputTensor mropeCosSinOut,

cudaStream_t stream,

bool imageOnly = false,

) noexcept override

Preprocess request with images and text.

Parameters:

• request – Generation request with prompts and images

• batchedInputIds – Output batched input token IDs

• tokenizer – Tokenizer instance

• mropeCosSinOut – Output MRope cos/sin cache. Required (has_value() == true) only for MRope-based multimodal runners (QwenViT, Qwen3OmniAudio), which write per-batch 3D position encodings into it. Pass std::nullopt when the base engine uses standard RoPE — runners with standard RoPE (InternViT, Phi4MMViT) do not read this parameter.

• stream – CUDA stream

• imageOnly – When true, only run image preprocessing (skip text tokenization and RoPE generation). Used for benchmarking where only the visual engine inputs need to be set up.

Returns:

True on success, false on failure

virtual bool infer(cudaStream_t stream) noexcept override

Run multimodal inference.

Parameters:

stream – CUDA stream

Returns:

True on success, false on failure

virtual bool validateAndFillConfig(

std::string const &engineDir,

) override

Validate and fill configuration from file.

Parameters:

engineDir – Path to engine directory

Returns:

True on success, false on failure

virtual bool allocateBuffer(cudaStream_t stream) override

Allocate device buffers.

Returns:

True on success, false on failure

struct Gemma4UnifiedAudioConfig

Runtime configuration for the encoder-free Gemma4 Unified audio embedder.

Public Members

int64_t minFrames = {0}

int64_t maxFrames = {0}

int64_t samplesPerFrame = {640}

int64_t outputHiddenSize = {0}

int32_t sampleRate = {16000}

int32_t audioTokenId = {0}

int32_t beginAudioTokenId = {0}

int32_t endAudioTokenId = {0}