Metrics

class BaseMetrics

Base class for performance metrics.

Provides common interface and total runs tracking.

Subclassed by trt_edgellm::metrics::EagleGenerationMetrics, trt_edgellm::metrics::LLMGenerationMetrics, trt_edgellm::metrics::LLMPrefillMetrics, trt_edgellm::metrics::MultimodalMetrics

Public Functions

virtual ~BaseMetrics() noexcept = default

Virtual destructor.

inline int64_t getTotalRuns() const noexcept

Get total number of runs.

Returns:

Total runs count

class LLMPrefillMetrics : public trt_edgellm::metrics::BaseMetrics

LLM prefill stage metrics.

Tracks reused and computed tokens during prefill.

Public Functions

inline void recordRun(int64_t reused, int64_t computed) noexcept

Record a prefill run.

Parameters:

• reused – Number of reused tokens

• computed – Number of computed tokens

Public Members

int64_t reusedTokens = {0}

Number of reused tokens from cache.

int64_t computedTokens = {0}

Number of newly computed tokens.

class LLMGenerationMetrics : public trt_edgellm::metrics::BaseMetrics

LLM generation stage metrics.

Tracks generated tokens during decoding.

Public Functions

inline void recordRun(int64_t generated) noexcept

Record a generation run.

Parameters:

generated – Number of generated tokens

Public Members

int64_t generatedTokens = {0}

Total number of generated tokens.

class MultimodalMetrics : public trt_edgellm::metrics::BaseMetrics

Multimodal processing stage metrics.

Tracks image and audio processing statistics.

Public Functions

inline void recordRun(

int64_t imageCount,

int64_t imageTokens,

int64_t audioCount = 0,

int64_t audioTokens = 0

) noexcept

Record a multimodal processing run.

Parameters:

• imageCount – Number of images processed

• imageTokens – Number of image tokens generated

• audioCount – Number of audio clips processed (optional, for Qwen3-Omni)

• audioTokens – Number of audio tokens generated (optional, for Qwen3-Omni)

Public Members

int64_t totalImages = {0}

Total number of processed images.

int64_t totalImageTokens = {0}

Total number of image tokens generated.

int64_t totalAudios = {0}

Total number of processed audio clips (Qwen3-Omni)

int64_t totalAudioTokens = {0}

Total number of audio tokens generated (Qwen3-Omni)

class EagleGenerationMetrics : public trt_edgellm::metrics::BaseMetrics

Eagle speculative decoding generation metrics.

Tracks iterations and tokens generated during Eagle spec-decode.

Public Functions

inline void recordRun(

int64_t iterations,

int64_t generatedTokens

) noexcept

Record an Eagle generation run.

Parameters:

• iterations – Number of iterations

• generatedTokens – Number of generated tokens

Public Members

int64_t totalIterations = {0}

Total number of Eagle iterations.

int64_t totalGeneratedTokens = {0}

Total number of generated tokens.