Metrics

class BaseMetrics

Base class for performance metrics.

Provides common interface and total runs tracking.

Subclassed by trt_edgellm::metrics::EagleGenerationMetrics, trt_edgellm::metrics::LLMGenerationMetrics, trt_edgellm::metrics::LLMPrefillMetrics, trt_edgellm::metrics::MultimodalMetrics

Public Functions

virtual ~BaseMetrics() = default

Virtual destructor.

inline int64_t getTotalRuns() const

Get total number of runs.

Returns:

Total runs count

class LLMPrefillMetrics : public trt_edgellm::metrics::BaseMetrics

LLM prefill stage metrics.

Tracks reused and computed tokens during prefill.

Public Functions

inline void recordRun(int64_t reused, int64_t computed)

Record a prefill run.

Parameters:

• reused – Number of reused tokens

• computed – Number of computed tokens

Public Members

int64_t reusedTokens = {0}

Number of reused tokens from cache.

int64_t computedTokens = {0}

Number of newly computed tokens.

class LLMGenerationMetrics : public trt_edgellm::metrics::BaseMetrics

LLM generation stage metrics.

Tracks generated tokens during decoding.

Public Functions

inline void recordRun(int64_t generated)

Record a generation run.

Parameters:

generated – Number of generated tokens

Public Members

int64_t generatedTokens = {0}

Total number of generated tokens.

class MultimodalMetrics : public trt_edgellm::metrics::BaseMetrics

Multimodal processing stage metrics.

Tracks image processing statistics.

Public Functions

inline void recordRun(int64_t imageCount, int64_t imageTokens)

Record a multimodal processing run.

Parameters:

• imageCount – Number of images processed

• imageTokens – Number of image tokens generated

Public Members

int64_t totalImages = {0}

Total number of processed images.

int64_t totalImageTokens = {0}

Total number of image tokens generated.

class EagleGenerationMetrics : public trt_edgellm::metrics::BaseMetrics

Eagle speculative decoding generation metrics.

Tracks iterations and tokens generated during Eagle spec-decode.

Public Functions

inline void recordRun(int64_t iterations, int64_t generatedTokens)

Record an Eagle generation run.

Parameters:

• iterations – Number of iterations

• generatedTokens – Number of generated tokens

Public Members

int64_t totalIterations = {0}

Total number of Eagle iterations.

int64_t totalGeneratedTokens = {0}

Total number of generated tokens.