PrometheusMetricsConfig#

class tensorrt_llm.llmapi.PrometheusMetricsConfig( *, e2e_request_latency_buckets: List[float] | None = None, time_to_first_token_buckets: List[float] | None = None, time_per_output_token_buckets: List[float] | None = None, request_queue_time_buckets: List[float] | None = None, request_prefill_time_buckets: List[float] | None = None, request_decode_time_buckets: List[float] | None = None, request_inference_time_buckets: List[float] | None = None, )[source]#

Bases: StrictBaseModel

Configuration for Prometheus metrics collection.

Groups all Prometheus-related parameters including custom histogram bucket boundaries for latency metrics.

field e2e_request_latency_buckets: List[float] | None = None#: Custom histogram bucket boundaries (in seconds) for trtllm_e2e_request_latency_seconds. Defaults to built-in values when unset.

field request_decode_time_buckets: List[float] | None = None#: Custom histogram bucket boundaries (in seconds) for trtllm_request_decode_time_seconds. Defaults to built-in values when unset.

field request_inference_time_buckets: List[float] | None = None#: Custom histogram bucket boundaries (in seconds) for trtllm_request_inference_time_seconds. Defaults to built-in values when unset.

field request_prefill_time_buckets: List[float] | None = None#: Custom histogram bucket boundaries (in seconds) for trtllm_request_prefill_time_seconds. Defaults to built-in values when unset.

field request_queue_time_buckets: List[float] | None = None#: Custom histogram bucket boundaries (in seconds) for trtllm_request_queue_time_seconds. Defaults to built-in values when unset.

field time_per_output_token_buckets: List[float] | None = None#: Custom histogram bucket boundaries (in seconds) for trtllm_time_per_output_token_seconds. Defaults to built-in values when unset.

field time_to_first_token_buckets: List[float] | None = None#: Custom histogram bucket boundaries (in seconds) for trtllm_time_to_first_token_seconds. Defaults to built-in values when unset.

__init__(**data: Any) → None#

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

validator validate_histogram_buckets » e2e_request_latency_buckets, request_queue_time_buckets, time_per_output_token_buckets, time_to_first_token_buckets, request_decode_time_buckets, request_inference_time_buckets, request_prefill_time_buckets[source]#: Validate that histogram bucket lists are non-empty and strictly increasing.