Request Profiler#
-
template<typename Clock = std::chrono::steady_clock>
class LLMRequestProfiler# Per-sequence wall timing, independent of aggregate GPU event profiling. The runtime supplies cumulative counts at host-visible phase boundaries.
Public Functions
-
inline void beginPrefill(bool enabled)#
Opt in before the first prefill; repeated calls preserve the start across prompt chunks.
-
inline void finishPrefill(int32_t generatedTokens)#
Finish after the last prefill chunk and its first-token sampling.
-
inline void recordGeneration(int32_t generatedTokens)#
Record accepted output, including multiple tokens from a speculative step.
- inline LLMRequestMetrics snapshot(
- int32_t processedPromptTokens
Use the same effective prompt length as the aggregate prefill metrics.
-
inline void beginPrefill(bool enabled)#