Vanilla Autoregressive Strategy#
-
template<typename DecodeState>
class VanillaAutoregressiveStrategy# Request-scoped autoregressive scheduling over borrowed, Initialization-prepared storage. The request context and borrowed storage outlive this object. DecodeState prepares a PendingStep without committing request progress, then commitLMStageOutputs publishes that validated step through a non-failing operation.
Public Types
-
using LMStageOutputs = typename DecodeState::LMStageOutputs#
-
using PendingStep = typename DecodeState::PendingStep#
Public Functions
- inline VanillaAutoregressiveStrategy(
- DecodeState &state,
- LMStageId prefillStage,
- LMStageId decodeStage,
- TokenIdBuffer generatedTokens
- VanillaAutoregressiveStrategy( ) = delete#
- VanillaAutoregressiveStrategy &operator=( ) = delete#
- inline SafetyStatus beginRequest(
- int64_t generatedTokenBudget,
- int32_t const *stopTokenIds,
- std::size_t stopTokenCount
Begins this object’s request after the caller has begun its request context. Stop-token storage stays valid for the whole request.
-
inline bool hasNextLMStage() const noexcept#
-
inline LMStageId nextLMStage() const noexcept#
- inline SafetyStatus acceptLMStageOutputs(
- LMStageOutputs const &outputs
-
inline SafetyStatus abortRequest(SafetyStatus status) noexcept#
Ends this request; the runtime Error Handler owns fatal rejection across requests.
-
inline TokenIdView generatedTokenIds() const noexcept#
-
inline FinishReason finishReason() const noexcept#
-
using LMStageOutputs = typename DecodeState::LMStageOutputs#