Sampling

struct SamplingParams

Structure to hold sampling parameters.

Public Functions

inline SamplingParams(

int32_t batchSize_,

int32_t vocabSize_,

float temperature_ = 1.0f,

int32_t topK_ = 0,

float topP_ = 1.0f

)

Constructor with default values.

Parameters:

• batchSize_ – Number of samples in the batch

• vocabSize_ – Size of the vocabulary

• temperature_ – Temperature parameter (default: 1.0f)

• topK_ – Top-K parameter (default: 0, disabled)

• topP_ – Top-P parameter (default: 1.0f, disabled)

Throws:

std::invalid_argument – if neither topK nor topP is set, or if temperature is invalid

Public Members

int32_t batchSize

Number of samples in the batch.

int32_t vocabSize

Size of the vocabulary.

float temperature

Temperature parameter for sampling (higher = more random)

int32_t topK

Top-K sampling parameter (0 = disabled)

float topP

Top-P (nucleus) sampling parameter (1.0 = disabled)

bool useTopK

Flag indicating if top-K sampling is enabled.

bool useTopP

Flag indicating if top-P sampling is enabled.