Model Inference Types

struct LMStageOutputs

Borrowed Runner output; the prepared context outlives request processing.

Public Members

LMStageContext const *context = {nullptr}

struct TokenIdView

Public Members

int32_t const *data = {nullptr}

std::size_t size = {0}

struct TokenIdBuffer

Storage is prepared during Initialization and outlives the request strategy.

Public Members

int32_t *data = {nullptr}

std::size_t capacity = {0}