Stage Registry

class StageRegistry

Runner name to declaration and factory.

builtinRunners() is the table of the runners this library ships. A caller that adds its own runner copies it, add()s a row and passes the copy to MultiStageRuntime::load().

Public Functions

StageRegistry() = default

StageRegistry(std::initializer_list<RunnerEntry> entries)

Throws:

std::runtime_error – as add() does for any entry.

void add(RunnerEntry entry)

Throws:

std::runtime_error – when the name is empty or already taken, or the entry has no factory.

RunnerEntry const *find(std::string const &name) const

nullptr when unknown. The pointee lives as long as the table.

std::vector<std::string> names() const

Sorted, so an “unknown runner” message can list every legal value.

struct RunnerDecl

What a runner promises, independent of any one deployment.

A RunnerDecl is the implementation’s half of the contract; a StageConfig is the deployment’s half. validateRunners() checks one against the other, so a typo in multistage_config.json fails at load naming the rule it broke instead of surfacing as a missing binding half way through a request.

Public Members

std::string name

Table key, e.g. “<family>.talker”. Matches StageConfig::runner.

StageKind kind = {StageKind::kHost}

std::vector<std::pair<std::string, PortKind>> ports

Ports this runner can produce. A deployment may emit a subset of them.

std::vector<std::string> inputs

Input names this runner understands. A deployment may bind a subset of them.

std::vector<std::string> requiredConfigKeys

Keys that must appear in StageConfig::config. Presence only – type and range stay with the runner, the only thing that knows what they mean.

bool batches = {false}

Whether this runner can hold more than one request at a time. A deployment may declare a wider max_batch than the implementation has; the executor caps the cohort at what the runner promises rather than handing it members it would step on top of each other.

std::vector<std::string> samplingKeys

Keys this runner reads from a JSON object bound to its sampling input.

std::vector<std::string> styleFields

Fields of its style input that a request file may also set at its top level or per request.

std::vector<std::pair<std::string, std::string>> requestAliases

Other front ends’ request-file names for a sampling key or an input name, as {spelling, name}.

struct RunnerEntry

One runner: its declaration and the factory that builds its stage.

Public Members

RunnerDecl decl

StageFactory create = {nullptr}

StageRegistry const &trt_edgellm::rt::multiStage::builtinRunners()

Every runner this library ships, its declaration next to its factory.

The runners this library ships, one row each; defined in stageFactories.cpp.

The only translation unit that names a concrete runner. Its references are also what keep each runner’s object file in a static link.

void trt_edgellm::rt::multiStage::validateRunners(

MultiStageConfig const &config,

StageRegistry const &runners

)

The half of load-time validation that needs the runners.

Split from multiStage::validate() on purpose: validate() is pure structure with no dependency on any implementation. This one answers a different question – does this config match what the runners in runners actually offer.

Checks, per stage: the runner is in the table; StageKind agrees with the declaration; every emitted port is declared with the same PortKind; every bound input name is accepted; every required config key is present. A port the runner declares but the config omits is fine – not every deployment wants every output.

Throws:

std::runtime_error – naming the stage id and the rule.