Stage Config#

struct PortSpec#

One output of a stage.

Public Members

std::string id#

PortKind kind = {PortKind::kText}#

Transport transport = {Transport::kDeviceTensor}#

std::string binding#: Engine tensor name.

bool terminal = {false}#: A caller may attach a sink here.

struct InputSpec#

One input of a stage.

Public Members

std::string name#

std::string from#: “<stageId>:<portId>” or “request:<field>”.

OnMissing onMissing = {OnMissing::kFail}#

struct SourceRef#

A parsed from. Exactly one of the two forms is valid.

Public Functions

std::string str() const#: Round-trips back to the original string. Used in diagnostics.

Public Members

bool fromRequest = {false}#: true -> requestField is set; false -> stageId/portId are set.

std::string stageId#

std::string portId#

std::string requestField#

struct StageConfig#

Public Functions

PortSpec const *findPort(std::string const &portId) const#

InputSpec const *findInput(std::string const &name) const#

Public Members

std::string id#: Unique.

std::string runner#: Registry key. Decides which IStage implementation is built.

std::string artifact#: Engine directory, relative to the config root. May be empty.

StageKind kind = {StageKind::kHost}#

std::vector<PortSpec> emits#

std::vector<InputSpec> inputs#

int32_t maxBatch = {1}#

int32_t maxSeq = {0}#

int32_t streamLane = {0}#

nlohmann::json config = nlohmann::json::object()#: Validated by the runner’s own schema, not here. An empty object rather than a null: a stage that names no options has no options, and every runner reads this with value(), which throws on a null.

struct MultiStageConfig#

The topology, as data.

A MultiStageConfig is the whole topology: which stages exist, what each one emits, and which port feeds which input. It is parsed from multistage_config.json and is pure data – no callables, and no path that escapes the config root.

Nothing here knows about a model family. A family is a set of values in this structure, not a branch in the code.

Public Functions

StageConfig const *findStage(std::string const &id) const#

int32_t stageIndex(std::string const &id) const#: -1 when absent.

Public Members

int32_t schemaVersion = {kSchemaVersion}#

std::string label#: Diagnostics only. Never dispatched on.

std::vector<StageConfig> stages#

std::vector<std::string> terminals#: Port ids (“<stageId>:<portId>”), NOT stage ids. Same key space as Sinks and wanted. Non-empty, and exactly the ports flagged terminal; each is of kind text, audio, or image.

Public Static Functions

static MultiStageConfig parse(nlohmann::json const &root)#: Parses and validates. Throws std::runtime_error naming the offending id on any failure.

static MultiStageConfig load(std::string const &configRoot)#: Reads <configRoot>/multistage_config.json.

Public Static Attributes

static constexpr int32_t kSchemaVersion = 1#

Warning

doxygenfunction: Unable to resolve function “trt_edgellm::rt::multiStage::toString” with arguments None in doxygen xml output for project “TensorRT Edge-LLM” from directory: ../cpp_docs/xml. Potential matches:

- char const *toString(FinishReason v)
- char const *toString(OnMissing v)
- char const *toString(PortKind v)
- char const *toString(StageKind v)
- char const *toString(StageStatus v)
- char const *toString(Transport v)

StageKind trt_edgellm::rt::multiStage::parseStageKind( std::string const &v )#

PortKind trt_edgellm::rt::multiStage::parsePortKind( std::string const &v )#

Transport trt_edgellm::rt::multiStage::parseTransport( std::string const &v )#

OnMissing trt_edgellm::rt::multiStage::parseOnMissing( std::string const &v )#

SourceRef trt_edgellm::rt::multiStage::parseSourceRef( std::string const &from )#: Parses “<stageId>:<portId>” or “request:<field>”. Throws on a malformed string.

void trt_edgellm::rt::multiStage::validate( MultiStageConfig const &config, std::string const &configRoot = {} )#

Structural checks that must hold before anything is constructed.

Every rule here is one a layering bug could otherwise turn into a runtime surprise. This is the authority at load; the Python builder runs its own, separate preflight before it writes a config.

Parameters:

config – Parsed topology.
configRoot – Used only to reject an artifact that escapes the root. Pass an empty string to skip the filesystem check.

Throws:

std::runtime_error – naming the stage, the input and the rule.