Refit Weights#

struct RefitWeightSet#

Temporary device storage and named views consumed together during engine initialization. Views are destroyed before their backing storage. Copying is prohibited by Tensor ownership.

Public Members

Tensor storage#: Owns the temporary refit arena.

std::vector<Tensor> weights#: Non-owning named views into storage.

struct EngineRefitOptions#

Optional pre-context refit; stream must be explicit when weights is non-null.

Public Members

std::unique_ptr<RefitWeightSet> weights#: Ownership transferred from the checkpoint loader.

cudaStream_t stream = {nullptr}#: Initialization stream, not the default stream.

bool trt_edgellm::rt::validateWeightlessConfig( nlohmann::json const &config )#: Validate mode metadata and binding destinations before checkpoint allocation. Returns whether refit is required; throws for unsupported builds or malformed contracts.

void trt_edgellm::rt::refitEngineIfNeeded( nvinfer1::ICudaEngine &engine, EngineRefitOptions refit = {} )#: Consume optional refit storage before context creation; reject stripped engines without it.

void trt_edgellm::rt::refitEngine( nvinfer1::ICudaEngine &engine, RefitWeightSet const &weights, cudaStream_t stream )#: Apply all supplied weights and wait for completion before returning. The caller must keep weights alive until this returns, including on error. Throws on missing names, incompatible prototypes, or failed refit; no context may exist yet.