Builder Utils#

nvinfer1::Dims trt_edgellm::builder::createDims(
std::vector<int64_t> const &shape
)#

Create TensorRT dimensions from a vector of shape values.

Create TensorRT dimensions from a vector of shape values.

Parameters:

  • shape – Vector of dimension sizes

  • shape – Vector of dimension sizes

Throws:

std::invalid_argument – if shape size exceeds MAX_DIMS

Returns:

TensorRT Dims object with the specified dimensions

Returns:

TensorRT Dims object with the specified dimensions

bool trt_edgellm::builder::checkOptimizationProfileDims(
nvinfer1::Dims const &minDims,
nvinfer1::Dims const &optDims,
nvinfer1::Dims const &maxDims
) noexcept#

Validate optimization profile dimensions. Ensures that min <= opt <= max for all dimensions and that all profiles have the same number of dimensions.

Parameters:

  • minDims – Minimum dimensions for the optimization profile

  • optDims – Optimal dimensions for the optimization profile

  • maxDims – Maximum dimensions for the optimization profile

Returns:

true if dimensions are valid, false otherwise

bool trt_edgellm::builder::setOptimizationProfile(
nvinfer1::IOptimizationProfile *profile,
char const *inputName,
nvinfer1::Dims const &minDims,
nvinfer1::Dims const &optDims,
nvinfer1::Dims const &maxDims
) noexcept#

Set optimization profile dimensions for a specific input. Validates the dimensions and sets them on the optimization profile.

Parameters:

  • profile – Optimization profile to configure

  • inputName – Name of the input tensor

  • minDims – Minimum dimensions for the input

  • optDims – Optimal dimensions for the input

  • maxDims – Maximum dimensions for the input

Returns:

true if setting was successful, false otherwise

std::string trt_edgellm::builder::printNetworkInfo(
nvinfer1::INetworkDefinition const *network,
std::string const &prefix
)#

Print detailed information about the TensorRT network. Shows input and output tensor names and shapes for debugging purposes.

Parameters:

  • network – TensorRT network definition to analyze

  • prefix – Optional prefix for the output string

Returns:

Formatted string containing network information

std::string trt_edgellm::builder::printOptimizationProfile(
nvinfer1::IOptimizationProfile const *profile,
std::string const &profileName,
nvinfer1::INetworkDefinition const *network
)#

Print detailed information about an optimization profile. Shows the min, optimal, and max dimensions for each input in the profile.

Parameters:

  • profile – Optimization profile to analyze

  • profileName – Name of the profile for display purposes

  • network – TensorRT network definition for input analysis

Returns:

Formatted string containing optimization profile information

std::pair<std::unique_ptr<nvinfer1::IBuilder>, std::unique_ptr<nvinfer1::INetworkDefinition>> trt_edgellm::builder::createBuilderAndNetwork(
)#

Create TensorRT builder and network definition with strongly typed flag.

Returns:

Pair of builder and network, or {nullptr, nullptr} on failure

std::unique_ptr<nvinfer1::IBuilderConfig> trt_edgellm::builder::createBuilderConfig(
nvinfer1::IBuilder *builder
)#

Create TensorRT builder config with optimized settings.

Parameters:

builder – TensorRT builder object

Returns:

Builder config with monitor memory flag enabled (TRT >= 10.6)

std::unique_ptr<nvonnxparser::IParser> trt_edgellm::builder::parseOnnxModel(
nvinfer1::INetworkDefinition *network,
std::string const &onnxFilePath
)#

Parse ONNX model and create parser.

Parameters:

  • network – TensorRT network definition to populate

  • onnxFilePath – Path to ONNX model file

Returns:

ONNX parser object, or nullptr on failure

bool trt_edgellm::builder::buildAndSerializeEngine(
nvinfer1::IBuilder *builder,
nvinfer1::INetworkDefinition *network,
nvinfer1::IBuilderConfig *config,
std::string const &engineFilePath
)#

Build and serialize TensorRT engine to file.

Parameters:

  • builder – TensorRT builder object

  • network – TensorRT network definition

  • config – TensorRT builder config

  • engineFilePath – Output path for serialized engine

Returns:

true if successful, false otherwise

bool trt_edgellm::builder::loadJsonConfig(
std::string const &configPath,
nlohmann::json &outConfig
)#

Load and parse JSON config file.

Parameters:

  • configPath – Path to config.json file

  • outConfig – Output JSON object

Returns:

true if successful, false otherwise

bool trt_edgellm::builder::saveConfigWithBuilderInfo(
std::filesystem::path const &engineDir,
nlohmann::json const &modelConfig,
nlohmann::json const &builderConfig
)#

Save model config with builder config to engine directory.

Parameters:

  • engineDir – Engine directory path

  • modelConfig – Original model configuration

  • builderConfig – Builder configuration as JSON

Returns:

true if successful, false otherwise