device_id
|
Type: Integer. CUDA device index used by the execution provider. Default: 0. A non-negative device index available to the process. |
has_user_compute_stream
|
Type: Boolean (0 or 1). Set to 1 when supplying a user-managed CUDA compute stream via user_compute_stream. Default: 0. 0 — use an internally managed stream. 1 — use the stream supplied via user_compute_stream. |
user_compute_stream
|
Type: String (decimal encoding of a 64-bit pointer; use reinterpret_cast<uintptr_t>(ptr) in C++). Pointer to a caller-owned cudaStream_t, encoded as a decimal string. Requires has_user_compute_stream=1. Default: 0. |
user_aux_stream_array
|
Type: String (decimal encoding of a 64-bit pointer; use reinterpret_cast<uintptr_t>(ptr) in C++). Pointer to a caller-provided array of cudaStream_t used as TensorRT-RTX auxiliary streams, encoded as a decimal string. Keeps auxiliary work inside the caller’s CUDA context, which is required for correct CIG graphics interop. Default: 0. |
nv_max_workspace_size
|
Type: Non-negative 64-bit integer (uint64_t range). Maximum GPU memory in bytes that TensorRT-RTX may use as a workspace during engine build. Default: 0 (TensorRT-RTX default). Non-negative value. 0 lets TensorRT-RTX choose the workspace size automatically. |
nv_max_shared_mem_size
|
Type: Non-negative 64-bit integer (uint64_t range). Maximum shared memory in bytes that TensorRT-RTX kernels are allowed to use. Default: 0 (device default). Non-negative value. 0 uses the device maximum. |
nv_length_aux_stream_array
|
Type: Integer. Number of entries in user_aux_stream_array; also sets the maximum number of TensorRT-RTX auxiliary streams. Default: 0. Non-negative integer. Must match the length of the array passed via user_aux_stream_array. |
nv_dump_subgraphs
|
Type: Boolean (0 or 1). Dump partitioned subgraphs to disk for debugging. Default: 0. 0 — disabled. 1 — dump subgraphs to the current working directory. |
nv_detailed_build_log
|
Type: Boolean (0 or 1). Enable verbose TensorRT-RTX engine build logging. Default: 0. 0 — disabled. 1 — emit detailed TensorRT-RTX build log to the ORT logger. |
nv_profile_min_shapes
|
Type: String. Minimum input shapes for TensorRT-RTX optimization profiles. Specified as input:dim0xdim1,...;input2:dim0xdim1,.... Default: (empty — inferred from model). |
nv_profile_max_shapes
|
Type: String. Maximum input shapes for TensorRT-RTX optimization profiles. Specified as input:dim0xdim1,...;input2:dim0xdim1,.... Default: (empty — inferred from model). |
nv_profile_opt_shapes
|
Type: String. Optimal input shapes for TensorRT-RTX optimization profiles. Specified as input:dim0xdim1,...;input2:dim0xdim1,.... Default: (empty — inferred from model). |
enable_cuda_graph
|
Type: Boolean (0 or 1). Enable CUDA graph capture to reduce kernel-launch overhead on repeated inference runs with fixed input shapes. Default: 0. 0 — disabled. 1 — enable CUDA graph capture. Input shapes must be fixed across runs. |
nv_multi_profile_enable
|
Type: Boolean (0 or 1). Enable multi-profile support to handle multiple TensorRT-RTX optimization profiles within a single session. Default: 0. 0 — single profile. 1 — enable multiple optimization profiles. |
nv_use_external_data_initializer
|
Type: Boolean (0 or 1). Use external data initializers for model weights instead of embedding them in the engine. Default: 0. 0 — disabled. 1 — use externally supplied weight initializers. |
nv_runtime_cache_path
|
Type: String (directory path). Directory path where compiled TensorRT-RTX engines are cached between sessions. An empty string disables caching. Default: (empty — caching disabled). |
VkExternalComputeQueueDataParamsNV_data
|
Type: String (decimal encoding of a 64-bit pointer; use reinterpret_cast<uintptr_t>(ptr) in C++). Pointer to a Vulkan VkExternalComputeQueueDataParamsNV blob used for CIG context creation, encoded as a decimal string. Default: 0. |
nv_weight_streaming_budget
|
Type: String. GPU memory budget for TensorRT RTX weight streaming. Default: 0 (weight streaming disabled). 0 disables streaming. -1 uses TensorRT-RTX’s automatic budget. 1 requests minimum VRAM mode. Bare integers greater than 1 set an explicit byte budget. Suffixed values (B, K, M, G) specify resident bytes (e.g. 512M). Percentage values 0%–100% specify the fraction of streamable weights to keep resident in VRAM. |
nv_op_types_to_exclude
|
Type: String. Comma-separated list of ONNX op types that the EP should leave to other execution providers during graph partitioning. Default: (empty). Comma-separated ONNX op type names, e.g. Conv,Relu. Case-sensitive. |
nv_enable_profiling
|
Type: Boolean (0 or 1). Enable per-layer GPU timing via TensorRT-RTX’s IProfiler. Incompatible with CUDA graph capture; enabling both auto-disables CUDA graphs with a warning. Default: 0. 0 — disabled. 1 — enable per-layer profiling. Output path set via nv_profiling_output_file. |
nv_profiling_output_file
|
Type: String (file path). File path for the Chrome tracing JSON profiling output. A timestamped path is generated automatically when not specified. Default: (auto-generated). |
nv_use_sync_gpu_allocator
|
Type: Boolean (0 or 1). Install a synchronous GPU allocator (cudaMalloc/cudaFree) on the TensorRT RTX runtime, replacing the default cudaMallocAsync path. Enable in environments where asynchronous CUDA allocation is unreliable. Default: 0. 0 — use default async allocator (cudaMallocAsync). 1 — use synchronous allocator (cudaMalloc). |
nv_multi_rotary_cache_concat_offset
|
Type: Integer. Token offset at which the EP switches from the short to the long rotary position embedding (RoPE) cache. Required for LongRoPE models such as Phi-4. Most users can leave this at the default of 0. Default: 0. Non-negative integer. |
nv_weight_stripped_engine_enable_experimental
|
Type: Boolean (0 or 1). Experimental: enable weight-stripped EPContext engine builds. The resulting engine can be refitted at load time from the original ONNX model weights. Requires TensorRT RTX 1.6+. Default: 0. 0 — disabled. 1 — enable weight-stripped engine builds (experimental). |