Executable¶
- class nvtripy.Executable[source]¶
Bases:
object
Represents a compiled executable generated by the compiler.
See also
- input_infos: Dict[str, InputInfo]¶
Stores metadata, like shapes and data types, for each input to the executable.
- load() Executable [source]¶
Loads a executable from the provided path.
- Parameters:
path (str) – The path from which to load the exectuable.
- Returns:
The executable object loaded from the file.
- Return type:
Example: Save and load executable
1import os 2 3 4def add(a, b): 5 return a + b 6 7 8compiled_add = tp.compile( 9 add, 10 args=[ 11 tp.InputInfo(shape=((1, 2, 3),), dtype=tp.float32), 12 tp.InputInfo(shape=((1, 2, 3),), dtype=tp.float32), 13 ], 14) 15 16 17# Assuming `out_dir` is the directory containing the executable: 18executable_file = os.path.join(out_dir, "executable.json") 19loaded_executable = tp.Executable.load(executable_file)
Local Variables¶>>> loaded_executable Executable(a: nvtripy.Tensor, b: nvtripy.Tensor) -> nvtripy.Tensor
Output¶[I] /tmp/tmpj1_xhnt4 does not exist, creating now.
- __call__(*args: Tensor, **kwargs: Tensor) Tensor | Sequence[Tensor] [source]¶
Invokes the executable with the specified tensor arguments.
Note
Inputs must be evaluated tensors in GPU memory.
You can use
nvtripy.copy()
ornvtripy.Tensor.eval()
to ensure this.- Parameters:
- Returns:
The output
Tensor
s of the compiled function.- Return type:
Example
1def add(a, b): 2 return a + b 3 4 5compiled_add = tp.compile( 6 add, 7 args=[ 8 tp.InputInfo((1,), dtype=tp.float32), 9 tp.InputInfo((1,), dtype=tp.float32), 10 ], 11) 12 13a = tp.ones((1,), dtype=tp.float32).eval() 14b = tp.ones((1,), dtype=tp.float32).eval() 15 16out = compiled_add(a, b)
Local Variables¶>>> a tensor([1], dtype=float32, loc=gpu:0, shape=(1,)) >>> b tensor([1], dtype=float32, loc=gpu:0, shape=(1,)) >>> out tensor([2], dtype=float32, loc=gpu:0, shape=(1,))
- save(path: str) None [source]¶
Saves this executable to the provided path.
- Parameters:
path (str) – The path at which to save the executable.
Example: Save executable
1import os 2 3 4def add(a, b): 5 return a + b 6 7 8compiled_add = tp.compile( 9 add, 10 args=[ 11 tp.InputInfo(shape=((1, 2, 3),), dtype=tp.float32), 12 tp.InputInfo(shape=((1, 2, 3),), dtype=tp.float32), 13 ], 14) 15 16# Assuming `out_dir` is the desired output directory: 17executable_file = os.path.join(out_dir, "executable.json") 18compiled_add.save(executable_file)
Output¶[I] /tmp/tmps2n56d3p does not exist, creating now.
- property serialized_tensorrt_engine: bytes¶
The serialized TensorRT engine, as
bytes
, from the executable.See also
Refer to the TensorRT developer guide for details on how to work with serialized TensorRT engines.
Example: TensorRT engine
1def add(a, b): 2 return a + b 3 4 5compiled_add = tp.compile( 6 add, 7 args=[ 8 tp.InputInfo(shape=((1, 2, 3),), dtype=tp.float32), 9 tp.InputInfo(shape=((1, 2, 3),), dtype=tp.float32), 10 ], 11) 12 13trt_engine = compiled_add.serialized_tensorrt_engine