TensorRT Model Optimizer

Getting Started

Overview
Installation
Quick Start: Quantization
Quick Start: Pruning
Quick Start: Distillation
Quick Start: Sparsity

Guides

Quantization
Pruning
NAS
Distillation
Sparsity
Saving & Restoring
Speculative Decoding

Deployment

TensorRT-LLM Deployment

Examples

All GitHub Examples
ResNet20 on CIFAR-10: Pruning
HF BERT: Prune, Distill & Quantize

Reference

Model Optimizer Changelog
modelopt API
- deploy
- onnx
- torch
  - distill
  - export
  - nas
    - algorithms
    - autonas
    - config
    - conversion
    - hparams
    - mode
    - modules
    - plugins
    - registry
    - search_space
    - traced_hp
    - utils
  - opt
  - prune
  - quantization
  - sparsity
  - speculative
  - trace
  - utils

Support

Contact us
FAQs

TensorRT Model Optimizer

modelopt API
torch
nas
modules
container
View page source

container

Dynamic sequential implementation (variable depth based on torch.nn.modules.container).

Previous Next

© Copyright 2023-2024, NVIDIA Corporation.

Built with Sphinx using a theme provided by Read the Docs.