Model Optimizer
/

Announcements

  • Announcements

Getting Started

  • Overview
  • Installation
    • Installation for Linux
    • Installation for Windows
      • Install ModelOpt-Windows as a Standalone Toolkit
      • Install ModelOpt-Windows with Olive
  • Quick Start: PTQ - PyTorch
  • Quick Start: PTQ - ONNX
  • Quick Start: PTQ - PyTorch to ONNX
  • Quick Start: PTQ - Windows
  • Quick Start: QAT
  • Quick Start: Pruning
  • Quick Start: Distillation
  • Quick Start: Speculative Decoding
  • Quick Start: Sparsity

Guides

  • Support Matrix
  • Recipes
  • ModelOpt Config System
  • Quantization
    • Basic Concepts
    • Best practices to choose the right quantization methods
    • PyTorch Quantization
    • Quantization Configuration (quant_cfg)
    • Guides to quantize a customized model from Hugging Face for TensorRT-LLM deployment
    • Compress quantized models
    • ONNX Quantization - Linux (Beta)
    • ONNX Quantization - Windows
  • Saving & Restoring
  • Pruning
  • Distillation
  • Speculative Decoding
  • Sparsity
  • NAS
  • AutoCast (ONNX)
  • Autotune (ONNX)

Deployment

  • TensorRT-LLM
  • Onnxruntime
  • Unified HuggingFace Checkpoint

Examples

  • All GitHub Examples

Reference

  • Changelog
  • modelopt API
    • deploy
      • llm
        • generate
    • onnx
      • autocast
        • convert
        • graphsanitizer
        • logging_config
        • nodeclassifier
        • precisionconverter
        • referencerunner
        • utils
      • export
      • graph_surgery
      • llm_export_utils
        • export_utils
        • quantization_utils
        • surgeon_utils
      • logging_config
      • op_types
      • quantization
        • autotune
        • calib_utils
        • extensions
        • fp8
        • graph_utils
        • gs_patching
        • int4
        • int8
        • operators
        • ort_patching
        • ort_utils
        • partitioning
        • qdq_utils
        • quant_utils
        • modelopt.onnx.quantization.quantize
      • trt_utils
      • utils
    • torch
      • distill
        • config
        • distillation
        • distillation_model
        • layerwise_distillation_model
        • loss_balancers
        • losses
        • mode
        • plugins
        • registry
      • export
        • convert_hf_config
        • diffusers_utils
        • distribute
        • hf_config_map
        • hf_export_handlers
        • layer_utils
        • mcore_config_map
        • model_config
        • model_config_export
        • model_config_utils
        • model_utils
        • moe_utils
        • plugins
        • postprocess
        • quant_aware_conversion
        • quant_utils
        • registry
        • tensorrt_llm_type
        • tensorrt_llm_utils
        • transformer_engine
        • unified_export_hf
        • unified_export_hf_streaming
        • unified_export_megatron
      • fastgen
        • config
        • discriminators
        • ema
        • factory
        • flow_matching
        • loader
        • losses
        • methods
        • pipeline
        • plugins
        • utils
      • kernels
        • common
        • quantization
        • sparsity
      • nas
        • algorithms
        • autonas
        • conversion
        • hparams
        • modules
        • patch
        • plugins
        • registry
        • search_space
        • traced_hp
        • utils
      • opt
        • config
        • config_loader
        • conversion
        • dynamic
        • hparam
        • mode
        • plugins
        • searcher
        • utils
      • peft
        • config
        • conversion
        • convert
        • custom
        • lora
        • mode
      • prune
        • fastnas
        • importance_hooks
        • plugins
        • pruning
      • puzzletron
        • activation_scoring
        • anymodel
        • block_config
        • build_library_and_stats
        • dataset
        • entrypoint
        • mip
        • plugins
        • pruning
        • puzzletron_nas_plugin
        • replacement_library
        • scoring
        • sewing_kit
        • subblock_stats
        • tools
        • utils
      • quantization
        • algorithms
        • backends
        • calib
        • modelopt.torch.quantization.compress
        • config
        • conversion
        • export_onnx
        • extensions
        • mode
        • model_calib
        • model_quant
        • nn
        • plugins
        • qtensor
        • tensor_quant
        • utils
      • sparsity
        • attention_sparsity
        • weight_sparsity
      • speculative
        • config
        • dflash
        • eagle
        • medusa
        • mode
        • plugins
        • speculative_decoding
        • utils
      • trace
        • analyzer
        • modules
        • symbols
        • tracer
      • utils
        • cpp_extension
        • dataset_utils
        • distributed
        • graph
        • import_utils
        • list
        • logging
        • loss_mask
        • memory_monitor
        • mlflow
        • nemotron_vlm_dataset_utils
        • network
        • perf
        • plugins
        • random
        • regex
        • robust_json
        • serialization
        • speech_dataset_utils
        • tensor
        • vlm_dataset_utils

Support

  • Contact us
  • FAQs
  1. Model Optimizer /
  2. Installation

Installation#

  • Installation for Linux
  • Installation for Windows
Previous
Overview
Next
Installation for Linux

2023-2025, NVIDIA Corporation

Made with Sphinx and Shibuya theme.