Apex (A PyTorch Extension)

This site contains the API documentation for Apex (https://github.com/nvidia/apex), a Pytorch extension with NVIDIA-maintained utilities to streamline mixed precision and distributed training. Some of the code here will be included in upstream Pytorch eventually. The intention of Apex is to make up-to-date utilities available to users as quickly as possible.

Installation instructions can be found here: https://github.com/NVIDIA/apex#quick-start.

AMP: Automatic Mixed Precision

• apex.amp

Distributed Training

• apex.parallel

Fused Optimizers

• apex.optimizers

Fused Layer Norm

• apex.normalization.fused_layer_norm

Indices and tables

• Index
• Module Index