Ecosystem and historical integrations#

Transformer Engine has also been adopted by community projects and integrated through past collaborations. The projects below are included as ecosystem and historical references; their presence here does not imply that the Transformer Engine project actively maintains or validates each integration.

  • DeepSpeed — NVIDIA and the DeepSpeed project have collaborated on Transformer Engine support in the past. This integration is not actively maintained by the Transformer Engine project.

  • MosaicML Composer — includes community support for training with Transformer Engine precision.

  • GPT-NeoX — includes community integration code for Transformer Engine layers.

  • Hugging Face Nanotron — includes community FP8 integration code using Transformer Engine.

  • Levanter — includes community integration with the Transformer Engine JAX API.

For currently highlighted integrations, see the Integrations section in the project README.