megatron-energon
Introduction
General
Quickstart
Installation
Usage
Simple Library Usage
FAQ
Command-Line Interface
Data Preparation
Remote Dataset
Basic Concepts
Dataset Format on Disk
Data Flow
Dataset Concepts
Task Encoders
Metadataset
Blending datasets
Advanced Usage
Custom Sample Loader
Crude Data and How to Cook It 👨🍳
Reproducible Scaling
Packing
Joining Datasets / Adding Columns to Metadataset
Developer's Manual
Contribution Guidelines
Packages and Modules
megatron.energon
megatron-energon
Packages and Modules
megatron.energon
View page source
Previous
megatron.energon