Command-Line Interface

Once the package is installed, a script called energon will be added to your PATH. Currently, it provides mainly commands to prepare, convert, or lint datasets on disk.

Later, it may be extended for model or checkpoint management for example.

Here’s a simple example:

energon prepare /mnt/data/my_captioning_webdataset

The above command will scan your existing off-the-shelf web dataset and add the needed metadata to make it compatible with Energon.

Below, you can see the available sub-commands under energon.

energon prepare

An interactive tool to generate metadata for your existing webdataset. This will help make the dataset compliant with our format.

The tool will ask you for a train/val/test split and how to assign the webdataset fields to the fields of the corresponding sample type in Energon.

See Data Preparation for more details on how to use this command.

energon lint

You can execute this tool on the prepared dataset to check if the data is valid and loadable. It will report any problems such as non-readable images.