Data Flow

The steps of how the data flows through those task encoder methods is explained in Task Encoder.

Dataset Flavors

The datasets are organized in “flavors”, i.e. each modality returned by the dataset is a “flavor”. A modality can for example be a CaptioningSample or an VQASample. The dataset class combines the source data format and the iterated sample format. For example, the CaptioningWebdataset combines the webdataset loader with the CaptioningSample.

For all types, see Available Sample Types