ncore.data_converter Package#

Package containing common and abstract functionality to implement NCore data-converters

class ncore.data_converter.BaseDataConverter(
config: BaseDataConverterConfig,
)#

Bases: ABC

Base preprocessing class used to preprocess AV datasets in a canonical representation as used in the Nvidia NCore project.

For adding a new dataset, please inherit this class, implement the required functions, and register a new CLI command.

The output data should follow the conventions defined in the conventions documentation.

Please use the facilities of the data writer modules, which simplifies adding new datasets.

classmethod convert(
config: BaseDataConverterConfig,
) → None#

Main entry-point to perform conversion of all sequences

abstractmethod convert_sequence(sequence_id: str) → None#

Runs dataset-specific conversion for a sequence referenced by a sequence ID

abstractmethod static from_config(
config,
) → BaseDataConverter#

Return an instance of the data converter

get_active_camera_ids(
all_camera_ids: list[str],
) → list[str]#

Returns config-specified subselection of active camera ids or all camera ids if no subselection was performed

get_active_lidar_ids(
all_lidar_ids: list[str],
) → list[str]#

Returns config-specified subselection of active lidar ids or all lidar ids if no subselection was performed

get_active_radar_ids(
all_radar_ids: list[str],
) → list[str]#

Returns config-specified subselection of active radar ids or all radar ids if no subselection was performed

abstractmethod static get_sequence_ids(config) → list[str]#

Return dataset-specific sequence IDs (e.g., path names) to process

class ncore.data_converter.BaseDataConverterConfig(
*,
output_dir: str,
no_cameras: bool,
camera_ids: Tuple[str, ...] | None,
no_lidars: bool,
lidar_ids: Tuple[str, ...] | None,
no_radars: bool,
radar_ids: Tuple[str, ...] | None,
verbose: bool,
debug: bool,
debug_port: int,
)#

Bases: object

Generic data converter parameters

camera_ids: Tuple[str, ...] | None#
debug: bool#
debug_port: int#
lidar_ids: Tuple[str, ...] | None#
no_cameras: bool#
no_lidars: bool#
no_radars: bool#
output_dir: str#
radar_ids: Tuple[str, ...] | None#
verbose: bool#
class ncore.data_converter.FileBasedDataConverter(
config: FileBasedDataConverterConfig,
)#

Bases: BaseDataConverter

Base class for converters that read source data from a local root directory.

Subclass this instead of BaseDataConverter when the converter needs self.root_dir. Pair with FileBasedDataConverterConfig (or a subclass of it) to get early validation of root_dir.

class ncore.data_converter.FileBasedDataConverterConfig(
*,
output_dir: str,
no_cameras: bool,
camera_ids: Tuple[str, ...] | None,
no_lidars: bool,
lidar_ids: Tuple[str, ...] | None,
no_radars: bool,
radar_ids: Tuple[str, ...] | None,
verbose: bool,
debug: bool,
debug_port: int,
root_dir: str,
)#

Bases: BaseDataConverterConfig

Config for converters that read from a local root directory.

Subclass this instead of BaseDataConverterConfig when the converter discovers its input sequences from --root-dir.

root_dir: str#