physicsnemo_curator.core#
Core pipeline framework for PhysicsNeMo Curator.
Classes#
Abstract filter/transform that processes a stream of T items. |
|
Descriptor for a configurable parameter on a pipeline component. |
|
Lazy pipeline that chains a source through filters into a sink. |
|
Abstract sink that persists items and returns output file paths. |
|
Abstract data source that yields items of type T. |
|
Metadata about a single pipeline database file. |
|
Metrics for one |
|
Aggregated metrics across all processed indices. |
|
SQLite-backed store combining checkpoint tracking, metrics, provenance, and worker progress. |
|
Metrics for a single pipeline stage (source, one filter, or sink). |
|
Global registry of submodules and their pipeline components. |
|
Registry entry for a single submodule (e.g. |