sdm.processing#

Processor API#

Processor

Base processor for tensor-aware table transformations.

InvertibleMixin

Extend a Processor by an inverse transformation.

EnsembleProcessor

Base processor for ensemble-aware table transformations.

EnsembleInvertibleMixin

Extend a EnsembleProcessor by an inverse transformation.

TargetAwareEnsembleProcessor

Base processor with access to unprocessed targets.

SeriesEnsembleProcessor

Base processor for consecutive, equally spaced time-series data.

Recipe

Processing contract around an external model boundary.

Common Processors#

Identity

Return inputs unchanged.

Callable

Adapt a stateless callable to the Processor interface.

Sequential

Apply processors and callables to a table in sequence.

DropStypes

Remove all columns for specific semantic types.

StypeDispatch

Apply separate processor pipelines to columns grouped by semantic type.

TaskDispatch

Apply separate processors based on the semantic type of the target.

TableDispatch

Apply separate feature processors to task and related tables.

EnsembleProcessorAdapter

Adapt an ordinary processor to ensemble-aware processing.

Choice

Route each table or ensemble member through one selected option.

ToNumerical

Move categorical columns into the numerical block.

ShuffleColumns

Permute numerical columns and their names.

SelectColumns

Select a subset of columns for each semantic type.

Numerical Processors#

Cast

Cast numerical columns to a floating-point dtype.

Clip

Clip numerical values to a fixed interval.

ClipQuantiles

Clamp each numerical column to its fitted quantile bounds.

ClipSigma

Two-stage z-score outlier clipping.

ImputeMean

Replace missing numerical values with fitted per-column means.

PowerTransform

Apply a Yeo-Johnson power transform to each numerical column.

QuantileTransform

Map numerical columns through their empirical quantiles.

RankGaussian

Map interpolated empirical mid-ranks to standard normal quantiles.

Standardize

Center and scale each numerical column.

RobustScale

Center and scale each numerical column with median and quantile range.

MinMaxScale

Scale each numerical column by its fitted min-max range.

FlipSign

Randomly negate numerical columns.

Asinh

Apply the inverse hyperbolic sine to numerical values.

DropConstantColumns

Remove non-informative numerical columns learned during fit.

PCA

Project numerical columns onto their principal components.

RandomProjection

Randomly project numerical columns.

Detrend

Remove a per-column linear trend over step coordinates.

Interpolate

Interpolate missing numerical values along the row dimension.

Categorical Processors#

AlignCategories

Align categorical columns to categories observed during fitting.

ShuffleCategories

Permute the integer codes of categorical columns.

ImputeMode

Replace missing categorical values with fitted per-column modes.

AddCategoryCounts

Add the log row count of each categorical value.

OneHotEncode

One-hot encode categorical columns.

Text Processors#

TFIDF

Encode text columns as character n-gram TF-IDF vectors.

SentenceTransformer

Transform text columns with a sentence_transformers.SentenceTransformer model.

Datetime Processors#

AddCalendarFields

Add numerical calendar fields derived from datetime columns.

Post-Processors#

AverageEstimators

Reduce the leading ensemble dimension of model outputs.

Softmax

Apply softmax to logits after temperature scaling.

SortQuantiles

Sort quantile predictions in ascending order.

ClipNonnegative

Clip outputs at zero for targets with nonnegative observations.