TabICLv2

class sdm.models.TabICLv2(task: Task | str | Iterable[Task | str] | None = None, pretrained: bool = True, device: device | str | None = None)

Bases: ICLModel

The tabular foundation model from the “TabICLv2: A Better, Faster, Scalable, and Open Tabular Foundation Model” paper.

TabICLv2 treats a table as an in-context learning problem: a set of labeled training rows provides context, and the model predicts targets for held-out test rows from the same table.

Feature columns are encoded via repeated feature grouping, where each feature participates in multiple shifted feature groups. This breaks symmetries between similarly distributed columns while preserving fine-grained feature information. Target-aware embeddings are then added to the training-row feature representations, injecting label information early without giving test rows access to their own targets.

Afterwards, the module applies three attention stages in sequence:

• Column-wise: Each grouped feature is processed as a set of row tokens with induced set attention. The inducing tokens summarize information from the in-context training rows, then pass it back to the row tokens, giving each feature group a target-aware representation of its values across examples. Query-Aware Scalable SoftMax (QASSMax) sharpens attention over long contexts and reduces attention fading as the number of rows grows.

• Row-wise: For each row, the feature-group embeddings are processed together with learnable readout tokens. Attention across the grouped features lets the model combine column evidence and feature interactions within that row. The readout token outputs are concatenated to form a fixed-size row embedding.

• Dataset-wise: The row embeddings are processed across the dataset for in-context prediction. Training-row embeddings are combined with target embeddings, and test rows attend to the labeled training rows. The resulting test-row states are mapped to task outputs, such as class probabilities for classification or quantile predictions for regression.

from sdm import TableTensor
from sdm.models import TabICLv2

table = TableTensor.from_columns(
    {
        "col0": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0],
        "col1": [1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0],
        "target": ["t", "f", "t", "f", "t", None, None, None],
    },
    stypes={
        "col0": "numerical",
        "col1": "numerical",
        "target": "categorical",
    },
    device="cuda",
)
model = TabICLv2(device="cuda")

# Default in-context learning forward pass:
out = model(
    x_context=table[:5].drop_columns("target"),
    y_context=table[:5, "target"],
    x_query=table[5:].drop_columns("target"),
)
assert out.size() == (3, 2)

# Fit+Predict forward pass via key/value caching:
model.fit(
    x=table[:5].drop_columns("target"),
    y=table[:5, "target"],
)
out = model.predict(table[5:].drop_columns("target"))
assert out.size() == (3, 2)

Parameters:

• task (TaskLike | Iterable[TaskLike] | None) – The tasks to initialize. If None, all tasks supported by this model are initialized.

• pretrained (bool) – Whether to load the pretrained checkpoint.

• device (torch.device | str | None) – The device.

Capabilities

Supported Input Semantic Types	numerical
Supported Target Semantic Types	categorical, numerical
Supported Prediction Tasks	classification, regression
Multi-Target Support	❌
Related Table Support	❌

Default Recipe

Recipe(
  features=Sequential(
    StypeDispatch(
      categorical=Sequential(
        AlignCategories(sort_by='value'),
        ToNumerical(),
      ),
    ),
    StypeDispatch(
      numerical=Sequential(
        ImputeMean(),
        DropConstantColumns(),
        Standardize(eps=1e-06),
        Clip(-100.0, 100.0),
        Choice(
          Identity(),
          PowerTransform(),
          method='round_robin',
        ),
        ClipSigma(threshold=4.0),
        ShuffleColumns(method='latin'),
      ),
    ),
  ),
  target=Sequential(
    StypeDispatch(
      numerical=Standardize(eps=0.0),
      categorical=Sequential(
        AlignCategories(),
        ShuffleCategories(method='shift'),
      ),
    ),
  ),
  output=Sequential(
    TaskDispatch(
      regression=SortQuantiles(),
    ),
    AverageEstimators(),
    TaskDispatch(
      classification=Softmax(temperature=0.9),
    ),
  ),
)