SentenceTransformer#
- class sdm.processing.text.SentenceTransformer(model_name: str, *, batch_size: int = 32, max_seq_length: int | None = None)#
Bases:
ProcessorTransform text columns with a
sentence_transformers.SentenceTransformermodel.- Parameters:
model_name (str) – Model name or local path passed to
sentence_transformers.SentenceTransformerbatch_size (int) – Batch size passed to
sentence_transformers.SentenceTransformer.encode(). Adjusting the batch size can significantly improve processing speed. The optimal value depends on your hardware, model size, precision, and input length.max_seq_length (int | None) – Maximum number of tokens per text, including special tokens. Longer inputs are truncated. When omitted, the model’s own limit is used, and a larger value is not raised above that limit.
Capabilities#
Handled Semantic Types |
|
Requires Fitting |
❌ |