NeMo Retriever API Reference

Error and failure contract

The Python API does not define a separate set of numeric NeMo Retriever extraction error codes. Depending on the run mode and failing stage, callers observe one or more of the following:

• Python configuration or dependency exceptions, such as ValueError, ImportError, or RuntimeError.
• GraphIngestionError for row-level failures from explicitly configured remote NIM stages in run_mode="inprocess" or "batch" when error_policy="raise" (the default).
• HTTP status codes or gRPC errors returned by a remote NIM or by the Retriever service. These are transport or upstream-service statuses, not NeMo Retriever-specific error codes.
• Per-document failures in ServiceIngestResult.failures when run_mode="service".

The generated API signatures and parameter models below are the API contract. Exception text and upstream response bodies can change between releases; do not parse them as stable codes. The stable text-generation codes documented in One-shot text generation apply to generation operator output columns, not to document extraction.

Choose raise or collect behavior

For graph run modes, error_policy="raise" raises GraphIngestionError when an explicitly configured remote NIM stage reports a row-level error. The exception retains the underlying records in exc.records. When available, its message identifies the stage, invoke URL, and HTTP status in a form similar to [stage=OCR NIM url=https://... http=503], followed by a troubleshooting hint.

Use error_policy="collect" when partial results are useful and your application inspects the error fields in every returned row. Alternatively, pass return_failures=True to .ingest() to receive a (result, failures) tuple. When no remote invoke URL is configured, return_failures=True scans all output columns for row-level error fields so local failures are still visible. In service mode, failures are also available from ServiceIngestResult.failures.

What the raise error policy covers

The strict policy applies only to stages where you explicitly configure a remote NIM invoke URL. It does not raise for local-only PDFium parsing, caption, audio or video, or ASR failures, even when those stages populate row-level error fields.

Configured invoke URL	DataFrame column scanned	Stage label in messages
page_elements_invoke_url	output_column (default page_elements_v3)	Page Elements NIM
ocr_invoke_url	ocr	OCR NIM
table_structure_invoke_url	table_structure_ocr_v1	Table Structure NIM
nemotron_parse_invoke_url or invoke_url	nemotron_parse_v1_2	Nemotron Parse NIM
embed_invoke_url or embedding_endpoint	output_column (default text_embeddings_1b_v2)	Embedding NIM

Caption and ASR use remote endpoints but are outside this raise path today. Remote caption failures can abort the whole ingest instead of returning a partial DataFrame. ASR failures can omit affected rows while logging a warning, which can look like an empty transcript unless you inspect logs.

Row-level error payloads

Most extraction stages write errors into the result row instead of raising immediately. The common nested shape is:

{
  "error": {
    "stage": "ocr_page_elements",
    "type": "HTTPError",
    "message": "HTTP 503 from https://example/v1/infer: ...",
    "traceback": "..."
  }
}

The stage string is a semi-stable operator identifier (for example remote_inference, nemotron_parse_pages, or split_pdf). It is not a product-wide error-code enum. HTTP status codes usually appear inside message text rather than as a separate status_code field; when a structured status is present, GraphIngestionError can include it in the rendered exception.

import os

from nemo_retriever import GraphIngestionError, create_ingestor
from nemo_retriever.common.params import ExtractParams

pipeline = (
    create_ingestor(run_mode="inprocess", error_policy="raise")
    .files(["document.pdf"])
    .extract(
        ExtractParams(
            method="ocr",
            ocr_invoke_url=os.environ["OCR_INVOKE_URL"],
        )
    )
)

try:
    result = pipeline.ingest()
except GraphIngestionError as exc:
    # Records can contain source paths, endpoint details, and upstream
    # response text. Extract only known-safe diagnostic fields before
    # logging or sending them to your support workflow.
    for record in exc.records:
        payload = record.get("error") if isinstance(record, dict) else record
        if isinstance(payload, dict):
            print(
                {
                    "column": record.get("column"),
                    "stage": payload.get("stage"),
                    "type": payload.get("type"),
                    "message": payload.get("message"),
                }
            )
        else:
            print(
                {
                    "column": record.get("column") if isinstance(record, dict) else None,
                    "message": str(payload),
                }
            )

For a support-oriented mapping of extraction paths, error signals, corrective actions, and escalation criteria, refer to Python API error triage.

Version-specific behavior

This reference describes the current NeMo Retriever Library. Older NV-Ingest releases, including 25.4.2, can use different exception text and result shapes and might not include enriched GraphIngestionError diagnostics. When troubleshooting an older deployment, use the package and container versions from that deployment and include them in the support case.

PDF pre-splitting for parallel ingest

Large PDFs are split into page batches before Ray processing so extraction can run in parallel. This happens on the default ingest path; you do not need extra configuration for typical workloads.

To tune splitter throughput from the CLI, use --pdf-split-batch-size (Ray actor batch size for the splitter stage). Refer to Text chunking and PDF page batches in the CLI reference.

Python client (pdf_split_config): Only create_ingestor(run_mode="service") implements .pdf_split_config(pages_per_chunk=...), which records page-chunking settings in the request pipeline spec for the remote gateway. Local graph ingest (run_mode="inprocess" or "batch") raises NotImplementedError if you call this method; PDFs are split automatically on the default ingest path without client-side configuration.

One-shot text generation

TextGenerationOperator is the reusable base for synchronous, one-request-per-row text generation. It is a provisional text-only API: it does not support tool calls, agent loops, streaming, multiple choices, or structured domain results.

Concrete operators construct an immutable TextGenerationTask and provide reconstructible constructor state. Runtime task and client objects must not be included in graph constructor arguments. A custom completion client must be safe for concurrent calls or report that it does not support concurrent calls so the operator serializes access.

Embedding and captioning remain separate operator families because they use modality grouping, native batching, and specialized CPU/GPU lifecycles.

Generic generation and summarization

Both operators consume a pandas DataFrame and add text, latency, model, and error columns without changing the input rows:

import pandas as pd

from nemo_retriever.common.params import TextGenerationParams
from nemo_retriever.operators.generation import GenericGenerationOperator, SummarizationOperator

summary_params = TextGenerationParams.from_kwargs(
    model="openai/gpt-4o-mini",
    api_key="os.environ/OPENAI_API_KEY",
    temperature=0.0,
    max_tokens=512,
)
summaries = SummarizationOperator(summary_params).run(
    pd.DataFrame({"text": ["A long document to summarize."]})
)

prompt_params = TextGenerationParams.from_kwargs(
    model="openai/gpt-4o-mini",
    api_key="os.environ/OPENAI_API_KEY",
    prompt="Write a {tone} title for: {text}",
)
titles = GenericGenerationOperator(
    prompt_params,
    input_columns={"tone": "style", "text": "document"},
    output_column="title",
).run(pd.DataFrame({"style": ["concise"], "document": ["Quarterly results"]}))

SummarizationOperator defaults to text, summary, summary_latency_s, summary_model, and summary_error. GenericGenerationOperator maps each named prompt placeholder to a physical DataFrame column and derives the metadata column names from output_column. Prompt contracts are validated when the operator is constructed, before any provider request runs.

To define another one-request/one-text-result task, subclass TextGenerationTask, declare required_inputs, and implement build_request(). Then construct it from a TextGenerationOperator subclass with explicit logical-input-to-DataFrame-column mappings. This abstraction is intentionally text-only; use a separate operator family for embeddings, captioning, tools, streaming, or structured domain results.

Generation failures are collected per row using stable error codes: empty_input, request_error, transport_error, unsupported_response, parse_error, empty_output, and the RAG-specific thinking_truncated. Raw provider exceptions and credentials are not written to DataFrame outputs.

Persisted graphs are trusted configuration

Graph loading imports operator classes and invokes their constructors. Load graph JSON only from trusted sources; do not expose graph payloads, callable references, or class names as model- or user-controlled agent tools.

Version 2 graph files preserve shared-node DAG identity and reject cycles. Constructor state must consist of supported JSON-native values, typed Pydantic models, paths, sets and tuples, or importable type/callable references. Runtime data such as DataFrames and opaque client objects is not persistable.

API keys are never written into graph JSON. Use an explicit environment reference in persisted configuration:

QAGenerationOperator(
    model="openai/gpt-4o-mini",
    api_key="os.environ/OPENAI_API_KEY",
)

Serializing a graph containing a literal API key fails with a contextual error instead of guessing which provider credential should be used on a worker.

Ingestor bucket: ingestion orchestration, planning, manifests and results.

The public ingestor API lives in :mod:nemo_retriever.ingestor.core and is re-exported here so that nemo_retriever.ingestor keeps the exact module-level surface it had before the reorganization (create_ingestor, ingestor / Ingestor, _merge_params and the re-exported param models such as IngestorCreateParams).

Ingestor = ingestor module-attribute

IngestorRunMode = Literal['inprocess', 'batch', 'service'] module-attribute

__all__ = ['create_ingestor', 'ingestor', 'Ingestor'] module-attribute

CaptionParams

Bases: LLMInferenceParams

Source code in nemo_retriever/common/params/models.py

class CaptionParams(LLMInferenceParams):
    endpoint_url: Optional[str] = None
    model_name: str = Field(
        default=DEFAULT_LOCAL_CAPTION_MODEL_ID,
        description=(
            "Caption model identifier. The default local BF16 checkpoint has approximately 62 GiB of weights; "
            "set this explicitly to select a smaller local model or an API model for a remote endpoint."
        ),
    )
    api_key: Optional[str] = None
    prompt: str = "Caption the content of this image:"
    system_prompt: Optional[str] = "/no_think"
    batch_size: int = 8
    device: Optional[str] = None
    hf_cache_dir: Optional[str] = None
    context_text_max_chars: int = 0
    tensor_parallel_size: int = 1
    gpu_memory_utilization: float = 0.5
    caption_infographics: bool = False
    extra_body: dict[str, Any] = Field(default_factory=dict)

    @field_validator("temperature")
    @classmethod
    def _require_temperature(cls, value: Optional[float]) -> float:
        if value is None:
            raise ValueError("temperature cannot be None for captioning")
        return value

api_key = None class-attribute instance-attribute

batch_size = 8 class-attribute instance-attribute

caption_infographics = False class-attribute instance-attribute

context_text_max_chars = 0 class-attribute instance-attribute

device = None class-attribute instance-attribute

endpoint_url = None class-attribute instance-attribute

extra_body = Field(default_factory=dict) class-attribute instance-attribute

gpu_memory_utilization = 0.5 class-attribute instance-attribute

hf_cache_dir = None class-attribute instance-attribute

model_name = Field(default=DEFAULT_LOCAL_CAPTION_MODEL_ID, description='Caption model identifier. The default local BF16 checkpoint has approximately 62 GiB of weights; set this explicitly to select a smaller local model or an API model for a remote endpoint.') class-attribute instance-attribute

prompt = 'Caption the content of this image:' class-attribute instance-attribute

system_prompt = '/no_think' class-attribute instance-attribute

tensor_parallel_size = 1 class-attribute instance-attribute

_require_temperature(value) classmethod

Source code in nemo_retriever/common/params/models.py

@field_validator("temperature")
@classmethod
def _require_temperature(cls, value: Optional[float]) -> float:
    if value is None:
        raise ValueError("temperature cannot be None for captioning")
    return value

DedupParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class DedupParams(_ParamsModel):
    content_hash: bool = True
    bbox_iou: bool = True
    iou_threshold: float = Field(default=0.45, ge=0.0, le=1.0)

bbox_iou = True class-attribute instance-attribute

content_hash = True class-attribute instance-attribute

iou_threshold = Field(default=0.45, ge=0.0, le=1.0) class-attribute instance-attribute

EmbedParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class EmbedParams(_ParamsModel):
    model_name: Optional[str] = None
    embedding_endpoint: Optional[str] = None
    embed_invoke_url: Optional[str] = None
    embed_model_name: Optional[str] = None
    embed_model_revision: Optional[str] = None
    embed_model_provider_prefix: Optional[str] = None
    api_key: Optional[str] = None
    input_type: str = "passage"
    embed_modality: str = "text"  # "text", "image", or "text_image" — default for all element types
    embed_granularity: Literal["element", "page"] = "element"  # "element" = per-element rows, "page" = one row per page
    text_elements_modality: Optional[str] = None  # per-type override for page-text rows
    structured_elements_modality: Optional[str] = None  # per-type override for table/chart/infographic rows
    text_column: str = "text"
    inference_batch_size: int = 32
    output_column: str = "text_embeddings_1b_v2"
    embedding_dim_column: str = "text_embeddings_1b_v2_dim"
    has_embedding_column: str = "text_embeddings_1b_v2_has_embedding"
    embed_output_column: str = "text_embeddings_1b_v2"
    embed_inference_batch_size: int = 16

    local_ingest_embed_backend: str = (
        "vllm"  # "vllm" or "hf" — selects ingest-time embedder backend for both text and VL models
    )
    query_max_length: int = 128
    dimensions: Optional[int] = None

    # Concurrent HTTP embedding requests per Ray batch (OpenAI-compatible NIM).
    nim_http_max_concurrent: int = 32
    request_timeout_s: float = 600.0

    runtime: ModelRuntimeParams = Field(default_factory=ModelRuntimeParams)
    batch_tuning: BatchTuningParams = Field(default_factory=BatchTuningParams)

    @field_validator("local_ingest_embed_backend", mode="before")
    @classmethod
    def _validate_local_ingest_embed_backend(cls, v: str) -> str:
        from nemo_retriever.models import (
            _LOCAL_INGEST_EMBED_BACKENDS,
            normalize_backend,
        )

        return normalize_backend(
            str(v) if v is not None else None,
            _LOCAL_INGEST_EMBED_BACKENDS,
            field_name="local_ingest_embed_backend",
            default="vllm",
        )

    @field_validator(
        "embed_modality",
        "text_elements_modality",
        "structured_elements_modality",
        mode="before",
    )
    @classmethod
    def _validate_modality(cls, v: str | None) -> str | None:
        if v is None:
            return None
        modality = str(v).strip()
        if modality == "image_text":
            raise ValueError("Use 'text_image' instead of 'image_text'.")
        if modality not in VALID_EMBED_MODALITIES:
            raise ValueError(f"Modality must be one of {sorted(VALID_EMBED_MODALITIES)}")
        return modality

    @model_validator(mode="after")
    def _warn_page_granularity_overrides(self) -> "EmbedParams":
        if self.embed_granularity == "page" and (
            self.text_elements_modality is not None or self.structured_elements_modality is not None
        ):
            warnings.warn(
                "text_elements_modality and structured_elements_modality are ignored when "
                "embed_granularity='page' (only embed_modality is used).",
                UserWarning,
                stacklevel=2,
            )
        return self

api_key = None class-attribute instance-attribute

batch_tuning = Field(default_factory=BatchTuningParams) class-attribute instance-attribute

dimensions = None class-attribute instance-attribute

embed_granularity = 'element' class-attribute instance-attribute

embed_inference_batch_size = 16 class-attribute instance-attribute

embed_invoke_url = None class-attribute instance-attribute

embed_modality = 'text' class-attribute instance-attribute

embed_model_name = None class-attribute instance-attribute

embed_model_provider_prefix = None class-attribute instance-attribute

embed_model_revision = None class-attribute instance-attribute

embed_output_column = 'text_embeddings_1b_v2' class-attribute instance-attribute

embedding_dim_column = 'text_embeddings_1b_v2_dim' class-attribute instance-attribute

embedding_endpoint = None class-attribute instance-attribute

has_embedding_column = 'text_embeddings_1b_v2_has_embedding' class-attribute instance-attribute

inference_batch_size = 32 class-attribute instance-attribute

input_type = 'passage' class-attribute instance-attribute

local_ingest_embed_backend = 'vllm' class-attribute instance-attribute

model_name = None class-attribute instance-attribute

nim_http_max_concurrent = 32 class-attribute instance-attribute

output_column = 'text_embeddings_1b_v2' class-attribute instance-attribute

query_max_length = 128 class-attribute instance-attribute

request_timeout_s = 600.0 class-attribute instance-attribute

runtime = Field(default_factory=ModelRuntimeParams) class-attribute instance-attribute

structured_elements_modality = None class-attribute instance-attribute

text_column = 'text' class-attribute instance-attribute

text_elements_modality = None class-attribute instance-attribute

_validate_local_ingest_embed_backend(v) classmethod

Source code in nemo_retriever/common/params/models.py

@field_validator("local_ingest_embed_backend", mode="before")
@classmethod
def _validate_local_ingest_embed_backend(cls, v: str) -> str:
    from nemo_retriever.models import (
        _LOCAL_INGEST_EMBED_BACKENDS,
        normalize_backend,
    )

    return normalize_backend(
        str(v) if v is not None else None,
        _LOCAL_INGEST_EMBED_BACKENDS,
        field_name="local_ingest_embed_backend",
        default="vllm",
    )

_validate_modality(v) classmethod

Source code in nemo_retriever/common/params/models.py

@field_validator(
    "embed_modality",
    "text_elements_modality",
    "structured_elements_modality",
    mode="before",
)
@classmethod
def _validate_modality(cls, v: str | None) -> str | None:
    if v is None:
        return None
    modality = str(v).strip()
    if modality == "image_text":
        raise ValueError("Use 'text_image' instead of 'image_text'.")
    if modality not in VALID_EMBED_MODALITIES:
        raise ValueError(f"Modality must be one of {sorted(VALID_EMBED_MODALITIES)}")
    return modality

_warn_page_granularity_overrides()

Source code in nemo_retriever/common/params/models.py

@model_validator(mode="after")
def _warn_page_granularity_overrides(self) -> "EmbedParams":
    if self.embed_granularity == "page" and (
        self.text_elements_modality is not None or self.structured_elements_modality is not None
    ):
        warnings.warn(
            "text_elements_modality and structured_elements_modality are ignored when "
            "embed_granularity='page' (only embed_modality is used).",
            UserWarning,
            stacklevel=2,
        )
    return self

ExtractParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class ExtractParams(_ParamsModel):
    # Extraction flags
    extract_text: bool = True
    extract_images: bool = True
    extract_tables: bool = True
    extract_charts: bool = True
    extract_infographics: bool = False
    extract_page_as_image: Optional[bool] = True

    # Extraction options
    method: str = "pdfium"
    # Run PageElementDetection (layout/yolox). Required by TableStructure and
    # OCR. Safe to disable for text-only ingests.
    use_page_elements: bool = True
    use_table_structure: bool = False
    table_output_format: Optional[Literal["pseudo_markdown", "markdown"]] = None
    dpi: int = 200
    image_format: str = "jpeg"
    jpeg_quality: int = 100
    render_mode: Literal["full_dpi", "fit_to_model"] = "fit_to_model"
    inference_batch_size: int = 8
    ocr_model_dir: Optional[str] = None
    ocr_version: Literal["v1", "v2"] = "v2"
    ocr_lang: Optional[Literal["multi", "english"]] = None

    # Service endpoints
    invoke_url: Optional[str] = None
    api_key: Optional[str] = None
    request_timeout_s: float = 60.0
    page_elements_invoke_url: Optional[str] = None
    page_elements_api_key: Optional[str] = None
    page_elements_request_timeout_s: Optional[float] = None
    ocr_invoke_url: Optional[str] = None
    ocr_api_key: Optional[str] = None
    ocr_request_timeout_s: Optional[float] = None
    table_structure_invoke_url: Optional[str] = None
    nemotron_parse_invoke_url: Optional[str] = None
    nemotron_parse_model: Optional[str] = None

    # Output columns
    output_column: str = "page_elements_v3"
    num_detections_column: str = "page_elements_v3_num_detections"
    counts_by_label_column: str = "page_elements_v3_counts_by_label"

    remote_retry: RemoteRetryParams = Field(default_factory=RemoteRetryParams)
    batch_tuning: BatchTuningParams = Field(default_factory=BatchTuningParams)

    @model_validator(mode="after")
    def _auto_enable_features(self) -> "ExtractParams":
        """Auto-configure feature flags from remote endpoints.

        * Enable ``use_table_structure`` when ``table_structure_invoke_url``
          is provided.
        * Default ``table_output_format`` to ``"markdown"`` when the stage is
          enabled and the caller did not explicitly choose a format.
        """
        if self.table_structure_invoke_url and not self.use_table_structure:
            self.use_table_structure = True
        if self.table_output_format is None:
            self.table_output_format = "markdown" if self.use_table_structure else "pseudo_markdown"
        if self.ocr_version == "v1" and self.ocr_lang is not None:
            raise ValueError("ocr_lang is only supported when ocr_version='v2'.")
        if self.method != "nemotron_parse" and (
            self.nemotron_parse_invoke_url is not None or self.nemotron_parse_model is not None
        ):
            raise ValueError(
                "`nemotron_parse_invoke_url` and `nemotron_parse_model` require "
                "`method='nemotron_parse'`; Parse-specific configuration is otherwise ignored."
            )
        if not self.use_page_elements:
            consumers = [("use_table_structure", self.use_table_structure and self.extract_tables)]
            enabled = [name for name, on in consumers if on]
            if enabled:
                raise ValueError(f"use_page_elements=False is incompatible with: {', '.join(enabled)}")
        return self

api_key = None class-attribute instance-attribute

batch_tuning = Field(default_factory=BatchTuningParams) class-attribute instance-attribute

counts_by_label_column = 'page_elements_v3_counts_by_label' class-attribute instance-attribute

dpi = 200 class-attribute instance-attribute

extract_charts = True class-attribute instance-attribute

extract_images = True class-attribute instance-attribute

extract_infographics = False class-attribute instance-attribute

extract_page_as_image = True class-attribute instance-attribute

extract_tables = True class-attribute instance-attribute

extract_text = True class-attribute instance-attribute

image_format = 'jpeg' class-attribute instance-attribute

inference_batch_size = 8 class-attribute instance-attribute

invoke_url = None class-attribute instance-attribute

jpeg_quality = 100 class-attribute instance-attribute

method = 'pdfium' class-attribute instance-attribute

nemotron_parse_invoke_url = None class-attribute instance-attribute

nemotron_parse_model = None class-attribute instance-attribute

num_detections_column = 'page_elements_v3_num_detections' class-attribute instance-attribute

ocr_api_key = None class-attribute instance-attribute

ocr_invoke_url = None class-attribute instance-attribute

ocr_lang = None class-attribute instance-attribute

ocr_model_dir = None class-attribute instance-attribute

ocr_request_timeout_s = None class-attribute instance-attribute

ocr_version = 'v2' class-attribute instance-attribute

output_column = 'page_elements_v3' class-attribute instance-attribute

page_elements_api_key = None class-attribute instance-attribute

page_elements_invoke_url = None class-attribute instance-attribute

page_elements_request_timeout_s = None class-attribute instance-attribute

remote_retry = Field(default_factory=RemoteRetryParams) class-attribute instance-attribute

render_mode = 'fit_to_model' class-attribute instance-attribute

request_timeout_s = 60.0 class-attribute instance-attribute

table_output_format = None class-attribute instance-attribute

table_structure_invoke_url = None class-attribute instance-attribute

use_page_elements = True class-attribute instance-attribute

use_table_structure = False class-attribute instance-attribute

_auto_enable_features()

Auto-configure feature flags from remote endpoints.

• Enable use_table_structure when table_structure_invoke_url is provided.
• Default table_output_format to "markdown" when the stage is enabled and the caller did not explicitly choose a format.

Source code in nemo_retriever/common/params/models.py

@model_validator(mode="after")
def _auto_enable_features(self) -> "ExtractParams":
    """Auto-configure feature flags from remote endpoints.

    * Enable ``use_table_structure`` when ``table_structure_invoke_url``
      is provided.
    * Default ``table_output_format`` to ``"markdown"`` when the stage is
      enabled and the caller did not explicitly choose a format.
    """
    if self.table_structure_invoke_url and not self.use_table_structure:
        self.use_table_structure = True
    if self.table_output_format is None:
        self.table_output_format = "markdown" if self.use_table_structure else "pseudo_markdown"
    if self.ocr_version == "v1" and self.ocr_lang is not None:
        raise ValueError("ocr_lang is only supported when ocr_version='v2'.")
    if self.method != "nemotron_parse" and (
        self.nemotron_parse_invoke_url is not None or self.nemotron_parse_model is not None
    ):
        raise ValueError(
            "`nemotron_parse_invoke_url` and `nemotron_parse_model` require "
            "`method='nemotron_parse'`; Parse-specific configuration is otherwise ignored."
        )
    if not self.use_page_elements:
        consumers = [("use_table_structure", self.use_table_structure and self.extract_tables)]
        enabled = [name for name, on in consumers if on]
        if enabled:
            raise ValueError(f"use_page_elements=False is incompatible with: {', '.join(enabled)}")
    return self

IngestExecuteParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class IngestExecuteParams(_ParamsModel):
    show_progress: bool = False
    return_failures: bool = False
    return_traces: bool = False
    return_results: bool = True
    result_schema: Literal["legacy", "compact"] = "legacy"
    return_embeddings: bool = False
    return_images: bool = False
    parallel: bool = False
    max_workers: Optional[int] = None
    gpu_devices: list[str] = Field(default_factory=list)
    page_chunk_size: int = 32
    runtime_metrics_dir: Optional[str] = None
    runtime_metrics_prefix: Optional[str] = None

gpu_devices = Field(default_factory=list) class-attribute instance-attribute

max_workers = None class-attribute instance-attribute

page_chunk_size = 32 class-attribute instance-attribute

parallel = False class-attribute instance-attribute

result_schema = 'legacy' class-attribute instance-attribute

return_embeddings = False class-attribute instance-attribute

return_failures = False class-attribute instance-attribute

return_images = False class-attribute instance-attribute

return_results = True class-attribute instance-attribute

return_traces = False class-attribute instance-attribute

runtime_metrics_dir = None class-attribute instance-attribute

runtime_metrics_prefix = None class-attribute instance-attribute

show_progress = False class-attribute instance-attribute

IngestorCreateParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class IngestorCreateParams(_ParamsModel):
    documents: list[str] = Field(default_factory=list)
    ray_address: Optional[str] = None
    ray_log_to_driver: bool = True
    debug: bool = False
    base_url: str = "http://localhost:7670"
    allow_no_gpu: bool = False
    node_overrides: Optional[dict[str, dict[str, Any]]] = None
    api_key: Optional[str] = None
    error_policy: Literal["raise", "collect"] = "raise"
    # service run mode: maximum number of concurrent page uploads.  Lower
    # values (e.g. 2-4) reduce burst pressure on Kubernetes NodePort /
    # kube-proxy paths that otherwise reset connections under heavy load.
    max_concurrency: Optional[int] = None

allow_no_gpu = False class-attribute instance-attribute

api_key = None class-attribute instance-attribute

base_url = 'http://localhost:7670' class-attribute instance-attribute

debug = False class-attribute instance-attribute

documents = Field(default_factory=list) class-attribute instance-attribute

error_policy = 'raise' class-attribute instance-attribute

max_concurrency = None class-attribute instance-attribute

node_overrides = None class-attribute instance-attribute

ray_address = None class-attribute instance-attribute

ray_log_to_driver = True class-attribute instance-attribute

StoreParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class StoreParams(_ParamsModel):
    storage_uri: str = "stored_images"
    storage_options: dict[str, Any] = Field(default_factory=dict)
    image_format: str = "png"
    strip_base64: bool = True
    batch_tuning: BatchTuningParams = Field(default_factory=BatchTuningParams)

    @model_validator(mode="after")
    def _resolve_local_storage_uri(self) -> "StoreParams":
        """Resolve relative local paths to absolute so they survive Ray serialization."""
        if not urlparse(self.storage_uri).scheme:
            self.storage_uri = str(UPath(self.storage_uri).resolve())
        return self

batch_tuning = Field(default_factory=BatchTuningParams) class-attribute instance-attribute

image_format = 'png' class-attribute instance-attribute

storage_options = Field(default_factory=dict) class-attribute instance-attribute

storage_uri = 'stored_images' class-attribute instance-attribute

strip_base64 = True class-attribute instance-attribute

_resolve_local_storage_uri()

Resolve relative local paths to absolute so they survive Ray serialization.

Source code in nemo_retriever/common/params/models.py

@model_validator(mode="after")
def _resolve_local_storage_uri(self) -> "StoreParams":
    """Resolve relative local paths to absolute so they survive Ray serialization."""
    if not urlparse(self.storage_uri).scheme:
        self.storage_uri = str(UPath(self.storage_uri).resolve())
    return self

VdbUploadParams

Bases: _ParamsModel

Post-graph vector DB upload configuration.

Sidecar metadata (meta_*) matches nv_ingest_client / metadata_and_filtered_search.ipynb: all three fields must be set together to merge columns into each chunk's content_metadata.

Source code in nemo_retriever/common/params/models.py

class VdbUploadParams(_ParamsModel):
    """Post-graph vector DB upload configuration.

    Sidecar metadata (``meta_*``) matches ``nv_ingest_client`` / ``metadata_and_filtered_search.ipynb``:
    all three fields must be set together to merge columns into each chunk's ``content_metadata``.
    """

    vdb_op: str = "lancedb"
    vdb_kwargs: dict[str, Any] = Field(default_factory=dict)
    meta_dataframe: Optional[Any] = None
    """Path to csv/json/parquet or an in-memory :class:`pandas.DataFrame`."""
    meta_source_field: Optional[str] = None
    meta_fields: Optional[list[str]] = None
    meta_join_key: MetaJoinKey = "auto"
    """How to match rows to documents: ``source_id`` (full path), ``source_name`` (basename), or ``auto`` (try both)."""

    @model_validator(mode="after")
    def _validate_sidecar_triplet(self) -> "VdbUploadParams":
        trio = (self.meta_dataframe, self.meta_source_field, self.meta_fields)
        if all(x is None for x in trio):
            return self
        if any(x is None for x in trio):
            raise ValueError(
                "meta_dataframe, meta_source_field, and meta_fields must all be set together "
                "when attaching sidecar metadata."
            )
        if not self.meta_fields:
            raise ValueError("meta_fields must be a non-empty list when sidecar metadata is enabled.")
        return self

    def to_ingest_operator_kwargs(self) -> dict[str, Any]:
        """Flatten into kwargs for :class:`~nemo_retriever.vdb.IngestVdbOperator`."""
        out = dict(self.vdb_kwargs or {})
        if self.meta_dataframe is not None:
            out["meta_dataframe"] = self.meta_dataframe
            out["meta_source_field"] = self.meta_source_field
            out["meta_fields"] = list(self.meta_fields or [])
            out["meta_join_key"] = self.meta_join_key
        return out

meta_dataframe = None class-attribute instance-attribute

Path to csv/json/parquet or an in-memory :class:pandas.DataFrame.

meta_fields = None class-attribute instance-attribute

meta_join_key = 'auto' class-attribute instance-attribute

How to match rows to documents: source_id (full path), source_name (basename), or auto (try both).

meta_source_field = None class-attribute instance-attribute

vdb_kwargs = Field(default_factory=dict) class-attribute instance-attribute

vdb_op = 'lancedb' class-attribute instance-attribute

_validate_sidecar_triplet()

Source code in nemo_retriever/common/params/models.py

@model_validator(mode="after")
def _validate_sidecar_triplet(self) -> "VdbUploadParams":
    trio = (self.meta_dataframe, self.meta_source_field, self.meta_fields)
    if all(x is None for x in trio):
        return self
    if any(x is None for x in trio):
        raise ValueError(
            "meta_dataframe, meta_source_field, and meta_fields must all be set together "
            "when attaching sidecar metadata."
        )
    if not self.meta_fields:
        raise ValueError("meta_fields must be a non-empty list when sidecar metadata is enabled.")
    return self

to_ingest_operator_kwargs()

Flatten into kwargs for :class:~nemo_retriever.vdb.IngestVdbOperator.

Source code in nemo_retriever/common/params/models.py

def to_ingest_operator_kwargs(self) -> dict[str, Any]:
    """Flatten into kwargs for :class:`~nemo_retriever.vdb.IngestVdbOperator`."""
    out = dict(self.vdb_kwargs or {})
    if self.meta_dataframe is not None:
        out["meta_dataframe"] = self.meta_dataframe
        out["meta_source_field"] = self.meta_source_field
        out["meta_fields"] = list(self.meta_fields or [])
        out["meta_join_key"] = self.meta_join_key
    return out

WebhookParams

Bases: _ParamsModel

Configuration for the webhook notification stage.

When endpoint_url is set, selected columns from the processed batch are serialised to JSON and HTTP-POSTed to that URL. If endpoint_url is None the stage is a no-op.

Source code in nemo_retriever/common/params/models.py

class WebhookParams(_ParamsModel):
    """Configuration for the webhook notification stage.

    When ``endpoint_url`` is set, selected columns from the processed batch
    are serialised to JSON and HTTP-POSTed to that URL.  If ``endpoint_url``
    is ``None`` the stage is a no-op.
    """

    endpoint_url: Optional[str] = None
    columns: list[str] = Field(default_factory=list)
    headers: dict[str, str] = Field(default_factory=dict)
    timeout_s: float = 30.0
    max_retries: int = 3

columns = Field(default_factory=list) class-attribute instance-attribute

endpoint_url = None class-attribute instance-attribute

headers = Field(default_factory=dict) class-attribute instance-attribute

max_retries = 3 class-attribute instance-attribute

timeout_s = 30.0 class-attribute instance-attribute

ingestor

Interface base class. All methods intentionally raise NotImplementedError.

Each runmode should subclass this and eventually provide working behavior.

Source code in nemo_retriever/ingestor/core.py

class ingestor:
    """
    Interface base class. All methods intentionally raise NotImplementedError.

    Each runmode should subclass this and eventually provide working behavior.
    """

    RUN_MODE: str = "interface"

    def __init__(self, documents: Optional[List[str]] = None) -> None:
        self._documents: List[str] = list(documents or [])
        self._buffers: List[Tuple[str, BytesIO]] = []

    def _not_implemented(self, method_name: str) -> "None":
        raise NotImplementedError(
            f"{self.__class__.__name__}.{method_name}() is not implemented yet " f"(run_mode={self.RUN_MODE})."
        )

    def files(self, documents: Union[str, List[str]]) -> "ingestor":
        """Add document paths/URIs for processing."""
        self._not_implemented("files")

    def texts(self, texts: Union[str, Sequence[str]]) -> Self:
        """Set raw inline text documents for processing."""
        self._not_implemented("texts")

    def buffers(self, buffers: Union[Tuple[str, BytesIO], List[Tuple[str, BytesIO]]]) -> "ingestor":
        """Add in-memory buffers for processing."""
        self._not_implemented("buffers")

    def load(self) -> "ingestor":
        """
        Placeholder for remote fetch/localization.

        The client-side Ingestor supports downloading remote URIs locally.
        In this system, each runmode may handle remote inputs differently.
        """
        self._not_implemented("load")

    def ingest(
        self,
        params: IngestExecuteParams | None = None,
        **kwargs: Any,
    ) -> Union[List[Any], Tuple[Any, ...]]:
        """Execute the configured ingestion pipeline (placeholder).

        In ``run_mode='service'``, ``return_results`` (default ``True``)
        controls whether completed rows are fetched into
        ``ServiceIngestResult.dataframe``.
        """
        _ = _merge_params(params, kwargs)
        self._not_implemented("ingest")

    def ingest_async(self, *, return_failures: bool = False, return_traces: bool = False) -> Any:
        """Asynchronously execute ingestion (placeholder)."""
        self._not_implemented("ingest_async")

    def all_tasks(self) -> "ingestor":
        """Record the default task chain (placeholder)."""
        self._not_implemented("all_tasks")

    def dedup(self, params: DedupParams | None = None, **kwargs: Any) -> "ingestor":
        """Record a dedup task configuration."""
        _ = _merge_params(params, kwargs)
        self._not_implemented("dedup")

    def embed(self, params: EmbedParams | None = None, **kwargs: Any) -> "ingestor":
        """Record an embedding task configuration."""
        _ = _merge_params(params, kwargs)
        self._not_implemented("embed")

    def extract(self, params: ExtractParams | None = None, **kwargs: Any) -> "ingestor":
        """Record an extract task configuration."""
        _ = _merge_params(params, kwargs)
        self._not_implemented("extract")

    def extract_image_files(self, params: ExtractParams | None = None, **kwargs: Any) -> "ingestor":
        """Record an extract-image-files task configuration."""
        _ = _merge_params(params, kwargs)
        self._not_implemented("extract_image_files")

    def filter(self) -> "ingestor":
        """Record a filter task configuration."""
        self._not_implemented("filter")

    def store(self, params: StoreParams | None = None, **kwargs: Any) -> "ingestor":
        """Record a store task configuration for extracted image assets."""
        _ = _merge_params(params, kwargs)
        self._not_implemented("store")

    def store_embed(self) -> "ingestor":
        """Record a store-embed task configuration."""
        self._not_implemented("store_embed")

    def udf(
        self,
        udf_function: str,
        udf_function_name: Optional[str] = None,
        phase: Optional[Union[int, str]] = None,
        target_stage: Optional[str] = None,
        run_before: bool = False,
        run_after: bool = False,
    ) -> "ingestor":
        """Record a UDF task configuration."""
        self._not_implemented("udf")

    def vdb_upload(self, params: VdbUploadParams | None = None, **kwargs: Any) -> "ingestor":
        """Record a vector DB upload configuration (execution TBD)."""
        _ = _merge_params(params, kwargs)
        self._not_implemented("vdb_upload")

    def save_intermediate_results(self, output_dir: str) -> "ingestor":
        """Record intermediate results persistence configuration."""
        self._not_implemented("save_intermediate_results")

    def caption(self, params: "CaptionParams | None" = None, **kwargs: Any) -> "ingestor":
        """Record a caption task configuration."""
        _ = _merge_params(params, kwargs)
        self._not_implemented("caption")

    def webhook(self, params: "WebhookParams | None" = None, **kwargs: Any) -> "ingestor":
        """Record a webhook notification configuration."""
        _ = _merge_params(params, kwargs)
        self._not_implemented("webhook")

    def pdf_split_config(self, pages_per_chunk: int = 32) -> "ingestor":
        """Record PDF split configuration (execution TBD)."""
        self._not_implemented("pdf_split_config")

    def completed_jobs(self) -> int:
        """Return completed job count (placeholder until backend populates job state)."""
        self._not_implemented("completed_jobs")

    def failed_jobs(self) -> int:
        """Return failed job count (placeholder until backend populates job state)."""
        self._not_implemented("failed_jobs")

    def cancelled_jobs(self) -> int:
        """Return cancelled job count (placeholder until backend populates job state)."""
        self._not_implemented("cancelled_jobs")

    def remaining_jobs(self) -> int:
        """Return remaining job count (placeholder until backend populates job state)."""
        self._not_implemented("remaining_jobs")

    def get_status(self) -> Dict[str, str]:
        """
        Return per-document status mapping (placeholder).

        Once Ray execution is wired, this should reflect actual job/task state.
        """
        self._not_implemented("get_status")

RUN_MODE = 'interface' class-attribute instance-attribute

_buffers = [] instance-attribute

_documents = list(documents or []) instance-attribute

__init__(documents=None)

Source code in nemo_retriever/ingestor/core.py

def __init__(self, documents: Optional[List[str]] = None) -> None:
    self._documents: List[str] = list(documents or [])
    self._buffers: List[Tuple[str, BytesIO]] = []

_not_implemented(method_name)

Source code in nemo_retriever/ingestor/core.py

def _not_implemented(self, method_name: str) -> "None":
    raise NotImplementedError(
        f"{self.__class__.__name__}.{method_name}() is not implemented yet " f"(run_mode={self.RUN_MODE})."
    )

all_tasks()

Record the default task chain (placeholder).

Source code in nemo_retriever/ingestor/core.py

def all_tasks(self) -> "ingestor":
    """Record the default task chain (placeholder)."""
    self._not_implemented("all_tasks")

buffers(buffers)

Add in-memory buffers for processing.

Source code in nemo_retriever/ingestor/core.py

def buffers(self, buffers: Union[Tuple[str, BytesIO], List[Tuple[str, BytesIO]]]) -> "ingestor":
    """Add in-memory buffers for processing."""
    self._not_implemented("buffers")

cancelled_jobs()

Return cancelled job count (placeholder until backend populates job state).

Source code in nemo_retriever/ingestor/core.py

def cancelled_jobs(self) -> int:
    """Return cancelled job count (placeholder until backend populates job state)."""
    self._not_implemented("cancelled_jobs")

caption(params=None, **kwargs)

Record a caption task configuration.

Source code in nemo_retriever/ingestor/core.py

def caption(self, params: "CaptionParams | None" = None, **kwargs: Any) -> "ingestor":
    """Record a caption task configuration."""
    _ = _merge_params(params, kwargs)
    self._not_implemented("caption")

completed_jobs()

Return completed job count (placeholder until backend populates job state).

Source code in nemo_retriever/ingestor/core.py

def completed_jobs(self) -> int:
    """Return completed job count (placeholder until backend populates job state)."""
    self._not_implemented("completed_jobs")

dedup(params=None, **kwargs)

Record a dedup task configuration.

Source code in nemo_retriever/ingestor/core.py

def dedup(self, params: DedupParams | None = None, **kwargs: Any) -> "ingestor":
    """Record a dedup task configuration."""
    _ = _merge_params(params, kwargs)
    self._not_implemented("dedup")

embed(params=None, **kwargs)

Record an embedding task configuration.

Source code in nemo_retriever/ingestor/core.py

def embed(self, params: EmbedParams | None = None, **kwargs: Any) -> "ingestor":
    """Record an embedding task configuration."""
    _ = _merge_params(params, kwargs)
    self._not_implemented("embed")

extract(params=None, **kwargs)

Record an extract task configuration.

Source code in nemo_retriever/ingestor/core.py

def extract(self, params: ExtractParams | None = None, **kwargs: Any) -> "ingestor":
    """Record an extract task configuration."""
    _ = _merge_params(params, kwargs)
    self._not_implemented("extract")

extract_image_files(params=None, **kwargs)

Record an extract-image-files task configuration.

Source code in nemo_retriever/ingestor/core.py

def extract_image_files(self, params: ExtractParams | None = None, **kwargs: Any) -> "ingestor":
    """Record an extract-image-files task configuration."""
    _ = _merge_params(params, kwargs)
    self._not_implemented("extract_image_files")

failed_jobs()

Return failed job count (placeholder until backend populates job state).

Source code in nemo_retriever/ingestor/core.py

def failed_jobs(self) -> int:
    """Return failed job count (placeholder until backend populates job state)."""
    self._not_implemented("failed_jobs")

files(documents)

Add document paths/URIs for processing.

Source code in nemo_retriever/ingestor/core.py

def files(self, documents: Union[str, List[str]]) -> "ingestor":
    """Add document paths/URIs for processing."""
    self._not_implemented("files")

filter()

Record a filter task configuration.

Source code in nemo_retriever/ingestor/core.py

def filter(self) -> "ingestor":
    """Record a filter task configuration."""
    self._not_implemented("filter")

get_status()

Return per-document status mapping (placeholder).

Once Ray execution is wired, this should reflect actual job/task state.

Source code in nemo_retriever/ingestor/core.py

def get_status(self) -> Dict[str, str]:
    """
    Return per-document status mapping (placeholder).

    Once Ray execution is wired, this should reflect actual job/task state.
    """
    self._not_implemented("get_status")

ingest(params=None, **kwargs)

Execute the configured ingestion pipeline (placeholder).

In run_mode='service', return_results (default True) controls whether completed rows are fetched into ServiceIngestResult.dataframe.

Source code in nemo_retriever/ingestor/core.py

def ingest(
    self,
    params: IngestExecuteParams | None = None,
    **kwargs: Any,
) -> Union[List[Any], Tuple[Any, ...]]:
    """Execute the configured ingestion pipeline (placeholder).

    In ``run_mode='service'``, ``return_results`` (default ``True``)
    controls whether completed rows are fetched into
    ``ServiceIngestResult.dataframe``.
    """
    _ = _merge_params(params, kwargs)
    self._not_implemented("ingest")

ingest_async(*, return_failures=False, return_traces=False)

Asynchronously execute ingestion (placeholder).

Source code in nemo_retriever/ingestor/core.py

def ingest_async(self, *, return_failures: bool = False, return_traces: bool = False) -> Any:
    """Asynchronously execute ingestion (placeholder)."""
    self._not_implemented("ingest_async")

load()

Placeholder for remote fetch/localization.

The client-side Ingestor supports downloading remote URIs locally. In this system, each runmode may handle remote inputs differently.

Source code in nemo_retriever/ingestor/core.py

def load(self) -> "ingestor":
    """
    Placeholder for remote fetch/localization.

    The client-side Ingestor supports downloading remote URIs locally.
    In this system, each runmode may handle remote inputs differently.
    """
    self._not_implemented("load")

remaining_jobs()

Return remaining job count (placeholder until backend populates job state).

Source code in nemo_retriever/ingestor/core.py

def remaining_jobs(self) -> int:
    """Return remaining job count (placeholder until backend populates job state)."""
    self._not_implemented("remaining_jobs")

save_intermediate_results(output_dir)

Record intermediate results persistence configuration.

Source code in nemo_retriever/ingestor/core.py

def save_intermediate_results(self, output_dir: str) -> "ingestor":
    """Record intermediate results persistence configuration."""
    self._not_implemented("save_intermediate_results")

store(params=None, **kwargs)

Record a store task configuration for extracted image assets.

Source code in nemo_retriever/ingestor/core.py

def store(self, params: StoreParams | None = None, **kwargs: Any) -> "ingestor":
    """Record a store task configuration for extracted image assets."""
    _ = _merge_params(params, kwargs)
    self._not_implemented("store")

store_embed()

Record a store-embed task configuration.

Source code in nemo_retriever/ingestor/core.py

def store_embed(self) -> "ingestor":
    """Record a store-embed task configuration."""
    self._not_implemented("store_embed")

texts(texts)

Set raw inline text documents for processing.

Source code in nemo_retriever/ingestor/core.py

def texts(self, texts: Union[str, Sequence[str]]) -> Self:
    """Set raw inline text documents for processing."""
    self._not_implemented("texts")

udf(udf_function, udf_function_name=None, phase=None, target_stage=None, run_before=False, run_after=False)

Record a UDF task configuration.

Source code in nemo_retriever/ingestor/core.py

def udf(
    self,
    udf_function: str,
    udf_function_name: Optional[str] = None,
    phase: Optional[Union[int, str]] = None,
    target_stage: Optional[str] = None,
    run_before: bool = False,
    run_after: bool = False,
) -> "ingestor":
    """Record a UDF task configuration."""
    self._not_implemented("udf")

vdb_upload(params=None, **kwargs)

Record a vector DB upload configuration (execution TBD).

Source code in nemo_retriever/ingestor/core.py

def vdb_upload(self, params: VdbUploadParams | None = None, **kwargs: Any) -> "ingestor":
    """Record a vector DB upload configuration (execution TBD)."""
    _ = _merge_params(params, kwargs)
    self._not_implemented("vdb_upload")

webhook(params=None, **kwargs)

Record a webhook notification configuration.

Source code in nemo_retriever/ingestor/core.py

def webhook(self, params: "WebhookParams | None" = None, **kwargs: Any) -> "ingestor":
    """Record a webhook notification configuration."""
    _ = _merge_params(params, kwargs)
    self._not_implemented("webhook")

create_ingestor(*, run_mode='inprocess', params=None, **kwargs)

Graph-only ingestion factory.

Source code in nemo_retriever/ingestor/core.py

def create_ingestor(
    *,
    run_mode: IngestorRunMode = "inprocess",
    params: IngestorCreateParams | None = None,
    **kwargs: Any,
) -> "Ingestor":
    """
    Graph-only ingestion factory.
    """
    merged = _merge_params(params, kwargs)
    if isinstance(merged, IngestorCreateParams):
        parsed = merged
    else:
        parsed = IngestorCreateParams(**merged)

    if run_mode == "service":
        from nemo_retriever.service.service_ingestor import ServiceIngestor

        service_kwargs: dict[str, Any] = {
            "base_url": parsed.base_url,
            "documents": parsed.documents,
            "api_token": parsed.api_key,
        }
        if parsed.max_concurrency is not None:
            service_kwargs["max_concurrency"] = parsed.max_concurrency
        return ServiceIngestor(**service_kwargs)

    if run_mode not in {"batch", "inprocess"}:
        raise ValueError(f"create_ingestor supports run modes 'inprocess', 'batch', and 'service'; got {run_mode!r}.")

    from nemo_retriever.ingestor.graph_ingestor import GraphIngestor

    return GraphIngestor(
        run_mode=run_mode,
        documents=parsed.documents,
        ray_address=parsed.ray_address,
        ray_log_to_driver=parsed.ray_log_to_driver,
        debug=parsed.debug,
        allow_no_gpu=parsed.allow_no_gpu,
        node_overrides=parsed.node_overrides,
        error_policy=parsed.error_policy,
    )

logger = logging.getLogger(__name__) module-attribute

retriever = Retriever module-attribute

Retriever dataclass

Graph-based query helper: batch embed → VDB retrieve [→ Nemotron rerank].

Configuration is passed through embed_kwargs (:class:~nemo_retriever.params.EmbedParams), vdb_kwargs (constructor kwargs for :class:~nemo_retriever.vdb.operators.RetrieveVdbOperator), and optional rerank_kwargs for :class:~nemo_retriever.rerank.rerank.NemotronRerankActor.

See retriever.md for examples.

Source code in nemo_retriever/graph/retriever.py

@dataclass
class Retriever:
    """Graph-based query helper: batch embed → VDB retrieve [→ Nemotron rerank].

    Configuration is passed through ``embed_kwargs`` (:class:`~nemo_retriever.params.EmbedParams`),
    ``vdb_kwargs`` (constructor kwargs for :class:`~nemo_retriever.vdb.operators.RetrieveVdbOperator`),
    and optional ``rerank_kwargs`` for :class:`~nemo_retriever.rerank.rerank.NemotronRerankActor`.

    See ``retriever.md`` for examples.
    """

    run_mode: Literal["local", "service"] = "local"
    """``local`` uses archetype batch embed resolution; ``service`` forces CPU HTTP embed."""

    top_k: int = 10
    rerank: bool = False
    """When ``True``, append :class:`~nemo_retriever.rerank.rerank.NemotronRerankActor` after retrieval."""

    graph: Any = None
    """Custom :class:`~nemo_retriever.graph.pipeline_graph.Graph`. When set, ``embed_kwargs`` /
    ``vdb_kwargs`` default-graph fields are ignored for construction (you still pass execute kwargs)."""

    embed_kwargs: dict[str, Any] = field(default_factory=dict)
    vdb_kwargs: dict[str, Any] = field(default_factory=dict)
    rerank_kwargs: dict[str, Any] = field(default_factory=dict)

    _cached_graph: Any = field(default=None, init=False, repr=False, compare=False)
    _cache_key: Any = field(default=None, init=False, repr=False, compare=False)
    _lancedb_capabilities_cache: dict[tuple[str, str], LanceTableCapabilities] = field(
        default_factory=dict, init=False, repr=False, compare=False
    )

    def __post_init__(self) -> None:
        if self.run_mode not in ("local", "service"):
            raise ValueError("run_mode must be 'local' or 'service'")

    def _merge_embed_params(self, extra: Optional[dict[str, Any]] = None) -> Any:
        from nemo_retriever.models import _LOCAL_INGEST_EMBED_BACKENDS, normalize_backend
        from nemo_retriever.common.params import EmbedParams

        base: dict[str, Any] = {
            "model_name": VL_EMBED_MODEL,
            "embed_model_name": VL_EMBED_MODEL,
            "input_type": "query",
            "text_column": "text",
            "inference_batch_size": 32,
            "embed_inference_batch_size": 32,
            "local_ingest_embed_backend": "hf",
        }
        overrides = {**dict(self.embed_kwargs or {}), **dict(extra or {})}
        merged = {**base, **overrides}
        endpoint = str(merged.get("embedding_endpoint") or merged.get("embed_invoke_url") or "").strip()
        if self.run_mode == "local" and not endpoint and overrides.get("local_ingest_embed_backend") is None:
            model_id = str(merged.get("embed_model_name") or merged.get("model_name") or "").strip()
            spec = resolve_embed_model_spec(model_id, revision=merged.get("embed_model_revision"))
            merged["local_ingest_embed_backend"] = "vllm" if spec.requires_vllm else "hf"
            merged["embed_model_revision"] = spec.revision
        if "local_ingest_embed_backend" in merged and merged["local_ingest_embed_backend"] is not None:
            merged["local_ingest_embed_backend"] = normalize_backend(
                str(merged["local_ingest_embed_backend"]),
                _LOCAL_INGEST_EMBED_BACKENDS,
                field_name="local_ingest_embed_backend",
                default="vllm",
            )
        params = EmbedParams.model_validate(merged)
        if self.run_mode == "service":
            url = (params.embedding_endpoint or params.embed_invoke_url or "").strip()
            if not url:
                raise ValueError(
                    "run_mode='service' requires a non-empty HTTP embedding URL. "
                    "Set ``embedding_endpoint`` or ``embed_invoke_url`` inside ``embed_kwargs``."
                )
        return params

    def _merge_rerank_actor_kwargs(self) -> dict[str, Any]:
        return {**_default_rerank_actor_kwargs(), **dict(self.rerank_kwargs or {})}

    def _refine_factor(self) -> int:
        if not self.rerank:
            return 1
        return int(self._merge_rerank_actor_kwargs().get("refine_factor", 4))

    def _build_default_graph(self, *, embed_extra: Optional[dict[str, Any]] = None) -> Any:
        from nemo_retriever.operators.rerank import NemotronRerankActor
        from nemo_retriever.operators.embed.cpu_operator import _BatchEmbedCPUActor
        from nemo_retriever.operators.embed.operators import _BatchEmbedActor

        embed_params = self._merge_embed_params(embed_extra)
        if self.run_mode == "service":
            embed_op = _BatchEmbedCPUActor(params=embed_params)
        else:
            embed_op = _BatchEmbedActor(params=embed_params)

        vdb_init = _coerce_vdb_init(self.vdb_kwargs)
        retrieve = RetrieveVdbOperator(
            explode_for_rerank=self.rerank,
            **vdb_init,
        )

        chain = embed_op >> retrieve
        if self.rerank:
            rk = self._merge_rerank_actor_kwargs()
            rk.pop("refine_factor", None)
            chain = chain >> NemotronRerankActor(**rk)

        return chain

    def _get_graph(self, *, embed_extra: Optional[dict[str, Any]] = None) -> Any:
        if self.graph is not None:
            return self.graph

        key = (
            self.run_mode,
            self.rerank,
            json.dumps(self.vdb_kwargs, sort_keys=True, default=str),
            json.dumps(self.embed_kwargs, sort_keys=True, default=str),
            json.dumps(self.rerank_kwargs, sort_keys=True, default=str),
            json.dumps(embed_extra or {}, sort_keys=True, default=str),
        )
        if self._cached_graph is not None and self._cache_key == key:
            return self._cached_graph
        g = self._build_default_graph(embed_extra=embed_extra)
        self._cached_graph = g
        self._cache_key = key
        return g

    def _execute_queries_graph(
        self,
        query_texts: list[str],
        *,
        effective_top_k: int,
        retrieval_top_k: int,
        vdb_call_kwargs: Optional[dict[str, Any]],
        embed_extra: Optional[dict[str, Any]],
    ) -> list[list[dict[str, Any]]]:
        embed_params = self._merge_embed_params(embed_extra)
        text_col = str(embed_params.text_column)
        df = pd.DataFrame({text_col: query_texts})

        # Hybrid retrieval relies on these ordered query strings staying aligned
        # with the embedded rows produced from ``df``. If this query graph grows
        # distributed/shuffled stages, carry row-local query text or IDs instead.
        graph = self._get_graph(embed_extra=embed_extra)

        exec_kwargs: dict[str, Any] = {
            **filter_retrieval_kwargs(dict(vdb_call_kwargs or {})),
            "top_k": int(retrieval_top_k),
            "query_texts": query_texts,
        }
        if self.graph is None:
            leaves = graph.execute_in_place(df, **exec_kwargs)
        else:
            # Preserve resolve-per-query behavior for caller-owned graphs, which
            # may be mutated between calls.
            resolve = getattr(graph, "resolve_for_local_execution", None)
            if not callable(resolve):
                raise TypeError("graph must provide resolve_for_local_execution() (e.g. pipeline_graph.Graph)")
            resolved = resolve()
            leaves = resolved.execute(df, **exec_kwargs)
        if len(leaves) != 1:
            raise RuntimeError(
                f"Retriever query graph must yield exactly one leaf output; got {len(leaves)}. "
                "Use a linear graph or adjust your custom ``graph``."
            )
        out = leaves[0]

        if isinstance(out, pd.DataFrame):
            if not self.rerank:
                raise TypeError(
                    "Graph returned a DataFrame but ``rerank`` is False; expected list[list[dict]] from retrieval."
                )
            rk = self._merge_rerank_actor_kwargs()
            score_col = str(rk.get("score_column", "rerank_score"))
            return rerank_long_dataframe_to_hits(
                out, query_texts=query_texts, top_k=int(effective_top_k), score_column=score_col
            )
        if not isinstance(out, list):
            raise TypeError(f"Unexpected query graph output type: {type(out).__name__}")
        return out

    def _inspect_lancedb_capabilities(self, uri: str, table_name: str) -> LanceTableCapabilities:
        key = (uri, table_name)
        caps = self._lancedb_capabilities_cache.get(key)
        if caps is None:
            caps = inspect_lancedb_table(uri, table_name)
            self._lancedb_capabilities_cache[key] = caps
        return caps

    def _resolve_lancedb_query_mode(
        self,
        runtime_vdb_kwargs: Optional[dict[str, Any]],
    ) -> tuple[str, LanceTableCapabilities, str, str, bool] | None:
        if self.graph is not None:
            return None

        lancedb_kwargs = dict(self.vdb_kwargs or {})
        if "vdb" in lancedb_kwargs:
            return None
        if "vdb_op" in lancedb_kwargs:
            if str(lancedb_kwargs.get("vdb_op") or "").strip().lower() != "lancedb":
                return None
            lancedb_kwargs = dict(lancedb_kwargs.get("vdb_kwargs") or {})
        lancedb_kwargs.update(dict(runtime_vdb_kwargs or {}))

        uri = str(
            lancedb_kwargs.get("table_path")
            or lancedb_kwargs.get("uri")
            or lancedb_kwargs.get("lancedb_uri")
            or "lancedb"
        )
        table_name = str(lancedb_kwargs.get("table_name") or lancedb_kwargs.get("lancedb_table") or "nv-ingest")
        caps = self._inspect_lancedb_capabilities(uri, table_name)

        mode_override = str(lancedb_kwargs.get("retrieval_mode") or "auto").strip().lower()
        if mode_override not in {"auto", "dense", "hybrid", "sparse"}:
            raise ValueError(
                f"Unsupported LanceDB retrieval mode {mode_override!r}; " "use 'auto', 'dense', 'hybrid', or 'sparse'."
            )
        if "hybrid" in lancedb_kwargs:
            mode_override = "hybrid" if bool(lancedb_kwargs["hybrid"]) else "dense"
        mode = caps.retrieval_mode if mode_override == "auto" else cast(LanceRetrievalMode, mode_override)

        if mode == "unknown":
            raise ValueError(
                f"LanceDB table {table_name!r} at {uri!r} is not queryable: "
                "no vector column or FTS index was detected."
            )
        if mode == "dense" and not caps.has_vector:
            raise ValueError(
                f"LanceDB table {table_name!r} at {uri!r} cannot run dense retrieval: " "no vector column was detected."
            )
        if mode == "hybrid" and (not caps.has_vector or not caps.has_fts):
            raise ValueError(
                f"LanceDB table {table_name!r} at {uri!r} cannot run hybrid retrieval: "
                "both a vector column and FTS index are required."
            )
        if mode == "sparse" and not caps.has_fts:
            raise ValueError(
                f"LanceDB table {table_name!r} at {uri!r} cannot run sparse retrieval: " "no FTS index was detected."
            )

        return mode, caps, uri, table_name, mode_override != "auto"

    @staticmethod
    def _embedding_model_from_kwargs(kwargs: Optional[dict[str, Any]]) -> str | None:
        values = dict(kwargs or {})
        for key in ("model_name", "embed_model_name"):
            value = str(values.get(key) or "").strip()
            if value:
                return value
        return None

    def _resolve_embed_kwargs(
        self,
        index_model: str | None,
        runtime_embed_kwargs: Optional[dict[str, Any]],
        index_revision: str | None = None,
    ) -> dict[str, Any]:
        """Choose the query model snapshot: explicit override, index metadata, or default."""
        resolved = dict(runtime_embed_kwargs or {})
        runtime_model = self._embedding_model_from_kwargs(runtime_embed_kwargs)
        configured_model = self._embedding_model_from_kwargs(self.embed_kwargs)
        explicit_model = runtime_model or configured_model
        model_name = explicit_model or index_model
        model_name = resolve_embed_model(model_name)
        resolved["model_name"] = model_name
        resolved["embed_model_name"] = model_name
        if runtime_model and "embed_model_revision" not in resolved:
            resolved_configured = resolve_embed_model(configured_model) if configured_model else None
            if resolved_configured != model_name:
                resolved["embed_model_revision"] = None
        if explicit_model is None and index_revision:
            resolved.setdefault("embed_model_revision", index_revision)
        return resolved

    def _execute_sparse_lancedb_queries(
        self,
        query_texts: list[str],
        *,
        retrieval_top_k: int,
        vdb_call_kwargs: Optional[dict[str, Any]],
        caps: LanceTableCapabilities,
        uri: str,
        table_name: str,
    ) -> list[list[dict[str, Any]]]:
        from nemo_retriever.common.vdb.lancedb import LanceDB

        text_column = caps.text_column or "text"
        retrieval_kwargs = {
            **filter_retrieval_kwargs(dict(vdb_call_kwargs or {})),
            "top_k": int(retrieval_top_k),
            "table_path": uri,
            "table_name": table_name,
            "text_column_name": text_column,
        }
        vdb = LanceDB(uri=uri, table_name=table_name, overwrite=False, sparse=True)
        return normalize_retrieval_results(vdb.sparse_retrieval(query_texts, **retrieval_kwargs))

    def query(
        self,
        query: str,
        *,
        top_k: Optional[int] = None,
        candidate_k: Optional[int] = None,
        page_dedup: bool = False,
        content_types: str | Sequence[str] | None = None,
        vdb_kwargs: Optional[dict[str, Any]] = None,
        embed_kwargs: Optional[dict[str, Any]] = None,
    ) -> list[RetrievalHit]:
        """Run one retrieval query and return shaped hits.

        ``top_k`` is the final number of hits to return. ``candidate_k`` is the
        wider pre-filter/pre-dedup candidate pool and must be greater than or
        equal to ``top_k``. Increase it when page deduplication or content-type
        filtering would otherwise reduce the final hit count. ``page_dedup``
        keeps the first hit per document page. ``content_types`` accepts a
        comma-separated string or sequence of content types to keep, such as
        ``"text,table"``, and normalizes values to the canonical content types
        stored in hit metadata. Hits with missing or unknown content types are
        excluded while this filter is active. Page deduplication and
        content-type filtering are applied after vector retrieval, preserving
        retriever ranking order.
        """
        return self.queries(
            [query],
            top_k=top_k,
            candidate_k=candidate_k,
            page_dedup=page_dedup,
            content_types=content_types,
            vdb_kwargs=vdb_kwargs,
            embed_kwargs=embed_kwargs,
        )[0]

    def queries(
        self,
        queries: Sequence[str],
        *,
        top_k: Optional[int] = None,
        candidate_k: Optional[int] = None,
        page_dedup: bool = False,
        content_types: str | Sequence[str] | None = None,
        vdb_kwargs: Optional[dict[str, Any]] = None,
        embed_kwargs: Optional[dict[str, Any]] = None,
    ) -> list[list[RetrievalHit]]:
        """Run retrieval for multiple query strings and return shaped hits.

        ``top_k`` is the final number of hits to return. ``candidate_k`` is the
        wider pre-filter/pre-dedup candidate pool and must be greater than or
        equal to ``top_k``. Increase it when page deduplication or content-type
        filtering would otherwise reduce the final hit count. ``page_dedup``
        keeps the first hit per document page. ``content_types`` accepts a
        comma-separated string or sequence of content types to keep, such as
        ``"text,table"``, and normalizes values to the canonical content types
        stored in hit metadata. Hits with missing or unknown content types are
        excluded while this filter is active. Page deduplication and
        content-type filtering are applied after vector retrieval, preserving
        retriever ranking order.
        """
        query_texts = [str(q) for q in queries]
        if not query_texts:
            return []

        effective_top_k = int(top_k) if top_k is not None else int(self.top_k)
        candidate_top_k = int(candidate_k) if candidate_k is not None else effective_top_k
        if candidate_top_k < effective_top_k:
            raise ValueError(
                f"candidate_k ({candidate_top_k}) must be greater than or equal to top_k ({effective_top_k})."
            )
        refine = self._refine_factor()
        retrieval_top_k = candidate_top_k * refine if self.rerank else candidate_top_k

        vdb_call_kwargs = dict(vdb_kwargs or {})
        index_model: str | None = None
        index_revision: str | None = None
        explicit_model = self._embedding_model_from_kwargs(embed_kwargs) or self._embedding_model_from_kwargs(
            self.embed_kwargs
        )
        if self.graph is None and explicit_model is None:
            metadata_reader = RetrieveVdbOperator(**_coerce_vdb_init(self.vdb_kwargs))
            index_model = metadata_reader.get_index_metadata("embedding_model_name", **vdb_call_kwargs)
            index_revision = metadata_reader.get_index_metadata("embedding_model_revision", **vdb_call_kwargs)

        lancedb_mode = self._resolve_lancedb_query_mode(vdb_call_kwargs)
        for key in _QUERY_ROUTING_VDB_KWARGS:
            vdb_call_kwargs.pop(key, None)
        if lancedb_mode is not None:
            mode, caps, uri, table_name, has_mode_override = lancedb_mode
            if mode == "sparse":
                raw_hits = self._execute_sparse_lancedb_queries(
                    query_texts,
                    retrieval_top_k=retrieval_top_k,
                    vdb_call_kwargs=vdb_call_kwargs,
                    caps=caps,
                    uri=uri,
                    table_name=table_name,
                )
                return [
                    shape_query_hits(
                        hits,
                        top_k=effective_top_k,
                        page_dedup=page_dedup,
                        content_types=content_types,
                    )
                    for hits in raw_hits
                ]
            if mode == "hybrid":
                vdb_call_kwargs["hybrid"] = True
            elif mode == "dense" and has_mode_override:
                vdb_call_kwargs["hybrid"] = False
            if caps.vector_column and caps.vector_column != "vector":
                vdb_call_kwargs.setdefault("vector_column_name", caps.vector_column)
        if self.graph is None:
            embed_kwargs = self._resolve_embed_kwargs(index_model, embed_kwargs, index_revision)

        raw_hits = self._execute_queries_graph(
            query_texts,
            effective_top_k=candidate_top_k,
            retrieval_top_k=retrieval_top_k,
            vdb_call_kwargs=vdb_call_kwargs,
            embed_extra=embed_kwargs,
        )
        return [
            shape_query_hits(
                hits,
                top_k=effective_top_k,
                page_dedup=page_dedup,
                content_types=content_types,
            )
            for hits in raw_hits
        ]

    def retrieve(
        self,
        query: str,
        top_k: Optional[int] = None,
        *,
        vdb_kwargs: Optional[dict[str, Any]] = None,
        embed_kwargs: Optional[dict[str, Any]] = None,
    ) -> "RetrievalResult":
        from nemo_retriever.models.llm.types import RetrievalResult

        hits = self.query(query, top_k=top_k, vdb_kwargs=vdb_kwargs, embed_kwargs=embed_kwargs)

        chunks: list[str] = []
        metadata: list[dict[str, Any]] = []
        for hit in hits:
            chunks.append(str(hit.get("text", "")))
            metadata.append({k: v for k, v in hit.items() if k != "text"})
        return RetrievalResult(chunks=chunks, metadata=metadata)

    def retrieve_batch(
        self,
        queries: Sequence[str],
        *,
        top_k: Optional[int] = None,
        vdb_kwargs: Optional[dict[str, Any]] = None,
        embed_kwargs: Optional[dict[str, Any]] = None,
    ) -> list["RetrievalResult"]:
        from nemo_retriever.models.llm.types import RetrievalResult

        query_texts = [str(q) for q in queries]
        if not query_texts:
            return []

        hits_per_query = self.queries(query_texts, top_k=top_k, vdb_kwargs=vdb_kwargs, embed_kwargs=embed_kwargs)

        results: list[RetrievalResult] = []
        for hits in hits_per_query:
            chunks = [str(hit.get("text", "")) for hit in hits]
            metadata = [{k: v for k, v in hit.items() if k != "text"} for hit in hits]
            results.append(RetrievalResult(chunks=chunks, metadata=metadata))
        return results

    def answer(
        self,
        query: str,
        *,
        llm: "LLMClient",
        judge: Optional["AnswerJudge"] = None,
        reference: Optional[str] = None,
        top_k: Optional[int] = None,
        reasoning_enabled: Optional[bool] = None,
        vdb_kwargs: Optional[dict[str, Any]] = None,
        embed_kwargs: Optional[dict[str, Any]] = None,
    ) -> "AnswerResult":
        from nemo_retriever.models.llm.types import (
            AnswerRequest,
            build_answer_result,
        )

        if judge is not None and reference is None:
            raise ValueError("judge requires reference")

        answer_req = AnswerRequest(
            query=query,
            top_k=int(top_k) if top_k is not None else int(self.top_k),
            reasoning_enabled=reasoning_enabled,
            reference=reference,
            judge_enabled=judge is not None,
        )
        retrieved = self.retrieve(
            answer_req.query,
            top_k=answer_req.top_k,
            vdb_kwargs=vdb_kwargs,
            embed_kwargs=embed_kwargs,
        )

        generate_kwargs: dict[str, Any] = {}
        if answer_req.reasoning_enabled is not None:
            generate_kwargs["reasoning_enabled"] = answer_req.reasoning_enabled
        gen = llm.generate(answer_req.query, retrieved.chunks, **generate_kwargs)

        return build_answer_result(
            query=answer_req.query,
            retrieval=retrieved,
            generation=gen,
            reference=answer_req.reference,
            judge=judge if answer_req.judge_enabled else None,
        )

    def pipeline(self, *, top_k: Optional[int] = None) -> "RetrieverPipelineBuilder":
        effective_top_k = int(top_k) if top_k is not None else int(self.top_k)
        return RetrieverPipelineBuilder(self, top_k=effective_top_k)

    def generate_sql(self, query: str) -> str:
        from nemo_retriever.tabular_data.retrieval import generate_sql

        return generate_sql(query)

embed_kwargs = field(default_factory=dict) class-attribute instance-attribute

graph = None class-attribute instance-attribute

Custom :class:~nemo_retriever.graph.pipeline_graph.Graph. When set, embed_kwargs / vdb_kwargs default-graph fields are ignored for construction (you still pass execute kwargs).

rerank = False class-attribute instance-attribute

When True, append :class:~nemo_retriever.rerank.rerank.NemotronRerankActor after retrieval.

rerank_kwargs = field(default_factory=dict) class-attribute instance-attribute

run_mode = 'local' class-attribute instance-attribute

local uses archetype batch embed resolution; service forces CPU HTTP embed.

top_k = 10 class-attribute instance-attribute

vdb_kwargs = field(default_factory=dict) class-attribute instance-attribute

__init__(run_mode='local', top_k=10, rerank=False, graph=None, embed_kwargs=dict(), vdb_kwargs=dict(), rerank_kwargs=dict())

__post_init__()

Source code in nemo_retriever/graph/retriever.py

def __post_init__(self) -> None:
    if self.run_mode not in ("local", "service"):
        raise ValueError("run_mode must be 'local' or 'service'")

answer(query, *, llm, judge=None, reference=None, top_k=None, reasoning_enabled=None, vdb_kwargs=None, embed_kwargs=None)

Source code in nemo_retriever/graph/retriever.py

def answer(
    self,
    query: str,
    *,
    llm: "LLMClient",
    judge: Optional["AnswerJudge"] = None,
    reference: Optional[str] = None,
    top_k: Optional[int] = None,
    reasoning_enabled: Optional[bool] = None,
    vdb_kwargs: Optional[dict[str, Any]] = None,
    embed_kwargs: Optional[dict[str, Any]] = None,
) -> "AnswerResult":
    from nemo_retriever.models.llm.types import (
        AnswerRequest,
        build_answer_result,
    )

    if judge is not None and reference is None:
        raise ValueError("judge requires reference")

    answer_req = AnswerRequest(
        query=query,
        top_k=int(top_k) if top_k is not None else int(self.top_k),
        reasoning_enabled=reasoning_enabled,
        reference=reference,
        judge_enabled=judge is not None,
    )
    retrieved = self.retrieve(
        answer_req.query,
        top_k=answer_req.top_k,
        vdb_kwargs=vdb_kwargs,
        embed_kwargs=embed_kwargs,
    )

    generate_kwargs: dict[str, Any] = {}
    if answer_req.reasoning_enabled is not None:
        generate_kwargs["reasoning_enabled"] = answer_req.reasoning_enabled
    gen = llm.generate(answer_req.query, retrieved.chunks, **generate_kwargs)

    return build_answer_result(
        query=answer_req.query,
        retrieval=retrieved,
        generation=gen,
        reference=answer_req.reference,
        judge=judge if answer_req.judge_enabled else None,
    )

generate_sql(query)

Source code in nemo_retriever/graph/retriever.py

def generate_sql(self, query: str) -> str:
    from nemo_retriever.tabular_data.retrieval import generate_sql

    return generate_sql(query)

pipeline(*, top_k=None)

Source code in nemo_retriever/graph/retriever.py

def pipeline(self, *, top_k: Optional[int] = None) -> "RetrieverPipelineBuilder":
    effective_top_k = int(top_k) if top_k is not None else int(self.top_k)
    return RetrieverPipelineBuilder(self, top_k=effective_top_k)

queries(queries, *, top_k=None, candidate_k=None, page_dedup=False, content_types=None, vdb_kwargs=None, embed_kwargs=None)

Run retrieval for multiple query strings and return shaped hits.

top_k is the final number of hits to return. candidate_k is the wider pre-filter/pre-dedup candidate pool and must be greater than or equal to top_k. Increase it when page deduplication or content-type filtering would otherwise reduce the final hit count. page_dedup keeps the first hit per document page. content_types accepts a comma-separated string or sequence of content types to keep, such as "text,table", and normalizes values to the canonical content types stored in hit metadata. Hits with missing or unknown content types are excluded while this filter is active. Page deduplication and content-type filtering are applied after vector retrieval, preserving retriever ranking order.

Source code in nemo_retriever/graph/retriever.py

def queries(
    self,
    queries: Sequence[str],
    *,
    top_k: Optional[int] = None,
    candidate_k: Optional[int] = None,
    page_dedup: bool = False,
    content_types: str | Sequence[str] | None = None,
    vdb_kwargs: Optional[dict[str, Any]] = None,
    embed_kwargs: Optional[dict[str, Any]] = None,
) -> list[list[RetrievalHit]]:
    """Run retrieval for multiple query strings and return shaped hits.

    ``top_k`` is the final number of hits to return. ``candidate_k`` is the
    wider pre-filter/pre-dedup candidate pool and must be greater than or
    equal to ``top_k``. Increase it when page deduplication or content-type
    filtering would otherwise reduce the final hit count. ``page_dedup``
    keeps the first hit per document page. ``content_types`` accepts a
    comma-separated string or sequence of content types to keep, such as
    ``"text,table"``, and normalizes values to the canonical content types
    stored in hit metadata. Hits with missing or unknown content types are
    excluded while this filter is active. Page deduplication and
    content-type filtering are applied after vector retrieval, preserving
    retriever ranking order.
    """
    query_texts = [str(q) for q in queries]
    if not query_texts:
        return []

    effective_top_k = int(top_k) if top_k is not None else int(self.top_k)
    candidate_top_k = int(candidate_k) if candidate_k is not None else effective_top_k
    if candidate_top_k < effective_top_k:
        raise ValueError(
            f"candidate_k ({candidate_top_k}) must be greater than or equal to top_k ({effective_top_k})."
        )
    refine = self._refine_factor()
    retrieval_top_k = candidate_top_k * refine if self.rerank else candidate_top_k

    vdb_call_kwargs = dict(vdb_kwargs or {})
    index_model: str | None = None
    index_revision: str | None = None
    explicit_model = self._embedding_model_from_kwargs(embed_kwargs) or self._embedding_model_from_kwargs(
        self.embed_kwargs
    )
    if self.graph is None and explicit_model is None:
        metadata_reader = RetrieveVdbOperator(**_coerce_vdb_init(self.vdb_kwargs))
        index_model = metadata_reader.get_index_metadata("embedding_model_name", **vdb_call_kwargs)
        index_revision = metadata_reader.get_index_metadata("embedding_model_revision", **vdb_call_kwargs)

    lancedb_mode = self._resolve_lancedb_query_mode(vdb_call_kwargs)
    for key in _QUERY_ROUTING_VDB_KWARGS:
        vdb_call_kwargs.pop(key, None)
    if lancedb_mode is not None:
        mode, caps, uri, table_name, has_mode_override = lancedb_mode
        if mode == "sparse":
            raw_hits = self._execute_sparse_lancedb_queries(
                query_texts,
                retrieval_top_k=retrieval_top_k,
                vdb_call_kwargs=vdb_call_kwargs,
                caps=caps,
                uri=uri,
                table_name=table_name,
            )
            return [
                shape_query_hits(
                    hits,
                    top_k=effective_top_k,
                    page_dedup=page_dedup,
                    content_types=content_types,
                )
                for hits in raw_hits
            ]
        if mode == "hybrid":
            vdb_call_kwargs["hybrid"] = True
        elif mode == "dense" and has_mode_override:
            vdb_call_kwargs["hybrid"] = False
        if caps.vector_column and caps.vector_column != "vector":
            vdb_call_kwargs.setdefault("vector_column_name", caps.vector_column)
    if self.graph is None:
        embed_kwargs = self._resolve_embed_kwargs(index_model, embed_kwargs, index_revision)

    raw_hits = self._execute_queries_graph(
        query_texts,
        effective_top_k=candidate_top_k,
        retrieval_top_k=retrieval_top_k,
        vdb_call_kwargs=vdb_call_kwargs,
        embed_extra=embed_kwargs,
    )
    return [
        shape_query_hits(
            hits,
            top_k=effective_top_k,
            page_dedup=page_dedup,
            content_types=content_types,
        )
        for hits in raw_hits
    ]

query(query, *, top_k=None, candidate_k=None, page_dedup=False, content_types=None, vdb_kwargs=None, embed_kwargs=None)

Run one retrieval query and return shaped hits.

top_k is the final number of hits to return. candidate_k is the wider pre-filter/pre-dedup candidate pool and must be greater than or equal to top_k. Increase it when page deduplication or content-type filtering would otherwise reduce the final hit count. page_dedup keeps the first hit per document page. content_types accepts a comma-separated string or sequence of content types to keep, such as "text,table", and normalizes values to the canonical content types stored in hit metadata. Hits with missing or unknown content types are excluded while this filter is active. Page deduplication and content-type filtering are applied after vector retrieval, preserving retriever ranking order.

Source code in nemo_retriever/graph/retriever.py

def query(
    self,
    query: str,
    *,
    top_k: Optional[int] = None,
    candidate_k: Optional[int] = None,
    page_dedup: bool = False,
    content_types: str | Sequence[str] | None = None,
    vdb_kwargs: Optional[dict[str, Any]] = None,
    embed_kwargs: Optional[dict[str, Any]] = None,
) -> list[RetrievalHit]:
    """Run one retrieval query and return shaped hits.

    ``top_k`` is the final number of hits to return. ``candidate_k`` is the
    wider pre-filter/pre-dedup candidate pool and must be greater than or
    equal to ``top_k``. Increase it when page deduplication or content-type
    filtering would otherwise reduce the final hit count. ``page_dedup``
    keeps the first hit per document page. ``content_types`` accepts a
    comma-separated string or sequence of content types to keep, such as
    ``"text,table"``, and normalizes values to the canonical content types
    stored in hit metadata. Hits with missing or unknown content types are
    excluded while this filter is active. Page deduplication and
    content-type filtering are applied after vector retrieval, preserving
    retriever ranking order.
    """
    return self.queries(
        [query],
        top_k=top_k,
        candidate_k=candidate_k,
        page_dedup=page_dedup,
        content_types=content_types,
        vdb_kwargs=vdb_kwargs,
        embed_kwargs=embed_kwargs,
    )[0]

retrieve(query, top_k=None, *, vdb_kwargs=None, embed_kwargs=None)

Source code in nemo_retriever/graph/retriever.py

def retrieve(
    self,
    query: str,
    top_k: Optional[int] = None,
    *,
    vdb_kwargs: Optional[dict[str, Any]] = None,
    embed_kwargs: Optional[dict[str, Any]] = None,
) -> "RetrievalResult":
    from nemo_retriever.models.llm.types import RetrievalResult

    hits = self.query(query, top_k=top_k, vdb_kwargs=vdb_kwargs, embed_kwargs=embed_kwargs)

    chunks: list[str] = []
    metadata: list[dict[str, Any]] = []
    for hit in hits:
        chunks.append(str(hit.get("text", "")))
        metadata.append({k: v for k, v in hit.items() if k != "text"})
    return RetrievalResult(chunks=chunks, metadata=metadata)

retrieve_batch(queries, *, top_k=None, vdb_kwargs=None, embed_kwargs=None)

Source code in nemo_retriever/graph/retriever.py

def retrieve_batch(
    self,
    queries: Sequence[str],
    *,
    top_k: Optional[int] = None,
    vdb_kwargs: Optional[dict[str, Any]] = None,
    embed_kwargs: Optional[dict[str, Any]] = None,
) -> list["RetrievalResult"]:
    from nemo_retriever.models.llm.types import RetrievalResult

    query_texts = [str(q) for q in queries]
    if not query_texts:
        return []

    hits_per_query = self.queries(query_texts, top_k=top_k, vdb_kwargs=vdb_kwargs, embed_kwargs=embed_kwargs)

    results: list[RetrievalResult] = []
    for hits in hits_per_query:
        chunks = [str(hit.get("text", "")) for hit in hits]
        metadata = [{k: v for k, v in hit.items() if k != "text"} for hit in hits]
        results.append(RetrievalResult(chunks=chunks, metadata=metadata))
    return results

RetrieverPipelineBuilder

Fluent builder for live-RAG batch operator graphs.

Returned from :meth:Retriever.pipeline. Each builder method appends an :class:~nemo_retriever.evaluation.eval_operator.EvalOperator to an internal list; :meth:run composes them into a graph via the existing >> chaining and executes it on a DataFrame built from the provided queries.

Example

builder = retriever.pipeline() # doctest: +SKIP df = builder.generate(llm).score().judge(judge).run( # doctest: +SKIP ... queries=["q1", "q2"], ... reference=["r1", "r2"], ... )

Source code in nemo_retriever/graph/retriever.py

class RetrieverPipelineBuilder:
    """Fluent builder for live-RAG batch operator graphs.

    Returned from :meth:`Retriever.pipeline`.  Each builder method appends
    an :class:`~nemo_retriever.evaluation.eval_operator.EvalOperator` to an
    internal list; :meth:`run` composes them into a graph via the existing
    ``>>`` chaining and executes it on a DataFrame built from the provided
    queries.

    Example:
        >>> builder = retriever.pipeline()  # doctest: +SKIP
        >>> df = builder.generate(llm).score().judge(judge).run(  # doctest: +SKIP
        ...     queries=["q1", "q2"],
        ...     reference=["r1", "r2"],
        ... )
    """

    def __init__(self, retriever: "Retriever", *, top_k: int = 5) -> None:
        self._retriever = retriever
        self._top_k = int(top_k)
        self._steps: list[Any] = []

    def with_retrieval(self, *, top_k: int) -> "RetrieverPipelineBuilder":
        """Override the ``top_k`` used for the live retrieval source."""
        self._top_k = int(top_k)
        return self

    def generate(
        self,
        llm: Optional[Any] = None,
        /,
        *,
        model: Optional[str] = None,
        **kwargs: Any,
    ) -> "RetrieverPipelineBuilder":
        """Append a :class:`QAGenerationOperator` step.

        Accepts either a pre-built
        :class:`~nemo_retriever.llm.clients.LiteLLMClient` (whose transport
        and sampling params are unpacked onto the operator) or the flat
        ``model=..., api_base=..., ...`` kwargs forwarded to the operator
        constructor directly.

        Raises:
            ValueError: If neither ``llm`` nor ``model`` is provided.
        """
        from nemo_retriever.tools.evaluation.generation import QAGenerationOperator

        if llm is None and model is None:
            raise ValueError("generate() requires either llm= or model=")

        if llm is not None:
            transport = llm.transport
            sampling = llm.sampling
            operator = QAGenerationOperator(
                model=transport.model,
                api_base=transport.api_base,
                api_key=transport.api_key,
                temperature=sampling.temperature,
                top_p=sampling.top_p,
                max_tokens=sampling.max_tokens,
                extra_params=dict(transport.extra_params) if transport.extra_params else None,
                num_retries=transport.num_retries,
                timeout=transport.timeout,
                rag_system_prompt=transport.rag_system_prompt,
                rag_system_prompt_prefix=transport.rag_system_prompt_prefix,
                reasoning_enabled=getattr(transport, "reasoning_enabled", True),
            )
        else:
            operator = QAGenerationOperator(model=model, **kwargs)

        self._steps.append(operator)
        return self

    def score(self) -> "RetrieverPipelineBuilder":
        """Append a :class:`ScoringOperator` step (Tier 1 + Tier 2)."""
        from nemo_retriever.operators.graph_ops.scoring_operator import ScoringOperator

        self._steps.append(ScoringOperator())
        return self

    def judge(
        self,
        judge: Optional[Any] = None,
        /,
        *,
        model: Optional[str] = None,
        **kwargs: Any,
    ) -> "RetrieverPipelineBuilder":
        """Append a :class:`JudgingOperator` step (Tier 3).

        Accepts either a pre-built
        :class:`~nemo_retriever.llm.clients.judge.LLMJudge` (whose transport params
        are unpacked onto the operator) or the flat ``model=...`` kwargs
        forwarded to the operator constructor.

        Raises:
            ValueError: If neither ``judge`` nor ``model`` is provided.
        """
        from nemo_retriever.tools.evaluation.judging import JudgingOperator

        if judge is None and model is None:
            raise ValueError("judge() requires either judge= or model=")

        if judge is not None:
            transport = judge.transport
            operator = JudgingOperator(
                model=transport.model,
                api_base=transport.api_base,
                api_key=transport.api_key,
                extra_params=dict(transport.extra_params) if transport.extra_params else None,
                num_retries=transport.num_retries,
                timeout=transport.timeout,
            )
        else:
            operator = JudgingOperator(model=model, **kwargs)

        self._steps.append(operator)
        return self

    def run(
        self,
        queries: Any,
        *,
        reference: Any = None,
    ) -> "pd.DataFrame":
        """Execute the composed graph on ``queries``.

        Args:
            queries: A single query string, a list of query strings, or a
                pre-built ``pandas.DataFrame`` (which must contain a
                ``query`` column and, when judging/scoring, a
                ``reference_answer`` column).
            reference: Optional ground-truth answer(s).  Accepts a single
                string (applied to all queries), a list aligned with
                ``queries``, or ``None``.  Ignored when ``queries`` is
                already a DataFrame.

        Returns:
            A ``pandas.DataFrame`` with the columns contributed by each
            appended step (always ``query``, ``context``, and
            ``context_metadata``; plus ``answer``/``latency_s``/... when
            ``.generate()`` ran, and so on).

        Raises:
            ValueError: If ``reference`` is a list whose length does not
                match ``queries``.
        """
        import pandas as pd

        from nemo_retriever.tools.evaluation.live_retrieval import LiveRetrievalOperator

        if isinstance(queries, str):
            query_list = [queries]
            df = pd.DataFrame({"query": query_list})
            if reference is not None:
                refs = reference if isinstance(reference, list) else [reference]
                if len(refs) != len(query_list):
                    raise ValueError("reference length must match queries length")
                df["reference_answer"] = refs
        elif isinstance(queries, list):
            df = pd.DataFrame({"query": list(queries)})
            if reference is not None:
                refs = reference if isinstance(reference, list) else [reference] * len(queries)
                if len(refs) != len(queries):
                    raise ValueError("reference length must match queries length")
                df["reference_answer"] = refs
        elif isinstance(queries, pd.DataFrame):
            df = queries.copy()
        else:
            raise TypeError("queries must be a str, list[str], or pandas.DataFrame; " f"got {type(queries).__name__}")

        retrieval_op = LiveRetrievalOperator(self._retriever, top_k=self._top_k)
        if not self._steps:
            out = retrieval_op.run(df)
        else:
            graph = retrieval_op
            for step in self._steps:
                graph = graph >> step
            # Linear live-RAG pipelines have exactly one leaf.
            leaves = graph.execute(df)
            if len(leaves) != 1:
                raise RuntimeError(f"Unexpected pipeline fan-out: got {len(leaves)} leaf outputs")
            out = leaves[0]

        # Expose the generation failure rate on ``df.attrs`` for downstream aggregators.
        if "gen_error" in out.columns and len(out) > 0:
            out.attrs["generation_failure_rate"] = float(out["gen_error"].notna().mean())

        return out

__init__(retriever, *, top_k=5)

Source code in nemo_retriever/graph/retriever.py

def __init__(self, retriever: "Retriever", *, top_k: int = 5) -> None:
    self._retriever = retriever
    self._top_k = int(top_k)
    self._steps: list[Any] = []

generate(llm=None, /, *, model=None, **kwargs)

Append a :class:QAGenerationOperator step.

Accepts either a pre-built :class:~nemo_retriever.llm.clients.LiteLLMClient (whose transport and sampling params are unpacked onto the operator) or the flat model=..., api_base=..., ... kwargs forwarded to the operator constructor directly.

Raises:

Type	Description
ValueError	If neither llm nor model is provided.

Source code in nemo_retriever/graph/retriever.py

def generate(
    self,
    llm: Optional[Any] = None,
    /,
    *,
    model: Optional[str] = None,
    **kwargs: Any,
) -> "RetrieverPipelineBuilder":
    """Append a :class:`QAGenerationOperator` step.

    Accepts either a pre-built
    :class:`~nemo_retriever.llm.clients.LiteLLMClient` (whose transport
    and sampling params are unpacked onto the operator) or the flat
    ``model=..., api_base=..., ...`` kwargs forwarded to the operator
    constructor directly.

    Raises:
        ValueError: If neither ``llm`` nor ``model`` is provided.
    """
    from nemo_retriever.tools.evaluation.generation import QAGenerationOperator

    if llm is None and model is None:
        raise ValueError("generate() requires either llm= or model=")

    if llm is not None:
        transport = llm.transport
        sampling = llm.sampling
        operator = QAGenerationOperator(
            model=transport.model,
            api_base=transport.api_base,
            api_key=transport.api_key,
            temperature=sampling.temperature,
            top_p=sampling.top_p,
            max_tokens=sampling.max_tokens,
            extra_params=dict(transport.extra_params) if transport.extra_params else None,
            num_retries=transport.num_retries,
            timeout=transport.timeout,
            rag_system_prompt=transport.rag_system_prompt,
            rag_system_prompt_prefix=transport.rag_system_prompt_prefix,
            reasoning_enabled=getattr(transport, "reasoning_enabled", True),
        )
    else:
        operator = QAGenerationOperator(model=model, **kwargs)

    self._steps.append(operator)
    return self

judge(judge=None, /, *, model=None, **kwargs)

Append a :class:JudgingOperator step (Tier 3).

Accepts either a pre-built :class:~nemo_retriever.llm.clients.judge.LLMJudge (whose transport params are unpacked onto the operator) or the flat model=... kwargs forwarded to the operator constructor.

Raises:

Type	Description
ValueError	If neither judge nor model is provided.

Source code in nemo_retriever/graph/retriever.py

def judge(
    self,
    judge: Optional[Any] = None,
    /,
    *,
    model: Optional[str] = None,
    **kwargs: Any,
) -> "RetrieverPipelineBuilder":
    """Append a :class:`JudgingOperator` step (Tier 3).

    Accepts either a pre-built
    :class:`~nemo_retriever.llm.clients.judge.LLMJudge` (whose transport params
    are unpacked onto the operator) or the flat ``model=...`` kwargs
    forwarded to the operator constructor.

    Raises:
        ValueError: If neither ``judge`` nor ``model`` is provided.
    """
    from nemo_retriever.tools.evaluation.judging import JudgingOperator

    if judge is None and model is None:
        raise ValueError("judge() requires either judge= or model=")

    if judge is not None:
        transport = judge.transport
        operator = JudgingOperator(
            model=transport.model,
            api_base=transport.api_base,
            api_key=transport.api_key,
            extra_params=dict(transport.extra_params) if transport.extra_params else None,
            num_retries=transport.num_retries,
            timeout=transport.timeout,
        )
    else:
        operator = JudgingOperator(model=model, **kwargs)

    self._steps.append(operator)
    return self

run(queries, *, reference=None)

Execute the composed graph on queries.

Parameters:

Name	Type	Description	Default
queries	Any	A single query string, a list of query strings, or a pre-built pandas.DataFrame (which must contain a query column and, when judging/scoring, a reference_answer column).	required
reference	Any	Optional ground-truth answer(s). Accepts a single string (applied to all queries), a list aligned with queries, or None. Ignored when queries is already a DataFrame.	None

Returns:

Type	Description
'pd.DataFrame'	A pandas.DataFrame with the columns contributed by each
'pd.DataFrame'	appended step (always query, context, and
'pd.DataFrame'	context_metadata; plus answer/latency_s/... when
'pd.DataFrame'	.generate() ran, and so on).

Raises:

Type	Description
ValueError	If reference is a list whose length does not match queries.

Source code in nemo_retriever/graph/retriever.py

def run(
    self,
    queries: Any,
    *,
    reference: Any = None,
) -> "pd.DataFrame":
    """Execute the composed graph on ``queries``.

    Args:
        queries: A single query string, a list of query strings, or a
            pre-built ``pandas.DataFrame`` (which must contain a
            ``query`` column and, when judging/scoring, a
            ``reference_answer`` column).
        reference: Optional ground-truth answer(s).  Accepts a single
            string (applied to all queries), a list aligned with
            ``queries``, or ``None``.  Ignored when ``queries`` is
            already a DataFrame.

    Returns:
        A ``pandas.DataFrame`` with the columns contributed by each
        appended step (always ``query``, ``context``, and
        ``context_metadata``; plus ``answer``/``latency_s``/... when
        ``.generate()`` ran, and so on).

    Raises:
        ValueError: If ``reference`` is a list whose length does not
            match ``queries``.
    """
    import pandas as pd

    from nemo_retriever.tools.evaluation.live_retrieval import LiveRetrievalOperator

    if isinstance(queries, str):
        query_list = [queries]
        df = pd.DataFrame({"query": query_list})
        if reference is not None:
            refs = reference if isinstance(reference, list) else [reference]
            if len(refs) != len(query_list):
                raise ValueError("reference length must match queries length")
            df["reference_answer"] = refs
    elif isinstance(queries, list):
        df = pd.DataFrame({"query": list(queries)})
        if reference is not None:
            refs = reference if isinstance(reference, list) else [reference] * len(queries)
            if len(refs) != len(queries):
                raise ValueError("reference length must match queries length")
            df["reference_answer"] = refs
    elif isinstance(queries, pd.DataFrame):
        df = queries.copy()
    else:
        raise TypeError("queries must be a str, list[str], or pandas.DataFrame; " f"got {type(queries).__name__}")

    retrieval_op = LiveRetrievalOperator(self._retriever, top_k=self._top_k)
    if not self._steps:
        out = retrieval_op.run(df)
    else:
        graph = retrieval_op
        for step in self._steps:
            graph = graph >> step
        # Linear live-RAG pipelines have exactly one leaf.
        leaves = graph.execute(df)
        if len(leaves) != 1:
            raise RuntimeError(f"Unexpected pipeline fan-out: got {len(leaves)} leaf outputs")
        out = leaves[0]

    # Expose the generation failure rate on ``df.attrs`` for downstream aggregators.
    if "gen_error" in out.columns and len(out) > 0:
        out.attrs["generation_failure_rate"] = float(out["gen_error"].notna().mean())

    return out

score()

Append a :class:ScoringOperator step (Tier 1 + Tier 2).

Source code in nemo_retriever/graph/retriever.py

def score(self) -> "RetrieverPipelineBuilder":
    """Append a :class:`ScoringOperator` step (Tier 1 + Tier 2)."""
    from nemo_retriever.operators.graph_ops.scoring_operator import ScoringOperator

    self._steps.append(ScoringOperator())
    return self

with_retrieval(*, top_k)

Override the top_k used for the live retrieval source.

Source code in nemo_retriever/graph/retriever.py

def with_retrieval(self, *, top_k: int) -> "RetrieverPipelineBuilder":
    """Override the ``top_k`` used for the live retrieval source."""
    self._top_k = int(top_k)
    return self

IngestorRunMode = Literal['inprocess', 'batch', 'service'] module-attribute

MetaJoinKey = Literal['auto', 'source_id', 'source_name'] module-attribute

NO_API_KEY = '' module-attribute

SPLIT_CONFIG_VALID_KEYS = frozenset({'text', 'html', 'pdf', 'audio', 'image', 'video'}) module-attribute

__all__ = ['ASRParams', 'AudioChunkParams', 'AudioVisualFuseParams', 'BatchTuningParams', 'CaptionParams', 'ChartParams', 'DedupParams', 'EmbedParams', 'ExtractParams', 'GpuAllocationParams', 'HtmlChunkParams', 'IngestExecuteParams', 'IngestorCreateParams', 'IngestorRunMode', 'LanceDbParams', 'LLMInferenceParams', 'LLMRemoteClientParams', 'LLMSamplingOverrides', 'ModelRuntimeParams', 'NO_API_KEY', 'OcrParams', 'PageElementsParams', 'PdfSplitParams', 'RemoteInvokeParams', 'RemoteRetryParams', 'SPLIT_CONFIG_VALID_KEYS', 'StoreParams', 'TabularExtractParams', 'TableParams', 'TextChunkParams', 'TextGenerationParams', 'MetaJoinKey', 'VdbUploadParams', 'VideoFrameParams', 'VideoFrameTextDedupParams', 'WebhookParams', 'build_embed_option_kwargs', 'resolve_split_params'] module-attribute

ASRParams

Bases: _ParamsModel

Params for ASR (Parakeet/Riva gRPC or local transformers backend).

Choice of remote-NIM vs local-model is made by the :class:ASRActor archetype (CPU variant = remote, GPU variant = local), not by a flag here. Pass audio_endpoints to force the remote variant on any host; leave them empty to let the archetype pick GPU (local) when a GPU is present and fall back to remote (NVCF default) when not.

Source code in nemo_retriever/common/params/models.py

class ASRParams(_ParamsModel):
    """Params for ASR (Parakeet/Riva gRPC or local transformers backend).

    Choice of remote-NIM vs local-model is made by the :class:`ASRActor`
    archetype (CPU variant = remote, GPU variant = local), not by a flag here.
    Pass ``audio_endpoints`` to force the remote variant on any host; leave
    them empty to let the archetype pick GPU (local) when a GPU is present
    and fall back to remote (NVCF default) when not.
    """

    audio_endpoints: Tuple[Optional[str], Optional[str]] = (None, None)
    audio_infer_protocol: str = "grpc"
    # ``auto``: streaming (online) for NVCF; offline recognize for other gRPC
    # endpoints (e.g. Helm Parakeet NIM with ``mode=ofl``).
    audio_infer_mode: Literal["auto", "online", "offline"] = "auto"
    function_id: Optional[str] = None
    auth_token: Optional[str] = None
    segment_audio: bool = False

audio_endpoints = (None, None) class-attribute instance-attribute

audio_infer_mode = 'auto' class-attribute instance-attribute

audio_infer_protocol = 'grpc' class-attribute instance-attribute

auth_token = None class-attribute instance-attribute

function_id = None class-attribute instance-attribute

segment_audio = False class-attribute instance-attribute

AudioChunkParams

Bases: _ParamsModel

Params for media chunking (audio/video split). Aligned with nemo_retriever.api dataloader.

Set enabled=False (when wired through VideoSplitActor) to skip audio chunking and ASR on a video pipeline — useful for visual-only recall benchmarks. MediaChunkActor ignores this flag for the audio-only pipeline since chunking is the whole point there.

audio_only=True on a video input extracts only the audio track, runs ASR over it, and skips the visual branch entirely — no frame extraction, no OCR, no audio/visual fusion.

video_audio_separate is accepted for compatibility but ignored by MediaChunkActor on video inputs: this ASR chunking path always demuxes videos to ASR-safe audio chunks and does not emit video-container chunks. Use VideoSplitActor or the video pipeline when you need audio+visual video processing.

Source code in nemo_retriever/common/params/models.py

class AudioChunkParams(_ParamsModel):
    """Params for media chunking (audio/video split). Aligned with `nemo_retriever.api` dataloader.

    Set ``enabled=False`` (when wired through ``VideoSplitActor``) to skip
    audio chunking and ASR on a video pipeline — useful for visual-only
    recall benchmarks. ``MediaChunkActor`` ignores this flag for the
    audio-only pipeline since chunking is the whole point there.

    ``audio_only=True`` on a video input extracts only the audio track,
    runs ASR over it, and skips the visual branch entirely — no frame
    extraction, no OCR, no audio/visual fusion.

    ``video_audio_separate`` is accepted for compatibility but ignored by
    ``MediaChunkActor`` on video inputs: this ASR chunking path always demuxes
    videos to ASR-safe audio chunks and does not emit video-container chunks.
    Use ``VideoSplitActor`` or the video pipeline when you need audio+visual
    video processing.
    """

    enabled: bool = True
    split_type: Literal["size", "time", "frame"] = "size"
    split_interval: int = 450
    audio_only: bool = False
    video_audio_separate: bool = False

audio_only = False class-attribute instance-attribute

enabled = True class-attribute instance-attribute

split_interval = 450 class-attribute instance-attribute

split_type = 'size' class-attribute instance-attribute

video_audio_separate = False class-attribute instance-attribute

AudioVisualFuseParams

Bases: _ParamsModel

Toggle for :class:~nemo_retriever.video.AudioVisualFuser.

Source code in nemo_retriever/common/params/models.py

class AudioVisualFuseParams(_ParamsModel):
    """Toggle for :class:`~nemo_retriever.video.AudioVisualFuser`."""

    enabled: bool = True

enabled = True class-attribute instance-attribute

BatchTuningParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class BatchTuningParams(_ParamsModel):
    debug_run_id: str = "unknown"
    pdf_split_batch_size: int = 1
    pdf_extract_batch_size: int = 4
    pdf_extract_num_cpus: float = 2
    pdf_extract_workers: Optional[int] = None
    page_elements_batch_size: int = 24
    detect_batch_size: int = 24
    ocr_inference_batch_size: Optional[int] = None
    page_elements_workers: Optional[int] = None
    ocr_workers: Optional[int] = None
    detect_workers: Optional[int] = None
    page_elements_cpus_per_actor: float = 1
    ocr_cpus_per_actor: float = 1
    table_structure_workers: Optional[int] = None
    table_structure_batch_size: Optional[int] = None
    table_structure_cpus_per_actor: float = 1
    embed_workers: Optional[int] = None
    embed_batch_size: int = 32
    embed_cpus_per_actor: float = 1
    gpu_page_elements: Optional[float] = None
    gpu_ocr: Optional[float] = None
    gpu_table_structure: Optional[float] = None
    gpu_embed: Optional[float] = None
    nemotron_parse_workers: Optional[int] = None
    gpu_nemotron_parse: Optional[float] = None
    nemotron_parse_batch_size: Optional[int] = None
    store_workers: Optional[int] = None
    inference_batch_size: int = 8

debug_run_id = 'unknown' class-attribute instance-attribute

detect_batch_size = 24 class-attribute instance-attribute

detect_workers = None class-attribute instance-attribute

embed_batch_size = 32 class-attribute instance-attribute

embed_cpus_per_actor = 1 class-attribute instance-attribute

embed_workers = None class-attribute instance-attribute

gpu_embed = None class-attribute instance-attribute

gpu_nemotron_parse = None class-attribute instance-attribute

gpu_ocr = None class-attribute instance-attribute

gpu_page_elements = None class-attribute instance-attribute

gpu_table_structure = None class-attribute instance-attribute

inference_batch_size = 8 class-attribute instance-attribute

nemotron_parse_batch_size = None class-attribute instance-attribute

nemotron_parse_workers = None class-attribute instance-attribute

ocr_cpus_per_actor = 1 class-attribute instance-attribute

ocr_inference_batch_size = None class-attribute instance-attribute

ocr_workers = None class-attribute instance-attribute

page_elements_batch_size = 24 class-attribute instance-attribute

page_elements_cpus_per_actor = 1 class-attribute instance-attribute

page_elements_workers = None class-attribute instance-attribute

pdf_extract_batch_size = 4 class-attribute instance-attribute

pdf_extract_num_cpus = 2 class-attribute instance-attribute

pdf_extract_workers = None class-attribute instance-attribute

pdf_split_batch_size = 1 class-attribute instance-attribute

store_workers = None class-attribute instance-attribute

table_structure_batch_size = None class-attribute instance-attribute

table_structure_cpus_per_actor = 1 class-attribute instance-attribute

table_structure_workers = None class-attribute instance-attribute

CaptionParams

Bases: LLMInferenceParams

Source code in nemo_retriever/common/params/models.py

class CaptionParams(LLMInferenceParams):
    endpoint_url: Optional[str] = None
    model_name: str = Field(
        default=DEFAULT_LOCAL_CAPTION_MODEL_ID,
        description=(
            "Caption model identifier. The default local BF16 checkpoint has approximately 62 GiB of weights; "
            "set this explicitly to select a smaller local model or an API model for a remote endpoint."
        ),
    )
    api_key: Optional[str] = None
    prompt: str = "Caption the content of this image:"
    system_prompt: Optional[str] = "/no_think"
    batch_size: int = 8
    device: Optional[str] = None
    hf_cache_dir: Optional[str] = None
    context_text_max_chars: int = 0
    tensor_parallel_size: int = 1
    gpu_memory_utilization: float = 0.5
    caption_infographics: bool = False
    extra_body: dict[str, Any] = Field(default_factory=dict)

    @field_validator("temperature")
    @classmethod
    def _require_temperature(cls, value: Optional[float]) -> float:
        if value is None:
            raise ValueError("temperature cannot be None for captioning")
        return value

api_key = None class-attribute instance-attribute

batch_size = 8 class-attribute instance-attribute

caption_infographics = False class-attribute instance-attribute

context_text_max_chars = 0 class-attribute instance-attribute

device = None class-attribute instance-attribute

endpoint_url = None class-attribute instance-attribute

extra_body = Field(default_factory=dict) class-attribute instance-attribute

gpu_memory_utilization = 0.5 class-attribute instance-attribute

hf_cache_dir = None class-attribute instance-attribute

model_name = Field(default=DEFAULT_LOCAL_CAPTION_MODEL_ID, description='Caption model identifier. The default local BF16 checkpoint has approximately 62 GiB of weights; set this explicitly to select a smaller local model or an API model for a remote endpoint.') class-attribute instance-attribute

prompt = 'Caption the content of this image:' class-attribute instance-attribute

system_prompt = '/no_think' class-attribute instance-attribute

tensor_parallel_size = 1 class-attribute instance-attribute

ChartParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class ChartParams(_ParamsModel):
    remote: RemoteInvokeParams = Field(default_factory=RemoteInvokeParams)
    remote_retry: RemoteRetryParams = Field(default_factory=RemoteRetryParams)
    inference_batch_size: int = 8

inference_batch_size = 8 class-attribute instance-attribute

remote = Field(default_factory=RemoteInvokeParams) class-attribute instance-attribute

remote_retry = Field(default_factory=RemoteRetryParams) class-attribute instance-attribute

DedupParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class DedupParams(_ParamsModel):
    content_hash: bool = True
    bbox_iou: bool = True
    iou_threshold: float = Field(default=0.45, ge=0.0, le=1.0)

bbox_iou = True class-attribute instance-attribute

content_hash = True class-attribute instance-attribute

iou_threshold = Field(default=0.45, ge=0.0, le=1.0) class-attribute instance-attribute

EmbedParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class EmbedParams(_ParamsModel):
    model_name: Optional[str] = None
    embedding_endpoint: Optional[str] = None
    embed_invoke_url: Optional[str] = None
    embed_model_name: Optional[str] = None
    embed_model_revision: Optional[str] = None
    embed_model_provider_prefix: Optional[str] = None
    api_key: Optional[str] = None
    input_type: str = "passage"
    embed_modality: str = "text"  # "text", "image", or "text_image" — default for all element types
    embed_granularity: Literal["element", "page"] = "element"  # "element" = per-element rows, "page" = one row per page
    text_elements_modality: Optional[str] = None  # per-type override for page-text rows
    structured_elements_modality: Optional[str] = None  # per-type override for table/chart/infographic rows
    text_column: str = "text"
    inference_batch_size: int = 32
    output_column: str = "text_embeddings_1b_v2"
    embedding_dim_column: str = "text_embeddings_1b_v2_dim"
    has_embedding_column: str = "text_embeddings_1b_v2_has_embedding"
    embed_output_column: str = "text_embeddings_1b_v2"
    embed_inference_batch_size: int = 16

    local_ingest_embed_backend: str = (
        "vllm"  # "vllm" or "hf" — selects ingest-time embedder backend for both text and VL models
    )
    query_max_length: int = 128
    dimensions: Optional[int] = None

    # Concurrent HTTP embedding requests per Ray batch (OpenAI-compatible NIM).
    nim_http_max_concurrent: int = 32
    request_timeout_s: float = 600.0

    runtime: ModelRuntimeParams = Field(default_factory=ModelRuntimeParams)
    batch_tuning: BatchTuningParams = Field(default_factory=BatchTuningParams)

    @field_validator("local_ingest_embed_backend", mode="before")
    @classmethod
    def _validate_local_ingest_embed_backend(cls, v: str) -> str:
        from nemo_retriever.models import (
            _LOCAL_INGEST_EMBED_BACKENDS,
            normalize_backend,
        )

        return normalize_backend(
            str(v) if v is not None else None,
            _LOCAL_INGEST_EMBED_BACKENDS,
            field_name="local_ingest_embed_backend",
            default="vllm",
        )

    @field_validator(
        "embed_modality",
        "text_elements_modality",
        "structured_elements_modality",
        mode="before",
    )
    @classmethod
    def _validate_modality(cls, v: str | None) -> str | None:
        if v is None:
            return None
        modality = str(v).strip()
        if modality == "image_text":
            raise ValueError("Use 'text_image' instead of 'image_text'.")
        if modality not in VALID_EMBED_MODALITIES:
            raise ValueError(f"Modality must be one of {sorted(VALID_EMBED_MODALITIES)}")
        return modality

    @model_validator(mode="after")
    def _warn_page_granularity_overrides(self) -> "EmbedParams":
        if self.embed_granularity == "page" and (
            self.text_elements_modality is not None or self.structured_elements_modality is not None
        ):
            warnings.warn(
                "text_elements_modality and structured_elements_modality are ignored when "
                "embed_granularity='page' (only embed_modality is used).",
                UserWarning,
                stacklevel=2,
            )
        return self

api_key = None class-attribute instance-attribute

batch_tuning = Field(default_factory=BatchTuningParams) class-attribute instance-attribute

dimensions = None class-attribute instance-attribute

embed_granularity = 'element' class-attribute instance-attribute

embed_inference_batch_size = 16 class-attribute instance-attribute

embed_invoke_url = None class-attribute instance-attribute

embed_modality = 'text' class-attribute instance-attribute

embed_model_name = None class-attribute instance-attribute

embed_model_provider_prefix = None class-attribute instance-attribute

embed_model_revision = None class-attribute instance-attribute

embed_output_column = 'text_embeddings_1b_v2' class-attribute instance-attribute

embedding_dim_column = 'text_embeddings_1b_v2_dim' class-attribute instance-attribute

embedding_endpoint = None class-attribute instance-attribute

has_embedding_column = 'text_embeddings_1b_v2_has_embedding' class-attribute instance-attribute

inference_batch_size = 32 class-attribute instance-attribute

input_type = 'passage' class-attribute instance-attribute

local_ingest_embed_backend = 'vllm' class-attribute instance-attribute

model_name = None class-attribute instance-attribute

nim_http_max_concurrent = 32 class-attribute instance-attribute

output_column = 'text_embeddings_1b_v2' class-attribute instance-attribute

query_max_length = 128 class-attribute instance-attribute

request_timeout_s = 600.0 class-attribute instance-attribute

runtime = Field(default_factory=ModelRuntimeParams) class-attribute instance-attribute

structured_elements_modality = None class-attribute instance-attribute

text_column = 'text' class-attribute instance-attribute

text_elements_modality = None class-attribute instance-attribute

ExtractParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class ExtractParams(_ParamsModel):
    # Extraction flags
    extract_text: bool = True
    extract_images: bool = True
    extract_tables: bool = True
    extract_charts: bool = True
    extract_infographics: bool = False
    extract_page_as_image: Optional[bool] = True

    # Extraction options
    method: str = "pdfium"
    # Run PageElementDetection (layout/yolox). Required by TableStructure and
    # OCR. Safe to disable for text-only ingests.
    use_page_elements: bool = True
    use_table_structure: bool = False
    table_output_format: Optional[Literal["pseudo_markdown", "markdown"]] = None
    dpi: int = 200
    image_format: str = "jpeg"
    jpeg_quality: int = 100
    render_mode: Literal["full_dpi", "fit_to_model"] = "fit_to_model"
    inference_batch_size: int = 8
    ocr_model_dir: Optional[str] = None
    ocr_version: Literal["v1", "v2"] = "v2"
    ocr_lang: Optional[Literal["multi", "english"]] = None

    # Service endpoints
    invoke_url: Optional[str] = None
    api_key: Optional[str] = None
    request_timeout_s: float = 60.0
    page_elements_invoke_url: Optional[str] = None
    page_elements_api_key: Optional[str] = None
    page_elements_request_timeout_s: Optional[float] = None
    ocr_invoke_url: Optional[str] = None
    ocr_api_key: Optional[str] = None
    ocr_request_timeout_s: Optional[float] = None
    table_structure_invoke_url: Optional[str] = None
    nemotron_parse_invoke_url: Optional[str] = None
    nemotron_parse_model: Optional[str] = None

    # Output columns
    output_column: str = "page_elements_v3"
    num_detections_column: str = "page_elements_v3_num_detections"
    counts_by_label_column: str = "page_elements_v3_counts_by_label"

    remote_retry: RemoteRetryParams = Field(default_factory=RemoteRetryParams)
    batch_tuning: BatchTuningParams = Field(default_factory=BatchTuningParams)

    @model_validator(mode="after")
    def _auto_enable_features(self) -> "ExtractParams":
        """Auto-configure feature flags from remote endpoints.

        * Enable ``use_table_structure`` when ``table_structure_invoke_url``
          is provided.
        * Default ``table_output_format`` to ``"markdown"`` when the stage is
          enabled and the caller did not explicitly choose a format.
        """
        if self.table_structure_invoke_url and not self.use_table_structure:
            self.use_table_structure = True
        if self.table_output_format is None:
            self.table_output_format = "markdown" if self.use_table_structure else "pseudo_markdown"
        if self.ocr_version == "v1" and self.ocr_lang is not None:
            raise ValueError("ocr_lang is only supported when ocr_version='v2'.")
        if self.method != "nemotron_parse" and (
            self.nemotron_parse_invoke_url is not None or self.nemotron_parse_model is not None
        ):
            raise ValueError(
                "`nemotron_parse_invoke_url` and `nemotron_parse_model` require "
                "`method='nemotron_parse'`; Parse-specific configuration is otherwise ignored."
            )
        if not self.use_page_elements:
            consumers = [("use_table_structure", self.use_table_structure and self.extract_tables)]
            enabled = [name for name, on in consumers if on]
            if enabled:
                raise ValueError(f"use_page_elements=False is incompatible with: {', '.join(enabled)}")
        return self

api_key = None class-attribute instance-attribute

batch_tuning = Field(default_factory=BatchTuningParams) class-attribute instance-attribute

counts_by_label_column = 'page_elements_v3_counts_by_label' class-attribute instance-attribute

dpi = 200 class-attribute instance-attribute

extract_charts = True class-attribute instance-attribute

extract_images = True class-attribute instance-attribute

extract_infographics = False class-attribute instance-attribute

extract_page_as_image = True class-attribute instance-attribute

extract_tables = True class-attribute instance-attribute

extract_text = True class-attribute instance-attribute

image_format = 'jpeg' class-attribute instance-attribute

inference_batch_size = 8 class-attribute instance-attribute

invoke_url = None class-attribute instance-attribute

jpeg_quality = 100 class-attribute instance-attribute

method = 'pdfium' class-attribute instance-attribute

nemotron_parse_invoke_url = None class-attribute instance-attribute

nemotron_parse_model = None class-attribute instance-attribute

num_detections_column = 'page_elements_v3_num_detections' class-attribute instance-attribute

ocr_api_key = None class-attribute instance-attribute

ocr_invoke_url = None class-attribute instance-attribute

ocr_lang = None class-attribute instance-attribute

ocr_model_dir = None class-attribute instance-attribute

ocr_request_timeout_s = None class-attribute instance-attribute

ocr_version = 'v2' class-attribute instance-attribute

output_column = 'page_elements_v3' class-attribute instance-attribute

page_elements_api_key = None class-attribute instance-attribute

page_elements_invoke_url = None class-attribute instance-attribute

page_elements_request_timeout_s = None class-attribute instance-attribute

remote_retry = Field(default_factory=RemoteRetryParams) class-attribute instance-attribute

render_mode = 'fit_to_model' class-attribute instance-attribute

request_timeout_s = 60.0 class-attribute instance-attribute

table_output_format = None class-attribute instance-attribute

table_structure_invoke_url = None class-attribute instance-attribute

use_page_elements = True class-attribute instance-attribute

use_table_structure = False class-attribute instance-attribute

GpuAllocationParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class GpuAllocationParams(_ParamsModel):
    gpu_devices: list[str] = Field(default_factory=list)
    startup_timeout: float = 600.0

gpu_devices = Field(default_factory=list) class-attribute instance-attribute

startup_timeout = 600.0 class-attribute instance-attribute

HtmlChunkParams

Bases: TextChunkParams

Source code in nemo_retriever/common/params/models.py

class HtmlChunkParams(TextChunkParams):
    pass

IngestExecuteParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class IngestExecuteParams(_ParamsModel):
    show_progress: bool = False
    return_failures: bool = False
    return_traces: bool = False
    return_results: bool = True
    result_schema: Literal["legacy", "compact"] = "legacy"
    return_embeddings: bool = False
    return_images: bool = False
    parallel: bool = False
    max_workers: Optional[int] = None
    gpu_devices: list[str] = Field(default_factory=list)
    page_chunk_size: int = 32
    runtime_metrics_dir: Optional[str] = None
    runtime_metrics_prefix: Optional[str] = None

gpu_devices = Field(default_factory=list) class-attribute instance-attribute

max_workers = None class-attribute instance-attribute

page_chunk_size = 32 class-attribute instance-attribute

parallel = False class-attribute instance-attribute

result_schema = 'legacy' class-attribute instance-attribute

return_embeddings = False class-attribute instance-attribute

return_failures = False class-attribute instance-attribute

return_images = False class-attribute instance-attribute

return_results = True class-attribute instance-attribute

return_traces = False class-attribute instance-attribute

runtime_metrics_dir = None class-attribute instance-attribute

runtime_metrics_prefix = None class-attribute instance-attribute

show_progress = False class-attribute instance-attribute

IngestorCreateParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class IngestorCreateParams(_ParamsModel):
    documents: list[str] = Field(default_factory=list)
    ray_address: Optional[str] = None
    ray_log_to_driver: bool = True
    debug: bool = False
    base_url: str = "http://localhost:7670"
    allow_no_gpu: bool = False
    node_overrides: Optional[dict[str, dict[str, Any]]] = None
    api_key: Optional[str] = None
    error_policy: Literal["raise", "collect"] = "raise"
    # service run mode: maximum number of concurrent page uploads.  Lower
    # values (e.g. 2-4) reduce burst pressure on Kubernetes NodePort /
    # kube-proxy paths that otherwise reset connections under heavy load.
    max_concurrency: Optional[int] = None

allow_no_gpu = False class-attribute instance-attribute

api_key = None class-attribute instance-attribute

base_url = 'http://localhost:7670' class-attribute instance-attribute

debug = False class-attribute instance-attribute

documents = Field(default_factory=list) class-attribute instance-attribute

error_policy = 'raise' class-attribute instance-attribute

max_concurrency = None class-attribute instance-attribute

node_overrides = None class-attribute instance-attribute

ray_address = None class-attribute instance-attribute

ray_log_to_driver = True class-attribute instance-attribute

LLMInferenceParams

Bases: _ParamsModel

Reusable LLM sampling / generation parameters.

Inherit from this model to add temperature, top_p, and max_tokens to any task that invokes an LLM (captioning, summarization, etc.).

Source code in nemo_retriever/common/params/models.py

class LLMInferenceParams(_ParamsModel):
    """Reusable LLM sampling / generation parameters.

    Inherit from this model to add temperature, top_p, and max_tokens
    to any task that invokes an LLM (captioning, summarization, etc.).
    """

    temperature: Optional[float] = 1.0
    top_p: Optional[float] = None
    max_tokens: int = 1024

    @field_validator("temperature")
    @classmethod
    def _check_temperature(cls, v: Optional[float]) -> Optional[float]:
        if v is not None and not (0.0 <= v <= 2.0):
            raise ValueError("temperature must be between 0.0 and 2.0")
        return v

    @field_validator("top_p")
    @classmethod
    def _check_top_p(cls, v: Optional[float]) -> Optional[float]:
        if v is not None and not (0.0 <= v <= 1.0):
            raise ValueError("top_p must be between 0.0 and 1.0")
        return v

    @field_validator("max_tokens")
    @classmethod
    def _check_max_tokens(cls, v: int) -> int:
        if v <= 0:
            raise ValueError("max_tokens must be > 0")
        return v

    def to_sampling_kwargs(self) -> dict[str, Any]:
        """Build a dict of sampling parameters suitable for LLM inference calls.

        ``top_p`` is only included when explicitly set (not ``None``), because
        many backends (vLLM, OpenAI, NIM) change behaviour when the key is
        present vs. absent.
        """
        kw: dict[str, Any] = {"max_tokens": self.max_tokens}
        if self.temperature is not None:
            kw["temperature"] = self.temperature
        if self.top_p is not None:
            kw["top_p"] = self.top_p
        return kw

max_tokens = 1024 class-attribute instance-attribute

temperature = 1.0 class-attribute instance-attribute

top_p = None class-attribute instance-attribute

to_sampling_kwargs()

Build a dict of sampling parameters suitable for LLM inference calls.

top_p is only included when explicitly set (not None), because many backends (vLLM, OpenAI, NIM) change behaviour when the key is present vs. absent.

Source code in nemo_retriever/common/params/models.py

def to_sampling_kwargs(self) -> dict[str, Any]:
    """Build a dict of sampling parameters suitable for LLM inference calls.

    ``top_p`` is only included when explicitly set (not ``None``), because
    many backends (vLLM, OpenAI, NIM) change behaviour when the key is
    present vs. absent.
    """
    kw: dict[str, Any] = {"max_tokens": self.max_tokens}
    if self.temperature is not None:
        kw["temperature"] = self.temperature
    if self.top_p is not None:
        kw["top_p"] = self.top_p
    return kw

LLMRemoteClientParams

Bases: _ParamsModel

Transport / connection parameters for any remote LLM client.

Pairs with :class:LLMInferenceParams (sampling) to fully specify a call. api_key=None is left unset so LiteLLM can perform provider-native environment lookup on the worker.

Source code in nemo_retriever/common/params/models.py

class LLMRemoteClientParams(_ParamsModel):
    """Transport / connection parameters for any remote LLM client.

    Pairs with :class:`LLMInferenceParams` (sampling) to fully specify a
    call. ``api_key=None`` is left unset so LiteLLM can perform provider-native
    environment lookup on the worker.
    """

    _auto_resolve_unset_api_keys: ClassVar[bool] = False

    model: str
    api_base: Optional[str] = None
    api_key: Optional[str] = None
    num_retries: int = 3
    timeout: float = 120.0
    extra_params: dict[str, Any] = Field(default_factory=dict)
    rag_system_prompt: Optional[str] = None
    rag_system_prompt_prefix: Optional[str] = None
    reasoning_enabled: bool = True

    @field_validator("extra_params")
    @classmethod
    def _check_extra_params(cls, value: dict[str, Any]) -> dict[str, Any]:
        validate_llm_extra_params(value, source="LLMRemoteClientParams.extra_params")
        return value

    @field_validator("num_retries")
    @classmethod
    def _check_retries(cls, v: int) -> int:
        if v < 0:
            raise ValueError("num_retries must be >= 0")
        return v

    @field_validator("timeout")
    @classmethod
    def _check_timeout(cls, v: float) -> float:
        if v <= 0:
            raise ValueError("timeout must be > 0")
        return v

api_base = None class-attribute instance-attribute

api_key = None class-attribute instance-attribute

extra_params = Field(default_factory=dict) class-attribute instance-attribute

model instance-attribute

num_retries = 3 class-attribute instance-attribute

rag_system_prompt = None class-attribute instance-attribute

rag_system_prompt_prefix = None class-attribute instance-attribute

reasoning_enabled = True class-attribute instance-attribute

timeout = 120.0 class-attribute instance-attribute

LLMSamplingOverrides

Bases: _ParamsModel

Partial sampling overrides resolved on top of task-specific defaults.

Source code in nemo_retriever/common/params/models.py

class LLMSamplingOverrides(_ParamsModel):
    """Partial sampling overrides resolved on top of task-specific defaults."""

    temperature: Optional[float] = None
    top_p: Optional[float] = None
    max_tokens: Optional[int] = None

    @field_validator("temperature")
    @classmethod
    def _check_temperature(cls, v: Optional[float]) -> Optional[float]:
        if v is not None and not (0.0 <= v <= 2.0):
            raise ValueError("temperature must be between 0.0 and 2.0")
        return v

    @field_validator("top_p")
    @classmethod
    def _check_top_p(cls, v: Optional[float]) -> Optional[float]:
        if v is not None and not (0.0 <= v <= 1.0):
            raise ValueError("top_p must be between 0.0 and 1.0")
        return v

    @field_validator("max_tokens")
    @classmethod
    def _check_max_tokens(cls, v: Optional[int]) -> Optional[int]:
        if v is not None and v <= 0:
            raise ValueError("max_tokens must be > 0")
        return v

    @model_validator(mode="after")
    def _reject_explicit_null_max_tokens(self) -> "LLMSamplingOverrides":
        if "max_tokens" in self.model_fields_set and self.max_tokens is None:
            raise ValueError("max_tokens cannot be None; omit it to inherit the task default")
        return self

    @model_serializer(mode="plain")
    def _serialize_only_explicit_overrides(self) -> dict[str, Any]:
        """Preserve omitted-vs-null state across model and JSON round trips."""
        return {
            name: getattr(self, name)
            for name in ("temperature", "top_p", "max_tokens")
            if name in self.model_fields_set
        }

    def __eq__(self, other: object) -> bool:
        if isinstance(other, LLMSamplingOverrides):
            return self.model_fields_set == other.model_fields_set and super().__eq__(other)
        return super().__eq__(other)

    def resolve(self, defaults: LLMInferenceParams) -> LLMInferenceParams:
        """Apply explicitly supplied fields to defaults."""
        values = defaults.model_dump()
        for name in self.model_fields_set:
            value = getattr(self, name)
            values[name] = value
        return LLMInferenceParams(**values)

max_tokens = None class-attribute instance-attribute

temperature = None class-attribute instance-attribute

top_p = None class-attribute instance-attribute

__eq__(other)

Source code in nemo_retriever/common/params/models.py

def __eq__(self, other: object) -> bool:
    if isinstance(other, LLMSamplingOverrides):
        return self.model_fields_set == other.model_fields_set and super().__eq__(other)
    return super().__eq__(other)

resolve(defaults)

Apply explicitly supplied fields to defaults.

Source code in nemo_retriever/common/params/models.py

def resolve(self, defaults: LLMInferenceParams) -> LLMInferenceParams:
    """Apply explicitly supplied fields to defaults."""
    values = defaults.model_dump()
    for name in self.model_fields_set:
        value = getattr(self, name)
        values[name] = value
    return LLMInferenceParams(**values)

LanceDbParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class LanceDbParams(_ParamsModel):
    lancedb_uri: str = "lancedb"
    table_name: str = "nv-ingest"
    overwrite: bool = True
    create_index: bool = True
    index_type: str = "IVF_HNSW_SQ"
    metric: str = "l2"
    num_partitions: int = 16
    num_sub_vectors: int = 256
    embedding_column: str = "text_embeddings_1b_v2"
    embedding_key: str = "embedding"
    include_text: bool = True
    text_column: str = "text"
    hybrid: bool = False
    fts_language: str = "English"

create_index = True class-attribute instance-attribute

embedding_column = 'text_embeddings_1b_v2' class-attribute instance-attribute

embedding_key = 'embedding' class-attribute instance-attribute

fts_language = 'English' class-attribute instance-attribute

hybrid = False class-attribute instance-attribute

include_text = True class-attribute instance-attribute

index_type = 'IVF_HNSW_SQ' class-attribute instance-attribute

lancedb_uri = 'lancedb' class-attribute instance-attribute

metric = 'l2' class-attribute instance-attribute

num_partitions = 16 class-attribute instance-attribute

num_sub_vectors = 256 class-attribute instance-attribute

overwrite = True class-attribute instance-attribute

table_name = 'nv-ingest' class-attribute instance-attribute

text_column = 'text' class-attribute instance-attribute

ModelRuntimeParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class ModelRuntimeParams(_ParamsModel):
    device: Optional[str] = None
    hf_cache_dir: Optional[str] = None
    normalize: bool = True
    max_length: int = 8192
    model_name: Optional[str] = None
    gpu_memory_utilization: float = 0.45
    enforce_eager: bool = False

device = None class-attribute instance-attribute

enforce_eager = False class-attribute instance-attribute

gpu_memory_utilization = 0.45 class-attribute instance-attribute

hf_cache_dir = None class-attribute instance-attribute

max_length = 8192 class-attribute instance-attribute

model_name = None class-attribute instance-attribute

normalize = True class-attribute instance-attribute

OcrParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class OcrParams(_ParamsModel):
    remote: RemoteInvokeParams = Field(default_factory=RemoteInvokeParams)
    remote_retry: RemoteRetryParams = Field(default_factory=RemoteRetryParams)
    inference_batch_size: int = 8
    extract_tables: bool = False
    extract_charts: bool = False
    extract_infographics: bool = False

extract_charts = False class-attribute instance-attribute

extract_infographics = False class-attribute instance-attribute

extract_tables = False class-attribute instance-attribute

inference_batch_size = 8 class-attribute instance-attribute

remote = Field(default_factory=RemoteInvokeParams) class-attribute instance-attribute

remote_retry = Field(default_factory=RemoteRetryParams) class-attribute instance-attribute

PageElementsParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class PageElementsParams(_ParamsModel):
    remote: RemoteInvokeParams = Field(default_factory=RemoteInvokeParams)
    remote_retry: RemoteRetryParams = Field(default_factory=RemoteRetryParams)
    inference_batch_size: int = 8
    output_column: str = "page_elements_v3"
    num_detections_column: str = "page_elements_v3_num_detections"
    counts_by_label_column: str = "page_elements_v3_counts_by_label"

counts_by_label_column = 'page_elements_v3_counts_by_label' class-attribute instance-attribute

inference_batch_size = 8 class-attribute instance-attribute

num_detections_column = 'page_elements_v3_num_detections' class-attribute instance-attribute

output_column = 'page_elements_v3' class-attribute instance-attribute

remote = Field(default_factory=RemoteInvokeParams) class-attribute instance-attribute

remote_retry = Field(default_factory=RemoteRetryParams) class-attribute instance-attribute

PdfSplitParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class PdfSplitParams(_ParamsModel):
    start_page: Optional[int] = None
    end_page: Optional[int] = None

end_page = None class-attribute instance-attribute

start_page = None class-attribute instance-attribute

RemoteInvokeParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class RemoteInvokeParams(_ParamsModel):
    invoke_url: Optional[str] = None
    api_key: Optional[str] = None
    request_timeout_s: float = 60.0

api_key = None class-attribute instance-attribute

invoke_url = None class-attribute instance-attribute

request_timeout_s = 60.0 class-attribute instance-attribute

RemoteRetryParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class RemoteRetryParams(_ParamsModel):
    remote_max_pool_workers: int = 32
    remote_max_retries: int = 5
    remote_max_429_retries: int = 3

remote_max_429_retries = 3 class-attribute instance-attribute

remote_max_pool_workers = 32 class-attribute instance-attribute

remote_max_retries = 5 class-attribute instance-attribute

StoreParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class StoreParams(_ParamsModel):
    storage_uri: str = "stored_images"
    storage_options: dict[str, Any] = Field(default_factory=dict)
    image_format: str = "png"
    strip_base64: bool = True
    batch_tuning: BatchTuningParams = Field(default_factory=BatchTuningParams)

    @model_validator(mode="after")
    def _resolve_local_storage_uri(self) -> "StoreParams":
        """Resolve relative local paths to absolute so they survive Ray serialization."""
        if not urlparse(self.storage_uri).scheme:
            self.storage_uri = str(UPath(self.storage_uri).resolve())
        return self

batch_tuning = Field(default_factory=BatchTuningParams) class-attribute instance-attribute

image_format = 'png' class-attribute instance-attribute

storage_options = Field(default_factory=dict) class-attribute instance-attribute

storage_uri = 'stored_images' class-attribute instance-attribute

strip_base64 = True class-attribute instance-attribute

TableParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class TableParams(_ParamsModel):
    remote: RemoteInvokeParams = Field(default_factory=RemoteInvokeParams)
    remote_retry: RemoteRetryParams = Field(default_factory=RemoteRetryParams)
    inference_batch_size: int = 8
    output_column: str = "table_structure_v1"
    num_detections_column: str = "table_structure_v1_num_detections"
    counts_by_label_column: str = "table_structure_v1_counts_by_label"

counts_by_label_column = 'table_structure_v1_counts_by_label' class-attribute instance-attribute

inference_batch_size = 8 class-attribute instance-attribute

num_detections_column = 'table_structure_v1_num_detections' class-attribute instance-attribute

output_column = 'table_structure_v1' class-attribute instance-attribute

remote = Field(default_factory=RemoteInvokeParams) class-attribute instance-attribute

remote_retry = Field(default_factory=RemoteRetryParams) class-attribute instance-attribute

TabularExtractParams

Bases: _ParamsModel

Params for step 1: extract schema metadata and write to Neo4j.

Covers SQLAlchemy reflection of a live database and/or parsing of pre-existing SQL DDL/query files. Produces Database, Schema, Table, Column, View and Query nodes together with their relationships. The Neo4j connection is provided by get_neo4j_conn() (see tabular_data.neo4j) and is not configured here.

Source code in nemo_retriever/common/params/models.py

class TabularExtractParams(_ParamsModel):
    """Params for step 1: extract schema metadata and write to Neo4j.

    Covers SQLAlchemy reflection of a live database and/or parsing of
    pre-existing SQL DDL/query files.  Produces Database, Schema, Table,
    Column, View and Query nodes together with their relationships.
    The Neo4j connection is provided by get_neo4j_conn() (see
    tabular_data.neo4j) and is not configured here.
    """

    model_config = ConfigDict(extra="forbid", arbitrary_types_allowed=True)

    connector: Optional[SQLDatabase] = None

connector = None class-attribute instance-attribute

model_config = ConfigDict(extra='forbid', arbitrary_types_allowed=True) class-attribute instance-attribute

TextChunkParams

Bases: _ParamsModel

Source code in nemo_retriever/common/params/models.py

class TextChunkParams(_ParamsModel):
    max_tokens: int = 1024
    overlap_tokens: int = 0
    tokenizer_model_id: Optional[str] = None
    encoding: str = "utf-8"
    tokenizer_cache_dir: Optional[str] = None

encoding = 'utf-8' class-attribute instance-attribute

max_tokens = 1024 class-attribute instance-attribute

overlap_tokens = 0 class-attribute instance-attribute

tokenizer_cache_dir = None class-attribute instance-attribute

tokenizer_model_id = None class-attribute instance-attribute

TextGenerationParams

Bases: _ParamsModel

Transport, task controls, and partial sampling for text generation.

Source code in nemo_retriever/common/params/models.py

class TextGenerationParams(_ParamsModel):
    """Transport, task controls, and partial sampling for text generation."""

    transport: LLMRemoteClientParams
    sampling: LLMSamplingOverrides = Field(default_factory=LLMSamplingOverrides)
    prompt: Optional[str] = None
    system_prompt: Optional[str] = None
    reasoning_enabled: Optional[bool] = None
    max_workers: int = Field(default=8, ge=1)

    def resolve_sampling(self, defaults: LLMInferenceParams) -> LLMInferenceParams:
        """Resolve explicit sampling fields over a task's defaults."""
        return self.sampling.resolve(defaults)

    @classmethod
    def from_kwargs(
        cls,
        *,
        model: str,
        api_base: Optional[str] = None,
        api_key: Optional[str] = None,
        temperature: Any = _SAMPLING_UNSET,
        top_p: Any = _SAMPLING_UNSET,
        max_tokens: Any = _SAMPLING_UNSET,
        extra_params: Optional[dict[str, Any]] = None,
        num_retries: int = 3,
        timeout: float = 120.0,
        rag_system_prompt: Optional[str] = None,
        rag_system_prompt_prefix: Optional[str] = None,
        reasoning_enabled: Optional[bool] = None,
        prompt: Optional[str] = None,
        system_prompt: Optional[str] = None,
        max_workers: int = 8,
    ) -> "TextGenerationParams":
        """Construct structured text-generation params from flat kwargs."""
        sampling_values: dict[str, Any] = {}
        for name, value in (
            ("temperature", temperature),
            ("top_p", top_p),
            ("max_tokens", max_tokens),
        ):
            if value is not _SAMPLING_UNSET:
                sampling_values[name] = value

        transport_reasoning = True if reasoning_enabled is None else reasoning_enabled
        return cls(
            transport=LLMRemoteClientParams(
                model=model,
                api_base=api_base,
                api_key=api_key,
                num_retries=num_retries,
                timeout=timeout,
                extra_params=extra_params or {},
                rag_system_prompt=rag_system_prompt,
                rag_system_prompt_prefix=rag_system_prompt_prefix,
                reasoning_enabled=transport_reasoning,
            ),
            sampling=LLMSamplingOverrides(**sampling_values),
            prompt=prompt,
            system_prompt=system_prompt,
            reasoning_enabled=reasoning_enabled,
            max_workers=max_workers,
        )

max_workers = Field(default=8, ge=1) class-attribute instance-attribute

prompt = None class-attribute instance-attribute

reasoning_enabled = None class-attribute instance-attribute

sampling = Field(default_factory=LLMSamplingOverrides) class-attribute instance-attribute

system_prompt = None class-attribute instance-attribute

transport instance-attribute

from_kwargs(*, model, api_base=None, api_key=None, temperature=_SAMPLING_UNSET, top_p=_SAMPLING_UNSET, max_tokens=_SAMPLING_UNSET, extra_params=None, num_retries=3, timeout=120.0, rag_system_prompt=None, rag_system_prompt_prefix=None, reasoning_enabled=None, prompt=None, system_prompt=None, max_workers=8) classmethod

Construct structured text-generation params from flat kwargs.

Source code in nemo_retriever/common/params/models.py

@classmethod
def from_kwargs(
    cls,
    *,
    model: str,
    api_base: Optional[str] = None,
    api_key: Optional[str] = None,
    temperature: Any = _SAMPLING_UNSET,
    top_p: Any = _SAMPLING_UNSET,
    max_tokens: Any = _SAMPLING_UNSET,
    extra_params: Optional[dict[str, Any]] = None,
    num_retries: int = 3,
    timeout: float = 120.0,
    rag_system_prompt: Optional[str] = None,
    rag_system_prompt_prefix: Optional[str] = None,
    reasoning_enabled: Optional[bool] = None,
    prompt: Optional[str] = None,
    system_prompt: Optional[str] = None,
    max_workers: int = 8,
) -> "TextGenerationParams":
    """Construct structured text-generation params from flat kwargs."""
    sampling_values: dict[str, Any] = {}
    for name, value in (
        ("temperature", temperature),
        ("top_p", top_p),
        ("max_tokens", max_tokens),
    ):
        if value is not _SAMPLING_UNSET:
            sampling_values[name] = value

    transport_reasoning = True if reasoning_enabled is None else reasoning_enabled
    return cls(
        transport=LLMRemoteClientParams(
            model=model,
            api_base=api_base,
            api_key=api_key,
            num_retries=num_retries,
            timeout=timeout,
            extra_params=extra_params or {},
            rag_system_prompt=rag_system_prompt,
            rag_system_prompt_prefix=rag_system_prompt_prefix,
            reasoning_enabled=transport_reasoning,
        ),
        sampling=LLMSamplingOverrides(**sampling_values),
        prompt=prompt,
        system_prompt=system_prompt,
        reasoning_enabled=reasoning_enabled,
        max_workers=max_workers,
    )

resolve_sampling(defaults)

Resolve explicit sampling fields over a task's defaults.

Source code in nemo_retriever/common/params/models.py

def resolve_sampling(self, defaults: LLMInferenceParams) -> LLMInferenceParams:
    """Resolve explicit sampling fields over a task's defaults."""
    return self.sampling.resolve(defaults)

VdbUploadParams

Bases: _ParamsModel

Post-graph vector DB upload configuration.

Sidecar metadata (meta_*) matches nv_ingest_client / metadata_and_filtered_search.ipynb: all three fields must be set together to merge columns into each chunk's content_metadata.

Source code in nemo_retriever/common/params/models.py

class VdbUploadParams(_ParamsModel):
    """Post-graph vector DB upload configuration.

    Sidecar metadata (``meta_*``) matches ``nv_ingest_client`` / ``metadata_and_filtered_search.ipynb``:
    all three fields must be set together to merge columns into each chunk's ``content_metadata``.
    """

    vdb_op: str = "lancedb"
    vdb_kwargs: dict[str, Any] = Field(default_factory=dict)
    meta_dataframe: Optional[Any] = None
    """Path to csv/json/parquet or an in-memory :class:`pandas.DataFrame`."""
    meta_source_field: Optional[str] = None
    meta_fields: Optional[list[str]] = None
    meta_join_key: MetaJoinKey = "auto"
    """How to match rows to documents: ``source_id`` (full path), ``source_name`` (basename), or ``auto`` (try both)."""

    @model_validator(mode="after")
    def _validate_sidecar_triplet(self) -> "VdbUploadParams":
        trio = (self.meta_dataframe, self.meta_source_field, self.meta_fields)
        if all(x is None for x in trio):
            return self
        if any(x is None for x in trio):
            raise ValueError(
                "meta_dataframe, meta_source_field, and meta_fields must all be set together "
                "when attaching sidecar metadata."
            )
        if not self.meta_fields:
            raise ValueError("meta_fields must be a non-empty list when sidecar metadata is enabled.")
        return self

    def to_ingest_operator_kwargs(self) -> dict[str, Any]:
        """Flatten into kwargs for :class:`~nemo_retriever.vdb.IngestVdbOperator`."""
        out = dict(self.vdb_kwargs or {})
        if self.meta_dataframe is not None:
            out["meta_dataframe"] = self.meta_dataframe
            out["meta_source_field"] = self.meta_source_field
            out["meta_fields"] = list(self.meta_fields or [])
            out["meta_join_key"] = self.meta_join_key
        return out

meta_dataframe = None class-attribute instance-attribute

Path to csv/json/parquet or an in-memory :class:pandas.DataFrame.

meta_fields = None class-attribute instance-attribute

meta_join_key = 'auto' class-attribute instance-attribute

How to match rows to documents: source_id (full path), source_name (basename), or auto (try both).

meta_source_field = None class-attribute instance-attribute

vdb_kwargs = Field(default_factory=dict) class-attribute instance-attribute

vdb_op = 'lancedb' class-attribute instance-attribute

to_ingest_operator_kwargs()

Flatten into kwargs for :class:~nemo_retriever.vdb.IngestVdbOperator.

Source code in nemo_retriever/common/params/models.py

def to_ingest_operator_kwargs(self) -> dict[str, Any]:
    """Flatten into kwargs for :class:`~nemo_retriever.vdb.IngestVdbOperator`."""
    out = dict(self.vdb_kwargs or {})
    if self.meta_dataframe is not None:
        out["meta_dataframe"] = self.meta_dataframe
        out["meta_source_field"] = self.meta_source_field
        out["meta_fields"] = list(self.meta_fields or [])
        out["meta_join_key"] = self.meta_join_key
    return out

VideoFrameParams

Bases: _ParamsModel

Params for video frame extraction (ffmpeg fps + perceptual-hash dedup).

Set enabled=False to skip frame extraction entirely; the video pipeline then produces only audio (ASR) rows — no frame OCR, no audio+visual fusion. Useful for ablating the visual modality or for audio-only recall benchmarks against video corpora.

dedup activates perceptual-hash (dhash) dedup before OCR. dhash catches visually-identical adjacent frames that byte-level hashing misses (encoder noise, brightness drift, etc.). On a 60s slide-heavy sample we measured ~91% duplicates collapsed at distance 5 vs ~11% for MD5 — a near-10x cut in OCR cost on slide content. Tune dedup_max_hamming_distance upward for more aggressive merging or down to 0 to require exact perceptual-hash matches.

Source code in nemo_retriever/common/params/models.py

class VideoFrameParams(_ParamsModel):
    """Params for video frame extraction (ffmpeg fps + perceptual-hash dedup).

    Set ``enabled=False`` to skip frame extraction entirely; the video
    pipeline then produces only audio (ASR) rows — no frame OCR, no
    audio+visual fusion. Useful for ablating the visual modality or for
    audio-only recall benchmarks against video corpora.

    ``dedup`` activates perceptual-hash (dhash) dedup before OCR. dhash
    catches visually-identical adjacent frames that byte-level hashing
    misses (encoder noise, brightness drift, etc.). On a 60s slide-heavy
    sample we measured ~91% duplicates collapsed at distance 5 vs ~11%
    for MD5 — a near-10x cut in OCR cost on slide content. Tune
    ``dedup_max_hamming_distance`` upward for more aggressive merging or
    down to 0 to require exact perceptual-hash matches.
    """

    enabled: bool = True
    fps: float = Field(default=1.0, gt=0.0)
    max_frames: Optional[int] = None
    dedup: bool = True
    dedup_max_hamming_distance: int = 5
    dedup_max_dropped_frames: int = 2

dedup = True class-attribute instance-attribute

dedup_max_dropped_frames = 2 class-attribute instance-attribute

dedup_max_hamming_distance = 5 class-attribute instance-attribute

enabled = True class-attribute instance-attribute

fps = Field(default=1.0, gt=0.0) class-attribute instance-attribute

max_frames = None class-attribute instance-attribute

VideoFrameTextDedupParams

Bases: _ParamsModel

Params for merging consecutive video_frame rows with identical OCR text.

After full-frame OCR, slides that are visible for many seconds produce a flood of frames with the same text (image-hash dedup misses them when encoder noise differs frame-to-frame). This stage groups by (source_path, text) and merges adjacent runs into a single row whose segment_start_seconds / segment_end_seconds cover the union of the run.

Tolerance is expressed in dropped frames, not seconds, so it scales with video_frame_fps: at runtime the dedup reads each group's metadata.fps and converts to max_gap_seconds = max_dropped_frames / fps. Default 2 means we bridge gaps of up to 2 missing frames in a run — a typical safety margin for image-hash dedup leaving small holes.

Source code in nemo_retriever/common/params/models.py

class VideoFrameTextDedupParams(_ParamsModel):
    """Params for merging consecutive video_frame rows with identical OCR text.

    After full-frame OCR, slides that are visible for many seconds produce a
    flood of frames with the same text (image-hash dedup misses them when
    encoder noise differs frame-to-frame). This stage groups by
    ``(source_path, text)`` and merges adjacent runs into a single row whose
    ``segment_start_seconds`` / ``segment_end_seconds`` cover the union of
    the run.

    Tolerance is expressed in **dropped frames**, not seconds, so it scales
    with ``video_frame_fps``: at runtime the dedup reads each group's
    ``metadata.fps`` and converts to ``max_gap_seconds = max_dropped_frames / fps``.
    Default 2 means we bridge gaps of up to 2 missing frames in a run —
    a typical safety margin for image-hash dedup leaving small holes.
    """

    enabled: bool = True
    max_dropped_frames: int = 2

enabled = True class-attribute instance-attribute

max_dropped_frames = 2 class-attribute instance-attribute

WebhookParams

Bases: _ParamsModel

Configuration for the webhook notification stage.

When endpoint_url is set, selected columns from the processed batch are serialised to JSON and HTTP-POSTed to that URL. If endpoint_url is None the stage is a no-op.

Source code in nemo_retriever/common/params/models.py

class WebhookParams(_ParamsModel):
    """Configuration for the webhook notification stage.

    When ``endpoint_url`` is set, selected columns from the processed batch
    are serialised to JSON and HTTP-POSTed to that URL.  If ``endpoint_url``
    is ``None`` the stage is a no-op.
    """

    endpoint_url: Optional[str] = None
    columns: list[str] = Field(default_factory=list)
    headers: dict[str, str] = Field(default_factory=dict)
    timeout_s: float = 30.0
    max_retries: int = 3

columns = Field(default_factory=list) class-attribute instance-attribute

endpoint_url = None class-attribute instance-attribute

headers = Field(default_factory=dict) class-attribute instance-attribute

max_retries = 3 class-attribute instance-attribute

timeout_s = 30.0 class-attribute instance-attribute

build_embed_option_kwargs(embed_invoke_url, embed_model_name, local_ingest_embed_backend=None, embed_api_key=None, embed_model_provider_prefix=None, embed_modality=None, text_elements_modality=None, structured_elements_modality=None, embed_granularity=None, embed_workers=None, embed_batch_size=None, embed_cpus_per_actor=None, embed_gpus_per_actor=None, embed_model_revision=None)

Build EmbedParams kwargs from CLI/request option values.

Source code in nemo_retriever/common/params/utils.py

def build_embed_option_kwargs(
    embed_invoke_url: str | None,
    embed_model_name: str | None,
    local_ingest_embed_backend: str | None = None,
    embed_api_key: str | None = None,
    embed_model_provider_prefix: str | None = None,
    embed_modality: str | None = None,
    text_elements_modality: str | None = None,
    structured_elements_modality: str | None = None,
    embed_granularity: str | None = None,
    embed_workers: int | None = None,
    embed_batch_size: int | None = None,
    embed_cpus_per_actor: float | None = None,
    embed_gpus_per_actor: float | None = None,
    embed_model_revision: str | None = None,
) -> Dict[str, Any]:
    """Build ``EmbedParams`` kwargs from CLI/request option values."""
    embed_kwargs: Dict[str, Any] = {}
    if embed_invoke_url is not None:
        embed_kwargs["embed_invoke_url"] = embed_invoke_url
    if embed_model_name is not None:
        # Remote HTTP embedding reads model_name; local/GPU paths read embed_model_name.
        embed_kwargs["model_name"] = embed_model_name
        embed_kwargs["embed_model_name"] = embed_model_name
    if embed_model_revision is not None:
        embed_kwargs["embed_model_revision"] = embed_model_revision
    if local_ingest_embed_backend is not None:
        embed_kwargs["local_ingest_embed_backend"] = local_ingest_embed_backend
    if embed_api_key is not None:
        embed_kwargs["api_key"] = embed_api_key
    if embed_model_provider_prefix is not None:
        embed_kwargs["embed_model_provider_prefix"] = embed_model_provider_prefix
    if embed_modality is not None:
        embed_kwargs["embed_modality"] = embed_modality
    if text_elements_modality is not None:
        embed_kwargs["text_elements_modality"] = text_elements_modality
    if structured_elements_modality is not None:
        embed_kwargs["structured_elements_modality"] = structured_elements_modality
    if embed_granularity is not None:
        embed_kwargs["embed_granularity"] = embed_granularity
    embed_tuning = _build_embed_batch_tuning(
        embed_workers=embed_workers,
        embed_batch_size=embed_batch_size,
        embed_cpus_per_actor=embed_cpus_per_actor,
        embed_gpus_per_actor=embed_gpus_per_actor,
    )
    if embed_tuning is not None:
        embed_kwargs["batch_tuning"] = embed_tuning
    return normalize_embed_kwargs(embed_kwargs)

resolve_split_params(split_config)

Resolve a user-supplied split_config dict into per-key effective params.

Returns a dict keyed by every entry in SPLIT_CONFIG_VALID_KEYS. Each value is one of: a TextChunkParams / HtmlChunkParams instance (chunking enabled for that key), None (key absent — chunking off via the default), or False (explicit opt-out sentinel).

Per-key values supplied by the caller may be a plain dict of chunk-params fields, a pre-built TextChunkParams / HtmlChunkParams instance (passed through verbatim), None, or False.

Source code in nemo_retriever/common/params/utils.py

def resolve_split_params(
    split_config: dict[str, Any] | None,
) -> dict[str, Any]:
    """Resolve a user-supplied split_config dict into per-key effective params.

    Returns a dict keyed by every entry in ``SPLIT_CONFIG_VALID_KEYS``. Each
    value is one of: a ``TextChunkParams`` / ``HtmlChunkParams`` instance
    (chunking enabled for that key), ``None`` (key absent — chunking off
    via the default), or ``False`` (explicit opt-out sentinel).

    Per-key values supplied by the caller may be a plain dict of
    chunk-params fields, a pre-built ``TextChunkParams`` /
    ``HtmlChunkParams`` instance (passed through verbatim), ``None``, or
    ``False``.
    """
    from nemo_retriever.common.params.models import HtmlChunkParams, TextChunkParams

    cfg = split_config or {}
    unknown = set(cfg) - SPLIT_CONFIG_VALID_KEYS
    if unknown:
        raise ValueError(
            f"Unknown split_config key(s): {sorted(unknown)}; " f"expected one of {sorted(SPLIT_CONFIG_VALID_KEYS)}"
        )

    out: dict[str, Any] = {}
    for key in SPLIT_CONFIG_VALID_KEYS:
        v = cfg.get(key)
        if v is None:
            out[key] = None
            continue
        if v is False:
            out[key] = False  # explicit opt-out (distinct from None / absent)
            continue
        if isinstance(v, TextChunkParams):  # HtmlChunkParams is a TextChunkParams subclass
            out[key] = v
            continue
        if isinstance(v, dict):
            cls = HtmlChunkParams if key == "html" else TextChunkParams
            out[key] = cls(**v)
            continue
        raise TypeError(
            f"split_config['{key}'] must be a TextChunkParams, dict, None, or False; got {type(v).__name__}"
        )
    return out