Customize & extend
NeMo Retriever Library ships with defaults tuned for strong recall on common document types. When those defaults are not enough, you can extend the library at several levels—from task keyword arguments on the fluent ingestor API through custom graph operators and vector-database adapters.
Use this page to choose an extension path and find the detailed guides in the repository.
The following table maps common needs to the right section:
| If you need to… | Start here |
|---|---|
| Tune extraction, chunking, embedding, or upload without new code | Start with task configuration |
| Add a small Python transformation between pipeline stages | User-defined functions (UDFs) |
| Build or reuse operators stage-by-stage | Custom graph pipelines |
| Store vectors in a backend other than LanceDB | Custom vector databases |
On this page
- Start with task configuration
- User-defined functions (UDFs)
- Custom graph pipelines
- Custom vector databases
- Related Topics
Start with task configuration
Most customization does not require new code. Chain tasks on create_ingestor(...) and pass keyword arguments to control extraction, chunking, embedding, and storage—for example extract_method, chunking and splitting options on .extract(), embed_modality on .embed(), and vdb_op / vdb_kwargs on .vdb_upload().
For parameter details, refer to the Python API guide. For chunking behavior and pipeline concepts, refer to Concepts.
User-defined functions (UDFs)
A user-defined function (UDF) wraps your Python logic as a first-class pipeline stage. In the graph model, UDFOperator turns a plain callable into an operator you can chain with built-in stages—for example to normalize HTML, apply a custom split, or call an external service between extract and embed steps.
Use UDFs when you need a small, self-contained transformation that is not covered by task keyword arguments.
Repository guides
- NeMo Retriever graph README —
UDFOperator— API, lifecycle, and when to useUDFOperatorversus a custom operator class - NimClient and custom NIM endpoints — call custom or self-hosted NIM microservices from UDF stages
Custom graph pipelines
When you need to compose pipelines stage-by-stage, reuse operators across workflows, or run the same graph in-process or with Ray Data, use the graph execution model instead of (or alongside) the fluent GraphIngestor API.
The graph package provides AbstractOperator, executors (InprocessExecutor, RayDataExecutor), and operator chaining with >>. Built-in ingestion operators live under nemo_retriever.operators; you can add your own operators or UDF stages anywhere in the chain.
For the full guide—including custom operator classes, executors, and graph shape constraints—refer to the NeMo Retriever graph README.
Custom vector databases
The supported user path for vector storage is LanceDB (vdb_op="lancedb"). That page covers upload, semantic retrieval, metadata filtering, and LanceDB deployment characteristics.
To integrate a different vector store, implement the VDB interface and wire it through graph IngestVdbOperator / RetrieveVdbOperator. NVIDIA validates the first-party LanceDB operator; you are responsible for testing and maintaining other backends.
Repository guides
- Vector DB package (source) —
VDBabstract base and LanceDB reference implementation
Partner and blueprint integrations (Elasticsearch, Pinecone, Teradata, and others) are summarized on Vector databases — Vector database partners.