nvalchemi.csp.comparison.deduplicate_stream#
- class nvalchemi.csp.comparison.deduplicate_stream(count, read_typed_batch, *, cutoff, threshold, type_vocabulary=None, priority_order=None, device='cpu', input_batch_size=None, pair_block_size=None, max_batch_atoms=200_000, max_memory_fraction=0.85, summary_coordinate_count=32, include_hydrogens=True, confirm=None)[source]#
Deduplicate a pool in priority order while bounding descriptor blocks.
Candidate structures are resolved in
priority_orderagainst earlier retained representatives. Each proposed pair passes untyped, center-typed, and center-plus-neighbor-typed radial screens in that order when types are supplied, and only the untyped screen otherwise. Retention is greedy and order-dependent; radial matches are approximate and need not be transitive.When
summary_coordinate_countis positive, a preliminary conservative rejection check uses up to that many selected coordinates; the selected count may be lower. Pairs that pass this check go to the radial index matcher; summary bounds can reject a pair but never establish a match. Setting the count to zero disables the pilot and outer check and sends all still-eligible pairs to the index matcher, which may apply its built-in conservative summary bounds before scoring full descriptors. Candidate chunks contain up toinput_batch_sizerows. Screening keeps each active candidate tile live while it scans earlier representative blocks of up topair_block_sizerows.- Parameters:
count (int) – Number of logical rows available from
read_typed_batch.read_typed_batch (callable) – Given CPU int64 row IDs, returns a
(Batch, atom_types)pair. The int32 or int64 type vector aligns with all atoms, or isNoneon every call for untyped comparison. Loader values must remain stable.type_vocabulary (sequence of int, optional) – Fixed set of signed int32 type IDs used across batches. Required when the loader returns atom types.
priority_order (sequence of int or torch.Tensor, optional) – Permutation of logical row IDs that defines candidate and representative priority. Defaults to logical row order.
cutoff (float) – Descriptor cutoff in angstroms.
threshold (float) – Inclusive finite nonnegative radial mismatch threshold.
device (torch.device or str) – Device used to build and score descriptors. Defaults to CPU.
input_batch_size (int, optional) – Maximum rows requested per summary read and per candidate chunk. Defaults to 1024 on CUDA and 64 on CPU.
pair_block_size (int, optional) – Maximum earlier representatives considered together; it does not limit candidate chunk size. Defaults to 256 on CUDA and 32 on CPU.
max_batch_atoms (int, default=200000) – Maximum retained comparison atoms in one descriptor tile. Hydrogens are not counted when
include_hydrogens=False.max_memory_fraction (float, default=0.85) – Maximum fraction of CUDA memory available to PyTorch assigned to descriptor construction and scoring workspace, including reusable caching-allocator bytes. This is an estimate, not a reservation.
summary_coordinate_count (int, default=32) – Maximum number of coordinates used by the conservative preliminary rejection check, from 0 through 128. The selected count may be lower. Pairs that pass go to the radial index matcher; summary bounds can reject a pair but never establish a match. Zero disables the pilot and outer check and sends all still-eligible pairs to the index matcher, which may use its built-in conservative summary bounds before scoring full descriptors.
include_hydrogens (bool, default=True) – Whether hydrogen atoms participate in comparison. False removes all atomic-number-1 atoms before neighbor lists and descriptors are built. Supply types aligned with original atoms; retained labels and original structure-row IDs are preserved.
confirm (callable, optional) – Receives ordered radial-match proposals as int32 [K, 2] original row IDs on the comparison device. Column zero is the candidate; column one is an earlier retained representative. Return an ordered subset on the same device. A rejected proposal does not discard the candidate; later representatives are tried. Without a callback, radial matches suffice. Calls may be grouped across candidates; decisions must depend on pair data rather than the global callback invocation order.
- Returns:
retained_indicescontains logical IDs in priority order.representative_indicesis indexed by original logical ID and stores each row’s representative logical ID.multiplicitiesaligns withretained_indices.- Return type:
- Raises:
TypeError – If the loader, returned batch/type vector, or confirm callback has an unsupported type.
ValueError – If the input ordering, type vocabulary, threshold, cutoff, block sizes, or summary coordinate count are invalid.