Qsa Indexer Jit Compiler#

struct QsaIndexerJitKey#

Key that identifies one NVRTC-compiled QSA indexer module variant.

One compilation produces a single cubin holding all extern “C” entry points (qsa_indexer_q_prep, qsa_indexer_k_compress, qsa_indexer_scores, qsa_indexer_ids_fill, qsa_indexer_expand, qsa_indexer_raw_k_tail_write_prefill, qsa_indexer_pre_decode, qsa_indexer_seal, qsa_indexer_scores_decode, qsa_indexer_topk_expand_decode, qsa_indexer_k_compress_append, qsa_indexer_scores_paged, qsa_indexer_tree_paths, qsa_indexer_tree_k_compress, qsa_indexer_tree_scores, qsa_indexer_tree_map_indices) specialized for (sm, dataType).

Public Functions

inline std::array<uint32_t, kNumWords> keyWords() const noexcept#

The fields in digest and hash order.

inline bool operator==(QsaIndexerJitKey const &other) const noexcept#

Public Members

int32_t sm = {}#
QsaIndexerJitDataType dataType = {QsaIndexerJitDataType::kHALF}#

Public Static Functions

static inline QsaIndexerJitKey fromKeyWords(
std::array<uint32_t, kNumWords> const &words
) noexcept#

Public Static Attributes

static constexpr size_t kNumWords = {2}#
bool trt_edgellm::canCompileQsaIndexerJitKernel(
QsaIndexerJitKey const &key
) noexcept#
QsaIndexerJitKernel trt_edgellm::compileQsaIndexerJitKernel(
QsaIndexerJitKey const &key
)#

Compile the QSA indexer kernels for one (sm, dataType) with NVRTC.

Results are shared process-wide through a compile cache keyed by the full key, so repeated calls (including concurrent ones) pay for at most one NVRTC compilation.

Throws:

std::invalid_argument – on an unsupported key, std::runtime_error on NVRTC failure.