Qsa Indexer Jit Compiler#
-
struct QsaIndexerJitKey#
Key that identifies one NVRTC-compiled QSA indexer module variant.
One compilation produces a single cubin holding all extern “C” entry points (qsa_indexer_q_prep, qsa_indexer_k_compress, qsa_indexer_scores, qsa_indexer_ids_fill, qsa_indexer_expand, qsa_indexer_raw_k_tail_write_prefill, qsa_indexer_pre_decode, qsa_indexer_seal, qsa_indexer_scores_decode, qsa_indexer_topk_expand_decode, qsa_indexer_k_compress_append, qsa_indexer_scores_paged, qsa_indexer_tree_paths, qsa_indexer_tree_k_compress, qsa_indexer_tree_scores, qsa_indexer_tree_map_indices) specialized for (sm, dataType).
Public Functions
-
inline std::array<uint32_t, kNumWords> keyWords() const noexcept#
The fields in digest and hash order.
-
inline bool operator==(QsaIndexerJitKey const &other) const noexcept#
Public Static Functions
- static inline QsaIndexerJitKey fromKeyWords(
- std::array<uint32_t, kNumWords> const &words
Public Static Attributes
-
static constexpr size_t kNumWords = {2}#
-
inline std::array<uint32_t, kNumWords> keyWords() const noexcept#
- bool trt_edgellm::canCompileQsaIndexerJitKernel(
- QsaIndexerJitKey const &key
- QsaIndexerJitKernel trt_edgellm::compileQsaIndexerJitKernel(
- QsaIndexerJitKey const &key
Compile the QSA indexer kernels for one (sm, dataType) with NVRTC.
Results are shared process-wide through a compile cache keyed by the full key, so repeated calls (including concurrent ones) pay for at most one NVRTC compilation.
- Throws:
std::invalid_argument – on an unsupported key, std::runtime_error on NVRTC failure.