Dart Pruner

class DartPruner : public trt_edgellm::rt::VisualTokenPruner

DART visual-token pruner, based on the paper “Stop Looking for Important Tokens in

Multimodal Language Models: Duplication Matters More” and registered as the default “dart”.

Each visual span keeps a proportional quota. Its highest-L1 image tokens and shared text tokens are pivots; remaining tokens with the lowest cosine similarity to those pivots are kept.

This embedding-level variant uses input embeddings because the paper’s decoder-layer-2 states are unavailable inside the monolithic engine. See the user guide for validation.

CUDA kernels compute row norms and batched pivot similarities; greedy selection runs on the host.

Public Functions

DartPruner(

VisualPrunerConfig const &config,

LLMEngineConfig const &engineConfig

)

Preallocates the selection work buffers from the engine limits.

Throws:

std::runtime_error – on invalid pivot configuration.

inline virtual char const *name() const noexcept override

Algorithm name (matches the registry key).