Initialize Cos Sin Cache#

void trt_edgellm::kernel::initializeNormalRopeCosSin(
float *cosSinCache,
float rotaryBaseFrequency,
float rotaryScale,
float partialRotaryFactor,
int32_t rotaryDim,
int32_t rotaryEmbeddingMaxPositions,
cudaStream_t stream
)#

Initialize normal RoPE cos/sin cache.

Precomputes cos/sin values for standard rotary position encoding.

Parameters:

  • cosSinCache – Output cos/sin cache

  • rotaryBaseFrequency – Base frequency for RoPE

  • rotaryScale – Scaling factor

  • partialRotaryFactor – Fraction of RoPE angle slots initialized with non-identity values

  • rotaryDim – Rotary embedding dimension

  • rotaryEmbeddingMaxPositions – Maximum positions

  • stream – CUDA stream

void trt_edgellm::kernel::initializeLongRopeCosSin(
float *shortCosSinCache,
float *longCosSinCache,
float *shortFactor,
float *longFactor,
float rotaryBaseFrequency,
int32_t rotaryDim,
int32_t rotaryEmbeddingMaxPositions,
int32_t maxPositionEmbeddings,
int32_t originalMaxPositionEmbeddings,
cudaStream_t stream
)#

Initialize long RoPE cos/sin cache with interpolation.

Precomputes cos/sin values for long-context RoPE with position interpolation. Used for extending context length beyond original training range.

Parameters:

  • shortCosSinCache – Short-range cos/sin cache

  • longCosSinCache – Long-range cos/sin cache

  • shortFactor – Short interpolation factors

  • longFactor – Long interpolation factors

  • rotaryBaseFrequency – Base frequency

  • rotaryDim – Rotary dimension

  • rotaryEmbeddingMaxPositions – Maximum positions

  • maxPositionEmbeddings – Maximum position embeddings

  • originalMaxPositionEmbeddings – Original max positions from training

  • stream – CUDA stream

void trt_edgellm::kernel::initializeYarnCosSin(
float *cosSinCache,
float *invFreq,
float mscale,
int32_t rotaryDim,
int32_t rotaryEmbeddingMaxPositions,
cudaStream_t stream
)#

Initialize YaRN RoPE cos/sin cache from a precomputed per-dim inverse frequency.

Fills cos/sin as cos(pos * invFreq[d]) * mscale (and sin likewise) on device. invFreq (length rotaryDim/2) is the NTK-by-parts blended inverse frequency computed once on the host; this kernel only does the position sweep, mirroring initializeNormalRopeCosSin for the common rotary dims.

Parameters:

  • cosSinCache – Output cos/sin cache, shape [1, rotaryEmbeddingMaxPositions, rotaryDim]

  • invFreq – Device array of per-dimension inverse frequencies (length rotaryDim/2)

  • mscale – YaRN attention magnitude scale applied to cos/sin

  • rotaryDim – Rotary dimension

  • rotaryEmbeddingMaxPositions – Maximum positions

  • stream – CUDA stream

Warning

doxygenfunction: Unable to resolve function “trt_edgellm::kernel::initializeMRopeCosSin” with arguments None in doxygen xml output for project “TensorRT Edge-LLM” from directory: ../cpp_docs/xml. Potential matches: 

- void initializeMRopeCosSin(float *cosSinCache, float *mropePositionIds, float rotaryBaseFrequency, int64_t rotaryDim, int64_t rotaryEmbeddingMaxPositions, int64_t batchSize, bool interleaved, int32_t sectionH, int32_t sectionW, cudaStream_t stream)
- void initializeMRopeCosSin(float *cosSinCache, int64_t *mropePositionIds, float rotaryBaseFrequency, int64_t rotaryDim, int64_t rotaryEmbeddingMaxPositions, int64_t batchSize, bool interleaved, int32_t sectionH, int32_t sectionW, cudaStream_t stream)
void trt_edgellm::kernel::initializeTextOnlyMRopeCosSin(
float *cosSinCache,
float rotaryBaseFrequency,
int64_t rotaryDim,
int64_t maxPositions,
int64_t batchSize,
cudaStream_t stream
)#

Initialize MRoPE cos/sin cache for text-only inputs with sequential positions.

Initializes the MRoPE cache using sequential position IDs (pos[i] = i) for all 3 MRoPE sections. This is the correct default for text-only and audio-only modes where no spatial position information is available.

Parameters:

  • cosSinCache – Output cos/sin cache, shape [batchSize, maxPositions, rotaryDim]

  • rotaryBaseFrequency – Base frequency

  • rotaryDim – Rotary dimension

  • maxPositions – Maximum number of positions

  • batchSize – Number of batch slots to initialize

  • stream – CUDA stream