Safety Attention Contract

struct AttentionParameters

Serialized identity is shared with the current exporter; only vanilla FP16 is admitted.

Public Members

AttentionRole role = {AttentionRole::kLLM}

int32_t qHeads = {}

int32_t kvHeads = {}

int32_t headSize = {}

float scale = {}

bool trt_edgellm::safety::parseAttentionFields(

AttentionRole role,

nvinfer1::PluginFieldCollection const *fields,

AttentionParameters &parameters

) noexcept

Parse the current export fields, rejecting unknown or non-vanilla features.

Warning

doxygenfunction: Unable to resolve function “trt_edgellm::safety::validateAttentionDescriptors” with arguments None in doxygen xml output for project “TensorRT Edge-LLM” from directory: ../cpp_docs/xml. Potential matches:

- bool validateAttentionDescriptors(AttentionParameters const &parameters, TensorDescriptor const *inputs, int32_t nbInputs, TensorDescriptor const *outputs, int32_t nbOutputs, AttentionLaunch &launch) noexcept
- bool validateAttentionDescriptors(AttentionParameters const &parameters, nvinfer2::safe::TensorDescriptor const *inputs, int32_t nbInputs, nvinfer2::safe::TensorDescriptor const *outputs, int32_t nbOutputs, AttentionLaunch &launch) noexcept

Warning

doxygenfunction: Unable to resolve function “trt_edgellm::safety::supportsAttentionFormat” with arguments None in doxygen xml output for project “TensorRT Edge-LLM” from directory: ../cpp_docs/xml. Potential matches:

- bool supportsAttentionFormat(AttentionParameters const &parameters, int32_t index, TensorDescriptor const &descriptor) noexcept
- bool supportsAttentionFormat(AttentionParameters const &parameters, int32_t index, nvinfer2::safe::TensorDescriptor const &descriptor) noexcept

Warning

doxygenfunction: Unable to resolve function “trt_edgellm::safety::validateAttentionBuildDescriptors” with arguments None in doxygen xml output for project “TensorRT Edge-LLM” from directory: ../cpp_docs/xml. Potential matches:

- bool validateAttentionBuildDescriptors(AttentionParameters const &parameters, TensorDescriptor const *inputs, int32_t nbInputs, TensorDescriptor const *outputs, int32_t nbOutputs, AttentionLaunch &launch) noexcept
- bool validateAttentionBuildDescriptors(AttentionParameters const &parameters, nvinfer2::safe::TensorDescriptor const *inputs, int32_t nbInputs, nvinfer2::safe::TensorDescriptor const *outputs, int32_t nbOutputs, AttentionLaunch &launch) noexcept

Warning

doxygenfunction: Unable to resolve function “trt_edgellm::safety::supportsAttentionTensor” with arguments None in doxygen xml output for project “TensorRT Edge-LLM” from directory: ../cpp_docs/xml. Potential matches:

- bool supportsAttentionTensor(AttentionParameters const &parameters, int32_t index, TensorDescriptor const &descriptor) noexcept
- bool supportsAttentionTensor(AttentionParameters const &parameters, int32_t index, nvinfer2::safe::TensorDescriptor const &descriptor) noexcept