Alpamayo1 Action Runner#

class Alpamayo1ActionRunner#

Standalone action / diffusion head for Alpamayo 1 trajectory prediction.

Consumes the VLM KV cache after generation and produces future trajectory waypoints via a flow-matching denoising loop.

Public Types

using KvAllGatherFn = std::function<bool(void const *sendBuffer, void *recvBuffer, int32_t sendCount, cudaStream_t stream)>#

Public Functions

Alpamayo1ActionRunner(
std::string const &engineDir,
std::string const &checkpointDir,
cudaStream_t stream,
KVCacheManager::Config const &kvCacheConfig,
int32_t tensorParallelSize,
int32_t tensorParallelRank,
std::vector<std::array<float, 2>> kvCacheScales
)#

Load action engine, config, and allocate tensors.

config.json must include rope_theta and num_hidden_layers (decoder layer count)

Parameters:
  • engineDir – Path to directory containing action.engine and config.json

  • checkpointDir – Original model checkpoint used for runtime weights

  • stream – CUDA stream for operations

  • kvCacheConfig – KV cache layout from the LLM (from KVCacheManager::Config())

Throws:

std::runtime_error – If engine loading, configuration parsing, or allocation fails.

~Alpamayo1ActionRunner() noexcept = default#
int64_t getRequiredContextMemorySize() const#

Get the required context memory size for this engine.

Returns:

Required context memory size in bytes

bool setContextMemory(rt::Tensor &sharedContextMemory)#

Set shared context memory for the execution context.

Note

The tensor size must be >= getRequiredContextMemorySize(). Must be called before infer().

Parameters:

sharedContextMemory – Tensor containing the shared device memory (must be on GPU)

Returns:

True on success, false if the tensor is too small

inline void setKvAllGather(KvAllGatherFn kvAllGather)#

Install the FP16 TP all-gather used to reconstruct full action KV tensors.

inline bool isRoot() const noexcept#
inline action::ActionModelType getModelType() const noexcept#

Get action head model type.

Returns:

Action model type

inline void setNoiseSeed(int32_t seed) noexcept#

Set the random seed used when initializing the diffusion noise trajectory.

Parameters:

seed – Random seed value

inline int32_t getMaxKVCacheCapacity() const noexcept#

Get the max KV cache capacity the action engine was built with.

Returns:

Maximum KV cache capacity (from builder_config in engine config.json)

std::vector<std::vector<FutureTrajectoryPoint>> sampleTrajectory(
cudaStream_t stream,
HybridCacheManager const &kvcache,
ActionKvBatchView const &batch
)#

Run one batched diffusion/flow-matching loop and return future trajectory waypoints for all batch items. Call preprocess() once per request before this (prefill path).

Parameters:
  • stream – CUDA stream for operations

  • kvcache – KV cache page pool containing the VLM outputs

  • batch – Dense action-local page table, lengths, and positional metadata

bool preprocess(
LLMGenerationRequest const &request,
std::vector<std::vector<int32_t>> &batchedInputIds,
tokenizer::Tokenizer const *tokenizer
)#

Preprocess batched token IDs for Alpamayo (e.g. replace <|traj_history|> pads with trajectory tokens), validate batch size against allocated action buffers, and initialize diffusion noise for the request.

Returns:

True on success, false if trajectory placeholder fill or noise setup fails

Public Static Functions

static inline void validateKVPoolDataType(
nvinfer1::DataType poolType,
nvinfer1::DataType bindingType,
std::string const &bindingName
)#

Validate a KV binding’s declared dtype before sharing the LLM pool address.

static inline void validateKVPoolPageCount(
int64_t actualPages,
int64_t minPages,
int64_t optPages,
int64_t maxPages
)#

Validate the fixed action input profile against the actual LLM physical pool size.

struct ActionConfig#

Configuration parsed from the action engine’s config.json.

Public Members

std::vector<std::array<float, 2>> kvCacheScales#

FP8 codec used by the exported action plugins.

float ropeTheta = {0.0F}#

RoPE base frequency (from rope_theta)

int32_t mropeSectionH = {0}#

MRoPE frequency pairs for height dimension (from rope_parameters.mrope_section[1])

int32_t mropeSectionW = {0}#

MRoPE frequency pairs for width dimension (from rope_parameters.mrope_section[2])

int32_t numDecoderLayers = {0}#

Number of transformer decoder layers (from num_hidden_layers)

int32_t numTrajTokens = {0}#

Number of trajectory history tokens (from num_traj_tokens)

int32_t trajTokenStart = {0}#

Vocabulary ID of the first trajectory token (from traj_token_start)

int32_t maxKVCacheCapacity = {0}#

Maximum KV cache sequence capacity (from builder_config.max_kv_cache_capacity)

int32_t maxActionBatchSize = {0}#

Action engine batch capacity (from builder_config.max_batch_size)

int32_t numKVHeads = {0}#

Number of key-value heads (from num_key_value_heads)

int32_t numActionTokens = {0}#

Number of action tokens from n_diffusion_tokens.

int32_t headDim = {0}#

Head dimension (from head_dim)