SchedulingParams#

class tensorrt_llm.llmapi.SchedulingParams(
*,
attention_dp_rank: int | None = None,
attention_dp_relax: bool = True,
agent_hierarchy: List[Tuple[str, int]] | None = None,
)[source]#

Bases: object

Schedule parameters.

Parameters:

  • attention_dp_rank (int) – The rank of target attention dp

  • attention_dp_relax (bool) – Whether to allow the request to be scheduled to other attention dp for better throughput. Defaults to True.

  • agent_hierarchy (AgentHierarchy) – Path of (agent_type, node_id) tuples identifying this request’s position in an agent execution tree. Used by the batch scheduler for hierarchy-aware scheduling.

__init__(
*,
attention_dp_rank: int | None = None,
attention_dp_relax: bool = True,
agent_hierarchy: List[Tuple[str, int]] | None = None,
) → None#
agent_hierarchy: List[Tuple[str, int]] | None#
attention_dp_rank: int | None#
attention_dp_relax: bool#