GuidedDecodingParams#

class tensorrt_llm.llmapi.GuidedDecodingParams(
*,
json: str | BaseModel | dict | None = None,
regex: str | None = None,
grammar: str | None = None,
json_object: bool = False,
structural_tag: str | None = None,
)[source]#

Bases: object

Guided decoding parameters for text generation. Only one of the fields could be effective.

Parameters:

  • json (str, pydantic.main.BaseModel, dict, optional) – The generated text is amenable to json format with additional user-specified restrictions, namely schema. Defaults to None.

  • regex (str, optional) – The generated text is amenable to the user-specified regular expression. Defaults to None.

  • grammar (str, optional) – The generated text is amenable to the user-specified extended Backus-Naur form (EBNF) grammar. Defaults to None.

  • json_object (bool) – If True, the generated text is amenable to json format. Defaults to False.

  • structural_tag (str, optional) – The generated text is amenable to the user-specified structural tag. Structural tag is supported by xgrammar backend only. Defaults to None.

__init__(
*,
json: str | BaseModel | dict | None = None,
regex: str | None = None,
grammar: str | None = None,
json_object: bool = False,
structural_tag: str | None = None,
) → None#
grammar: str | None#
json: str | BaseModel | dict | None#
json_object: bool#
regex: str | None#
structural_tag: str | None#