`deepseek_v2` model family

Tasks

Text Generation

Supported architectures and task heads

The Hugging Face values below are copied from checkpoint metadata at the recorded revision. The TRTMC task contract comes from the exact E2E recipe. Architecture identity selects or describes a source graph; it does not imply that TRTMC reproduces every Hugging Face head with the same model_type. For example, an encoder recipe may consume the base model and intentionally return hidden states instead of the checkpoint's pretraining or classification logits.

HF `model_type`	HF architecture / pipeline class	TRTMC task contract	Exact recipe profiles
`deepseek_v2`	`DeepseekV2ForCausalLM` Source: `config.architectures` Metadata: `config.json` at `604d5664dddd`	`text_generation_causal`	`deepseek-v2-lite` `deepseek-v2-lite-tp4`
`deepseek_v3`	`DeepseekV3ForCausalLM` Source: `config.architectures` Metadata: `config.json` at `ba144b0d3331`	`text_generation_causal`	`deepseek-v2-tiny` `deepseek-v2-tiny-tp2`

Declared recipes

Each row comes from a model-owned E2E manifest. It declares a test recipe; it is not a live pass receipt for every hardware target.

Recipe	Exact Hugging Face checkpoint	Task	Build configuration	Declared E2E cases	Manifest
`deepseek-v2-lite`	`deepseek-ai/DeepSeek-V2-Lite`	Text Generation	fp16; single device	deepseek-v2-lite	`tests/e2e/models/deepseek_v2/manifests/deepseek-v2-lite.json`
`deepseek-v2-lite-tp4`	`deepseek-ai/DeepSeek-V2-Lite`	Text Generation	family default; TP4	deepseek-v2-lite-tp4	`tests/e2e/models/deepseek_v2/manifests/deepseek-v2-lite-tp4.json`
`deepseek-v2-tiny`	`katuni4ka/tiny-random-deepseek-v3` Revision: `ba144b0d3331a5892aa588d82722d382be2b6e6b`	Text Generation	fp16; single device	deepseek-v2-tiny	`tests/e2e/models/deepseek_v2/manifests/deepseek-v2-tiny.json`
`deepseek-v2-tiny-tp2`	`katuni4ka/tiny-random-deepseek-v3` Revision: `ba144b0d3331a5892aa588d82722d382be2b6e6b`	Text Generation	family default; TP2	deepseek-v2-tiny-tp2	`tests/e2e/models/deepseek_v2/manifests/deepseek-v2-tiny-tp2.json`

Family-owned configuration

This family does not declare a family-owned --set namespace. Use the explicit CLI options shown below and the shared configuration namespaces documented in Configure Runtime Behavior.

Family-specific CLI contracts

Inputs and options below are filtered by the declared E2E task and the methods and configuration fields used by that family's native runtime implementation. The global CLI parser accepts a wider union of flags; flags absent here are not declared for this family.

`trtmc run`

Generate text from a text prompt.

Declared recipes: deepseek-v2-lite, deepseek-v2-lite-tp4, deepseek-v2-tiny, deepseek-v2-tiny-tp2

trtmc run <bundle.bundle> --prompt "<text>" [generation options]

Supported input or option	Requirement	Runtime behavior
`--prompt <TEXT>`	Required	Text input for this causal language-model recipe.
`--max-new-tokens <N>`	Optional	Limit the number of generated tokens or audio frames.
`--temperature <F>`	Optional	Set sampling temperature.
`--top-k <N>`	Optional	Restrict sampling to the top K tokens.
`--top-p <F>`	Optional	Enable nucleus sampling at probability P.
`--min-p <F>`	Optional	Filter tokens below min-p times the maximum probability.
`--seed <N>`	Optional	Set the sampling or diffusion seed.
`--chat-template`	Optional	Apply the chat template packaged with the bundle.
`--no-thinking`	Optional	Disable a supported reasoning or thinking mode.
`--greedy`	Optional	Select deterministic greedy decoding by setting temperature to zero.
`--num-samples <N>`	Optional	Run N independent text generations.
`--output <PATH>`	Optional	Write generated samples as JSON Lines.
`--benchmark <N>`	Optional	Run N timed generation iterations.
`--warmup <N>`	Optional with --benchmark	Warm-up iterations before generation timing.

Code basis

Runtime provider: deepseek_v2

include/trtmc/pipeline.h
src/cli/main.cpp
src/runtime/models/deepseek_v2/pipeline.cpp
src/runtime/models/deepseek_v2/sampler.cpp
tests/e2e/models/deepseek_v2/manifests/deepseek-v2-lite-tp4.json
tests/e2e/models/deepseek_v2/manifests/deepseek-v2-lite.json
tests/e2e/models/deepseek_v2/manifests/deepseek-v2-tiny-tp2.json
tests/e2e/models/deepseek_v2/manifests/deepseek-v2-tiny.json

Bundle lifecycle commands

These commands apply to every declared recipe in this family; they do not add task inputs or model capabilities.

`trtmc build`

Build one exact checkpoint into a TensorRT-Model-Connect bundle.

trtmc build <hf-id> --output <bundle.bundle>

`trtmc inspect`

Inspect bundle metadata, runtime identity, and packaged sections.

trtmc inspect <bundle.bundle>

See the CLI Reference for all options and limitations.

deepseek_v2 model family