Model Recipes

Quick Start for Popular Models

The table below contains trtllm-serve commands that can be used to easily deploy popular models including DeepSeek-R1, gpt-oss, Llama 4, Qwen3, and more.

We maintain LLM API configuration files for these models containing recommended performance settings in two locations:

• Curated Examples: examples/configs/curated - Hand-picked configurations for common scenarios.

• Comprehensive Database: examples/configs/database - A more comprehensive set of known-good configurations for various GPUs and traffic patterns.

The TensorRT LLM Docker container makes these config files available at /app/tensorrt_llm/examples/configs/curated and /app/tensorrt_llm/examples/configs/database respectively. You can reference them as needed:

export TRTLLM_DIR="/app/tensorrt_llm" # path to the TensorRT LLM repo in your local environment

Note

The configs here are specifically optimized for a target ISL/OSL (Input/Output Sequence Length) of 1024/1024. If your traffic pattern is different, refer to the Comprehensive Configuration Database section below which covers a larger set of traffic patterns and performance profiles.

This table is designed to provide a straightforward starting point; for detailed model-specific deployment guides, check out the guides below.

Model Name	GPU	Inference Scenario	Config	Command
DeepSeek-R1	H100, H200	Max Throughput	deepseek-r1-throughput.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/curated/deepseek-r1-throughput.yaml
DeepSeek-R1	B200, GB200	Max Throughput	deepseek-r1-deepgemm.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/curated/deepseek-r1-deepgemm.yaml
DeepSeek-R1 (NVFP4)	B200, GB200	Max Throughput	deepseek-r1-throughput.yaml	trtllm-serve nvidia/DeepSeek-R1-FP4 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/curated/deepseek-r1-throughput.yaml
DeepSeek-R1 (NVFP4)	B200, GB200	Min Latency	deepseek-r1-latency.yaml	trtllm-serve nvidia/DeepSeek-R1-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/curated/deepseek-r1-latency.yaml
gpt-oss-120b	Any	Max Throughput	gpt-oss-120b-throughput.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/curated/gpt-oss-120b-throughput.yaml
gpt-oss-120b	Any	Min Latency	gpt-oss-120b-latency.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/curated/gpt-oss-120b-latency.yaml
Qwen3-Next-80B-A3B-Thinking	Any	Max Throughput	qwen3-next.yaml	trtllm-serve Qwen/Qwen3-Next-80B-A3B-Thinking --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/curated/qwen3-next.yaml
Qwen3 family (e.g. Qwen3-30B-A3B)	Any	Max Throughput	qwen3.yaml	trtllm-serve Qwen/Qwen3-30B-A3B --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/curated/qwen3.yaml (swap to another Qwen3 model name as needed)
Llama-3.3-70B (FP8)	Any	Max Throughput	llama-3.3-70b.yaml	trtllm-serve nvidia/Llama-3.3-70B-Instruct-FP8 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/curated/llama-3.3-70b.yaml
Llama 4 Scout (FP8)	Any	Max Throughput	llama-4-scout.yaml	trtllm-serve nvidia/Llama-4-Scout-17B-16E-Instruct-FP8 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/curated/llama-4-scout.yaml

Model-Specific Deployment Guides

The deployment guides below provide more detailed instructions for serving specific models with TensorRT LLM.

• Deployment Guide for DeepSeek R1 on TensorRT LLM - Blackwell & Hopper Hardware
• Deployment Guide for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware
• Deployment Guide for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware
• Deployment Guide for GPT-OSS on TensorRT-LLM - Blackwell Hardware
• Deployment Guide for Qwen3 on TensorRT LLM - Blackwell & Hopper Hardware
• Deployment Guide for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
• Deployment Guide for Kimi K2 Thinking on TensorRT LLM - Blackwell

Comprehensive Configuration Database

The table below lists all available pre-configured model scenarios in the TensorRT LLM configuration database. Each row represents a specific model, GPU, and performance profile combination with recommended request settings.

Note

Traffic Patterns: The ISL (Input Sequence Length) and OSL (Output Sequence Length) values in each configuration represent the maximum supported values for that config. Requests exceeding these limits may result in errors.

To handle requests with input sequences longer than the configured ISL, add the following to your config file:

enable_chunked_prefill: true

This enables chunked prefill, which processes long input sequences in chunks rather than requiring them to fit within a single prefill operation. Note that enabling chunked prefill does not guarantee optimal performance—these configs are tuned for the specified ISL/OSL.

DeepSeek-R1

GPU	Performance Profile	ISL / OSL	Concurrency	Config	Command
8xB200_NVL	Min Latency	1024 / 1024	4	1k1k_tp8_conc4.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/B200/1k1k_tp8_conc4.yaml
8xB200_NVL	Low Latency	1024 / 1024	8	1k1k_tp8_conc8.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/B200/1k1k_tp8_conc8.yaml
8xB200_NVL	Balanced	1024 / 1024	16	1k1k_tp8_conc16.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/B200/1k1k_tp8_conc16.yaml
8xB200_NVL	High Throughput	1024 / 1024	32	1k1k_tp8_conc32.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/B200/1k1k_tp8_conc32.yaml
8xB200_NVL	Max Throughput	1024 / 1024	64	1k1k_tp8_conc64.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/B200/1k1k_tp8_conc64.yaml
8xB200_NVL	Min Latency	8192 / 1024	4	8k1k_tp8_conc4.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/B200/8k1k_tp8_conc4.yaml
8xB200_NVL	Low Latency	8192 / 1024	8	8k1k_tp8_conc8.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/B200/8k1k_tp8_conc8.yaml
8xB200_NVL	Balanced	8192 / 1024	16	8k1k_tp8_conc16.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/B200/8k1k_tp8_conc16.yaml
8xB200_NVL	High Throughput	8192 / 1024	32	8k1k_tp8_conc32.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/B200/8k1k_tp8_conc32.yaml
8xB200_NVL	Max Throughput	8192 / 1024	64	8k1k_tp8_conc64.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/B200/8k1k_tp8_conc64.yaml
8xH200_SXM	Min Latency	1024 / 1024	4	1k1k_tp8_conc4.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/H200/1k1k_tp8_conc4.yaml
8xH200_SXM	Low Latency	1024 / 1024	8	1k1k_tp8_conc8.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/H200/1k1k_tp8_conc8.yaml
8xH200_SXM	Balanced	1024 / 1024	16	1k1k_tp8_conc16.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/H200/1k1k_tp8_conc16.yaml
8xH200_SXM	High Throughput	1024 / 1024	32	1k1k_tp8_conc32.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/H200/1k1k_tp8_conc32.yaml
8xH200_SXM	Max Throughput	1024 / 1024	64	1k1k_tp8_conc64.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/H200/1k1k_tp8_conc64.yaml
8xH200_SXM	Min Latency	8192 / 1024	4	8k1k_tp8_conc4.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/H200/8k1k_tp8_conc4.yaml
8xH200_SXM	Low Latency	8192 / 1024	8	8k1k_tp8_conc8.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/H200/8k1k_tp8_conc8.yaml
8xH200_SXM	Balanced	8192 / 1024	16	8k1k_tp8_conc16.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/H200/8k1k_tp8_conc16.yaml
8xH200_SXM	High Throughput	8192 / 1024	32	8k1k_tp8_conc32.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/H200/8k1k_tp8_conc32.yaml
8xH200_SXM	Max Throughput	8192 / 1024	64	8k1k_tp8_conc64.yaml	trtllm-serve deepseek-ai/DeepSeek-R1-0528 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/deepseek-ai/DeepSeek-R1-0528/H200/8k1k_tp8_conc64.yaml

DeepSeek-R1 (NVFP4)

GPU	Performance Profile	ISL / OSL	Concurrency	Config	Command
4xB200_NVL	Min Latency	1024 / 1024	4	1k1k_tp4_conc4.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp4_conc4.yaml
8xB200_NVL	Low Latency	1024 / 1024	4	1k1k_tp8_conc4.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp8_conc4.yaml
4xB200_NVL	Low Latency	1024 / 1024	8	1k1k_tp4_conc8.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp4_conc8.yaml
8xB200_NVL	Low Latency	1024 / 1024	8	1k1k_tp8_conc8.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp8_conc8.yaml
4xB200_NVL	Low Latency	1024 / 1024	16	1k1k_tp4_conc16.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp4_conc16.yaml
8xB200_NVL	Low Latency	1024 / 1024	16	1k1k_tp8_conc16.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp8_conc16.yaml
4xB200_NVL	Low Latency	1024 / 1024	32	1k1k_tp4_conc32.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp4_conc32.yaml
8xB200_NVL	High Throughput	1024 / 1024	32	1k1k_tp8_conc32.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp8_conc32.yaml
4xB200_NVL	High Throughput	1024 / 1024	64	1k1k_tp4_conc64.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp4_conc64.yaml
8xB200_NVL	High Throughput	1024 / 1024	64	1k1k_tp8_conc64.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp8_conc64.yaml
4xB200_NVL	High Throughput	1024 / 1024	128	1k1k_tp4_conc128.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp4_conc128.yaml
8xB200_NVL	High Throughput	1024 / 1024	128	1k1k_tp8_conc128.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp8_conc128.yaml
4xB200_NVL	High Throughput	1024 / 1024	256	1k1k_tp4_conc256.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp4_conc256.yaml
8xB200_NVL	Max Throughput	1024 / 1024	256	1k1k_tp8_conc256.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/1k1k_tp8_conc256.yaml
4xB200_NVL	Min Latency	8192 / 1024	4	8k1k_tp4_conc4.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp4_conc4.yaml
8xB200_NVL	Low Latency	8192 / 1024	4	8k1k_tp8_conc4.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp8_conc4.yaml
4xB200_NVL	Low Latency	8192 / 1024	8	8k1k_tp4_conc8.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp4_conc8.yaml
8xB200_NVL	Low Latency	8192 / 1024	8	8k1k_tp8_conc8.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp8_conc8.yaml
4xB200_NVL	Low Latency	8192 / 1024	16	8k1k_tp4_conc16.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp4_conc16.yaml
8xB200_NVL	Low Latency	8192 / 1024	16	8k1k_tp8_conc16.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp8_conc16.yaml
4xB200_NVL	Low Latency	8192 / 1024	32	8k1k_tp4_conc32.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp4_conc32.yaml
8xB200_NVL	High Throughput	8192 / 1024	32	8k1k_tp8_conc32.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp8_conc32.yaml
4xB200_NVL	High Throughput	8192 / 1024	64	8k1k_tp4_conc64.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp4_conc64.yaml
8xB200_NVL	High Throughput	8192 / 1024	64	8k1k_tp8_conc64.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp8_conc64.yaml
4xB200_NVL	High Throughput	8192 / 1024	128	8k1k_tp4_conc128.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp4_conc128.yaml
8xB200_NVL	High Throughput	8192 / 1024	128	8k1k_tp8_conc128.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp8_conc128.yaml
4xB200_NVL	High Throughput	8192 / 1024	256	8k1k_tp4_conc256.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp4_conc256.yaml
8xB200_NVL	Max Throughput	8192 / 1024	256	8k1k_tp8_conc256.yaml	trtllm-serve nvidia/DeepSeek-R1-0528-FP4-v2 --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/nvidia/DeepSeek-R1-0528-FP4-v2/B200/8k1k_tp8_conc256.yaml

gpt-oss-120b

GPU	Performance Profile	ISL / OSL	Concurrency	Config	Command
B200_NVL	Min Latency	1024 / 1024	4	1k1k_tp1_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp1_conc4.yaml
2xB200_NVL	Low Latency	1024 / 1024	4	1k1k_tp2_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp2_conc4.yaml
4xB200_NVL	Low Latency	1024 / 1024	4	1k1k_tp4_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp4_conc4.yaml
8xB200_NVL	Low Latency	1024 / 1024	4	1k1k_tp8_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp8_conc4.yaml
B200_NVL	Low Latency	1024 / 1024	8	1k1k_tp1_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp1_conc8.yaml
2xB200_NVL	Low Latency	1024 / 1024	8	1k1k_tp2_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp2_conc8.yaml
4xB200_NVL	Low Latency	1024 / 1024	8	1k1k_tp4_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp4_conc8.yaml
8xB200_NVL	Low Latency	1024 / 1024	8	1k1k_tp8_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp8_conc8.yaml
B200_NVL	Low Latency	1024 / 1024	16	1k1k_tp1_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp1_conc16.yaml
2xB200_NVL	Low Latency	1024 / 1024	16	1k1k_tp2_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp2_conc16.yaml
4xB200_NVL	High Throughput	1024 / 1024	16	1k1k_tp4_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp4_conc16.yaml
8xB200_NVL	High Throughput	1024 / 1024	16	1k1k_tp8_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp8_conc16.yaml
B200_NVL	High Throughput	1024 / 1024	32	1k1k_tp1_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp1_conc32.yaml
2xB200_NVL	High Throughput	1024 / 1024	32	1k1k_tp2_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp2_conc32.yaml
4xB200_NVL	High Throughput	1024 / 1024	32	1k1k_tp4_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp4_conc32.yaml
8xB200_NVL	High Throughput	1024 / 1024	32	1k1k_tp8_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp8_conc32.yaml
B200_NVL	High Throughput	1024 / 1024	64	1k1k_tp1_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp1_conc64.yaml
2xB200_NVL	High Throughput	1024 / 1024	64	1k1k_tp2_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp2_conc64.yaml
4xB200_NVL	High Throughput	1024 / 1024	64	1k1k_tp4_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp4_conc64.yaml
8xB200_NVL	Max Throughput	1024 / 1024	64	1k1k_tp8_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k1k_tp8_conc64.yaml
B200_NVL	Min Latency	1024 / 8192	4	1k8k_tp1_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp1_conc4.yaml
2xB200_NVL	Low Latency	1024 / 8192	4	1k8k_tp2_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp2_conc4.yaml
4xB200_NVL	Low Latency	1024 / 8192	4	1k8k_tp4_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp4_conc4.yaml
8xB200_NVL	Low Latency	1024 / 8192	4	1k8k_tp8_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp8_conc4.yaml
B200_NVL	Low Latency	1024 / 8192	8	1k8k_tp1_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp1_conc8.yaml
2xB200_NVL	Low Latency	1024 / 8192	8	1k8k_tp2_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp2_conc8.yaml
4xB200_NVL	Low Latency	1024 / 8192	8	1k8k_tp4_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp4_conc8.yaml
8xB200_NVL	Low Latency	1024 / 8192	8	1k8k_tp8_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp8_conc8.yaml
B200_NVL	Low Latency	1024 / 8192	16	1k8k_tp1_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp1_conc16.yaml
2xB200_NVL	Low Latency	1024 / 8192	16	1k8k_tp2_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp2_conc16.yaml
4xB200_NVL	High Throughput	1024 / 8192	16	1k8k_tp4_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp4_conc16.yaml
8xB200_NVL	High Throughput	1024 / 8192	16	1k8k_tp8_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp8_conc16.yaml
B200_NVL	High Throughput	1024 / 8192	32	1k8k_tp1_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp1_conc32.yaml
2xB200_NVL	High Throughput	1024 / 8192	32	1k8k_tp2_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp2_conc32.yaml
4xB200_NVL	High Throughput	1024 / 8192	32	1k8k_tp4_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp4_conc32.yaml
8xB200_NVL	High Throughput	1024 / 8192	32	1k8k_tp8_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp8_conc32.yaml
B200_NVL	High Throughput	1024 / 8192	64	1k8k_tp1_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp1_conc64.yaml
2xB200_NVL	High Throughput	1024 / 8192	64	1k8k_tp2_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp2_conc64.yaml
4xB200_NVL	High Throughput	1024 / 8192	64	1k8k_tp4_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp4_conc64.yaml
8xB200_NVL	Max Throughput	1024 / 8192	64	1k8k_tp8_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/1k8k_tp8_conc64.yaml
B200_NVL	Min Latency	8192 / 1024	4	8k1k_tp1_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp1_conc4.yaml
2xB200_NVL	Low Latency	8192 / 1024	4	8k1k_tp2_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp2_conc4.yaml
4xB200_NVL	Low Latency	8192 / 1024	4	8k1k_tp4_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp4_conc4.yaml
8xB200_NVL	Low Latency	8192 / 1024	4	8k1k_tp8_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp8_conc4.yaml
B200_NVL	Low Latency	8192 / 1024	8	8k1k_tp1_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp1_conc8.yaml
2xB200_NVL	Low Latency	8192 / 1024	8	8k1k_tp2_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp2_conc8.yaml
4xB200_NVL	Low Latency	8192 / 1024	8	8k1k_tp4_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp4_conc8.yaml
8xB200_NVL	Low Latency	8192 / 1024	8	8k1k_tp8_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp8_conc8.yaml
B200_NVL	Low Latency	8192 / 1024	16	8k1k_tp1_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp1_conc16.yaml
2xB200_NVL	Low Latency	8192 / 1024	16	8k1k_tp2_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp2_conc16.yaml
4xB200_NVL	High Throughput	8192 / 1024	16	8k1k_tp4_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp4_conc16.yaml
8xB200_NVL	High Throughput	8192 / 1024	16	8k1k_tp8_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp8_conc16.yaml
B200_NVL	High Throughput	8192 / 1024	32	8k1k_tp1_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp1_conc32.yaml
2xB200_NVL	High Throughput	8192 / 1024	32	8k1k_tp2_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp2_conc32.yaml
4xB200_NVL	High Throughput	8192 / 1024	32	8k1k_tp4_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp4_conc32.yaml
8xB200_NVL	High Throughput	8192 / 1024	32	8k1k_tp8_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp8_conc32.yaml
B200_NVL	High Throughput	8192 / 1024	64	8k1k_tp1_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp1_conc64.yaml
2xB200_NVL	High Throughput	8192 / 1024	64	8k1k_tp2_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp2_conc64.yaml
4xB200_NVL	High Throughput	8192 / 1024	64	8k1k_tp4_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp4_conc64.yaml
8xB200_NVL	Max Throughput	8192 / 1024	64	8k1k_tp8_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/B200/8k1k_tp8_conc64.yaml
H200_SXM	Min Latency	1024 / 1024	4	1k1k_tp1_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp1_conc4.yaml
2xH200_SXM	Low Latency	1024 / 1024	4	1k1k_tp2_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp2_conc4.yaml
4xH200_SXM	Low Latency	1024 / 1024	4	1k1k_tp4_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp4_conc4.yaml
8xH200_SXM	Low Latency	1024 / 1024	4	1k1k_tp8_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp8_conc4.yaml
H200_SXM	Low Latency	1024 / 1024	8	1k1k_tp1_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp1_conc8.yaml
2xH200_SXM	Low Latency	1024 / 1024	8	1k1k_tp2_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp2_conc8.yaml
4xH200_SXM	Low Latency	1024 / 1024	8	1k1k_tp4_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp4_conc8.yaml
8xH200_SXM	Low Latency	1024 / 1024	8	1k1k_tp8_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp8_conc8.yaml
H200_SXM	Low Latency	1024 / 1024	16	1k1k_tp1_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp1_conc16.yaml
2xH200_SXM	Low Latency	1024 / 1024	16	1k1k_tp2_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp2_conc16.yaml
4xH200_SXM	High Throughput	1024 / 1024	16	1k1k_tp4_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp4_conc16.yaml
8xH200_SXM	High Throughput	1024 / 1024	16	1k1k_tp8_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp8_conc16.yaml
H200_SXM	High Throughput	1024 / 1024	32	1k1k_tp1_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp1_conc32.yaml
2xH200_SXM	High Throughput	1024 / 1024	32	1k1k_tp2_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp2_conc32.yaml
4xH200_SXM	High Throughput	1024 / 1024	32	1k1k_tp4_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp4_conc32.yaml
8xH200_SXM	High Throughput	1024 / 1024	32	1k1k_tp8_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp8_conc32.yaml
H200_SXM	High Throughput	1024 / 1024	64	1k1k_tp1_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp1_conc64.yaml
2xH200_SXM	High Throughput	1024 / 1024	64	1k1k_tp2_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp2_conc64.yaml
4xH200_SXM	High Throughput	1024 / 1024	64	1k1k_tp4_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp4_conc64.yaml
8xH200_SXM	Max Throughput	1024 / 1024	64	1k1k_tp8_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k1k_tp8_conc64.yaml
H200_SXM	Min Latency	1024 / 8192	4	1k8k_tp1_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp1_conc4.yaml
2xH200_SXM	Low Latency	1024 / 8192	4	1k8k_tp2_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp2_conc4.yaml
4xH200_SXM	Low Latency	1024 / 8192	4	1k8k_tp4_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp4_conc4.yaml
8xH200_SXM	Low Latency	1024 / 8192	4	1k8k_tp8_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp8_conc4.yaml
H200_SXM	Low Latency	1024 / 8192	8	1k8k_tp1_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp1_conc8.yaml
2xH200_SXM	Low Latency	1024 / 8192	8	1k8k_tp2_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp2_conc8.yaml
4xH200_SXM	Low Latency	1024 / 8192	8	1k8k_tp4_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp4_conc8.yaml
8xH200_SXM	Low Latency	1024 / 8192	8	1k8k_tp8_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp8_conc8.yaml
H200_SXM	Low Latency	1024 / 8192	16	1k8k_tp1_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp1_conc16.yaml
2xH200_SXM	Low Latency	1024 / 8192	16	1k8k_tp2_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp2_conc16.yaml
4xH200_SXM	High Throughput	1024 / 8192	16	1k8k_tp4_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp4_conc16.yaml
8xH200_SXM	High Throughput	1024 / 8192	16	1k8k_tp8_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp8_conc16.yaml
H200_SXM	High Throughput	1024 / 8192	32	1k8k_tp1_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp1_conc32.yaml
2xH200_SXM	High Throughput	1024 / 8192	32	1k8k_tp2_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp2_conc32.yaml
4xH200_SXM	High Throughput	1024 / 8192	32	1k8k_tp4_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp4_conc32.yaml
8xH200_SXM	High Throughput	1024 / 8192	32	1k8k_tp8_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp8_conc32.yaml
H200_SXM	High Throughput	1024 / 8192	64	1k8k_tp1_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp1_conc64.yaml
2xH200_SXM	High Throughput	1024 / 8192	64	1k8k_tp2_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp2_conc64.yaml
4xH200_SXM	High Throughput	1024 / 8192	64	1k8k_tp4_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp4_conc64.yaml
8xH200_SXM	Max Throughput	1024 / 8192	64	1k8k_tp8_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/1k8k_tp8_conc64.yaml
H200_SXM	Min Latency	8192 / 1024	4	8k1k_tp1_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp1_conc4.yaml
2xH200_SXM	Low Latency	8192 / 1024	4	8k1k_tp2_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp2_conc4.yaml
4xH200_SXM	Low Latency	8192 / 1024	4	8k1k_tp4_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp4_conc4.yaml
8xH200_SXM	Low Latency	8192 / 1024	4	8k1k_tp8_conc4.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp8_conc4.yaml
H200_SXM	Low Latency	8192 / 1024	8	8k1k_tp1_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp1_conc8.yaml
2xH200_SXM	Low Latency	8192 / 1024	8	8k1k_tp2_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp2_conc8.yaml
4xH200_SXM	Low Latency	8192 / 1024	8	8k1k_tp4_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp4_conc8.yaml
8xH200_SXM	Low Latency	8192 / 1024	8	8k1k_tp8_conc8.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp8_conc8.yaml
H200_SXM	Low Latency	8192 / 1024	16	8k1k_tp1_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp1_conc16.yaml
2xH200_SXM	Low Latency	8192 / 1024	16	8k1k_tp2_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp2_conc16.yaml
4xH200_SXM	High Throughput	8192 / 1024	16	8k1k_tp4_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp4_conc16.yaml
8xH200_SXM	High Throughput	8192 / 1024	16	8k1k_tp8_conc16.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp8_conc16.yaml
H200_SXM	High Throughput	8192 / 1024	32	8k1k_tp1_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp1_conc32.yaml
2xH200_SXM	High Throughput	8192 / 1024	32	8k1k_tp2_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp2_conc32.yaml
4xH200_SXM	High Throughput	8192 / 1024	32	8k1k_tp4_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp4_conc32.yaml
8xH200_SXM	High Throughput	8192 / 1024	32	8k1k_tp8_conc32.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp8_conc32.yaml
H200_SXM	High Throughput	8192 / 1024	64	8k1k_tp1_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp1_conc64.yaml
2xH200_SXM	High Throughput	8192 / 1024	64	8k1k_tp2_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp2_conc64.yaml
4xH200_SXM	High Throughput	8192 / 1024	64	8k1k_tp4_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp4_conc64.yaml
8xH200_SXM	Max Throughput	8192 / 1024	64	8k1k_tp8_conc64.yaml	trtllm-serve openai/gpt-oss-120b --extra_llm_api_options ${TRTLLM_DIR}/examples/configs/database/openai/gpt-oss-120b/H200/8k1k_tp8_conc64.yaml