Skip to main content

Build from Source

Use this path on Linux x86_64 or aarch64 for the first Qwen inference from source. Start at the repository root.

1. Select the GPU and start the container​

Change only GPU. The commands derive the SM used by CMake and select the matching development Dockerfile. Repository CI continues to use Dockerfile.

GPU=0
SM="$(
  nvidia-smi -i "$GPU" \
    --query-gpu=compute_cap \
    --format=csv,noheader,nounits |
  tr -d '.[:space:]'
)"
IMAGE="trtmc-quickstart"

case "$(uname -m)" in
  x86_64) DOCKERFILE=Dockerfile.dev.x86 ;;
  aarch64) DOCKERFILE=Dockerfile.dev.aarch64 ;;
  *) echo "Unsupported host architecture: $(uname -m)" >&2; exit 1 ;;
esac

docker build \
  -f "$DOCKERFILE" \
  -t "$IMAGE" .

SOURCE_DIR="$(git rev-parse --show-toplevel)"

docker run --rm -it \
  --gpus "device=${GPU}" \
  --ipc=host \
  --mount "type=bind,source=${SOURCE_DIR},target=/src" \
  --workdir /src \
  --env TRTMC_SM="$SM" \
  "$IMAGE" \
  bash

Run the remaining commands inside the container.

2. Build the native runtime​

python -m pip install --no-deps -e . -C py-only=true

TRTMC_BUILD_DIR="build-sm${TRTMC_SM}"

cmake -S . -B "$TRTMC_BUILD_DIR" -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CUDA_ARCHITECTURES="${TRTMC_SM}-real" \
  -DTRTMC_BUILD_BACKEND_TRT=ON \
  -DTRTMC_BUILD_BACKEND_RTX=OFF \
  -DTRTMC_BUILD_TESTS=OFF \
  -DTRTMC_BUILD_BENCHMARKS=OFF \
  -DTRTMC_ENABLE_LIBTORCH_MULTINOMIAL=OFF

cmake --build "$TRTMC_BUILD_DIR" --parallel "$(nproc)" --target \
  trtmc \
  trtmc_backend_trt \
  trtmc_model_qwen

export TRTMC_MODEL_PLUGIN_DIR="$TRTMC_BUILD_DIR/models"
export PATH="$PWD/$TRTMC_BUILD_DIR:$PATH"

This path skips CI-only Python profiles and unrelated model DSOs. Continue to Quick Start in the same container shell. Full-repository and advanced backend options belong in the Build System reference.