Using Dev Containers#

The TensorRT-LLM repository contains a Dev Containers configuration in .devcontainer. These files are intended for use with Visual Studio Code.

Due to the various container options supported by TensorRT-LLM (see Building from Source Code on Linux and NVIDIA/TensorRT-LLM), the Dev Container configuration also offers some degree of customization.

Generally, the initializeCommand in devcontainer.json will run make_env.py to generate an .env file for docker-compose. Most importantly, the docker-compose.yml uses ${DEV_CONTAINER_IMAGE} as base image. The generated .devcontainer/.env is not tracked by Git and combines data from the following sources:

  • jenkins/current_image_tags.properties which contains the image tags currently used by CI.

  • .devcontainer/devcontainer.env which contains common configuration settings and is tracked by Git.

  • .devcontainer/devcontainer.env.user (optional) which is ignored by Git and can be edited to customize the Dev Container behavior.

The source files are processed using sh, in the order in which they are listed above. Thus, features like command substitution are supported.

The following sections provide more detail on particular Dev Container configuration parameters which can be customized.

Note

After editing any of the configuration files, it may be necessary to execute the “Dev Containers: Reopen Folder in SSH” (if applicable) and “Dev Containers: Rebuild and Reopen in Container” Visual Studio Code commands.

Container image selection#

By default, make_env.py will attempt to auto-select a suitable container image as follows:

  1. Reuse the development container image used by CI. This requires access to the NVIDIA internal artifact repository.

  2. Use the most recent NGC Development container image associated with a Git tag which is reachable from the currently checked out commit.

  3. Build a development image locally.

Set DEV_CONTAINER_IMAGE=<some_uri> to bypass the aforementioned discovery mechanism if desired.

By setting LOCAL_BUILD=0, the local image build can be disabled. In this case, execution fails if no suitable pre-built image is found.

Setting LOCAL_BUILD=1 forces building of a local image, even if a pre-built image is available.

Volume Mounts#

Docker volume mounts can be customized by editing docker-compose.yml, which allows using any variables defined in .env.

By default, the Dev Container configuration mounts the VS Code workspace into /workspaces/tensorrt_llm and ~/.cache/huggingface into /huggingface. The source paths can be overridden by setting SOURCE_DIR and HOME_DIR in .devcontainer/devcontainer.env.user, respectively. This is of particular relevance when using Docker Rootless Mode, which requires configuring UID/GID translation using a tool like bindfs. The Dev Container scripts contain heuristics to detect Docker Rootless Mode and will issue an error if these variables are not set. An analogous logic is applied to HF_HOME.

Overriding Docker Compose configuration#

When starting the container, .devcontainer/docker-compose.yml is merged with .devcontainer/docker-compose.override.yml. The latter file is not tracked by Git and will be created by make_env.py if it does not exist.

This mechanism can be used, e.g., to add custom volume mounts:

# Example .devcontainer/docker-compose.override.yml
version: "3.9"
services:
  tensorrt_llm-dev:
    volumes:
      # Uncomment the following lines to enable
      # # Mount TRTLLM data volume:
      # - /home/scratch.trt_llm_data/:/home/scratch.trt_llm_data/:ro

It is possible to conditionally mount volumes by combining, e.g., [this method] (https://stackoverflow.com/a/61954812) and shell command substitution in .devcontainer/devcontainer.env.user.

If no .devcontainer/docker-compose.override.yml file is found, the Dev Container initialization script will create one with the contents listed above.