<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
<!-- SPDX-License-Identifier: Apache-2.0 -->

<a id="gamepad-plugin"></a>

# Gamepad Plugin

C++ plugin that streams a **gamepad** (USB or Bluetooth) into the standard DeviceIO path in one
layout for every pad: buttons named by position and the two sticks and triggers. It reads the pad
through [SDL3](https://github.com/libsdl-org/SDL), whose drivers and
[SDL_GameControllerDB](https://github.com/mdqinc/SDL_GameControllerDB) mappings resolve each pad’s
own numbering, and pushes the state at 90 Hz. Source and plugin README:
[src/plugins/gamepad/README.md](https://github.com/NVIDIA/IsaacCapture/blob/main/src/plugins/gamepad/README.md).

Supported pads: Xbox (360, One, Series, USB and Bluetooth), PlayStation (DualShock 4, DualSense),
Nintendo Switch Pro, Steam Controller, and any pad with an SDL_GameControllerDB mapping. Validated:
Xbox 360 Controller.

This plugin follows the [Add a New Device](add_device.md) pattern (FlatBuffer schema → plugin `SchemaPusher` →
tracker → MCAP).

> ##### On this page
> 
> * [Components](#components)
> * [Standard layout](#standard-layout)
> * [Device access](#device-access)
> * [Build](#build)
> * [Usage](#usage)
>   * [Multiple pads](#multiple-pads)
> * [Published data](#published-data)
> * [Retargeters](#retargeters)
> * [Device status](#device-status)
> * [Troubleshooting](#troubleshooting)
> * [Tests](#tests)

<a id="components"></a>

## Components

- **Schema** — [src/core/schema/fbs/gamepad.fbs](https://github.com/NVIDIA/IsaacCapture/blob/main/src/core/schema/fbs/gamepad.fbs): `GamepadInputState`, the standard
  layout as named fields, inside `GamepadSnapshot` / `GamepadSnapshotRecord`.
- **Plugin** — [src/plugins/gamepad](https://github.com/NVIDIA/IsaacCapture/tree/main/src/plugins/gamepad): `core/` reads the pad through SDL3 and reopens it
  after an unplug; `app/` is the OpenXR push loop.
- **Tracker** — `GamepadTracker`, generated at configure time from the `gamepad` entry in
  [src/core/deviceio_trackers/trackers.toml](https://github.com/NVIDIA/IsaacCapture/blob/main/src/core/deviceio_trackers/trackers.toml).
- **Source node** — `GamepadSource` turns the tracker’s data into a `gamepad` tensor group
  (`GamepadInput`, indexed by `GamepadInputIndex`) for retargeters.

<a id="standard-layout"></a>

## Standard layout

`GamepadInputState` is a generic controller split in two, with the same naming as
`ControllerInputState`. Face buttons are named by position, so a binding means the same on every
pad:

| Field                       | Xbox        | PlayStation      | Nintendo   |
|-----------------------------|-------------|------------------|------------|
| `south_click`               | A           | Cross            | B          |
| `east_click`                | B           | Circle           | A          |
| `west_click`                | X           | Square           | Y          |
| `north_click`               | Y           | Triangle         | X          |
| `view_click` / `menu_click` | View / Menu | Create / Options | − / +      |

The other buttons are `shoulder_[left|right]_click`, `thumbstick_[left|right]_click` and
`dpad_[up|down|left|right]_click`; the axes are `thumbstick_[left|right]_[x|y]` in `[-1, 1]`
(right and up positive, as in OpenXR) and `trigger_[left|right]_value` in `[0, 1]`. These are the
inputs every standard pad has, the same set as OpenXR’s `xbox_controller` profile; pad-specific
extras such as the guide button, paddles and touchpad are not reported.

<a id="device-access"></a>

## Device access

On a desktop session udev grants the logged-in user access to gamepad nodes, so no setup is
needed. Without a local seat (SSH, a service, a container), add the user to the `input` group,
and pass the pad’s `/dev/input/event*` node into a container.

<a id="build"></a>

## Build

Linux only. Complete the [source-build prerequisites](../development/build_from_source/index.md)
first. The plugin builds with the default plugin set and fetches SDL3 (zlib license) at configure
time, linking it statically with only its joystick and HIDAPI subsystems:

```bash
cmake -B build
cmake --build build --target gamepad_plugin --parallel
```

<a id="usage"></a>

## Usage

Start the plugin with the session through `TeleopSessionConfig.plugins`, from the copy bundled
with `isaaccapture` (wheel installs included), and read it with `GamepadSource`, as
[examples/teleop/python/isaaccapture_examples/teleop/gamepad_printer_example.py](https://github.com/NVIDIA/IsaacCapture/blob/main/examples/teleop/python/isaaccapture_examples/teleop/gamepad_printer_example.py) does:

```python
from isaaccapture.retargeting_engine.deviceio_source_nodes import GamepadSource
from isaaccapture.teleop_session_manager import (
    BUNDLED_PLUGINS_DIR,
    PluginConfig,
    TeleopSessionConfig,
)

config = TeleopSessionConfig(
    app_name="MyApp",
    pipeline=pipeline_reading(GamepadSource(name="gamepad")),
    plugins=[
        PluginConfig(
            plugin_name="gamepad",
            plugin_root_id="gamepad",
            search_paths=[BUNDLED_PLUGINS_DIR],
        )
    ],
)
```

The plugin reads the first connected pad, keeps it until it disconnects, and then takes the first
one again. Its name, type and USB IDs are logged when it is opened.

The plugin can also run on its own, for example on another machine pushing to the same runtime,
next to any reader of the same collection:

```bash
./install/plugins/gamepad/gamepad_plugin
```

<a id="multiple-pads"></a>

### Multiple pads

Run one plugin per pad. `--player=N` reads the Nth connected pad, and each plugin pushes to its
own collection, read by a `GamepadSource` with the same `collection_id`:

```python
plugins = [
    PluginConfig(
        plugin_name="gamepad",
        plugin_root_id=f"gamepad_{n}",
        search_paths=[BUNDLED_PLUGINS_DIR],
        plugin_args=[f"--player={n}", f"--collection-id=gamepad_{n}"],
    )
    for n in (1, 2)
]
sources = [GamepadSource(name=f"gamepad_{n}", collection_id=f"gamepad_{n}") for n in (1, 2)]
```

<a id="published-data"></a>

## Published data

The plugin pushes `GamepadSnapshot` to collection `gamepad` (`--collection-id` changes it;
match it in `GamepadSource` / `GamepadTracker`):

- `inputs` — the `GamepadInputState` above.

While no pad is open the plugin keeps pushing snapshots without `inputs`, so a
reader never holds the last stick deflection of an unplugged pad; `GamepadSource` then reports
its outputs as absent. Recordings use MCAP channels `gamepad` and `gamepad_tracked`.

<a id="retargeters"></a>

## Retargeters

`GamepadToSe3RelRetargeter` (end-effector delta from the sticks and D-pad),
`GamepadToSe2Retargeter` (base velocity from the sticks) and `GamepadGripperRetargeter`
(toggle on `WEST`) consume `GamepadSource`. Their dead zone defaults to 0.15: an idle pad’s
sticks rest up to 0.14 off center.

<a id="device-status"></a>

## Device status

When `TeleopSession` starts the plugin, it reports device `<plugin_root_id>/gamepad`
(`gamepad/gamepad` by default) through [Device provider monitoring](../references/device_provider_monitoring.md):
`CONNECTED`, `DISCONNECTED` when no pad is connected for its player, or `FAILED` when SDL
finds the pad but cannot open it.

<a id="troubleshooting"></a>

## Troubleshooting

- **Status DISCONNECTED with the pad connected** — SDL lists a device as a gamepad only when it
  has a mapping. Find the pad’s line in SDL_GameControllerDB (or write one) and set it in the
  `SDL_GAMECONTROLLERCONFIG` environment variable of the process that starts the plugin.
- **Status DISCONNECTED over SSH or in a container** — the user cannot read the pad’s
  `/dev/input/event*` node; see [Device access](#device-access).
- **The robot drifts with the pad at rest** — raise the retargeter’s `dead_zone`.

<a id="tests"></a>

## Tests

`gamepad_plugin_tests` reads SDL virtual gamepads: the standard layout, stick and trigger
ranges, a pad plugged in late, an unplug, and player selection.
