Gamepad Plugin

C++ plugin that streams a gamepad (USB or Bluetooth) into the standard DeviceIO path in one layout for every pad: buttons named by position and the two sticks and triggers. It reads the pad through SDL3, whose drivers and SDL_GameControllerDB mappings resolve each pad’s own numbering, and pushes the state at 90 Hz. Source and plugin README: src/plugins/gamepad/README.md.

Supported pads: Xbox (360, One, Series, USB and Bluetooth), PlayStation (DualShock 4, DualSense), Nintendo Switch Pro, Steam Controller, and any pad with an SDL_GameControllerDB mapping. Validated: Xbox 360 Controller.

This plugin follows the Add a New Device pattern (FlatBuffer schema → plugin SchemaPusher → tracker → MCAP).

Components

• Schema — src/core/schema/fbs/gamepad.fbs: GamepadInputState, the standard layout as named fields, inside GamepadSnapshot / GamepadSnapshotRecord.

• Plugin — src/plugins/gamepad: core/ reads the pad through SDL3 and reopens it after an unplug; app/ is the OpenXR push loop.

• Tracker — GamepadTracker, generated at configure time from the gamepad entry in src/core/deviceio_trackers/trackers.toml.

• Source node — GamepadSource turns the tracker’s data into a gamepad tensor group (GamepadInput, indexed by GamepadInputIndex) for retargeters.

Standard layout

GamepadInputState is a generic controller split in two, with the same naming as ControllerInputState. Face buttons are named by position, so a binding means the same on every pad:

Field	Xbox	PlayStation	Nintendo
south_click	A	Cross	B
east_click	B	Circle	A
west_click	X	Square	Y
north_click	Y	Triangle	X
view_click / menu_click	View / Menu	Create / Options	− / +

The other buttons are shoulder_[left|right]_click, thumbstick_[left|right]_click and dpad_[up|down|left|right]_click; the axes are thumbstick_[left|right]_[x|y] in [-1, 1] (right and up positive, as in OpenXR) and trigger_[left|right]_value in [0, 1]. These are the inputs every standard pad has, the same set as OpenXR’s xbox_controller profile; pad-specific extras such as the guide button, paddles and touchpad are not reported.

Device access

On a desktop session udev grants the logged-in user access to gamepad nodes, so no setup is needed. Without a local seat (SSH, a service, a container), add the user to the input group, and pass the pad’s /dev/input/event* node into a container.

Build

Linux only. The plugin builds with the default plugin set and fetches SDL3 (zlib license) at configure time, linking it statically with only its joystick and HIDAPI subsystems:

cmake -B build
cmake --build build --target gamepad_plugin --parallel

Usage

Start the plugin with the session through TeleopSessionConfig.plugins, from the copy bundled with isaaccapture (wheel installs included), and read it with GamepadSource, as examples/teleop/python/gamepad_printer_example.py does:

from isaaccapture.retargeting_engine.deviceio_source_nodes import GamepadSource
from isaaccapture.teleop_session_manager import (
    BUNDLED_PLUGINS_DIR,
    PluginConfig,
    TeleopSessionConfig,
)

config = TeleopSessionConfig(
    app_name="MyApp",
    pipeline=pipeline_reading(GamepadSource(name="gamepad")),
    plugins=[
        PluginConfig(
            plugin_name="gamepad",
            plugin_root_id="gamepad",
            search_paths=[BUNDLED_PLUGINS_DIR],
        )
    ],
)

The plugin reads the first connected pad, keeps it until it disconnects, and then takes the first one again. Its name, type and USB IDs are logged when it is opened.

The plugin can also run on its own, for example on another machine pushing to the same runtime, next to any reader of the same collection:

./install/plugins/gamepad/gamepad_plugin

Multiple pads

Run one plugin per pad. --player=N reads the Nth connected pad, and each plugin pushes to its own collection, read by a GamepadSource with the same collection_id:

plugins = [
    PluginConfig(
        plugin_name="gamepad",
        plugin_root_id=f"gamepad_{n}",
        search_paths=[BUNDLED_PLUGINS_DIR],
        plugin_args=[f"--player={n}", f"--collection-id=gamepad_{n}"],
    )
    for n in (1, 2)
]
sources = [GamepadSource(name=f"gamepad_{n}", collection_id=f"gamepad_{n}") for n in (1, 2)]

Published data

The plugin pushes GamepadSnapshot to collection gamepad (--collection-id changes it; match it in GamepadSource / GamepadTracker):

• inputs — the GamepadInputState above.

While no pad is open the plugin keeps pushing snapshots without inputs, so a reader never holds the last stick deflection of an unplugged pad; GamepadSource then reports its outputs as absent. Recordings use MCAP channels gamepad and gamepad_tracked.

Retargeters

GamepadToSe3RelRetargeter (end-effector delta from the sticks and D-pad), GamepadToSe2Retargeter (base velocity from the sticks) and GamepadGripperRetargeter (toggle on WEST) consume GamepadSource. Their dead zone defaults to 0.15: an idle pad’s sticks rest up to 0.14 off center.

Device status

When TeleopSession starts the plugin, it reports device <plugin_root_id>/gamepad (gamepad/gamepad by default) through Device provider monitoring: CONNECTED, DISCONNECTED when no pad is connected for its player, or FAILED when SDL finds the pad but cannot open it.

Troubleshooting

• Status DISCONNECTED with the pad connected — SDL lists a device as a gamepad only when it has a mapping. Find the pad’s line in SDL_GameControllerDB (or write one) and set it in the SDL_GAMECONTROLLERCONFIG environment variable of the process that starts the plugin.

• Status DISCONNECTED over SSH or in a container — the user cannot read the pad’s /dev/input/event* node; see Device access.

• The robot drifts with the pad at rest — raise the retargeter’s dead_zone.

Tests

gamepad_plugin_tests reads SDL virtual gamepads: the standard layout, stick and trigger ranges, a pad plugged in late, an unplug, and player selection.