Generic Joint-Space Device#

A reusable device path for any joint-encoder source – leader arms, exoskeletons, haptic gloves, or other articulated input devices. A device streams a name-keyed JointStateOutput FlatBuffer over the OpenXR tensor transport; one schema, one tracker, one source, and one retargeter serve them all, so adding a new joint-space device is just a new plugin plus a small config.

The SO-101 leader arm (TheRobotStudio SO-ARM100, 6 Feetech STS3215 bus servos) is the reference instance; the reBot DevArm leader (Seeed reBot DevArm, 7 Damiao DM-series motors) is a second instance built from the same recipe.

At a glance#

Layer	Component
Schema	src/core/schema/fbs/joint_state.fbs – `JointState` (name + position + optional velocity/effort) and `JointStateOutput` (a vector of joints + `device_id`).
Plugin	src/plugins/so101_leader – pushes `JointStateOutput` via `SchemaPusher`. Reads the FEETECH STS3215 servos over serial (`FeetechBus`); synthetic fallback when no device path is given.
Tracker	`JointStateTracker` (facade) with live (`LiveJointStateTrackerImpl`) and MCAP-replay (`ReplayJointStateTrackerImpl`) backends, registered in the live/replay factories.
Source	`JointStateSource` (`IDeviceIOSource`) – converts the FlatBuffer into a name-keyed group of joint positions for the retargeting graph.
Retargeter	`JointStateRetargeter` – `joint` (mirror) or `ee_pose` (URDF FK) mode. See Retargeter: Joint-Space Device.

Data schema#

Joints are modeled as name -> value records so consumers read them by name, independent of wire order:

table JointState {
  name: string (id: 0, key);   // e.g. "shoulder_pan", "gripper"
  position: float (id: 1);     // [rad] revolute, [m] prismatic
  velocity: float (id: 2);     // optional (JointStateOutput.has_velocity)
  effort: float (id: 3);       // optional (JointStateOutput.has_effort)
  valid: bool = true (id: 4);
}

table JointStateOutput {
  joints: [JointState] (id: 0);
  device_id: string (id: 1);
  has_velocity: bool (id: 2);
  has_effort: bool (id: 3);
  ee_pose: Pose (id: 4);       // RESERVED: device-side FK; not consumed yet
  ee_pose_valid: bool (id: 5);
}

The gripper is just another named DOF (conventionally "gripper"). velocity, effort, and ee_pose are optional/reserved: the reference plugin and JointStateSource populate and surface joint positions only.

The SO-101 leader plugin#

so101_leader reads the six SO-101 servos (shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper) and pushes them to a tensor collection. With a serial device path it talks to the FEETECH STS3215 bus servos directly via FeetechBus – the same SMS/STS wire protocol the FEETECH SCServo SDK / LeRobot’s FeetechMotorsBus use, with no SDK dependency: it disables torque (so the leader can be back-driven) and reads Present_Position each frame, converting ticks to radians with per-joint calibration. With no device path it falls back to a synthetic trajectory so the pipeline runs hardware-free (CI and the headless example).

# Synthetic backend (no hardware), default collection id "so101_leader":
./install/plugins/so101_leader/so101_leader_plugin

# Real SO-101 leader on a serial port (Linux), optional calibration file:
./install/plugins/so101_leader/so101_leader_plugin /dev/ttyACM0 so101_leader so101_leader.calib

See the plugin README for hardware setup (unique servo ids, gear removal, back-driving) and the calibration file format.

The consumer side creates a JointStateSource(name=..., collection_id="so101_leader", joint_names=[...]) on the same collection_id; TeleopSession discovers and polls the JointStateTracker each frame.

The reBot DevArm leader plugin#

rebot_devarm_leader reads the seven joints of the Seeed reBot DevArm (joint1 .. joint6, gripper – 7 Damiao DM-series MIT-protocol motors: DM4340P on joints 1-3, DM4310 on joints 4-6 and the gripper) and pushes them to a tensor collection, mirroring the SO-101 plugin’s structure and CLI shape. The motors sit on a CAN bus behind a Damiao USB-to-CAN serial adapter (USB CDC-ACM); DamiaoBus speaks the adapter’s fixed-size binary framing directly – no SDK dependency. As a leader, the plugin sends the disable control frame so the arm can be back-driven by hand (Damiao motors keep answering feedback requests while disabled), then requests one feedback frame per motor per cycle (command 0xCC addressed via CAN id 0x7FF) and decodes the fixed-point position/velocity feedback, which lands directly in radians – no tick conversion, only an optional per-joint sign and zero offset from a calibration file. With no device path it falls back to a synthetic trajectory, exactly like so101_leader.

# Synthetic backend (no hardware), default collection id "rebot_devarm_leader":
./install/plugins/rebot_devarm_leader/rebot_devarm_leader_plugin

# Real reBot DevArm on the Damiao USB-to-CAN adapter (Linux), optional calibration file:
./install/plugins/rebot_devarm_leader/rebot_devarm_leader_plugin /dev/ttyACM0 rebot_devarm_leader rebot_devarm.calib

# Probe wiring, motor ids, and the decode path -- no OpenXR runtime needed:
./install/plugins/rebot_devarm_leader/rebot_devarm_leader_plugin probe /dev/ttyACM0

probe exits 0 when every motor replied, and 3 when the motors replied but the gripper reads outside its physical travel: the Damiao multi-turn counter is volatile across power cycles, so the gripper (whose geared travel exceeds one turn) can wake up reading physical + 2*pi*k and must be re-homed (closed against the mechanical stop and re-zeroed) before teleoperating. While wrapped, the running plugin streams the gripper joint with valid = false so consumers hold it instead of executing garbage.

See the plugin README for the calibration file format (name, command/feedback CAN ids, motor model, sign, zero offset) and hardware notes.

Record and replay#

The live tracker records to MCAP, and ReplayJointStateTrackerImpl replays it back with no OpenXR runtime, so a recorded session drives the retargeting graph headlessly:

from isaacteleop.deviceio import McapRecordingConfig, McapReplayConfig
from isaacteleop.teleop_session_manager import SessionMode, TeleopSession, TeleopSessionConfig

# Record (live): TeleopSessionConfig(..., mcap_config=McapRecordingConfig("leader.mcap"))
# Replay (headless): TeleopSessionConfig(..., mode=SessionMode.REPLAY,
#                                        mcap_config=McapReplayConfig("leader.mcap"))

Add another joint-space device#

Reuse everything above by writing only:

A plugin that reads your hardware and fills JointStateOutput (positions; optionally velocity/effort), modeled on src/plugins/so101_leader (or src/plugins/rebot_devarm_leader, a second instance of the same recipe on a different motor bus).
A config: a collection_id, the device joint names, and – for ee_pose mode – a URDF and end-effector link.

The schema, JointStateTracker, JointStateSource, and JointStateRetargeter are unchanged.