Retargeters: SO-101 (5-DOF arm)#

The SO-101 is a low-cost 5-DOF arm with a single-jaw gripper. The controller is meant to feel like holding the leader arm: the gripper pose follows the controller pose directly. A full SE3 pose is commanded and a single differential IK solves all 5 arm joints, tracking position exactly and orientation best-effort (a 5-DOF arm is over-determined by one DOF on a 6-DOF pose). These two retargeters provide the pieces for comfortable XR controller teleoperation of the arm (used by the Isaac Lab Isaac-Stack-Cube-SO101-IK-Abs-v0 cube-stacking task):

SO101ClutchRetargeter – absolute EE pose (position + orientation) with clutch-style position rebasing (no teleport on engage).
SO101GripperRetargeter – proportional (analog) jaw closedness from the controller trigger.

Together they flatten (via TensorReorderer) into an 8-D action [pos_x, pos_y, pos_z, quat_x, quat_y, quat_z, quat_w, gripper].

At a glance#

Retargeter	Output	What it does
`SO101ClutchRetargeter`	7-D `ee_pose` (xyz + xyzw quat)	Same output contract as `Se3AbsRetargeter`, but rebases controller motion around an origin captured on engage: `pos = home + scale * (p_ctrl - p0)`. The orientation is the controller grip orientation composed with a fixed calibration offset; it drives the SE3 IK.
`SO101GripperRetargeter`	1 float `gripper_command` `c` in `[0, 1]`	Trigger -> jaw closedness (`0` = open, `1` = closed), with a released-end deadzone.

Why a clutch#

Se3AbsRetargeter maps the controller’s absolute position straight to the EE target, so engaging teleop teleports the arm to wherever the controller happens to be. The clutch instead re-arms whenever teleop is not RUNNING and latches a controller origin p0 on the first RUNNING frame (the headset “Play”). From then on the EE position is driven by the delta relative to p0, so engaging with a steady controller does not move the arm. On the latching frame p_ctrl == p0, so the emitted position is exactly the home (no jump). The last pose is held on a dropped frame.

The clutch keeps position-control IK (use_relative_mode=False): it emits an absolute target, just rebased.

Frames and the home#

The controller stream reaching the clutch is already expressed in the robot base frame: the Isaac Lab IsaacTeleopDevice rebases it upstream via its target_frame_prim_path (set to the robot base), composing base_T_world onto the XR anchor before the controllers are transformed. The clutch therefore applies the controller delta to the home directly, with no world->base rotation of its own, and needs no live end-effector or base feed.

The home is the clutch’s own running home: it is seeded on reset / first engage from the static home_base_T_ee reset-origin (the gripper’s pose in the base frame at the reset pose, only its translation is used), and thereafter holds the last commanded pose so a mid-task re-clutch resumes from where the arm was left.

Note

The fallback home, the position sign/scale knobs, and the orientation calibration offset carry TODO(tune-in-sim) markers: the rebasing math is exact and unit-tested, but the end-to-end controller->EE handedness and the neutral-controller -> neutral-gripper offset should be confirmed in simulation.

Orientation calibration#

The clutch emits the controller grip orientation composed with a fixed calibration offset: q_cmd = orientation_offset (x) q_grip (base-frame left multiply, renormalized). The offset defaults to identity (passthrough); a non-identity offset maps a neutrally-held controller to a sensible neutral gripper orientation for the SE3 IK. This single rotational offset replaces the per-DOF roll/pitch calibration hacks of earlier revisions.

Gripper#

SO101GripperRetargeter maps the analog trigger to a jaw closedness c in [0, 1] (0 = open, 1 = closed) with a small released-end deadzone, so a half-pressed trigger leaves the jaw half-closed. Downstream, an order-locked JointPositionActionCfg applies the affine joint = offset + scale * c mapping c onto the open/close joint angles. This is deliberately independent of the shared GripperRetargeter’s binary +1 = open / -1 = closed sign.

Use it from Python#

from isaacteleop.retargeters import (
    SO101ClutchRetargeter,
    SO101GripperRetargeter,
    TensorReorderer,
)
from isaacteleop.retargeting_engine.deviceio_source_nodes import ControllersSource
from isaacteleop.retargeting_engine.interface import OutputCombiner, ValueInput
from isaacteleop.retargeting_engine.tensor_types import TransformMatrix

def build_so101_stack_pipeline():
    controllers = ControllersSource(name="controllers")
    world_T_anchor = ValueInput("world_T_anchor", TransformMatrix())
    # The device rebases controller poses into the robot base frame upstream via
    # target_frame_prim_path, so the clutch needs no live EE / base feed.
    xformed = controllers.transformed(world_T_anchor.output(ValueInput.VALUE))

    clutch = SO101ClutchRetargeter(name="ee_pose", input_device=ControllersSource.RIGHT)
    connected_clutch = clutch.connect({
        ControllersSource.RIGHT: xformed.output(ControllersSource.RIGHT),
    })

    gripper = SO101GripperRetargeter(name="gripper", input_device=ControllersSource.RIGHT)
    connected_gripper = gripper.connect(
        {ControllersSource.RIGHT: xformed.output(ControllersSource.RIGHT)}
    )

    # Keep all 7 pose names and pass the full pose (xyz + quat) plus gripper through.
    ee_elements = ["pos_x", "pos_y", "pos_z", "quat_x", "quat_y", "quat_z", "quat_w"]
    reorderer = TensorReorderer(
        input_config={
            "ee_pose": ee_elements,
            "gripper_command": ["gripper_value"],
        },
        output_order=ee_elements + ["gripper_value"],
        name="action_reorderer",
        input_types={"ee_pose": "array", "gripper_command": "scalar"},
    )
    connected = reorderer.connect({
        "ee_pose": connected_clutch.output("ee_pose"),
        "gripper_command": connected_gripper.output("gripper_command"),
    })
    return OutputCombiner({"action": connected.output("output")})

See Build a Retargeting Pipeline for the general pipeline-builder pattern and Retargeting Interface for the full retargeting interface.

Validate#

The retargeters ship with sim-free unit tests (trigger/clutch math plus per-frame compute behavior):

$ ctest --test-dir build -R retargeting_test_so101_retargeters --output-on-failure