MCAP Recording & Replay#

Isaac Teleop supports recording live tracking data to MCAP files and replaying them offline through the same retargeting pipeline — no headset or OpenXR runtime required during replay.

Recording a Live Session#

Pass an McapRecordingConfig in the mcap_config field while keeping the default SessionMode.LIVE. TeleopSession automatically discovers trackers from the pipeline, so you only need to provide the output filename:

from isaacteleop.teleop_session_manager import TeleopSession, TeleopSessionConfig
from isaacteleop.deviceio import McapRecordingConfig

config = TeleopSessionConfig(
    app_name="MyApp",
    pipeline=pipeline,
    mcap_config=McapRecordingConfig("recording.mcap"),
)

with TeleopSession(config) as session:
    while True:
        result = session.step() # The tracker state is recorded to the MCAP file

When the context manager exits, the MCAP file is finalized and closed.

Auto-population and appending of tracker names#

TeleopSession always auto-discovers trackers from the pipeline’s DeviceIO sources and uses each source’s name as the MCAP channel name. For example, a pipeline with HeadSource(name="head") and HandsSource(name="hands") produces channels "head" and "hands".

Any (tracker, channel_name) pairs you pass explicitly via tracker_names are appended after the auto-discovered sources. This is useful for recording additional trackers that are not part of the pipeline:

from isaacteleop.deviceio import McapRecordingConfig

extra_tracker = deviceio.HandTracker()

mcap_config = McapRecordingConfig(
    "recording.mcap",
    [(extra_tracker, "extra_hands")],
)

# Final tracker list will be:
#   [(head_source.tracker, "head"), (hands_source.tracker, "hands"),
#    (extra_tracker, "extra_hands")]

Replaying a Recording#

Set mode=SessionMode.REPLAY and pass an McapReplayConfig. No OpenXR session or headset connection is created — data is read directly from the MCAP file:

from isaacteleop.teleop_session_manager import (
    TeleopSession,
    TeleopSessionConfig,
    SessionMode,
)
from isaacteleop.deviceio import McapReplayConfig

config = TeleopSessionConfig(
    app_name="MyApp",
    pipeline=pipeline,
    mode=SessionMode.REPLAY,
    mcap_config=McapReplayConfig("recording.mcap"),
)

with TeleopSession(config) as session:
    while True:
        result = session.step()
        action = result["action"]
        # Process replayed action ...

Just like recording, trackers are auto-discovered from the pipeline and any explicit tracker_names you provide are appended after them. Use this to add extra trackers that aren’t part of the pipeline:

McapReplayConfig(
    "recording.mcap",
    [(extra_tracker, "extra_hands")],
)

Important

mcap_config is required when mode is SessionMode.REPLAY. Omitting it raises ValueError.

Runnable Example#

A complete record / replay example lives at examples/mcap_record_replay/python/:

  • common.py — builds a single-source pipeline (HandsSource only) and defines a small HandJoints retargeter that exposes per-hand joint positions for downstream consumers.

  • record_hand.py — records the hands channel from a live OpenXR session.

  • replay_hand.py — replays the recording and visualizes both hands (joint cloud + skeleton) in a browser via viser.

Because only HandsSource is wired into the pipeline, TeleopSession auto-discovers a single HandTracker and the resulting MCAP contains exactly one channel (hands). To capture more data, add additional source nodes (HeadSource, ControllersSource, …) in common.py.

Recording#

From the installed example directory:

cd examples/mcap_record_replay/python
uv sync
uv run python record_hand.py            # 5 s → ../recordings/hands_<timestamp>.mcap
uv run python record_hand.py 10         # record for 10 seconds
uv run python record_hand.py 10 out.mcap  # custom output path

An active OpenXR runtime / headset must be connected, just like any other live TeleopSession.

Replaying#

Replay runs headless — no headset required:

uv run python replay_hand.py                       # newest file in ../recordings/
uv run python replay_hand.py path/to/file.mcap     # explicit file
uv run python replay_hand.py --loop                # repeat until Ctrl+C
uv run python replay_hand.py --port 8090           # change viser port

Open the printed URL (default http://localhost:8080) in a browser to see the left (green) and right (blue) hand skeletons update each frame.

API Reference#

SessionMode#

class SessionMode(Enum):
    LIVE = "live"
    REPLAY = "replay"

Determines whether TeleopSession creates a live OpenXR session or replays from an MCAP file.

McapRecordingConfig#

McapRecordingConfig(filename, tracker_names=None)

  • filename — Path to the output MCAP file.

  • tracker_names — Optional list of (tracker, channel_name) pairs. When empty, TeleopSession auto-populates from discovered sources.

McapReplayConfig#

McapReplayConfig(filename, tracker_names=None)

  • filename — Path to an existing MCAP file to replay.

  • tracker_names — Optional list of (tracker, channel_name) pairs. When empty, TeleopSession auto-populates from discovered sources.