Welcome to Isaac Capture

NVIDIA Isaac Capture is a shared library for teleoperation and egocentric data capture. It brings device plugins, retargeters, and data utilities together so applications can collect and use human demonstration data across robot and simulation environments.

Isaac Teleop is the robot-in-the-loop component within Isaac Capture. It provides teleoperation sessions and workflows for controlling robots and collecting demonstrations. The Ego Pipeline covers human and egocentric data capture and post-processing. Both build on the shared capture infrastructure.

See Architecture for the component relationships and Ecosystem for devices and integrations. Source code and Python packages are distributed through GitHub and PyPI.

Tip

To run Isaac Teleop, follow Quick Start for installation, headset setup, and your first session. Existing users can consult Migrating to isaaccapture for the Python package migration.

Table of Contents

Overview

• Architecture
• Isaac Teleop
• Ecosystem

Isaac Teleop

• Quick Start
  • 1. Check out code base (for examples)
  • 2. Install the isaaccapture pip package
  • 3. Configure CloudXR (optional)
  • 4. Whitelist ports for Firewall
  • 5. Connect an XR headset
  • 6. Run a teleop example
  • Next steps
  • More Information
• Teleop Session
  • Overview
  • Quick Start
  • Configuration Classes
  • API Reference
  • Examples
  • Benefits
  • Advanced Features
• Teleop Control State Machine
  • Overview
  • Default State Machine Behavior
  • Connecting Inputs
  • Using Context in High-Level App Logic

Development

• Build from Source
  • Prerequisites
  • One time setup
  • 1. Clone the repository
  • 2. CMake: Configure and build
  • 3. Running tests
  • 4. Install the isaaccapture pip package
• Televiz
  • Installation
  • Overview
  • Display modes
  • Quick start
  • Session configuration
  • Layers
  • Frame loop
  • Performance and diagnostics
  • Session state
  • Sharing the XR session
  • API reference
  • C++ API
  • More information
• Migrating to isaaccapture
  • In short
  • Upgrading an existing install
  • Imports
  • What importing the old name does
  • Installing
  • Logging
  • The asset cache
• Contributing
  • Previewing documentation changes
• License
  • Known exceptions
  • License headers

Integrations

• Isaac Lab & Sim
• Isaac ROS
• LeRobot and SO-101

Devices

• XR headsets
• Hand tracking
  • MANUS Gloves
  • Wuji Glove
  • Sharpa AvatarExo Glove
  • Haptikos Exoskeletons
• Body tracking
  • SOMA Pose Transport
• Cameras
  • OAK Camera Plugin
• Joint-space devices
• Tactile devices
  • OGLO Tactile Glove Plugin
• Spatial controllers
  • SpaceMouse Plugin
• Gamepads
  • Gamepad Plugin
• Spatial trackers
• Add a New Device

Device Conformance

• Overview
• Set Up the Tools
• Hand Suite
• Full-Body Suite
• Submit Your Results

References

• System Requirements
  • Teleoperation to Robots with Input Devices
  • Teleoperation with Isaac Sim and Isaac Lab
  • Human-centric Data Collection
• Build System
  • Project structure
  • CMake integration
  • Using the Python wheel
  • Installing from source with pip (scikit-build-core)
  • Build directory layout and per-Python-version builds
  • Output locations
  • Troubleshooting
  • Reference
• Generated Tracker Code
  • How it works
  • What each entry produces
  • Generated today
  • Still hand-written
  • Future work
• Retargeting Interface
  • Source Nodes
  • Available Retargeters
  • Build a Retargeting Pipeline
  • Add a New Retargeter
• Device provider monitoring
  • Reading status
  • Identity and inventory
  • Provider states
  • Device states
  • Plugin status transport
  • OpenXR ownership
• Camera Streaming
  • Requirements
  • Setup
  • First run — a recording, no camera required
  • Supported sources
  • Running with a real camera
  • Display modes
  • Display surfaces
  • CloudXR runtime flags
  • Split mode — robot → workstation over RTP
  • Configuration
  • Troubleshooting
  • How it works
  • Sharing the XR session with TeleopSession
• MCAP Recording & Replay
  • Recording a Live Session
  • Replaying a Recording
  • Runnable Example
  • API Reference
• CloudXR Service
  • Start the service
  • Manage the service
  • Optional launch modes
  • USB-local startup checks
  • Re-open the client on the headset
  • Load CloudXR environment variables
  • Run teleop examples against the service
  • Use a system OpenXR runtime (no CloudXR attach)
  • Run the service in a container or CI
  • Connect the OOB headset after application startup
  • Configuration
  • Troubleshooting
• Out-of-Band Teleop Control
  • Quick start
  • ADB automation
  • Architecture
  • WebSocket protocol
  • HTTP API
  • Authentication
  • Web client integration
  • Environment variables
  • USB-local mode (--setup-oob --usb-local)
  • Troubleshooting
• Haptic Feedback
  • Architecture
  • Example
  • Running
  • Adding a new haptic device
  • See also
• Egocentric Hand Reconstruction
  • Video Capture
  • Setup
  • Hand Reconstruction
  • Limitations

Indices and tables

• Index

• Search Page