OVXR installation#

OVXR provides optional XR presentation for applications that use Isaac Capture for teleoperation input. Select the ovxr extra when installing Isaac Capture to include OVXR and its rendering dependencies.

Install XR support#

Follow Build from Source to build Isaac Capture and select a Python environment matching the generated wheel. Include ovxr in the installation requirement:

uv pip install "isaaccapture[ovxr]==X.Y+local" --find-links=./install/wheels/
uv pip check

Replace X.Y with the version of your source build, as described in the build guide. Base installations and imports do not require OVXR.

OVXR declares its compatible OVRTX and OVStage versions in its package metadata. The consuming application’s dependency constraints must accept those versions. See the package pages below for supported Python versions and platforms.

Optional packages and licenses#

The ovxr extra installs OVXR and its OVRTX/OVStage dependencies separately from the Isaac Capture wheel. These packages retain their own licenses:

  • OVXR, OVRTX and OVStage are NVIDIA proprietary software. Review the license included with each distribution before use.

  • OVUI is Apache-2.0 and is installed by the independent desktop-mirror extra. Review its package metadata for supported Python versions and platforms.

CloudXR native libraries are bundled in Isaac Capture wheels and remain subject to their separate NVIDIA license and CloudXR EULA acceptance.

Optional camera panels#

Camera picture-in-picture panels require the native camera helper. The Ubuntu release workflow includes it in its wheels and checks the packaged artifact. Installing those wheels requires no CMake configuration.

For a source build, follow the source-build guide for your application’s Python version, then enable camera support and build the wheel:

cmake -S . -B build -DBUILD_OVXR_CAMERA_QUADS=ON
cmake --build build --target python_package python_stubs python_wheel --parallel

Install the generated wheel with its ovxr extra using the instructions above. The extra requires CUDA 13 Python bindings for GPU camera output; BUILD_OVXR_CAMERA_QUADS controls whether the wheel contains the native helper. The helper is disabled by default.

Use a CUDA 13-compatible NVIDIA driver (R580 or newer), meeting any stricter requirements of the selected bindings and renderer. See the CUDA compatibility requirements. Installing Python packages or a CUDA Toolkit does not update the host driver.

Install the Vulkan development SDK before configuring camera support. Linux builds also require the CUDA Toolkit. CMake discovers these installed dependencies. Use CUDAToolkit_ROOT to select a CUDA Toolkit installation and VULKAN_SDK to help locate a Vulkan SDK. The GPU driver and Vulkan runtime loader are required on the host.

Optional desktop mirror#

The desktop mirror requires Python 3.12 or newer. Isaac Capture’s core continues to support Python 3.11; on that version the extra does not install OVUI and the desktop mirror is unavailable. Use a Python 3.12+ environment and select both extras with isaaccapture[ovxr,desktop-mirror] in the installation requirement or the application’s optional dependencies to include the OVUI mirror. The mirror requires CUDA 13 Python bindings, a compatible NVIDIA driver as described above, and a desktop display associated with a CUDA device. The native camera helper is required only when camera panels are also enabled.

Initialize the renderer before starting the mirror. OVUI uses its default Vulkan backend; no OMNIUI_BACKEND setting is required. Capture does not change the process-wide backend. GPU images stay on their producing CUDA device, and OVUI copies them into its Vulkan image. The mirror requires the active OVUI backend to be Vulkan. Remove conflicting OMNIUI_BACKEND settings before OVUI starts. This selection does not change headset rendering.

The pinned OVUI 0.3.0 Vulkan upload copies on its import device’s default CUDA stream. The mirror drains that stream before reusing the source buffer, including upload failures; the source image can reside on a different GPU. This depends on the synchronous ByteImageProvider.set_bytes_data_from_gpu implementation, not OVUI’s separate asynchronous upload API. See the OVUI upload implementation.

Use from another application#

isaaccapture.ovxr_session_manager.ovxr_session provides OvxrSession and OvxrSessionConfig. Implement OvxrApplicationInterface to supply the application’s renderer, Stage and ordinal ledger in an OvxrSessionBinding. The binding also supplies optional frame_consumers after the application has resolved its render-product paths. Capture does not depend on an Isaac Lab task or scene configuration. camera_paths is required and must contain both stereo cameras. Set up_axis="Y" for a Y-up Stage; the default is "Z".

CameraPanelConsumer and OwnedDesktopMirror implement the same OvxrFrameConsumer lifecycle. Configure panels with a tuple of CameraPanelConfig values, each pairing a render-product path with its placement and image bound. Configure the mirror’s camera choices with MirrorProduct values. Either consumer can be enabled independently.

Camera-panel attachment failure disables panels and logs the reason without blocking teleoperation input. An incomplete GPU copy disables the affected panel; the consumer rechecks its copy stream before releasing the borrowed output. If completion still cannot be proved, the session reports a failure and retains the borrowed resources. Other camera mailboxes are still given a cleanup attempt.

Consumers receive FrameOutputs on the XR owner thread. Use latest_color to select an output, then copy_host_rgba or map_gpu to consume it without unpacking native result structures. Selected handles are valid only during that callback. CPU snapshots are shared within that frame; GPU resources retain their completion and retirement rules. A consumer may contribute composition layers that remain valid through submission. on_openxr_ready may be called repeatedly and must be idempotent. finish_frame releases borrowed outputs before submission or cancellation; close must tolerate partial startup and retain resources if safe retirement fails. The camera consumer owns one native panel collection, which is loaded directly and is not an OpenXR API layer.

For a uv-managed application, declare Isaac Capture with its ovxr extra and select the source-built wheel in the application’s project configuration:

[project.optional-dependencies]
ovxr = ["isaaccapture[ovxr]"]

[tool.uv.sources]
isaaccapture = { path = "path/to/your/local-wheel.whl", extra = "ovxr" }

Replace the wheel path with the file produced by your build. Refresh the application’s lock file after changing the wheel or dependency selection, then run with uv run --extra ovxr. Installing packages with uv pip install alone does not update the application’s lock file.

Application-specific OVXR versions#

Isaac Capture pins the OVXR version used for compatibility testing. uv respects that pin and reports a conflict if the application requires incompatible OVRTX or OVStage versions.

To validate another OVXR build with the same Isaac Capture release, select an OVXR release whose declared dependencies match the application’s OVRTX/OVStage requirements. Add an override in the application’s root pyproject.toml:

[tool.uv]
override-dependencies = ["ovxr==<compatible-version>"]

Replace <compatible-version> with the selected release. If the application already has an override-dependencies list, append this entry to that list. Then refresh the lock file and synchronize the environment using the ovxr extra from the example above:

uv lock --upgrade-package ovxr
uv sync --extra ovxr

An override changes dependency resolution; it does not establish compatibility with Isaac Capture’s OVXR integration. Validate the application’s XR workflow before relying on the selected build.

Applications using different provider versions need separate project environments and lock files. Each can install the same Isaac Capture wheel with its own OVXR/OVRTX/OVStage combination.

Session lifetime#

Use OvxrSession as a context manager, or call close() in a finally block. Stop application steps and input borrowers before closing. Startup failure before handles are borrowed retires its resources on the owner thread; successful startup requires explicit closure. Do not rely on interpreter exit or object finalizers to stop native rendering.

The session schedules work; its frame pump owns native frame processing and retirement, and its runtime owner stops only the CloudXR process it launched. Applications may supply runtime_launcher_factory for a compatible launcher; it receives the environment path and accept_eula and returns an object with stop(). Externally managed runtimes remain externally owned.

OvxrScenePublication and OvxrResourceOwner describe the two parts of the application contract. OvxrApplicationInterface combines them, so existing adapters can continue implementing the combined interface.

If native context destruction fails without proving the context was consumed, closure reports the failure and retains its borrowed resources and runtime. Do not treat this as successful retirement or start a replacement session over those resources. There is no automatic retry of uncertain native destruction.