<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
<!-- SPDX-License-Identifier: Apache-2.0 -->

<a id="out-of-band-teleop-control"></a>

# Out-of-Band Teleop Control

The **OOB (out-of-band) teleop control hub** lets you coordinate Isaac Teleop
from outside the headset — read streaming metrics, inspect connected clients,
and push configuration changes — over the **same TLS port** as the CloudXR
proxy.

The hub shares the proxy TLS port (default **48322**, override with
`PROXY_PORT`).

<a id="quick-start"></a>

## Quick start

**Step 1 — Start the streaming host with OOB enabled**

On first use, it is recommended to run once **without** `--setup-oob` to
confirm `adb devices` sees the headset, verify USB debugging is enabled, and
accept the self-signed certificate in the headset browser manually (both the
web client page and the `https://<host>:48322` proxy page). Once that
baseline works, add `--setup-oob` to automate the full flow.

Launch the CloudXR runtime with the `--setup-oob` flag (add `--accept-eula`
on first run):

```bash
python -m isaaccapture.cloudxr.service run --accept-eula --setup-oob
```

This will:

1. Check host prerequisites and start the WSS proxy with the OOB control hub
2. Wait for one ready headset, even if none is attached at startup
3. Open the teleop page on the headset via `adb shell am start`
4. Accept the self-signed certificate and click CONNECT automatically
   via Chrome DevTools Protocol (CDP)

When the OpenXR application starts after the service, add
`--defer-oob-connect`: steps 2-4 wait for `GET /api/oob/v1/connect`, so
the headset connects once there is a stream instead of retrying until one
appears.

```bash
python -m isaaccapture.cloudxr.service run --accept-eula --setup-oob --defer-oob-connect
# once the application has its OpenXR session:
curl -k -H "X-Control-Token: <CONTROL_TOKEN>" https://localhost:48322/api/oob/v1/connect
```

Ordinary headset absence, an `offline` or `unauthorized` transport, and
browser recovery failures leave the host running with a degraded OOB status.
The lifecycle retries for 60 seconds at a 5-second cadence, then keeps
observing every 5 seconds. A meaningful device, network, rule, or browser
change opens a fresh 60-second recovery episode. Override the positive,
finite values with `TELEOP_OOB_RECOVERY_TIMEOUT_SEC` and
`TELEOP_OOB_RETRY_INTERVAL_SEC`; invalid values fail host preflight.

At INFO, OOB progress reports headset waiting and connection, browser readiness,
and streaming loss or recovery. ADB authorization, missing headset Wi-Fi, and
USB-health blocks remain visible with operator action. An unchanged wait for
an absent headset is announced once; routine setup steps are DEBUG messages.
Progress uses the `isaaccapture.cloudxr.oob_teleop_env.oob-progress` logger.
It remains visible through filters selecting its parent logger, while embedded
applications can also select progress alone.

To start the hub **without** any `adb` interaction — useful in containers,
CI, or wireless-only environments — set `TELEOP_OOB_HUB_ONLY=1` and open
the teleop URL on the headset manually. This mode supports WiFi setup only
and is **not compatible with** `--usb-local`:

```bash
TELEOP_OOB_HUB_ONLY=1 python -m isaaccapture.cloudxr.service run --accept-eula --setup-oob
```

You should see output confirming the hub is running:

```text
CloudXR WSS proxy: running, log file: /home/<user>/.cloudxr/logs/wss.2026-04-13T202133Z.log
        oob:       enabled  (hub + USB adb automation — see OOB TELEOP block)
```

#### NOTE
The headset must be:

- **Connected via USB cable** for adb commands (opening the teleop URL)
- **Connected to WiFi** on the same network as the streaming host (for web
  page access and CloudXR streaming)

Streaming and web page access use WiFi, not USB tethering.
`adb forward` is used only temporarily for CDP automation.

<!-- Keep quick-start labels bold so they do not introduce a new RST title level. -->

**Automatic presence recovery**

While `--setup-oob` owns the selected headset, automatic power recovery runs
in both Wi-Fi and USB-local streaming, with normal menus or headless clients.
It stops with the OOB service. It requires authorized ADB access to that pinned
device; a network-only connection does not provide automatic wake.

On supported PICO hardware, the live `stk_stk3x1x` wakeup proximity sensor
must report NEAR and Android must report Asleep before a wake request is sent.
FAR, missing sensor data, or sensor privacy leave PICO wake to the operator.
Quest presence probing is currently unsupported; positively identified Meta/Oculus
headsets retain the existing one-shot wake request before browser automation,
which still waits for confirmed Awake. Unknown devices require manual wake. The worker
rechecks presence before each attempt, confirms Awake afterward, and allows
at most three attempts five seconds apart. A brief Awake bounce does not reset
that budget. Remove and put on the headset again, or press its power button,
after a failed recovery. No headset settings are changed.

Power recovery preserves the browser page and XR session. Awake does not prove
that tracking resumed: the client reports paused/waking/recovery-failed status,
and confirms restored head tracking only after fresh visible XR frames and head
poses. Controller pose ages remain separate; absent controllers do not imply
that head-only or hand-tracking modes failed.
The hub’s `xrPresence` reports these independently of cached SDK metrics.
The `xr` cadence contains `sessionActive`, `generation`, `visibility`
(2 visible, 1 blurred, 0 hidden, -1 unknown), `frameAgeMs`, `headAgeMs`,
`leftAgeMs`, `rightAgeMs`, and controller configuration flags. Client ages
use a monotonic clock and -1 for unknown; the hub exposes unknown ages as null
and adds monotonic receipt age. Session and visibility changes reset freshness.
This heartbeat does not refresh stream-metric timestamps.

**Step 2 — (Manual fallback) Open the web client on the headset**

If the adb automation fails (e.g. headset not paired), you can manually open
the client URL on the headset browser with **all three** required query
parameters — `oobEnable`, `serverIP`, and `port` — plus the hub’s
`controlToken`:

```default
https://nvidia.github.io/IsaacCapture/client/main/?oobEnable=1&serverIP=<HOST_IP>&port=48322&controlToken=<CONTROL_TOKEN>
```

Replace `<HOST_IP>` with the streaming host’s LAN IP. The `port` must
match the proxy port (default 48322). `<CONTROL_TOKEN>` is the value of
`CONTROL_TOKEN`, or the generated token printed in the startup banner; the
banner’s bookmark shows it as `<REDACTED>`, so substitute it before pasting.
Without the token the hub rejects the headset and the client does not retry.

#### NOTE
All three parameters are required. If `serverIP` or `port` is missing,
the OOB control channel is silently skipped — the client will still work for
streaming but will not register with the hub or report metrics.

**Step 3 — Verify the headset registered with the hub**

From a PC on the same network, query the hub state API (`-k` skips the
self-signed certificate check):

```bash
curl -k -H "X-Control-Token: <CONTROL_TOKEN>" https://<HOST_IP>:48322/api/oob/v1/state
```

You should see the headset listed under `"headsets"` with
`"connected": true`:

```json
{
  "updatedAt": 1776112022900,
  "configVersion": 0,
  "config": {"serverIP": "<HOST_IP>", "port": 48322},
  "headsets": [
    {
      "clientId": "193f3758-281e-4292-8c36-6541b58963ef",
      "connected": true,
      "deviceLabel": null,
      "registeredAt": 1776112022805,
      "lastSeenAt": null,
      "lastMetricsAt": null,
      "metricsByCadence": {}
    }
  ],
  "lifecycle": {"health": "browser_ready", "state": "ACTIVE"}
}
```

If `"headsets"` is empty, double-check that the URL on the headset includes
both `serverIP` and `port` and that the headset can reach the host over the
network.

**Step 4 — (Optional) Push config to the headset**

Before or after the headset connects to the CloudXR stream, you can push
configuration overrides via the HTTP config API:

```bash
curl -k -H "X-Control-Token: <CONTROL_TOKEN>" "https://<HOST_IP>:48322/api/oob/v1/config?serverIP=<HOST_IP>&port=48322&codec=av1"
```

See `GET /api/oob/v1/config` below for all supported keys.

**Step 5 — Stream and poll for metrics**

With `--setup-oob`, CONNECT is clicked automatically via CDP.  If running
without it, press **CONNECT** on the headset manually.  Once streaming begins,
the headset reports metrics to the hub every 500 ms.  Poll the state endpoint
from a PC to collect them:

```bash
# Poll every 2 seconds (adjust to taste)
watch -n 2 'curl -sk -H "X-Control-Token: <CONTROL_TOKEN>" https://<HOST_IP>:48322/api/oob/v1/state | python3 -m json.tool'
```

The `metricsByCadence` field on each headset entry will now contain live streaming metrics.

<a id="adb-automation"></a>

## ADB automation

The `--setup-oob` flag automates headset setup via USB `adb`:

1. **adb devices** selects one ready headset and pins its serial for the service lifetime
2. **am start** opens the teleop bookmark URL in the headset browser with
   the correct `oobEnable=1`, `serverIP`, and `port` parameters
3. **CDP connect** forwards the browser’s DevTools socket over `adb`,
   accepts the self-signed certificate interstitial, and clicks CONNECT
   via Chrome DevTools Protocol (`Input.dispatchMouseEvent`)

Streaming and web page access use WiFi, not USB tethering.  The headset
reaches the streaming host directly over WiFi.  `adb forward` is used only
temporarily during CDP automation to reach the browser’s DevTools socket.

Prerequisites:

- `adb` must be on `PATH` (Android SDK Platform Tools)
- The headset must be connected via USB with USB debugging enabled
- The headset must be on the same WiFi network as the streaming host

If any device step fails, the hub keeps serving and retries. Fall back to
`chrome://inspect/#devices` from the PC or tap CONNECT on the headset
directly.  To re-open the page later without restarting the launcher, pass the
bookmark URL above to `python -m isaaccapture.cloudxr.webclient` (see
[CloudXR Service](cloudxr.md)).

<a id="architecture"></a>

## Architecture

| Role                   | Software                                                 | What it does                                                                                                                     |
|------------------------|----------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------|
| **XR headset**         | Isaac Teleop web client in the device browser            | Registers with the hub via WebSocket, reports streaming metrics<br/>periodically (default every 500 ms), receives config pushes. |
| **Streaming host**     | `python -m isaaccapture.cloudxr.service run --setup-oob` | Runs CloudXR runtime + WSS proxy + OOB hub on a single TLS port.<br/>Opens the teleop page and clicks CONNECT via USB adb + CDP. |
| **Operator / scripts** | `curl`, browser, or custom tooling                       | Reads state via HTTP, optionally pushes config via HTTP.                                                                         |

<a id="websocket-protocol"></a>

## WebSocket protocol

Endpoint: `wss://<host>:<port>/oob/v1/ws`

All messages are JSON text frames with `{"type": ..., "payload": ...}`.

<a id="registration-first-message"></a>

### Registration (first message)

```json
{
  "type": "register",
  "payload": {
    "role": "headset",
    "deviceLabel": "Quest 3",
    "token": "<CONTROL_TOKEN>"
  }
}
```

`role` must be `"headset"`. The hub replies with:

```json
{
  "type": "hello",
  "payload": {
    "clientId": "<uuid>",
    "configVersion": 0,
    "config": {"serverIP": "...", "port": 48322}
  }
}
```

<a id="headset-hub-clientmetrics"></a>

### Headset → hub: `clientMetrics`

```json
{
  "type": "clientMetrics",
  "payload": {
    "t": 1712800000000,
    "sampleAgeMs": 125,
    "cadence": "frame",
    "metrics": {
      "streaming.framerate": 72.0,
      "render.pose_to_render_time": 18.5
    }
  }
}
```

One message is sent per cadence per tick, carrying the last known value of every metric
reported so far in that cadence. Metric names are the CloudXR.js `MetricsName` values,
so they match the SDK documentation. The hub stores them as an arbitrary
`{name: value}` map per cadence, so metrics added by a future SDK need no hub change.
`sampleAgeMs` is the elapsed client monotonic time since the last SDK observation
in that cadence. Repeating cached metrics increases their age rather than renewing
freshness. The hub subtracts this age from receipt time, independently of the device’s
wall clock. Older clients that omit the field retain receipt-time freshness; they
must be updated to distinguish cached samples from new observations.

<a id="browser-health-exchange"></a>

### Browser health exchange

After each automated CONNECT, the hub sends
`healthProbe {probeId, lifecycleGeneration}` to a newly registered
browser page. That page answers
`healthReport {probeId, lifecycleGeneration, pageTimestamp, streamStatus,
streamPhase, terminalEventId, terminalDetail, lastMetricsAt, metricCadences}`.
`streamPhase` is one of `idle`, `connecting`, `retrying`, `streaming`,
or `terminal`. A terminal event ID remains stable while that failure is
replayed and changes only for a later, distinct terminal failure. Older clients
that send only the boolean `streamStatus` remain supported. The hub accepts
only the matching probe and generation from the target connection. It uses its
own receive time for freshness; the headset clock is diagnostic only. A
matching reply means the UI, WSS, and bidirectional browser control path are
ready. The lifecycle
reports `browser_ready` while waiting for a stream. In USB-local relay
mode, `active` additionally requires `streamStatus=true` and fresh
post-CONNECT client metrics; coturn listening and ADB rules alone are only
TURN prerequisites.

If CONNECT was dispatched but a fresh browser report has not arrived, the
lifecycle remains degraded in `VERIFYING_BROWSER` and continues probing
without clicking CONNECT again. After a USB cable replug, the lifecycle enters
`VERIFYING_EXISTING_BROWSER` and gives the client’s configured reconnect loop
time to finish. Fresh post-repair stream metrics return it to `ACTIVE` without
closing, reloading, or navigating the surviving tab. One terminal event permits
one same-tab CONNECT click; a full browser launch is reserved for the case where
no usable tab survives the bounded grace period. The lifecycle is the sole
owner of CDP/browser mutations; its error monitor only reports evidence.

A lost browser/control connection or broken device-side prerequisite starts a
new recovery attempt. The browser bundle must implement `healthProbe` /
`healthReport`; an older non-empty
`TELEOP_WEB_CLIENT_STATIC_DIR/bundle.js` is not replaced automatically. OOB
startup checks for the protocol and fails with an asset diagnostic if the local
bundle is too old. For source-tree testing, run
`npm run build` in `deps/cloudxr/webxr_client` and point
`TELEOP_WEB_CLIENT_STATIC_DIR` at its `build` directory before starting
the service. USB-local static responses use `Cache-Control: no-store`;
the lifecycle also clears this UI origin’s browser storage once per selected
headset session. Before each local-client CONNECT, CDP reloads the page with
HTTP caching disabled so a previously cached `bundle.js` cannot mask the
updated client.

The same redacted lifecycle snapshot appears in `service status`, through
`CloudXRLauncher.oob_status()`, and at
`<cloudxr-install-dir>/run/oob_status.json`. A normal shutdown removes the
status file. The `selectedSerial` and `explicitSerial` fields identify the
session’s headset selection, and `ignoredSerials` lists other observed ADB
devices (up to eight display-safe serials). Selection is scoped to the current
service session; a previous status file never pins a new session.

The snapshot exposes separate readiness facts: `socketBound` means the WSS
listener is serving; `transportReady` means the selected headset’s network
and, in USB-local mode, TURN and reverse rules have been verified;
`clientLoaded` means the browser’s CONNECT control is actionable or a
matching browser health report was received; `connectDispatched` records a
host CONNECT click; and `streamConfirmed` requires a streaming report with
fresh metrics. `readinessStage` and `readinessSince` show the latest
milestone and when it was entered. `lastMetricsAt` is the latest metric observation
time in host epoch milliseconds, derived from receipt time minus sample age;
`lastSeenAt` remains the browser message receipt time. The launcher accepts `streamConfirmed`
only while those metrics and the lifecycle status are less than five seconds
old. Recovery can move the stage backward, and a surviving browser can resume
a stream without another host CONNECT click.

`CloudXRLauncher.wait_for_oob_stage("streamConfirmed", timeout_sec=90)`
waits for live status and raises on a fatal worker failure or timeout. Call it
after creating the OpenXR session when the application needs a confirmed
headset stream before announcing startup. Starting the standalone CloudXR
service confirms its listener only; headset setup continues with bounded
retries and is visible through the OOB status.

If the OOB lifecycle worker or listening WSS proxy exits unexpectedly, the
service publishes a terminal snapshot with `health=fatal` and `state=FATAL`,
clears readiness and streaming flags, and stops the runtime. Unlike a normal
shutdown, it retains `oob_status.json` so `service status` can show the
failure reason after the runtime stops. Foreground `service run` prints one
actionable `OOB stopped` message and exits nonzero. The per-session WSS
log contains the worker traceback, or a clear message if the worker returned
without an exception. This also applies in hub-only mode. A blocked status
writer does not keep the runtime alive after a fatal worker exit. A writer
stall can delay the status file update; the in-memory fatal snapshot remains
available while the service process is running. An intentional shutdown does
not create a fatal snapshot.

#### Metrics reported per cadence (CloudXR.js 6.3.0)

| Cadence   | Metric                                      | Meaning                                                                                                                          |
|-----------|---------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------|
| `render`  | `render.framerate`                          | Client render rate (FPS), rolling average.                                                                                       |
| `render`  | `pose.send_framerate`                       | Rate at which poses are sent upstream (FPS), rolling average. For teleop this is<br/>the rate operator intent reaches the robot. |
| `render`  | `latency.xr_pose_age_ms`                    | Age of the XR pose at send time.                                                                                                 |
| `frame`   | `streaming.framerate`                       | Streamed video rate (FPS), rolling average.                                                                                      |
| `frame`   | `streaming.frame_count`                     | Monotonic count of streamed frames.                                                                                              |
| `frame`   | `render.pose_to_render_time`                | Pose-to-render latency (ms), rolling average.                                                                                    |
| `frame`   | `latency.pose_upload_ms`                    | Time spent uploading the pose.                                                                                                   |
| `frame`   | `latency.pose_to_frame_received_ms`         | Time from pose send to frame received.                                                                                           |
| `frame`   | `frame_pipeline.compositor_skipped_percent` | Share of frames the compositor skipped.                                                                                          |
| `frame`   | `frame_pipeline.out_of_order_percent`       | Share of frames that arrived out of order.                                                                                       |
| `frame`   | `frame_pipeline.mismatched_percent`         | Share of frames whose pose did not match the rendered pixels.                                                                    |
| `network` | `network.streaming_rate_mbps`               | Current streaming rate.                                                                                                          |
| `network` | `network.available_bandwidth_mbps`          | Estimated available bandwidth.                                                                                                   |
| `network` | `network.rtt_ms`                            | Round-trip time.                                                                                                                 |
| `network` | `network.packet_loss`                       | Packet loss ratio.                                                                                                               |
| `network` | `network.avg_decode_time_ms`                | Average video decode time.                                                                                                       |
| `network` | `network.quality_score`                     | Composite network quality, 0-4.                                                                                                  |
| `network` | `network.bandwidth_score`                   | Bandwidth component of the quality score, 0-4.                                                                                   |
| `network` | `network.network_loss_score`                | Loss component of the quality score, 0-4.                                                                                        |
| `network` | `network.latency_score`                     | Latency component of the quality score, 0-4.                                                                                     |
| `network` | `session.quality`                           | Overall session quality, 0-4. Also drives the in-XR quality bars.                                                                |

Score metrics use the SDK’s `QualityScore` scale: 0 NoData, 1 Unsustainable,
2 Degraded, 3 Good, 4 Excellent.

Metrics are cleared when the stream stops, so a new session never reports the previous
session’s last known values.

<a id="http-api"></a>

## HTTP API

All endpoints use **GET** with query parameters on the proxy TLS port.

<a id="get-api-oob-v1-state"></a>

### `GET /api/oob/v1/state`

Returns the current hub state: connected headsets, latest metrics, and config
version.

```bash
curl -k -H "X-Control-Token: <CONTROL_TOKEN>" https://localhost:48322/api/oob/v1/state
```

Example response:

```json
{
  "updatedAt": 1712800000000,
  "configVersion": 0,
  "config": {"serverIP": "10.0.0.1", "port": 48322},
  "headsets": [
    {
      "clientId": "abc-123",
      "connected": true,
      "deviceLabel": "Quest 3",
      "registeredAt": 1712799990000,
      "metricsByCadence": {
        "frame": {
          "at": 1712800000000,
          "metrics": {"streaming.framerate": 72.0}
        }
      }
    }
  ]
}
```

<a id="get-api-oob-v1-config"></a>

### `GET /api/oob/v1/config`

Push config to connected headsets via query parameters:

```bash
curl -k -H "X-Control-Token: <CONTROL_TOKEN>" "https://localhost:48322/api/oob/v1/config?serverIP=10.0.0.5&port=48322"
```

Example response:

```json
{
  "ok": true,
  "changed": true,
  "configVersion": 1,
  "targetCount": 1
}
```

Supported query keys: `serverIP`, `port`, `panelHiddenAtStart`, `codec`.
Optional `targetClientId` restricts the push to a single headset (returns 404
if not connected).

<a id="get-api-oob-v1-connect"></a>

### `GET /api/oob/v1/connect`

Releases headset setup on a service started with `--defer-oob-connect`.
Returns `{"connectRequested": true, "alreadyRequested": false}`; repeating
it is harmless (`"alreadyRequested": true`). A service that was not started
deferred answers 409.

<a id="authentication"></a>

## Authentication

Every hub operation requires a token. Set `CONTROL_TOKEN=<secret>` to choose
it; if it is unset or empty, the hub generates a random token for the session and
prints it in the startup banner. Requests without the token, and WebSocket
`register` frames without it, are rejected. Pass it as:

- WebSocket: `"token"` field in the `register` payload
- HTTP: `?token=<secret>` query parameter or `X-Control-Token` header

The headset bookmark carries the token as `controlToken` in the URL query, so
the hosting server and any proxy logs can see it. Treat bookmark URLs as
secrets and prefer a client served from the streaming host (`--host-client`).

<a id="web-client-integration"></a>

## Web client integration

The web client connects to the hub when the page URL contains
`oobEnable=1` plus `serverIP` and `port`:

```default
https://nvidia.github.io/IsaacCapture/client/main/?oobEnable=1&serverIP=10.0.0.1&port=48322&controlToken=<CONTROL_TOKEN>
```

The client builds `wss://{serverIP}:{port}/oob/v1/ws` and:

1. Registers as role `"headset"`
2. Reports `clientMetrics` periodically (default every 500 ms)
3. Receives `config` pushes from operator

<a id="url-query-parameter-overrides"></a>

### URL query parameter overrides

The following URL parameters override their corresponding form fields (and
`localStorage` values) so that bookmarked links always take priority over
previously saved settings:

- `serverIP` CloudXR server IP address
- `port` CloudXR server port
- `codec` video codec
- `panelHiddenAtStart` hide the control panel on load

When no URL override is present, form fields restore from `localStorage`, with one
exception: a saved device profile other than `Custom` is re-applied from the current
profile table at startup, so profile default updates (frame rate, bitrate, codec)
reach returning clients. Only `Custom` keeps manually edited values.

<a id="environment-variables"></a>

## Environment variables

| Variable                                | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
|-----------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `PROXY_PORT`                            | WSS proxy port (default `48322`). Also the hosted `/client/`<br/>origin for both `--host-client` and `--usb-local`: the page<br/>and signaling share this socket. In `--usb-local` mode<br/>`adb reverse` maps it to the headset.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `CONTROL_TOKEN`                         | Auth token for hub access; a random per-session token is generated if unset                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `TELEOP_STREAM_SERVER_IP`               | Override the auto-detected LAN IP in hub initial config                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| `TELEOP_PROXY_HOST`                     | Override the LAN IP used for headset bookmark URLs                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| `TELEOP_WEB_CLIENT_BASE`                | Override the web client origin URL                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| `TELEOP_STREAM_PORT`                    | Override the signaling port (default same as proxy port)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `TELEOP_CLIENT_CODEC`                   | Default video codec for headset bookmarks                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| `TELEOP_CLIENT_PANEL_HIDDEN_AT_START`   | Hide control panel on load (`true` / `false`)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `TELEOP_CLIENT_IMMERSIVE_MODE`          | WebXR session mode: `vr` or `ar`. PICO 4 Ultra needs `vr`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| `TELEOP_CLIENT_REFERENCE_SPACE`         | XR reference space: `auto`, `local-floor`, `local`, `viewer` or<br/>`unbounded`. Pin it to compare replays across runs.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| `TELEOP_CLIENT_REPLAY_PACING`           | Replay pacing: `frame` (one recorded frame per XR frame, independent<br/>of server speed) or `time`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| `TELEOP_CLIENT_SHOW_RECORDING_CONTROLS` | Enable the recording and replay controls (`true` / `false`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| `TELEOP_CLIENT_ROUTE`                   | HashRouter fragment appended to the bookmark URL. Default empty<br/>(no fragment — the web client picks its own landing route). Set<br/>to e.g. `/real/gear/dexmate` to force a specific route. A<br/>leading `#` is stripped automatically.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| `TELEOP_OOB_HUB_ONLY`                   | Set to any non-empty value (e.g. `1`) to start the OOB hub<br/>without any `adb` interaction. The hub starts normally and<br/>accepts WebSocket connections, but the launcher skips<br/>`adb devices`, `adb shell` wakefulness checks, CDP port<br/>forwarding, and the automated browser open + CONNECT click.<br/>Useful in environments where `adb` is unavailable or<br/>unreliable (CI, containers, wireless-only setups). The operator<br/>must open the teleop page on the headset manually with the<br/>correct `oobEnable=1&serverIP=<HOST>&port=<PORT>` parameters.<br/>Only meaningful together with `--setup-oob`; has no effect<br/>without it. **Not compatible with** `--usb-local` — hub-only<br/>mode supports WiFi setup only; the launcher rejects the<br/>combination at startup.                                                                                                                                                                                                         |
| `ANDROID_SERIAL`                        | Pin the intended headset before starting OOB when more than one ADB<br/>transport is visible. Without it, the first observation must contain<br/>exactly one serial; that serial is pinned even while `unauthorized`<br/>or `offline`. An ambiguous first observation fails with a restart<br/>instruction, rather than selecting by list order or by which device<br/>becomes ready first. Every device command then uses `-s <serial>`.<br/>If the selected serial disappears or loses authorization, the service<br/>waits for it and ignores later devices. Once accessible, the selected<br/>device must identify as a Meta Quest or PICO headset through Android<br/>manufacturer/brand and a nonempty model property. Connect a supported<br/>HMD and set `ANDROID_SERIAL` if the wrong device was selected.<br/>Before each new browser launch, ADB must confirm<br/>`mWakefulness=Awake`; wear or unlock the headset (or cover its<br/>proximity sensor) when the service reports that it is asleep. |
| `TELEOP_OOB_RECORDING`                  | Recording to select in the client before CONNECT: a host path, pushed to<br/>the headset, or `latest`, the newest<br/>`isaacteleop-input-recording*.json` (the client’s Save name) in its<br/>Download folder.<br/>Starting the replay is left to the operator. Unset by default.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `TELEOP_HEADSET_BROWSER_PACKAGE`        | Headset browser package. Unset, the vendor’s browser is used. On PICO,<br/>set `com.pico.browser`: in hand-tracking mode the OS holds VR from<br/>non-system browsers behind a “requires controllers” dialog.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `TELEOP_OOB_RECOVERY_TIMEOUT_SEC`       | Positive finite recovery-episode duration in seconds (default `60`).<br/>Expiry changes to observation mode; it never stops the host. If this is<br/>shorter than the browser reconnect budget, the host caps browser grace<br/>early enough to leave one retry interval for a bounded fallback.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `TELEOP_OOB_RETRY_INTERVAL_SEC`         | Positive finite retry and observation interval in seconds (default `5`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `TELEOP_CLIENT_RECONNECT_MAX_ATTEMPTS`  | Non-negative number of browser-local reconnect attempts (default `10`).<br/>The WebXR client’s stream reconnect loop is opt-in — off unless an<br/>explicit `reconnectEnabled=true` URL param or the 2D debug UI’s<br/>checkbox turns it on — and OOB has no way to know which, so the host<br/>always waits out this budget before taking over, whether or not the<br/>client is actually retrying. The value is also included in the<br/>generated headset URL.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `TELEOP_CLIENT_RECONNECT_DELAY_MS`      | Non-negative delay between browser-local attempts in milliseconds<br/>(default `3000`). The host derives its bounded recovery grace from<br/>this value and the maximum-attempt count.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| `USB_BACKEND_PORT`                      | CloudXR backend port the headset reaches via `adb reverse` in<br/>`--usb-local` mode (default `49100`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| `USB_TURN_PORT`                         | coturn TURN-server port for WebRTC ICE relay in `--usb-local`<br/>mode (default `3478`). `adb reverse`-mapped to the headset.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |

<a id="usb-local-mode-setup-oob-usb-local"></a>

## USB-local mode (`--setup-oob --usb-local`)

`--usb-local` routes teleop signalling, the web client, and WebRTC media
over the USB cable on the headset’s loopback via `adb reverse`.  It requires
`--setup-oob` because the OOB hub is the only path that delivers TURN relay
configuration to the client (without it the headset has no way to discover the
coturn endpoint).  Always use both flags together:

```bash
python -m isaaccapture.cloudxr.service run --accept-eula --setup-oob --usb-local
```

On startup the launcher:

1. Pre-flights host tools, ports, and static assets. It can start with no
   headset attached.
2. Resolves the WebXR static directory from
   `TELEOP_WEB_CLIENT_STATIC_DIR` (default `~/.cloudxr/static-client`)
   and syncs missing `index.html`, `bundle.js`, and `bundle.emulator.js` from
   the published client (see [Build WebXR Client](../development/build_from_source/webxr.md)).
3. Serves that directory at `https://localhost:48322/client/` on the WSS
   proxy’s existing TLS listener.
4. After the selected headset is ready, checks its non-loopback network,
   starts/verifies coturn, and creates `adb reverse` rules for 48322
   (WSS and static UI), 49100 (backend), and 3478 (TURN).
5. Launches the teleop URL and clicks CONNECT via CDP. After cable loss,
   replugging the same serial first rebuilds all three reverse rules, verifies
   coturn, and repairs CDP forwarding without mutating a surviving browser tab.
   The WebXR client owns its bounded reconnect attempts. Fresh post-repair
   stream metrics resume the existing session; a stable terminal event allows
   one same-tab CONNECT click, and only a missing/unusable tab falls back to
   full browser launch automation. ADB loss, missing reverse rules, TURN
   failure or restart, and control/CDP/stream transport loss all enter this
   preservation-first repair path. While preserving an active browser, the
   launcher checks recovery prerequisites at most once per second instead of
   waiting for the ordinary five-second observation cadence.

In `--usb-local` mode the launcher also wipes localStorage / IndexedDB /
cookies / HTTP cache for the teleop UI origin (`https://localhost:<proxy_port>`)
once for the selected session — the SDK and web client both cache settings
(e.g. `general.iceTransportPolicy` for ICE transport policy,
`cxr.isaac.teleopPath` for the last-used project) in localStorage, and a
stale value can silently win over a fresh URL param. The origin is owned
by the launcher so clearing it has no collateral effect; the step is a
no-op when the headset browser isn’t running yet.

Required apt packages: `adb` (`android-tools-adb`) and `coturn`.
No Node.js / `npm` is required at runtime.

<a id="troubleshooting"></a>

## Troubleshooting

<a id="teleop-client-error-no-local-connection-candidates-0xc0f2220f"></a>

### Teleop client error: “No local connection candidates” (0xC0F2220F)

**Cause:** Wi-Fi must stay associated on the headset throughout the
session, even in `--usb-local` mode. No teleop traffic actually flows
over Wi-Fi — every byte goes over the USB cable via `adb reverse` —
but Chromium’s WebRTC `rtc::NetworkManager` excludes loopback
interfaces when enumerating networks for ICE. If the only active
network on the headset is `lo`, ICE gathering hangs at `gathering`
forever (no local candidates emitted, no error fires) until the CloudXR
session times out with this code.

**Fix:** Associate the headset with any Wi-Fi network and retry.
Internet is **not** required — a phone hotspot with no SIM works, an
open AP you never authenticate to works. The packets still route over
USB (the kernel short-circuits loopback regardless of source
interface); the Wi-Fi interface just needs to *exist* with an IP so
WebRTC’s enumeration is non-empty.

The lifecycle checks this via `adb shell ip -o -4 addr show` after the
host is listening. A successful command with no non-loopback interface
reports a network block; a failed ADB command reports cable/transport loss
and never claims that Wi-Fi dropped. Wi-Fi restoration resumes recovery.

<a id="web-client-ui-has-no-text-oob-host-client"></a>

### Web client UI has no text (OOB / `--host-client`)

**Cause:** Production webpack emits `bundle.js` (main app, including UIKit MSDF text)
and `bundle.emulator.js` (desktop emulation only). If the static cache under
`TELEOP_WEB_CLIENT_STATIC_DIR` is missing or outdated — especially a truncated
`bundle.js` — in-VR text does not render.

**Fix:** Ensure `index.html`, `bundle.js`, and `bundle.emulator.js` are present
under the static dir — run the launcher once with network, copy a full `npm run build`
output, or use the GitHub Pages URL directly. See
[Build WebXR Client](../development/build_from_source/webxr.md).

<a id="cdp-startbutton-marked-failed-not-actionable"></a>

### CDP: startButton marked failed / not actionable

**Cause:** The web client’s capability check (`App.tsx`) sets the button
label to `CONNECT (capability check failed)` and disables it when a
required feature is missing (WebGL2, `requestVideoFrameCallback`,
immersive-VR support). ADB automation detects this and aborts instead of
clicking a dead button.

**Fix:** Open the teleop URL on the headset manually and read the
`errorMessageBox` — it names the specific missing capability. Common
causes: launched in WebLayer instead of Meta Quest Browser (no WebXR
support → IWER fallback silently activates); WebGL2 disabled by device
policy.

<a id="coturn-not-found-turn-server-failed-to-start"></a>

### `coturn` not found / TURN server failed to start

**Cause:** `--usb-local` requires coturn to relay WebRTC media between
the headset (TCP via adb reverse) and the CloudXR backend (UDP on
loopback).

**Fix:** `sudo apt-get install -y coturn`. The launcher starts its own
`turnserver` process on 127.0.0.1:3478; no systemd service is needed
(and may conflict — stop the system `coturn.service` if enabled).

Inspect `/tmp/coturn-cloudxr-3478.log` for bind errors or credential
mismatches; the launcher truncates this file on every start so only the
current session’s output is present.

<a id="tab-not-found-within-timeout"></a>

### Tab not found within timeout

**Cause:** The headset’s default URL handler is something other than Meta
Quest Browser (e.g. WebLayer on Meta Quest) and did not open the teleop
URL in a browser with remote-debugging exposed.

**Fix:** Open `chrome://inspect#devices` on this PC, inspect the
headset tab manually, and click CONNECT. Or set a different default
browser on the headset.

<a id="webxr-static-download-fails-offline-proxy"></a>

### WebXR static download fails (offline / proxy)

**Cause:** The launcher syncs `index.html`, `bundle.js`, and
`bundle.emulator.js` into the static dir on first run. Behind a proxy or with no
internet, this fails and `--usb-local` aborts.

**Fix:** Pre-stage the full client `build/` tree (or the published
`/client/<version>/` directory) into the static dir, then re-run.  The
launcher only downloads missing or empty files.  Override the target directory
via `TELEOP_WEB_CLIENT_STATIC_DIR`.

**Fix:** Set the SDK versions in `deps/cloudxr/.env` (copy from
`.env.default`) so the download script can resolve the right version,
or stage `nvidia-cloudxr-<version>.tgz` in `deps/cloudxr/` manually.
