<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
<!-- SPDX-License-Identifier: Apache-2.0 -->

<a id="soma-pose-transport"></a>

# SOMA Pose Transport

Isaac Capture can transport SOMA body and hand data through FlatBuffers and
`SchemaPusher`. Producers publish either joint rotations or evaluated joint
poses. Consumer-side sources normalize both representations to global joint
poses for retargeting or visualization.

The schemas and examples are defined and verified against the SOMA-X v0.3.1
pose interface. See the [SOMA-X repository](https://github.com/NVlabs/SOMA-X)
for the upstream model and assets.

> ##### On this page
> 
> * [Data flow](#data-flow)
> * [Choose a representation](#choose-a-representation)
> * [Build the examples](#build-the-examples)
> * [Run the body example](#run-the-body-example)
> * [Run the hand example](#run-the-hand-example)
> * [Collection IDs and schema identifiers](#collection-ids-and-schema-identifiers)
> * [Vendor integration](#vendor-integration)
> * [Current limitations](#current-limitations)
> * [Troubleshooting](#troubleshooting)
>   * [No module named `isaaccapture_examples`](#no-module-named-isaaccapture-examples)
>   * [The publisher cannot connect](#the-publisher-cannot-connect)
>   * [The viewer receives no skeleton](#the-viewer-receives-no-skeleton)
> * [See also](#see-also)

<a id="data-flow"></a>

## Data flow

Body and hand data use separate schemas and collections. Applications can
select their body and hand sources independently.

```text
vendor body SDK -> SOMA body FlatBuffer -> SchemaPusher -> SOMA body tracker
vendor hand SDK -> SOMA hand FlatBuffer -> SchemaPusher -> SOMA hand tracker
```

The body schema contains SOMA’s complete body control set, including its hand
controls. The standalone hand schemas provide independent left and right hand
streams, identified by `hand_side` (`SomaHandSide.LEFT` or
`SomaHandSide.RIGHT`). Isaac Capture does not automatically fuse standalone
hands with the hand controls in a body payload.

<a id="choose-a-representation"></a>

## Choose a representation

Use the representation closest to the producer’s native data.

| Representation    | FlatBuffer types                                      | Behavior                                                                                                                                                         |
|-------------------|-------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `joint-rotations` | `SomaBodyJointRotations` and `SomaHandJointRotations` | Publishes SOMA joint rotations and the global Hips or Wrist translation.<br/>The consumer evaluates forward kinematics against a prepared SOMA<br/>identity.     |
| `joint-poses`     | `SomaBodyJointPoses` and `SomaHandJointPoses`         | Publishes global position and orientation for each provided joint. The<br/>consumer maps these poses directly and does not run per-frame forward<br/>kinematics. |

`joint-rotations` most directly matches the SOMA-X pose-control interface.
`joint-poses` is an Isaac Capture transport profile for producers that already
have evaluated global joint transforms.

Both representations use positions in meters and unit XYZW quaternions. They
use a right-handed reference frame with positive Y up and positive Z forward.
The publisher must convert its native skeleton and reference frame before
serialization. See [Tracker Reference](../trackers/index.md#tracker-reference) for the complete field and
validity contracts.

Consumers normalize finite quaternions whose norm is greater than `1e-12`. A
zero, degenerate, or non-finite quaternion, position, or global translation is
unavailable. Quaternion signs are not canonicalized, so `q` and `-q` remain
equivalent.

Rotation consumers resolve the SOMA v0.3.1 reference through SOMA-X and pass it
explicitly during forward kinematics. The prepared identity and subject scale
remain caller-selected calibration and are not constrained by this check.

Body and hand joint values are sorted, unique keyed vectors. Every entry carries
an explicit `SomaBodyJoint` or `SomaHandJoint` identifier. A missing joint is
unavailable for that frame; producers do not serialize placeholder values for
joints they cannot provide. Consumers expand the keyed values into SOMA-X’s
canonical dense order when invoking its body or hand helpers.

<a id="build-the-examples"></a>

## Build the examples

Start with a configured Isaac Capture checkout. See
[Build from Source](../../development/build_from_source/index.md) for the complete build
prerequisites. Install the live viewer and both deterministic publishers into a venv
set up as in [step 4 of Build from source](../../development/build_from_source/index.md#build-from-source-install-package):

```bash
uv pip install -e ./examples/deviceio_live_view --no-sources
uv pip install -e ./examples/soma_body_publisher --no-sources
uv pip install -e ./examples/soma_hand_publisher --no-sources
```

The example packages install `py-soma-x==0.3.1`.
The publishers use the Python-accessible `TensorPushTracker` and its native
`SchemaPusher` transport; no separate sender executable is needed. Demo samples
use the session’s monotonic clock for both timestamps.

<a id="run-the-body-example"></a>

## Run the body example

Start the viewer first. It launches the CloudXR runtime and prints the viewer
URL. Open that URL before starting the publisher. On a remote host, replace
`localhost` in the URL with the host name or IP address.

```bash
uv run --no-sync python -m isaaccapture_examples.deviceio_live_view \
  --accept-eula \
  --body-schema soma
```

In another terminal, load the runtime environment and start the body
publisher:

```bash
source ~/.cloudxr/run/cloudxr.env
uv run --no-sync python -m isaaccapture_examples.soma_body_publisher \
  --loop
```

These commands use `joint-rotations`. To publish evaluated joint poses, add
`--body-representation joint-poses` to the publisher and
`--soma-body-representation joint-poses` to the viewer.

<a id="run-the-hand-example"></a>

## Run the hand example

Start the viewer with SOMA hands:

```bash
uv run --no-sync python -m isaaccapture_examples.deviceio_live_view \
  --accept-eula \
  --hand-schema soma
```

In another terminal, load the runtime environment and start the hand
publisher:

```bash
source ~/.cloudxr/run/cloudxr.env
uv run --no-sync python -m isaaccapture_examples.soma_hand_publisher \
  --loop
```

These commands use `joint-rotations`. To publish evaluated joint poses, add
`--hand-representation joint-poses` to the publisher and
`--soma-hand-representation joint-poses` to the viewer.

<a id="collection-ids-and-schema-identifiers"></a>

## Collection IDs and schema identifiers

A collection ID pairs a producer with a consumer and is not encoded in the
FlatBuffer payload. The tensor identifier selects the representation carried by
that collection. The demo publishers use these fixed collection IDs; another
producer can choose different IDs as long as its consumers use the same values.

| Data       | Collection ID          | Viewer override                   |
|------------|------------------------|-----------------------------------|
| Body       | `soma_body_demo`       | `--soma-body-collection-id`       |
| Left hand  | `soma_hand_left_demo`  | `--soma-left-hand-collection-id`  |
| Right hand | `soma_hand_right_demo` | `--soma-right-hand-collection-id` |

The representation selected by the publisher and consumer must match. The
transport identifiers are `soma_body_joint_rotations`,
`soma_body_joint_poses`, `soma_hand_joint_rotations`, and
`soma_hand_joint_poses`.

<a id="vendor-integration"></a>

## Vendor integration

A vendor integration converts its native skeleton to one of the checked-in
SOMA contracts and publishes it through `SchemaPusher`. Use these schemas as
the serialization source:

- [src/core/schema/fbs/soma_common.fbs](https://github.com/NVIDIA/IsaacCapture/blob/main/src/core/schema/fbs/soma_common.fbs)
- [src/core/schema/fbs/soma_body_common.fbs](https://github.com/NVIDIA/IsaacCapture/blob/main/src/core/schema/fbs/soma_body_common.fbs)
- [src/core/schema/fbs/soma_body_joint_rotations.fbs](https://github.com/NVIDIA/IsaacCapture/blob/main/src/core/schema/fbs/soma_body_joint_rotations.fbs)
- [src/core/schema/fbs/soma_body_joint_poses.fbs](https://github.com/NVIDIA/IsaacCapture/blob/main/src/core/schema/fbs/soma_body_joint_poses.fbs)
- [src/core/schema/fbs/soma_hand_joint_rotations.fbs](https://github.com/NVIDIA/IsaacCapture/blob/main/src/core/schema/fbs/soma_hand_joint_rotations.fbs)
- [src/core/schema/fbs/soma_hand_joint_poses.fbs](https://github.com/NVIDIA/IsaacCapture/blob/main/src/core/schema/fbs/soma_hand_joint_poses.fbs)

The FlatBuffer transport and generated trackers do not depend on SOMA-X or
PyTorch. A rotation consumer uses SOMA-X and its resolved neutral identity to
evaluate forward kinematics. A joint-pose consumer does not need SOMA-X for
per-frame evaluation.

<a id="current-limitations"></a>

## Current limitations

- The examples evaluate rotations against a neutral identity. They do not
  perform subject calibration.
- Body and standalone hand streams are not fused automatically.
- A producer and consumer must select the same representation.
- The live viewer selects sources at launch. It does not switch schemas while
  running.

<a id="troubleshooting"></a>

## Troubleshooting

<a id="no-module-named-isaaccapture-examples"></a>

### No module named `isaaccapture_examples`

Install the corresponding example with `uv pip install -e ./examples/<name> --no-sources` and launch it
with `uv run --no-sync` from the repository root.

<a id="the-publisher-cannot-connect"></a>

### The publisher cannot connect

Start the viewer, open its client, and then source
`~/.cloudxr/run/cloudxr.env` in the publisher terminal. The environment file
identifies the active CloudXR OpenXR runtime.

<a id="the-viewer-receives-no-skeleton"></a>

### The viewer receives no skeleton

Confirm that the publisher and viewer use the same representation and
collection names.

<a id="see-also"></a>

## See also

- [examples/soma_body_publisher/README.md](https://github.com/NVIDIA/IsaacCapture/blob/main/examples/soma_body_publisher/README.md)
- [examples/soma_hand_publisher/README.md](https://github.com/NVIDIA/IsaacCapture/blob/main/examples/soma_hand_publisher/README.md)
- [examples/deviceio_live_view/README.md](https://github.com/NVIDIA/IsaacCapture/blob/main/examples/deviceio_live_view/README.md)
- [Spatial Trackers and Device Data](../trackers.md)
