Deploy Backend Operator#

Deploying the backend operator will register your compute backend with OSMO, making its resources available for running workflows. Follow these steps to deploy and connect your backend to OSMO.

Prerequisites

Install OSMO CLI before you begin
Replace osmo.example.com with your domain name in the commands below

Step 1: Create OSMO Service Token#

Create a service access token using OSMO CLI for backend operator authentication:

$ osmo login https://osmo.example.com

$ export OSMO_SERVICE_TOKEN=$(osmo token set backend-token --expires-at <insert-date> --description "Backend Operator Token" --service --roles osmo-backend -t json | jq -r '.token')

Note

Replace <insert-date> with an expiration date in UTC format (YYYY-MM-DD). Save the token securely as it will not be shown again.

Step 2: Create K8s Namespaces and Secrets#

Create Kubernetes namespaces and secrets necessary for the backend deployment.

  # Create namespaces for osmo operator and osmo workflows
  $ kubectl create namespace osmo-operator
  $ kubectl create namespace osmo-workflows

  # Create the secret used to authenticate with osmo
  $ kubectl create secret generic osmo-operator-token -n osmo-operator \
      --from-literal=token=$OSMO_SERVICE_TOKEN

Step 3: Deploy Backend Operator#

Deploy the backend operator to the backend kubernetes cluster.

Prepare the backend_operator_values.yaml file:

Deploy the backend operator:

$ helm repo add osmo https://helm.ngc.nvidia.com/nvidia/osmo

$ helm repo update

$ helm upgrade --install osmo-operator osmo/backend-operator \
  -f ./backend_operator_values.yaml \
  --version <insert-chart-version> \
  --namespace osmo-operator

Step 4: Configure KAI Scheduler#

Install the KAI scheduler (see Install KAI Scheduler), then configure it for the backend:

$ cat << EOF > /tmp/scheduler_settings.json
{
  "scheduler_settings": {
    "scheduler_type": "kai",
    "scheduler_name": "kai-scheduler",
    "coscheduling": true,
    "scheduler_timeout": 30
  }
}
EOF

$ export BACKEND_NAME=default  #Update with your backend name
$ osmo config update BACKEND $BACKEND_NAME --file /tmp/scheduler_settings.json

Note

See Scheduler Configuration for detailed scheduler configuration options.

Step 5: Validate Deployment#

Use the OSMO CLI to validate the backend configuration

$ export BACKEND_NAME=default  # Update with your backend name

$ osmo config show BACKEND $BACKEND_NAME

Alternatively, visit http://osmo.example.com/api/configs/backend in your browser.

Ensure the backend is online (see the highlighted line in the JSON output):

{
  "backends": [
      {
          "name": "default",
          "description": "Default backend",
          "version": "6.0.0",
          "k8s_uid": "6bae3562-6d32-4ff1-9317-09dd973c17a2",
          "k8s_namespace": "osmo-workflows",
          "dashboard_url": "",
          "grafana_url": "",
          "tests": [],
          "scheduler_settings": {
              "scheduler_type": "kai",
              "scheduler_name": "kai-scheduler",
              "coscheduling": true,
              "scheduler_timeout": 30
          },
          "node_conditions": {
              "rules": null,
              "prefix": "osmo.example.com/"
          },
          "last_heartbeat": "2025-11-15T02:35:17.957569",
          "created_date": "2025-09-03T19:48:21.969688",
          "router_address": "wss://osmo.example.com",
          "online": true
      }
  ]
}

Troubleshooting#

Token Expiration Error#

Connection failed with error: {OSMOUserError: Token is expired, but no refresh token is present}

Check the osmo token list --service command to see if the token is expired. Follow Step 1: Create OSMO Service Token to create a new token.