Deploy Backend Operator

Deploying the backend operator will register your compute backend with OSMO, making its resources available for running workflows. Follow these steps to deploy and connect your backend to OSMO.

Prerequisites

• Install OSMO CLI before you begin

• Replace osmo.example.com with your domain name in the commands below

Step 1: Create Service Account for Backend Operator

Create a service account and access token using OSMO CLI for backend operator authentication.

First, log in to OSMO:

$ osmo login https://osmo.example.com

Create a service account user for the backend operator:

$ osmo user create backend-operator --roles osmo-backend

Create a Access Token for the service account with the osmo-backend role:

$ export OSMO_SERVICE_TOKEN=$(osmo token set backend-token \
    --user backend-operator \
    --expires-at <insert-date> \
    --description "Backend Operator Token" \
    --roles osmo-backend \
    -t json | jq -r '.token')

Note

Replace <insert-date> with an expiration date in UTC format (YYYY-MM-DD). Save the token securely as it will not be shown again.

Tip

The --roles osmo-backend option limits the token to only the osmo-backend role. If omitted, the token inherits all roles from the user.

See also

See Service Accounts for more details on creating and managing service accounts.

Step 2: Create K8s Namespaces and Secrets

Create Kubernetes namespaces and secrets necessary for the backend deployment.

  # Create namespaces for osmo operator and osmo workflows
  $ kubectl create namespace osmo-operator
  $ kubectl create namespace osmo-workflows

  # Create the secret used to authenticate with osmo
  $ kubectl create secret generic osmo-operator-token -n osmo-operator \
      --from-literal=token=$OSMO_SERVICE_TOKEN

Step 3: Deploy Backend Operator

Deploy the backend operator to the backend kubernetes cluster.

Prepare the backend_operator_values.yaml file:

backend_operator_values.yaml

global:
  osmoImageTag: <insert-osmo-image-tag>  # REQUIRED: Update with OSMO image tag
  serviceUrl: https://osmo.example.com
  agentNamespace: osmo-operator
  backendNamespace: osmo-workflows
  backendName: default  # REQUIRED: Update with your backend name
  accountTokenSecret: osmo-operator-token
  loginMethod: token

  services:
    backendListener:
      resources:
        requests:
            cpu: "1"
            memory: "1Gi"
        limits:
            memory: "1Gi"
    backendWorker:
      resources:
        requests:
            cpu: "1"
            memory: "1Gi"
        limits:
            memory: "1Gi"

Note

If you plan to use group templates that create ConfigMaps, CRDs, or other Kubernetes objects, you must grant the backend worker permission for those resource kinds via services.backendWorker.extraRBACRules. See Required Backend Permissions for details and examples.

Deploy the backend operator:

$ helm repo add osmo https://helm.ngc.nvidia.com/nvidia/osmo

$ helm repo update

$ helm upgrade --install osmo-operator osmo/backend-operator \
  -f ./backend_operator_values.yaml \
  --version <insert-chart-version> \
  --namespace osmo-operator

Step 4: Validate Deployment

Use the OSMO CLI to validate the backend configuration

$ export BACKEND_NAME=default  # Update with your backend name

$ osmo config show BACKEND $BACKEND_NAME

Alternatively, visit http://osmo.example.com/api/configs/backend in your browser.

Ensure the backend is online (see the highlighted line in the JSON output):

{
  "backends": [
      {
          "name": "default",
          "description": "Default backend",
          "version": "6.0.0",
          "k8s_uid": "6bae3562-6d32-4ff1-9317-09dd973c17a2",
          "k8s_namespace": "osmo-workflows",
          "dashboard_url": "",
          "grafana_url": "",
          "tests": [],
          "scheduler_settings": {
              "scheduler_type": "kai",
              "scheduler_name": "kai-scheduler",
              "scheduler_timeout": 30
          },
          "node_conditions": {
              "rules": null,
              "prefix": "osmo.example.com/"
          },
          "last_heartbeat": "2025-11-15T02:35:17.957569",
          "created_date": "2025-09-03T19:48:21.969688",
          "router_address": "wss://osmo.example.com",
          "online": true
      }
  ]
}

See also

See /api/configs/backend for more information

Troubleshooting

Token Expiration Error

Connection failed with error: {OSMOUserError: Token is expired, but no refresh token is present}

Check if the token is expired by listing the service account’s tokens:

$ osmo token list --user backend-operator

If the token is expired, create a new one following Step 1: Create Service Account for Backend Operator. Remember to update the Kubernetes secret with the new token:

$ kubectl delete secret osmo-operator-token -n osmo-operator
$ kubectl create secret generic osmo-operator-token -n osmo-operator \
    --from-literal=token=$OSMO_SERVICE_TOKEN