Minimal Deployment#

This guide provides instructions for deploying OSMO in a minimal configuration suitable for testing, development, and evaluation purposes. This setup of OSMO creates the service and backend operator in the same kubernetes cluster, is suitable for single-tenant, has no authentication, and is designed for quick setup and experimentation.

Warning

Minimal deployment is not recommended for production use as it lacks authentication and has limited features.

Overview#

The minimal OSMO deployment includes:

API Service
Web UI
Router
External PostgreSQL database (configurable)
External Redis cache (configurable)
Default admin authentication (no identity provider required)
Single namespace deployment
Single replica per service
Minimal resource requirements

Prerequisites#

Refer to Cloud for the setup of the Kubernetes cluster, PostgreSQL database, and Redis instance.

Step 1: Create Namespace#

Create a dedicated namespace to deploy OSMO service:

$ kubectl create namespace osmo-minimal

Step 2: Add Helm Repository#

Add the NVIDIA OSMO Helm repository:

$ helm repo add osmo https://helm.ngc.nvidia.com/nvidia/osmo

$ helm repo update

Step 3: Create K8s Secrets#

Create secret for database and redis passwords:

$ kubectl create secret generic db-secret --from-literal=db-password=<your-db-password> --namespace osmo-minimal
$ kubectl create secret generic redis-secret --from-literal=redis-password=<your-redis-password> --namespace osmo-minimal

Create the master encryption key (MEK) for database encryption:

Generate a new master encryption key:

The MEK should be a JSON Web Key (JWK) with the following format:
```
{"k":"<base64-encoded-32-byte-key>","kid":"key1","kty":"oct"}
```

Generate the key using OpenSSL:

# Generate a 32-byte (256-bit) random key and base64 encode it
RANDOM_KEY=$(openssl rand -base64 32 | tr -d '\n')

# Create the JWK format
echo "{\"k\":\"$RANDOM_KEY\",\"kid\":\"key1\",\"kty\":\"oct\"}"

Base64 encode the entire JWK:

# Take the JWK output from step 2 and base64 encode it
JWK_JSON='{"k":"<your-base64-key>","kid":"key1","kty":"oct"}'
ENCODED_JWK=$(echo -n "$JWK_JSON" | base64 | tr -d '\n')
echo $ENCODED_JWK

Create the ConfigMap with your generated MEK:

$ kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: mek-config
  namespace: osmo-minimal
data:
  mek.yaml: |
    currentMek: key1
    meks:
      key1: $ENCODED_JWK
EOF

Security Considerations

Store the original JWK securely as you’ll need it for backups and recovery
Never commit the MEK to version control
Use a secure key management system, such as Vault in production
The MEK is used to encrypt sensitive data in the database

Example MEK generation script:

#!/bin/bash
# Generate MEK for OSMO

# Generate random 32-byte key
RANDOM_KEY=$(openssl rand -base64 32 | tr -d '\n')

# Create JWK
JWK_JSON="{\"k\":\"$RANDOM_KEY\",\"kid\":\"key1\",\"kty\":\"oct\"}"

# Base64 encode the JWK
ENCODED_JWK=$(echo -n "$JWK_JSON" | base64 | tr -d '\n')

echo "Generated JWK: $JWK_JSON"
echo "Encoded JWK: $ENCODED_JWK"

# Create ConfigMap
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: mek-config
  namespace: osmo-minimal
data:
  mek.yaml: |
    currentMek: key1
    meks:
      key1: $ENCODED_JWK
EOF

Step 4: Configure PostgreSQL#

Create a database for OSMO using the following command.

$ export OSMO_DB_HOST=<your-db-host>
$ export OSMO_PGPASSWORD=<your-postgres-password>

$ kubectl apply -f - <<EOF
  apiVersion: v1
  kind: Pod
  metadata:
    name: osmo-db-ops
  spec:
    containers:
      - name: osmo-db-ops
        image: alpine/psql:17.5
        command: ["/bin/sh", "-c"]
        args:
          - "PGPASSWORD=$OSMO_PGPASSWORD psql -U postgres -h $OSMO_DB_HOST -p 5432 -d postgres -c 'CREATE DATABASE osmo_db;'"
    restartPolicy: Never
  EOF

Note

Ignore export OSMO_PGPASSWORD=<your-postgres-password> and PGPASSWORD=$OSMO_PGPASSWORD if your PostgreSQL was configured without a password.

Verify that the process Completed with kubectl get pod osmo-db-ops. Then delete the pod with:

$ kubectl delete pod osmo-db-ops

Step 5: Prepare Service Values#

Create the following values files for the minimal deployment:

API Service Values (osmo_values.yaml):

UI Service Values (ui_values.yaml):

Router Service Values (router_values.yaml):

Important

Replace <insert-osmo-image-tag> with the desired OSMO version you want to deploy
Update the serviceName for postgres and redis to match your external services

Step 6: Helm Deploy#

Deploy the OSMO components using the minimal configuration:

Deploy OSMO Service:

$ helm upgrade --install osmo-minimal osmo/service \
  -f ./osmo_values.yaml \
  --namespace osmo-minimal

Deploy OSMO UI:

$ helm upgrade --install ui-minimal osmo/web-ui \
  -f ./ui_values.yaml \
  --namespace osmo-minimal

Deploy OSMO Router:

$ helm upgrade --install router-minimal osmo/router \
  -f ./router_values.yaml \
  --namespace osmo-minimal

Step 7: Verify Deployment#

Verify that all pods are running:

$ kubectl get pods -n osmo-minimal

You should see pods similar to the following example:

NAME                                    READY   STATUS    RESTARTS   AGE
osmo-agent-xxx                          1/1     Running   0          2m
osmo-delayed-job-monitor-xxx            1/1     Running   0          2m
osmo-service-xxx                        1/1     Running   0          2m
osmo-worker-xxx                         1/1     Running   0          2m
osmo-logger-xxx                         1/1     Running   0          2m
osmo-ui-xxx                             1/1     Running   0          2m
osmo-router-xxx                         1/1     Running   0          2m

Verify that all services are running:
```
$ kubectl get services -n osmo-minimal
```
Port forward the gateway to access the OSMO UI:
```
$ kubectl port-forward service/osmo-gateway 9000:80 -n osmo-minimal
```
Visit http://localhost:9000 in your web browser to access the OSMO UI dashboard as a guest user.

Step 8: Install Backend Operator#

Prepare backend_operator_values.yaml file:

Login to OSMO:

Port forward the gateway and login to OSMO:

$ kubectl port-forward service/osmo-gateway 9000:80 -n osmo-minimal

$ osmo login http://localhost:9000 --method=dev --username=testuser

Create the service account and token:

Create a service account user and generate a token for the backend operator with OSMO CLI:

# Create the service account user
$ osmo user create backend-operator --roles osmo-backend

# Generate a token for the service account with the osmo-backend role
$ export BACKEND_TOKEN=$(osmo token set backend-token \
    --user backend-operator \
    --expires-at <insert-date> \
    --description "Backend Operator Token" \
    --roles osmo-backend \
    -t json | jq -r '.token')

# Create the Kubernetes secret
$ kubectl create secret generic osmo-operator-token --from-literal=token=$BACKEND_TOKEN --namespace osmo-operator

Deploy the backend operator:

$ helm upgrade --install osmo-operator osmo/backend-operator \
  -f ./backend_operator_values.yaml \
  --namespace osmo-operator

Step 9: Access OSMO#

Port forward the gateway to access all OSMO services if you have not already:

$ kubectl port-forward service/osmo-gateway 9000:80 -n osmo-minimal

Access OSMO Service API:

Access the OSMO API at http://localhost:9000/api/docs in your web browser. You can interact with the API using the OSMO CLI.

$ osmo login http://localhost:9000 --method=dev --username=testuser

$ osmo resource list -p default
Node             Pool      Platform      Storage [Gi]   CPU [#]   Memory [Gi]   GPU [#]
========================================================================================
<node-name>       default   default        0/2028         0/2       1/32         0/8
========================================================================================

Access OSMO UI:

Access the OSMO UI at http://localhost:9000 in your web browser. You should be able to see the OSMO UI dashboard as a guest user.

Step 10: Basic Configuration#

After deployment, you need to configure a central storage for workflow spec, workflow logs, and task’s artifacts data before you can start running workflows:

Follow the Configure Data Storage guide to setup data storage.
Follow the Install Dependencies guide to install the KAI scheduler for running workflows.

Set the service base URL so that workflow pods can reach the gateway:

$ cat << EOF > /tmp/osmo_service_config.json
{
  "service_base_url": "http://osmo-gateway.osmo-minimal.svc.cluster.local"
}
EOF

$ osmo config update SERVICE --file /tmp/osmo_service_config.json

Testing Your Deployment#

Follow the Run Workflows guide to test basic OSMO functionality.

What’s Next#

Once you have tested OSMO with the minimal deployment and are ready for production use, consider the following steps:

Consider upgrading to production deployment (Deploy Service)
Configure authentication and authorization (Authentication and Authorization)
Configure persistent storage (Configure Data Storage)
Add observability and monitoring solutions (Add Observability (Optional))

Cleanup#

To remove the minimal deployment:

# Uninstall all helm releases
$ helm uninstall osmo-minimal --namespace osmo-minimal
$ helm uninstall ui-minimal --namespace osmo-minimal
$ helm uninstall router-minimal --namespace osmo-minimal
$ helm uninstall osmo-operator --namespace osmo-operator

# Delete the namespace
$ kubectl delete namespace osmo-minimal
$ kubectl delete namespace osmo-operator

Troubleshooting#

Common Issues#

Pods not starting: Check resource availability and image pull secrets
Database connection issues: Verify PostgreSQL database is accessible from the OSMO service and you have the correct credentials
Redis connection issues: Verify Redis is accessible from the OSMO service and you have the correct credentials, common issues are:
- TLS is enabled but tlsEnabled is set to false in the values file
Port forwarding issues: Ensure no other services are using the same port