Minimal Deployment#

This guide provides instructions for deploying OSMO in a minimal configuration suitable for testing, development, and evaluation purposes. This setup of OSMO creates the service and backend operator in the same kubernetes cluster, is suitable for single-tenant, has no authentication, and is designed for quick setup and experimentation.

Warning

Minimal deployment is not recommended for production use as it lacks authentication and has limited features.

Overview#

The minimal OSMO deployment includes:

API Service
Web UI
Router
External PostgreSQL database (configurable)
External Redis cache (configurable)
No authentication or authorization
Single namespace deployment
Single replica per service
Minimal resource requirements

Prerequisites#

Refer to Cloud for the setup of the Kubernetes cluster, PostgreSQL database, and Redis instance.

Note

Ingress must be installed as part of networking setup to use exec or port forwarding features in OSMO workflows.

Step 1: Create Namespace#

Create a dedicated namespace to deploy OSMO service:

$ kubectl create namespace osmo-minimal

Step 2: Add Helm Repository#

Add the NVIDIA OSMO Helm repository:

$ helm repo add osmo https://helm.ngc.nvidia.com/nvidia/osmo

$ helm repo update

Step 3: Create K8s Secrets#

Create secret for database and redis passwords:

$ kubectl create secret generic db-secret --from-literal=db-password=<your-db-password> --namespace osmo-minimal
$ kubectl create secret generic redis-secret --from-literal=redis-password=<your-redis-password> --namespace osmo-minimal

Create the master encryption key (MEK) for database encryption:

Generate a new master encryption key:

The MEK should be a JSON Web Key (JWK) with the following format:
```
{"k":"<base64-encoded-32-byte-key>","kid":"key1","kty":"oct"}
```

Generate the key using OpenSSL:

# Generate a 32-byte (256-bit) random key and base64 encode it
RANDOM_KEY=$(openssl rand -base64 32 | tr -d '\n')

# Create the JWK format
echo "{\"k\":\"$RANDOM_KEY\",\"kid\":\"key1\",\"kty\":\"oct\"}"

Base64 encode the entire JWK:

# Take the JWK output from step 2 and base64 encode it
JWK_JSON='{"k":"<your-base64-key>","kid":"key1","kty":"oct"}'
ENCODED_JWK=$(echo -n "$JWK_JSON" | base64 | tr -d '\n')
echo $ENCODED_JWK

Create the ConfigMap with your generated MEK:

$ kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: mek-config
  namespace: osmo-minimal
data:
  mek.yaml: |
    currentMek: key1
    meks:
      key1: $ENCODED_JWK
EOF

Security Considerations

Store the original JWK securely as you’ll need it for backups and recovery
Never commit the MEK to version control
Use a secure key management system, such as Vault in production
The MEK is used to encrypt sensitive data in the database

Example MEK generation script:

#!/bin/bash
# Generate MEK for OSMO

# Generate random 32-byte key
RANDOM_KEY=$(openssl rand -base64 32 | tr -d '\n')

# Create JWK
JWK_JSON="{\"k\":\"$RANDOM_KEY\",\"kid\":\"key1\",\"kty\":\"oct\"}"

# Base64 encode the JWK
ENCODED_JWK=$(echo -n "$JWK_JSON" | base64 | tr -d '\n')

echo "Generated JWK: $JWK_JSON"
echo "Encoded JWK: $ENCODED_JWK"

# Create ConfigMap
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: mek-config
  namespace: osmo-minimal
data:
  mek.yaml: |
    currentMek: key1
    meks:
      key1: $ENCODED_JWK
EOF

Step 4: Configure PostgreSQL#

Create a database for OSMO using the following command.

$ export OSMO_DB_HOST=<your-db-host>
$ export OSMO_PGPASSWORD=<your-postgres-password>

$ kubectl apply -f - <<EOF
  apiVersion: v1
  kind: Pod
  metadata:
    name: osmo-db-ops
  spec:
    containers:
      - name: osmo-db-ops
        image: alpine/psql:17.5
        command: ["/bin/sh", "-c"]
        args:
          - "PGPASSWORD=$OSMO_PGPASSWORD psql -U postgres -h $OSMO_DB_HOST -p 5432 -d postgres -c 'CREATE DATABASE osmo_db;'"
    restartPolicy: Never
  EOF

Note

Ignore export OSMO_PGPASSWORD=<your-postgres-password> and PGPASSWORD=$OSMO_PGPASSWORD if your PostgreSQL was configured without a password.

Verify that the process Completed with kubectl get pod osmo-db-ops. Then delete the pod with:

$ kubectl delete pod osmo-db-ops

Step 5: Prepare Service Values#

Create the following values files for the minimal deployment:

API Service Values (osmo_values.yaml):

UI Service Values (ui_values.yaml):

Router Service Values (router_values.yaml):

Important

Replace <insert-osmo-image-tag> with the desired OSMO version you want to deploy
Replace <your-domain> with your domain name (e.g., osmo.example.com) if you want to enable ingress for external access
Update the serviceName for postgres and redis to match your external services
Update ingressClass and ingress to match your cluster’s ingress controller and ingress configuration
Ensure your DNS points to the ingress controller’s load balancer if you want to enable ingress for external access

Step 6: Helm Deploy#

Deploy the OSMO components using the minimal configuration:

Deploy OSMO Service:

$ helm upgrade --install osmo-minimal osmo/service \
  -f ./osmo_values.yaml \
  --namespace osmo-minimal

Deploy OSMO UI:

$ helm upgrade --install ui-minimal osmo/web-ui \
  -f ./ui_values.yaml \
  --namespace osmo-minimal

Deploy OSMO Router:

$ helm upgrade --install router-minimal osmo/router \
  -f ./router_values.yaml \
  --namespace osmo-minimal

Step 7: Verify Deployment#

Verify that all pods are running:

$ kubectl get pods -n osmo-minimal

You should see pods similar to the following example:

NAME                                    READY   STATUS    RESTARTS   AGE
osmo-agent-xxx                          1/1     Running   0          2m
osmo-delayed-job-monitor-xxx            1/1     Running   0          2m
osmo-service-xxx                        1/1     Running   0          2m
osmo-worker-xxx                         1/1     Running   0          2m
osmo-logger-xxx                         1/1     Running   0          2m
osmo-ui-xxx                             1/1     Running   0          2m
osmo-router-xxx                         1/1     Running   0          2m

Verify that all services are running:
```
$ kubectl get services -n osmo-minimal
```

Verify that ingress configuration is set up correctly:

Note

If you have enabled ingress for external access, you should see ingress resources similar to the following:

$ kubectl get ingress -n osmo-minimal

You should see ingress resources similar to:

NAME                     CLASS   HOSTS              ADDRESS         PORTS   AGE
osmo-agent-ingress       nginx   <your-domain>      <lb-ip>         80      2m
osmo-service-ingress     nginx   <your-domain>      <lb-ip>         80      2m
osmo-logger-ingress      nginx   <your-domain>      <lb-ip>         80      2m
osmo-ui-ingress          nginx   <your-domain>      <lb-ip>         80      2m
osmo-router-ingress      nginx   <your-domain>      <lb-ip>         80      2m

Port forward to access the OSMO UI:
```
$ kubectl port-forward service/osmo-ui 3000:80 -n osmo-minimal
```
Visit http://localhost:3000 in your web browser to access the OSMO UI dashboard as a guest user.

Step 8: Install Backend Operator#

Prepare backend_operator_values.yaml file:

Login to OSMO:

If ingress is disabled, you can port forward the OSMO API server and login to OSMO

$ kubectl port-forward service/osmo-service 9000:80 -n osmo-minimal

$ osmo login http://localhost:9000 --method=dev --username=testuser

If ingress is enabled, you can login to OSMO through your domain:

$ osmo login https://<your-domain> --method=dev --username=testuser

Create the account token secret:

Generate a token for the backend operator with OSMO CLI:

$ export BACKEND_TOKEN=$(osmo token set backend-token --expires-at <insert-date> --description "Backend Operator Token" --service --roles osmo-backend -t json | jq -r '.token')

$ kubectl create secret generic osmo-operator-token --from-literal=token=$BACKEND_TOKEN --namespace osmo-operator

Deploy the backend operator:

$ helm upgrade --install osmo-operator osmo/backend-operator \
  -f ./backend_operator_values.yaml \
  --namespace osmo-operator

Step 9: Access OSMO#

With Ingress#

Access OSMO Service API:

Visit https://<your-domain>/api/docs in your web browser to access the OSMO API.
Access OSMO UI:

Visit https://<your-domain> in your web browser to access the OSMO UI.

Note

Replace <your-domain> with your domain name or use the alternative approach below
Ensure your DNS is configured to point to your ingress controller’s load balancer
If you need to test without DNS setup, you can use port forwarding as an alternative as well

Without Ingress#

If you haven’t set up DNS yet, you can access OSMO using port forwarding as an alternative:

Access OSMO Service API:

$ kubectl port-forward service/osmo-service 9000:80 -n osmo-minimal

Then access the OSMO API at http://localhost:9000/api/docs in your web browser. You can interact with the API using the OSMO CLI.

$ osmo login http://localhost:9000 --method=dev --username=testuser

$ osmo resource list -p default
Node             Pool      Platform      Storage [Gi]   CPU [#]   Memory [Gi]   GPU [#]
========================================================================================
<node-name>       default   default        0/2028         0/2       1/32         0/8
========================================================================================

Access OSMO UI (in a separate terminal):
```
$ kubectl port-forward service/osmo-ui 3000:80 -n osmo-minimal
```
Then access the OSMO UI at http://localhost:3000 in your web browser. You should be able to see the OSMO UI dashboard as a guest user.

Important

If you are accessing OSMO with port forwarding, the router service will not be accessible
Consequently, osmo workflow port-forward or osmo workflow exec commands are not expected to work

Step 10: Basic Configuration#

After deployment, you need to configure a central storage for workflow spec, workflow logs, and task’s artifacts data before you can start running workflows:

Follow the Configure Data Storage guide to setup data storage.
Follow the Install Dependencies guide to install the KAI scheduler for running workflows.

(Optional) If you your ingress is not configured, you will need to set the following config with OSMO CLI for workflows to run:

$ cat << EOF > /tmp/osmo_logger_config.json
{
"service_base_url": "http://osmo-logger.osmo-minimal.svc.cluster.local:80"
}
EOF

$ osmo config update SERVICE --file /tmp/osmo_logger_config.json

Testing Your Deployment#

Follow the Run Workflows guide to test basic OSMO functionality.

Warning

When accessing OSMO through port forwarding, you will not be able to run interactive workflows involving port-forwarding or exec functionality.

What’s Next#

Once you have tested OSMO with the minimal deployment and are ready for production use, consider the following steps:

Consider upgrading to production deployment (Deploy Service)
Setup SSO authentication and authorization (Advanced Keycloak Configuration)
Configure persistent storage (Configure Data Storage)
Add observability and monitoring solutions (Add Observability (Optional))

Cleanup#

To remove the minimal deployment:

# Uninstall all helm releases
$ helm uninstall osmo-minimal --namespace osmo-minimal
$ helm uninstall ui-minimal --namespace osmo-minimal
$ helm uninstall router-minimal --namespace osmo-minimal
$ helm uninstall osmo-operator --namespace osmo-operator

# Delete the namespace
$ kubectl delete namespace osmo-minimal
$ kubectl delete namespace osmo-operator

Troubleshooting#

Common Issues#

Pods not starting: Check resource availability and image pull secrets
Database connection issues: Verify PostgreSQL database is accessible from the OSMO service and you have the correct credentials
Redis connection issues: Verify Redis is accessible from the OSMO service and you have the correct credentials, common issues are:
- TLS is enabled but tlsEnabled is set to false in the values file
Port forwarding issues: Ensure no other services are using the same port