Connect to a managed Kubernetes cluster

Launch a managed Kubernetes cluster on your reserved Ornn GPUs, download the namespace-scoped kubeconfig, and run your first GPU workload. A managed Kubernetes cluster turns the GPU nodes you already reserved into a Kubernetes cluster you reach with a namespace-scoped kubeconfig. Ornn runs the control plane; you get full control inside your own namespace. You launch the cluster from the console, download a kubeconfig, and connect with kubectl.

When to use Kubernetes

You run containerized training or inference and want orchestration, not a single host.
You want to schedule multiple GPU jobs across your reserved nodes.
You already have Kubernetes manifests, Helm charts, or operators to deploy.

For a single machine you SSH into, VM access or Bare Metal access is simpler. Choose Kubernetes when you want a cluster.

Prerequisites

The bid is promoted to a confirmed reservation that is visible in My GPUs.
Checkout and payment for the reservation are complete.
You have kubectl installed locally.

Launch the cluster

Open Orchestration

From the console, go to Orchestration (/controllers) and start a new Kubernetes cluster. Cluster launch is not on the reservation access-mode toggle in My GPUs.

Choose Kubernetes and a network mode

Pick Kubernetes, then a network mode:

Public — the API is reachable over the internet.
Private — the API is reachable only over your reservation’s WireGuard VPN (nothing is exposed publicly).

Launch

Launch the cluster. The first launch after an enrollment takes a few extra minutes while the GPU images download. The manage page shows live progress and, once the cluster is active, a Download kubeconfig action plus a Need help connecting? link to this guide.

CLI equivalent:

ornn clusters eligible-nodes <reservation-id> --type kubernetes --network public
ornn clusters create <reservation-id> --type kubernetes --node <node-id> --wait
ornn clusters kubeconfig <reservation-id> --output kubeconfig.yaml

Connect with kubeconfig

Download the kubeconfig

On the cluster panel, click Download kubeconfig. It contains a short-lived, namespace-scoped token and the API endpoint.

Point kubectl at it

export KUBECONFIG="$HOME/Downloads/ornn-<reservation-id>-kubeconfig.yaml"

Confirm the connection

Your access is scoped to your namespace, so namespace commands work:

kubectl get ns tenant-<id>
kubectl get pods

Cluster-wide commands like kubectl get nodes and kubectl cluster-info return Forbidden. That is the security boundary working as intended — your token is scoped to your namespace — not a connection failure.

Run a GPU smoke test

Your reserved GPU nodes are dedicated to your reservation, and your cluster automatically pins your pods to them — you don’t set any node selector or toleration yourself. Just request nvidia.com/gpu and the pod lands on your hardware. This pod runs nvidia-smi and cleans up after itself:

kubectl apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: ornn-smoke
  namespace: tenant-<id>
spec:
  restartPolicy: Never
  containers:
    - name: smoke
      image: nvidia/cuda:12.4.1-base-ubuntu22.04
      command: ["nvidia-smi"]
      resources:
        limits:
          nvidia.com/gpu: 1
EOF

kubectl wait --for=jsonpath='{.status.phase}'=Succeeded pod/ornn-smoke --timeout=300s || true
kubectl logs ornn-smoke || true
kubectl delete pod ornn-smoke --ignore-not-found

You should see your allocated GPU in the nvidia-smi table. Your real workloads only need to request nvidia.com/gpu — the cluster pins them to your reserved nodes for you.

What your namespace can’t do

Your namespace is isolated from other tenants by admission-time policy. A pod or PVC that hits one of the rules below is rejected before it schedules, with a message that starts ValidatingAdmissionPolicy 'ornn-tenant-security-baseline'. In the current rollout you may see this as a warning first, then as a hard denial — treat both the same.

No hostPath volumes. Node filesystem paths belong to the cluster, not the tenant. Use a PVC or your own object storage.
No privileged containers or host namespaces (securityContext.privileged, hostPID, hostIPC, hostNetwork). Drop the privilege escalation; if you need GPU profiling capabilities, contact support to be added to the profiling-allowed group.
No pinning a PVC to an existing PV by name. Leave spec.volumeName unset and let the default StorageClass provision a fresh volume inside your namespace.
No custom nodeName or nodeSelector targeting other reservations. The cluster already pins your pods to your reserved nodes. Either omit node targeting entirely, or match the reservation label the cluster injects for you.

Private clusters: bring up WireGuard first

Private clusters expose nothing publicly — the API is reachable only over your reservation’s WireGuard tunnel. Set it up once:

# 1. Generate a WireGuard keypair locally.
wg genkey | tee privatekey | wg pubkey

# 2. Paste the printed PUBLIC key into the cluster panel's Connect section and
#    generate a config.
# 3. Save the returned config (fill in your private key) and bring the tunnel up.
sudo wg-quick up ./ornn-wg.conf

Once the tunnel is up, the downloaded kubeconfig reaches the API and the steps above are unchanged. Your existing WireGuard config is preserved when the cluster toggles between Public and Private — you don’t need to regenerate it after switching modes.

Replace or remove a WireGuard config

Each WireGuard public key is registered once per gateway. To rotate the key or move the tunnel to a different machine, remove the existing peer from the Connect section first, then paste the new public key and generate a fresh config. If you paste a key that another reservation on the same gateway already owns, the console returns a WireGuard public key already in use error — generate a new keypair with wg genkey and try again, or remove the key from the reservation that currently owns it.

Tear down

Tearing down drains all workers, revokes the kubeconfig, and returns the GPU nodes to your reservation. A torn-down or failed cluster relaunches in place.

Teardown interrupts running workloads and can’t be undone. Persist checkpoints and data to your own object storage first.

Troubleshooting

`kubectl get nodes` is Forbidden

Expected. Your token is namespace-scoped, so cluster-wide reads (get nodes, cluster-info, other namespaces) are denied by design. Use namespace-scoped commands like kubectl get pods.

Commands return Unauthorized

The kubeconfig token is short-lived. Download a fresh kubeconfig from the cluster panel and re-export KUBECONFIG.

My pod is stuck Pending

Your pods are pinned to your reserved GPU nodes automatically. If a pod stays Pending, confirm it requests nvidia.com/gpu and that your reservation still has free GPU capacity (each node has a fixed number of GPUs). Don’t add your own nodeSelector or toleration — the cluster sets those for you.

Image pull fails with 403 Forbidden / DENIED

A failed to fetch anonymous token: ... 403 Forbidden (or DENIED) on kubectl describe pod means the container registry refused an anonymous pull of that image — the image is private or the path doesn’t exist. This is not an Ornn credential issue; no Ornn-provided credential unlocks a third-party registry. Fixes:

Use a public image (the smoke test above uses nvidia/cuda:12.4.1-base-ubuntu22.04 from Docker Hub, which pulls with no credentials).
For gated NVIDIA images, pull from NGC (nvcr.io/nvidia/...) and create an imagePullSecret from your own NGC API key, then reference it in the pod’s imagePullSecrets.
Double-check the registry and repository path — some images are published on NGC (nvcr.io) rather than other registries.

A private cluster won't connect

Make sure the WireGuard tunnel is up (sudo wg show) before using the kubeconfig. Re-generate the peer config from the Connect section if your public key changed.

My pod or PVC is rejected by `ornn-tenant-security-baseline`

The cluster runs an admission policy that keeps tenants isolated. Common causes:

hostPath volume in the pod spec — remove it and use a PVC instead.
securityContext.privileged: true, or hostPID, hostIPC, or hostNetwork set to true — drop the privileged setting.
PVC with spec.volumeName set — remove it so a new volume is provisioned dynamically.
Explicit spec.nodeName or a spec.nodeSelector that targets a node outside your reservation — remove your own node targeting and let the cluster pin the pod for you.

See What your namespace can’t do for the full list.

What’s next

Slurm access

Prefer batch scheduling? Run a managed Slurm cluster on your reserved GPUs instead.

Access overview

Compare the ways to access your reserved Ornn compute.

Get Started

Reservations

Accessing Compute

Storage

My GPUs

Account

Connect to a managed Kubernetes cluster

When to use Kubernetes

Prerequisites

Launch the cluster

Connect with kubeconfig

Run a GPU smoke test

What your namespace can’t do

Private clusters: bring up WireGuard first

Replace or remove a WireGuard config

Tear down

Troubleshooting

What’s next

Slurm access

Access overview

​When to use Kubernetes

​Prerequisites

​Launch the cluster

​Connect with kubeconfig

​Run a GPU smoke test

​What your namespace can’t do

​Private clusters: bring up WireGuard first

​Replace or remove a WireGuard config

​Tear down

​Troubleshooting

​What’s next

Slurm access

Access overview

When to use Kubernetes

Prerequisites

Launch the cluster

Connect with kubeconfig

Run a GPU smoke test

What your namespace can’t do

Private clusters: bring up WireGuard first

Replace or remove a WireGuard config

Tear down

Troubleshooting

What’s next