Skip to content

Join a cluster to a network

A network is a group of clusters that share one set of management services and one set of rules. This page explains how to add your Kubernetes cluster to an existing network, so that its machines can run AI workloads for the network.

At a glance

Who this is for: cluster owners and administrators.
You need from the network operator: the Parser URL and the Cluster Controller Gateway URL.
Result: your cluster is registered in the network, and the AIOS cluster services (controller, metrics, gateway routes) are installed on it.

Overview

flowchart LR
    A["Prepare the cluster<br/><small>ports, Kubernetes, Ambassador, node labels</small>"] --> B["Describe the cluster<br/><small>cluster.json</small>"]
    B --> C["Register it<br/><small>Parser: createCluster</small>"]
    C --> D["Install cluster services<br/><small>Gateway: create-cluster-infra</small>"]
    D --> E["Optional: add-ons<br/><small>registries, inference server, ...</small>"]
  1. Prepare the network and the cluster.
  2. Describe your cluster in a JSON file.
  3. Register it with the network's Parser. The network's add_cluster governance policy decides whether it is accepted.
  4. Install the cluster services through the Cluster Controller Gateway.
  5. Optionally, deploy add-on services and offer them to the network.

Network prerequisites

What you must open depends on whether machines outside your own network will be allowed to join your cluster.

If all nodes are on your own internal network, you only need to expose the NodePort range so the network can reach AIOS services:

Port Protocol Purpose
30000–32767 TCP/UDP NodePort services

When a node joins with kubeadm join, it must reach your control plane over the internet. Open these ports:

Control plane (master) node

Port Protocol Component Purpose
6443 TCP kube-apiserver Main API server port. Required for kubeadm join and all cluster operations.
2379–2380 TCP etcd etcd server and peer communication. Required only if you run etcd yourself.
10250 TCP kubelet Lets the control plane talk to the kubelet on each node.
10257 TCP kube-controller-manager Internal metrics. Usually not needed externally.
10259 TCP kube-scheduler Internal metrics. Usually not needed externally.

Worker nodes

Port Protocol Component Purpose
10250 TCP kubelet Communication with the control plane.
30000–32767 TCP/UDP NodePort services Services exposed through NodePort.

Minimum for external nodes to join: 6443/TCP (API server) and 10250/TCP (kubelet) on the control plane node. Ports 2379–2380 (self-managed etcd) and 10257/10259 (metrics, debugging) are optional.

Also set config.allowExternalNodes to true in your cluster description.

Security considerations

Exposing control plane ports to the internet is high risk

If you allow external nodes, we strongly recommend that you:

  • Put control plane access behind a VPN, such as WireGuard or Tailscale. Only nodes with valid VPN credentials can then join.
  • Restrict access with firewall rules that allow only trusted IP addresses.

VPN setup and management happen outside AIOS.


Cluster prerequisites

1. Kubernetes

AIOS runs only on Kubernetes, but it does not depend on a particular Kubernetes version, as long as the cluster is API-compliant.

  • A working Kubernetes cluster, set up with kubeadm, with kubectl access.
  • Helm v3 or later on the administrator's machine.
  • Network access to the NodePort range.

2. Ambassador gateway

The Ambassador gateway is the entry point for traffic between AIOS and your cluster. It must run inside the cluster and be exposed on the fixed NodePort 32000.

# Add the Helm repository
helm repo add datawire https://app.getambassador.io
helm repo update

# Create a namespace
kubectl create namespace ambassador

# Install Ambassador, exposed on NodePort 32000
helm install ambassador datawire/ambassador \
  --namespace ambassador \
  --set service.type=NodePort \
  --set service.nodePorts.http=32000 \
  --set service.nodePorts.https=32000

Check that it is running:

kubectl get svc -n ambassador
kubectl get pods -n ambassador

Make sure that:

  • the ambassador service is of type NodePort
  • both HTTP and HTTPS are mapped to port 32000
  • all Ambassador pods are Running or Ready

3. Label each node with a nodeID

Every node needs a unique nodeID label. AIOS uses it to identify the node, and it must match the node id in your cluster description.

# List nodes
kubectl get nodes

# Label a node
kubectl label node <node-name> nodeID=<unique-id>

# Verify
kubectl get nodes --show-labels

# Change an existing label
kubectl label node <node-name> nodeID=<new-id> --overwrite

4. Publish GPU information (GPU nodes only)

If any node has GPUs, run this script on each GPU node. It adds a gpu.aios/info annotation to the node, which the onboarding script reads.

cd k8s/onboarding/cluster
sh ./gpu_info.sh

Step 1: Describe your cluster

The cluster description is a JSON file with your cluster's ID, its resources, configuration and contact information. All sizes are in MB.

A starting file is provided at k8s/onboarding/cluster/config.json. For the meaning of every field, see the Cluster spec.

Full example: a cluster with two GPU nodes
{
  "id": "cluster-west-vision-001",
  "regionId": "us-west-2",
  "status": "live",
  "nodes": {
    "count": 2,
    "nodeData": [
      {
        "id": "node-1", // must match the node's nodeID label
        "gpus": {
          "count": 2,
          "memory": 32768,
          "gpus": [
            { "modelName": "NVIDIA A100", "memory": 16384 },
            { "modelName": "NVIDIA A100", "memory": 16384 }
          ],
          "features": ["fp16", "tensor_cores"],
          "modelNames": ["NVIDIA A100"]
        },
        "vcpus": { "count": 32 },
        "memory": 131072,
        "swap": 8192,
        "storage": {
          "disks": 2,
          "size": 1048576
        },
        "network": {
          "interfaces": 2,
          "txBandwidth": 10000,
          "rxBandwidth": 10000
        }
      },
      {
        "id": "node-2", // must match the node's nodeID label
        "gpus": {
          "count": 1,
          "memory": 16384,
          "gpus": [
            { "modelName": "NVIDIA V100", "memory": 16384 }
          ],
          "features": ["fp16"],
          "modelNames": ["NVIDIA V100"]
        },
        "vcpus": { "count": 16 },
        "memory": 65536,
        "swap": 4096,
        "storage": {
          "disks": 1,
          "size": 524288
        },
        "network": {
          "interfaces": 1,
          "txBandwidth": 5000,
          "rxBandwidth": 5000
        }
      }
    ]
  },
  "gpus": {
    "count": 3,
    "memory": 49152
  },
  "vcpus": {
    "count": 48
  },
  "memory": 196608,
  "swap": 12288,
  "storage": {
    "disks": 3,
    "size": 1572864
  },
  "network": {
    "interfaces": 3,
    "txBandwidth": 15000,
    "rxBandwidth": 15000
  },
  "config": {
    "allowExternalNodes": true,
    "policyExecutorId": "policy-exec-007",
    "policyExecutionMode": "local",
    "customPolicySystem": {
      "name": "AdvancedPolicyRunner",
      "version": "2.1.0"
    },
    "publicHostname": "cluster-west-vision-001.company.net",
    "useGateway": true,
    "actionsPolicyMap": {
      "onScaleUp": "evaluate-gpu-availability",
      "onFailure": "notify-admin"
    }
  },
  "tags": ["gpu", "production", "ml", "vision", "us-west"],

  // This is just an example; you may add more fields
  "clusterMetadata": {
    "name": "Sample cluster",
    "description": "Dedicated to serving large-scale computer vision models in production.",
    "owner": "AI Infrastructure Team",
    "email": "ai-infra@company.net",
    "countries": ["USA", "Canada"],
    "miscContactInfo": {
      "pagerDuty": "https://sample-website/ai-clusters",
      "slack": "#ml-infra"
    },
    "additionalInfo": {
      "kubeadmVersion": "v28.0.1",
      "kubectlVersion": "v28.0.01",
      // kubeadm join command, used by nodes that join later
      "joinCommand": ""
    }
  },
  "reputation": 0
}

Tip

You don't need to fill in node resources by hand. The onboarding script in option A reads them from Kubernetes for you.


Step 2: Register the cluster

Registering adds your cluster to the network's cluster registry. It does not yet install anything on your cluster; that happens in step 3. Your cluster is accepted only if it passes the network's add_cluster governance policy.

The onboarding script checks your JSON file, reads every node's resources (including the GPU annotations) from the Kubernetes API, and submits the result to the network's Parser.

cd k8s/onboarding/cluster

# Preview: fills in node resources and writes <cluster_id>_populated.json, without registering
./onboard.sh config.json

# Register with the network
./onboard.sh config.json --onboard --api-url <parser-url>

The script needs python3 and a kubeconfig at ~/.kube/config.

Option B: Call the Parser API directly

If you prepared a complete JSON file yourself:

curl -X POST http://<parser-host>:<port>/api/createCluster \
  -H "Content-Type: application/json" \
  -d @cluster.json

Step 3: Install the cluster services

Now ask the network's Cluster Controller Gateway to install the AIOS cluster services on your cluster: the cluster controller, metrics and gateway routes. It needs your cluster's kubeconfig to do so.

curl -X POST "http://<cluster-controller-gateway-url>/create-cluster-infra" \
  -H "Content-Type: application/json" \
  -d '{
    "cluster_id": "cluster-west-vision-001",
    "kube_config_data": {}
}'

kube_config_data must contain the JSON form of your kubeconfig file, usually $HOME/.kube/config.

Warning

The kubeconfig contains credentials for your cluster. Send it only to a network you trust, over a secure connection.

The following script reads your kubeconfig and makes the call for you:

create_cluster_infra.py
import os
import json
import argparse
import requests

# Constants
DEFAULT_KUBE_CONFIG_PATH = os.path.join(os.path.expanduser("~"), ".kube", "config")
API_URL = "http://<server-url>/create-cluster-infra"  # Replace with actual server URL

def read_kube_config(file_path):
    try:
        with open(file_path, "r") as f:
            return f.read()
    except Exception as e:
        print(f"Error reading Kubernetes config file: {e}")
        exit(1)

def create_cluster_infra(cluster_id, kube_config_data):
    payload = {
        "cluster_id": cluster_id,
        "kube_config_data": kube_config_data
    }

    response = requests.post(API_URL, json=payload)

    if response.status_code == 200:
        print("Cluster creation scheduled successfully:", response.json())
    else:
        print("Error creating cluster:", response.json())

def main():
    parser = argparse.ArgumentParser(description="Create Cluster Infrastructure via API")
    parser.add_argument("cluster_id", help="Cluster ID")
    parser.add_argument("--config", default=DEFAULT_KUBE_CONFIG_PATH,
                        help="Path to Kubernetes config file")

    args = parser.parse_args()
    kube_config_data = read_kube_config(args.config)
    create_cluster_infra(args.cluster_id, kube_config_data)

if __name__ == "__main__":
    main()

Run it with python3 create_cluster_infra.py cluster-west-vision-001.

What gets exposed

Once the services are running, they are reachable both on NodePorts and through the Ambassador gateway (port 32000) under a path suffix.

Service NodePort Gateway path Purpose
Cluster controller 32300 /controller Cluster controller service
Cluster metrics 32301 /metrics Metrics server for the cluster
Block transactions 32302 /blocks Query block-related data
Management commands 32303 /mgmt Send management commands to blocks and the cluster
Public gateway 32000 / Public ingress

Optional services appear only if they are deployed:

Service Address Purpose
Policy executor NodePort 32380 Runs policy functions and jobs
Assets registry NodePort 32390 Stores and queries model and policy assets
Model split runner NodePort 32286 Runs model-splitting jobs
Inference server Gateway /tasks/<server-id> Submit inference tasks
Block endpoint Gateway /block/<block-id> A block's services
vDAG controller Gateway /<vdag-controller-id> A vDAG controller's service

Deploying add-on services

Add-ons are optional services that your cluster can host and offer to the whole network.

Add-on What it offers
Container registry Storage for container images. You can contribute space to the community.
Assets registry Storage for models, policy code and other files.
Inference server A gateway that accepts tasks and routes them to blocks.
Policy executor Runs policies as jobs and functions.
Model splitter Runs model-splitting jobs.

Container registry

Run an OCI-compliant image store, such as registry:2, and list it in the network's container registry index.

cd v1_deploy/container-registry

kubectl create ns registry
kubectl create -f pv.yaml
kubectl create -f pvc.yaml
kubectl create -f deployment.yaml
kubectl create -f svc.yaml

Note

If your copy of the repository doesn't include v1_deploy/container-registry, you can run any OCI-compliant registry instead, for example docker run -d -p 5000:5000 registry:2. See Container registry.

Register it:

curl -X POST http://registry-db-url/container_registry \
     -H "Content-Type: application/json" \
     -d @entry.json

See Container registry for the entry format.

Assets registry

Choose an object storage backend:

cd k8s/installer/addons
./deploy_ceph_operator.sh
./deploy_ceph_obs.sh
./create_ceph_user.sh
cd k8s/installer/addons
./install_minio.sh

Install the database and the registry service:

cd k8s/installer/addons/assets-registry
./install_db.sh 3 10Gi
./install_registry.sh

Register it with the network:

curl -X POST "http://<assets-network-registry>/asset_registry" \
     -H "Content-Type: application/json" \
     -d '{
           "asset_registry_name": "ResNet Models registry",
           "asset_registry_metadata": { "framework": "PyTorch" },
           "asset_registry_tags": ["image-classification"],
           "asset_registry_public_url": "http://192.168.0.106:31700"
         }'

See Assets registry for details.

Policy executor

  1. Write an executor document (executor.json). See Policies for the format.
  2. Register it:

    curl -X POST http://<policy-system-url>/executor \
         -H "Content-Type: application/json" \
         -d @executor.json
    
  3. Create (or remove) its infrastructure:

    # Create
    curl -X POST http://<policy-system-url>/executor/executor-001/create-infra \
      -H "Content-Type: application/json" \
      -d '{ "cluster_config": <kubeconfig JSON> }'
    
    # Remove
    curl -X DELETE http://<policy-system-url>/executor/executor-001/remove-infra \
      -H "Content-Type: application/json" \
      -d '{ "cluster_config": { "provider": "kubernetes" } }'
    

The executor is then available at:

  • External: http://<public-ip>:30900
  • Internal: http://executor-001.policies-system.svc.cluster.local:10250

Model splitter

curl -X POST http://<server-url>:8001/split-runner \
  -H "Content-Type: application/json" \
  -d '{
    "cluster_k8s_config": <kubeconfig JSON>,
    "split_runner_public_host": "192.168.0.106",
    "split_runner_metadata": {"key": "value"},
    "split_runner_tags": ["tag1", "tag2"]
  }'

If it succeeds, the splitter is exposed on NodePort 31701. See Model splitting.

Inference server

  1. (Optional) Install a database for request logs:

    cd k8s/installer/addons/inference-server
    ./install_db.sh 5Gi 3 logging username password
    
  2. Install the inference server:

    ./install_inference_server.sh server-pr-home-123 192.168.0.101 logging-primary.inference-server.svc.cluster.local 5432 username password
    
  3. Register it so that users can find it:

    curl -X POST <server-url>/inference_server \
         -H "Content-Type: application/json" \
         -d '{
               "inference_server_name": "server-pr-home-123",
               "inference_server_metadata": {
                   "region": "us-east-1",
                   "gpu_available": true
               },
               "inference_server_tags": ["NLP", "Transformer"],
               "inference_server_public_url": "http://192.168.0.101:31500"
             }'
    

It exposes NodePort 31500 (gRPC) and 31501 (REST). See Inference server.


Remove a cluster's services

To uninstall the AIOS cluster services, call remove-cluster-infra with the same kind of payload:

remove_cluster_infra.py
import os
import json
import argparse
import requests

DEFAULT_KUBE_CONFIG_PATH = os.path.join(os.path.expanduser("~"), ".kube", "config")
API_URL = "http://<server-url>/remove-cluster-infra"  # Replace with actual server URL

def read_kube_config(file_path):
    try:
        with open(file_path, "r") as f:
            return f.read()
    except Exception as e:
        print(f"Error reading Kubernetes config file: {e}")
        exit(1)

def remove_cluster_infra(cluster_id, kube_config_data):
    payload = {
        "cluster_id": cluster_id,
        "kube_config_data": kube_config_data
    }
    response = requests.post(API_URL, json=payload)
    if response.status_code == 200:
        print("Cluster removal scheduled successfully:", response.json())
    else:
        print("Error removing cluster:", response.json())

def main():
    parser = argparse.ArgumentParser(description="Remove Cluster Infrastructure via API")
    parser.add_argument("cluster_id", help="Cluster ID")
    parser.add_argument("--config", default=DEFAULT_KUBE_CONFIG_PATH,
                        help="Path to Kubernetes config file")
    args = parser.parse_args()
    kube_config_data = read_kube_config(args.config)
    remove_cluster_infra(args.cluster_id, kube_config_data)

if __name__ == "__main__":
    main()

Next steps