Join a cluster to a network¶
A network is a group of clusters that share one set of management services and one set of rules. This page explains how to add your Kubernetes cluster to an existing network, so that its machines can run AI workloads for the network.
At a glance
Who this is for: cluster owners and administrators.
You need from the network operator: the Parser URL and the Cluster Controller Gateway URL.
Result: your cluster is registered in the network, and the AIOS cluster services (controller, metrics, gateway routes) are installed on it.
Overview¶
flowchart LR
A["Prepare the cluster<br/><small>ports, Kubernetes, Ambassador, node labels</small>"] --> B["Describe the cluster<br/><small>cluster.json</small>"]
B --> C["Register it<br/><small>Parser: createCluster</small>"]
C --> D["Install cluster services<br/><small>Gateway: create-cluster-infra</small>"]
D --> E["Optional: add-ons<br/><small>registries, inference server, ...</small>"]
- Prepare the network and the cluster.
- Describe your cluster in a JSON file.
- Register it with the network's Parser. The network's
add_clustergovernance policy decides whether it is accepted. - Install the cluster services through the Cluster Controller Gateway.
- Optionally, deploy add-on services and offer them to the network.
Network prerequisites¶
What you must open depends on whether machines outside your own network will be allowed to join your cluster.
If all nodes are on your own internal network, you only need to expose the NodePort range so the network can reach AIOS services:
| Port | Protocol | Purpose |
|---|---|---|
| 30000–32767 | TCP/UDP | NodePort services |
When a node joins with kubeadm join, it must reach your control plane over the internet. Open these ports:
Control plane (master) node
| Port | Protocol | Component | Purpose |
|---|---|---|---|
| 6443 | TCP | kube-apiserver | Main API server port. Required for kubeadm join and all cluster operations. |
| 2379–2380 | TCP | etcd | etcd server and peer communication. Required only if you run etcd yourself. |
| 10250 | TCP | kubelet | Lets the control plane talk to the kubelet on each node. |
| 10257 | TCP | kube-controller-manager | Internal metrics. Usually not needed externally. |
| 10259 | TCP | kube-scheduler | Internal metrics. Usually not needed externally. |
Worker nodes
| Port | Protocol | Component | Purpose |
|---|---|---|---|
| 10250 | TCP | kubelet | Communication with the control plane. |
| 30000–32767 | TCP/UDP | NodePort services | Services exposed through NodePort. |
Minimum for external nodes to join: 6443/TCP (API server) and 10250/TCP (kubelet) on the control plane node. Ports 2379–2380 (self-managed etcd) and 10257/10259 (metrics, debugging) are optional.
Also set config.allowExternalNodes to true in your cluster description.
Security considerations¶
Exposing control plane ports to the internet is high risk
If you allow external nodes, we strongly recommend that you:
- Put control plane access behind a VPN, such as WireGuard or Tailscale. Only nodes with valid VPN credentials can then join.
- Restrict access with firewall rules that allow only trusted IP addresses.
VPN setup and management happen outside AIOS.
Cluster prerequisites¶
1. Kubernetes¶
AIOS runs only on Kubernetes, but it does not depend on a particular Kubernetes version, as long as the cluster is API-compliant.
- A working Kubernetes cluster, set up with
kubeadm, withkubectlaccess. - Helm v3 or later on the administrator's machine.
- Network access to the NodePort range.
2. Ambassador gateway¶
The Ambassador gateway is the entry point for traffic between AIOS and your cluster. It must run inside the cluster and be exposed on the fixed NodePort 32000.
# Add the Helm repository
helm repo add datawire https://app.getambassador.io
helm repo update
# Create a namespace
kubectl create namespace ambassador
# Install Ambassador, exposed on NodePort 32000
helm install ambassador datawire/ambassador \
--namespace ambassador \
--set service.type=NodePort \
--set service.nodePorts.http=32000 \
--set service.nodePorts.https=32000
Check that it is running:
Make sure that:
- the
ambassadorservice is of typeNodePort - both HTTP and HTTPS are mapped to port
32000 - all Ambassador pods are
RunningorReady
3. Label each node with a nodeID¶
Every node needs a unique nodeID label. AIOS uses it to identify the node, and it must match the node id in your cluster description.
# List nodes
kubectl get nodes
# Label a node
kubectl label node <node-name> nodeID=<unique-id>
# Verify
kubectl get nodes --show-labels
# Change an existing label
kubectl label node <node-name> nodeID=<new-id> --overwrite
4. Publish GPU information (GPU nodes only)¶
If any node has GPUs, run this script on each GPU node. It adds a gpu.aios/info annotation to the node, which the onboarding script reads.
Step 1: Describe your cluster¶
The cluster description is a JSON file with your cluster's ID, its resources, configuration and contact information. All sizes are in MB.
A starting file is provided at k8s/onboarding/cluster/config.json. For the meaning of every field, see the Cluster spec.
Full example: a cluster with two GPU nodes
{
"id": "cluster-west-vision-001",
"regionId": "us-west-2",
"status": "live",
"nodes": {
"count": 2,
"nodeData": [
{
"id": "node-1", // must match the node's nodeID label
"gpus": {
"count": 2,
"memory": 32768,
"gpus": [
{ "modelName": "NVIDIA A100", "memory": 16384 },
{ "modelName": "NVIDIA A100", "memory": 16384 }
],
"features": ["fp16", "tensor_cores"],
"modelNames": ["NVIDIA A100"]
},
"vcpus": { "count": 32 },
"memory": 131072,
"swap": 8192,
"storage": {
"disks": 2,
"size": 1048576
},
"network": {
"interfaces": 2,
"txBandwidth": 10000,
"rxBandwidth": 10000
}
},
{
"id": "node-2", // must match the node's nodeID label
"gpus": {
"count": 1,
"memory": 16384,
"gpus": [
{ "modelName": "NVIDIA V100", "memory": 16384 }
],
"features": ["fp16"],
"modelNames": ["NVIDIA V100"]
},
"vcpus": { "count": 16 },
"memory": 65536,
"swap": 4096,
"storage": {
"disks": 1,
"size": 524288
},
"network": {
"interfaces": 1,
"txBandwidth": 5000,
"rxBandwidth": 5000
}
}
]
},
"gpus": {
"count": 3,
"memory": 49152
},
"vcpus": {
"count": 48
},
"memory": 196608,
"swap": 12288,
"storage": {
"disks": 3,
"size": 1572864
},
"network": {
"interfaces": 3,
"txBandwidth": 15000,
"rxBandwidth": 15000
},
"config": {
"allowExternalNodes": true,
"policyExecutorId": "policy-exec-007",
"policyExecutionMode": "local",
"customPolicySystem": {
"name": "AdvancedPolicyRunner",
"version": "2.1.0"
},
"publicHostname": "cluster-west-vision-001.company.net",
"useGateway": true,
"actionsPolicyMap": {
"onScaleUp": "evaluate-gpu-availability",
"onFailure": "notify-admin"
}
},
"tags": ["gpu", "production", "ml", "vision", "us-west"],
// This is just an example; you may add more fields
"clusterMetadata": {
"name": "Sample cluster",
"description": "Dedicated to serving large-scale computer vision models in production.",
"owner": "AI Infrastructure Team",
"email": "ai-infra@company.net",
"countries": ["USA", "Canada"],
"miscContactInfo": {
"pagerDuty": "https://sample-website/ai-clusters",
"slack": "#ml-infra"
},
"additionalInfo": {
"kubeadmVersion": "v28.0.1",
"kubectlVersion": "v28.0.01",
// kubeadm join command, used by nodes that join later
"joinCommand": ""
}
},
"reputation": 0
}
Tip
You don't need to fill in node resources by hand. The onboarding script in option A reads them from Kubernetes for you.
Step 2: Register the cluster¶
Registering adds your cluster to the network's cluster registry. It does not yet install anything on your cluster; that happens in step 3. Your cluster is accepted only if it passes the network's add_cluster governance policy.
Option A: Use the onboarding script (recommended)¶
The onboarding script checks your JSON file, reads every node's resources (including the GPU annotations) from the Kubernetes API, and submits the result to the network's Parser.
cd k8s/onboarding/cluster
# Preview: fills in node resources and writes <cluster_id>_populated.json, without registering
./onboard.sh config.json
# Register with the network
./onboard.sh config.json --onboard --api-url <parser-url>
The script needs python3 and a kubeconfig at ~/.kube/config.
Option B: Call the Parser API directly¶
If you prepared a complete JSON file yourself:
curl -X POST http://<parser-host>:<port>/api/createCluster \
-H "Content-Type: application/json" \
-d @cluster.json
Step 3: Install the cluster services¶
Now ask the network's Cluster Controller Gateway to install the AIOS cluster services on your cluster: the cluster controller, metrics and gateway routes. It needs your cluster's kubeconfig to do so.
curl -X POST "http://<cluster-controller-gateway-url>/create-cluster-infra" \
-H "Content-Type: application/json" \
-d '{
"cluster_id": "cluster-west-vision-001",
"kube_config_data": {}
}'
kube_config_data must contain the JSON form of your kubeconfig file, usually $HOME/.kube/config.
Warning
The kubeconfig contains credentials for your cluster. Send it only to a network you trust, over a secure connection.
The following script reads your kubeconfig and makes the call for you:
create_cluster_infra.py
import os
import json
import argparse
import requests
# Constants
DEFAULT_KUBE_CONFIG_PATH = os.path.join(os.path.expanduser("~"), ".kube", "config")
API_URL = "http://<server-url>/create-cluster-infra" # Replace with actual server URL
def read_kube_config(file_path):
try:
with open(file_path, "r") as f:
return f.read()
except Exception as e:
print(f"Error reading Kubernetes config file: {e}")
exit(1)
def create_cluster_infra(cluster_id, kube_config_data):
payload = {
"cluster_id": cluster_id,
"kube_config_data": kube_config_data
}
response = requests.post(API_URL, json=payload)
if response.status_code == 200:
print("Cluster creation scheduled successfully:", response.json())
else:
print("Error creating cluster:", response.json())
def main():
parser = argparse.ArgumentParser(description="Create Cluster Infrastructure via API")
parser.add_argument("cluster_id", help="Cluster ID")
parser.add_argument("--config", default=DEFAULT_KUBE_CONFIG_PATH,
help="Path to Kubernetes config file")
args = parser.parse_args()
kube_config_data = read_kube_config(args.config)
create_cluster_infra(args.cluster_id, kube_config_data)
if __name__ == "__main__":
main()
Run it with python3 create_cluster_infra.py cluster-west-vision-001.
What gets exposed¶
Once the services are running, they are reachable both on NodePorts and through the Ambassador gateway (port 32000) under a path suffix.
| Service | NodePort | Gateway path | Purpose |
|---|---|---|---|
| Cluster controller | 32300 |
/controller |
Cluster controller service |
| Cluster metrics | 32301 |
/metrics |
Metrics server for the cluster |
| Block transactions | 32302 |
/blocks |
Query block-related data |
| Management commands | 32303 |
/mgmt |
Send management commands to blocks and the cluster |
| Public gateway | 32000 |
/ |
Public ingress |
Optional services appear only if they are deployed:
| Service | Address | Purpose |
|---|---|---|
| Policy executor | NodePort 32380 |
Runs policy functions and jobs |
| Assets registry | NodePort 32390 |
Stores and queries model and policy assets |
| Model split runner | NodePort 32286 |
Runs model-splitting jobs |
| Inference server | Gateway /tasks/<server-id> |
Submit inference tasks |
| Block endpoint | Gateway /block/<block-id> |
A block's services |
| vDAG controller | Gateway /<vdag-controller-id> |
A vDAG controller's service |
Deploying add-on services¶
Add-ons are optional services that your cluster can host and offer to the whole network.
| Add-on | What it offers |
|---|---|
| Container registry | Storage for container images. You can contribute space to the community. |
| Assets registry | Storage for models, policy code and other files. |
| Inference server | A gateway that accepts tasks and routes them to blocks. |
| Policy executor | Runs policies as jobs and functions. |
| Model splitter | Runs model-splitting jobs. |
Container registry¶
Run an OCI-compliant image store, such as registry:2, and list it in the network's container registry index.
cd v1_deploy/container-registry
kubectl create ns registry
kubectl create -f pv.yaml
kubectl create -f pvc.yaml
kubectl create -f deployment.yaml
kubectl create -f svc.yaml
Note
If your copy of the repository doesn't include v1_deploy/container-registry, you can run any OCI-compliant registry instead, for example docker run -d -p 5000:5000 registry:2. See Container registry.
Register it:
curl -X POST http://registry-db-url/container_registry \
-H "Content-Type: application/json" \
-d @entry.json
See Container registry for the entry format.
Assets registry¶
Choose an object storage backend:
Install the database and the registry service:
Register it with the network:
curl -X POST "http://<assets-network-registry>/asset_registry" \
-H "Content-Type: application/json" \
-d '{
"asset_registry_name": "ResNet Models registry",
"asset_registry_metadata": { "framework": "PyTorch" },
"asset_registry_tags": ["image-classification"],
"asset_registry_public_url": "http://192.168.0.106:31700"
}'
See Assets registry for details.
Policy executor¶
- Write an executor document (
executor.json). See Policies for the format. -
Register it:
-
Create (or remove) its infrastructure:
# Create curl -X POST http://<policy-system-url>/executor/executor-001/create-infra \ -H "Content-Type: application/json" \ -d '{ "cluster_config": <kubeconfig JSON> }' # Remove curl -X DELETE http://<policy-system-url>/executor/executor-001/remove-infra \ -H "Content-Type: application/json" \ -d '{ "cluster_config": { "provider": "kubernetes" } }'
The executor is then available at:
- External:
http://<public-ip>:30900 - Internal:
http://executor-001.policies-system.svc.cluster.local:10250
Model splitter¶
curl -X POST http://<server-url>:8001/split-runner \
-H "Content-Type: application/json" \
-d '{
"cluster_k8s_config": <kubeconfig JSON>,
"split_runner_public_host": "192.168.0.106",
"split_runner_metadata": {"key": "value"},
"split_runner_tags": ["tag1", "tag2"]
}'
If it succeeds, the splitter is exposed on NodePort 31701. See Model splitting.
Inference server¶
-
(Optional) Install a database for request logs:
-
Install the inference server:
-
Register it so that users can find it:
curl -X POST <server-url>/inference_server \ -H "Content-Type: application/json" \ -d '{ "inference_server_name": "server-pr-home-123", "inference_server_metadata": { "region": "us-east-1", "gpu_available": true }, "inference_server_tags": ["NLP", "Transformer"], "inference_server_public_url": "http://192.168.0.101:31500" }'
It exposes NodePort 31500 (gRPC) and 31501 (REST). See Inference server.
Remove a cluster's services¶
To uninstall the AIOS cluster services, call remove-cluster-infra with the same kind of payload:
remove_cluster_infra.py
import os
import json
import argparse
import requests
DEFAULT_KUBE_CONFIG_PATH = os.path.join(os.path.expanduser("~"), ".kube", "config")
API_URL = "http://<server-url>/remove-cluster-infra" # Replace with actual server URL
def read_kube_config(file_path):
try:
with open(file_path, "r") as f:
return f.read()
except Exception as e:
print(f"Error reading Kubernetes config file: {e}")
exit(1)
def remove_cluster_infra(cluster_id, kube_config_data):
payload = {
"cluster_id": cluster_id,
"kube_config_data": kube_config_data
}
response = requests.post(API_URL, json=payload)
if response.status_code == 200:
print("Cluster removal scheduled successfully:", response.json())
else:
print("Error removing cluster:", response.json())
def main():
parser = argparse.ArgumentParser(description="Remove Cluster Infrastructure via API")
parser.add_argument("cluster_id", help="Cluster ID")
parser.add_argument("--config", default=DEFAULT_KUBE_CONFIG_PATH,
help="Path to Kubernetes config file")
args = parser.parse_args()
kube_config_data = read_kube_config(args.config)
remove_cluster_infra(args.cluster_id, kube_config_data)
if __name__ == "__main__":
main()
Next steps¶
- Set your cluster's own rules with governance and stability policies: Cluster admin guide.
- Add more machines: Join a node.
- Run something on it: Quickstart: deploy a block.