Create a network¶
A network starts with a management cluster: a dedicated Kubernetes cluster that hosts the services which create, schedule and govern everything in the network, plus the registries that record blocks, clusters, vDAGs, metrics, policies and more.
This page walks you through installing that management cluster.
At a glance
Who this is for: whoever will operate a new network (the network admin).
Time: about 30–60 minutes on a prepared Kubernetes cluster.
Result: a running Parser and Cluster Controller Gateway, the URLs other people use to join and deploy.
What gets installed¶
You install five groups of services, in this order. Each group depends on the ones before it.
| Step | Group | Kubernetes namespace | Contents |
|---|---|---|---|
| 1 | Registry database | registries |
The database that backs all registries. |
| 2 | Registry services | registries |
APIs for blocks, clusters, vDAGs, policies, components, assets, containers and more. |
| 3 | Utility services | utils |
Search, resource allocation, vDAG processing and failure notification. |
| 4 | Metrics system | metrics-system |
The global metrics database and services. |
| 5 | Core services | services |
The Parser and the Cluster Controller Gateway. |
You can install everything with one script or step by step.
Prerequisites¶
Cluster¶
- A working Kubernetes cluster, installed with
kubeadmor another tool. - One master node. A multi-master setup is optional, for high availability.
- At least 3 worker nodes, each with at least 40 GB disk and 8 GB RAM.
- A CNI plugin for pod networking, such as Weave, Flannel, Calico or Cilium.
-
kubectlconfigured to access the cluster.
Network ports¶
Open these ports.
Control plane (master) node
| Port | Protocol | Component | Purpose |
|---|---|---|---|
| 6443 | TCP | kube-apiserver | Main API server port. Required for kubeadm join and all cluster operations. |
| 2379–2380 | TCP | etcd | etcd server and peer communication. Required only if you run etcd yourself. |
| 10250 | TCP | kubelet | Lets the control plane talk to the kubelet on each node. |
| 10257 | TCP | kube-controller-manager | Internal metrics. Usually not needed externally. |
| 10259 | TCP | kube-scheduler | Internal metrics. Usually not needed externally. |
Worker nodes
| Port | Protocol | Component | Purpose |
|---|---|---|---|
| 10250 | TCP | kubelet | Communication with the control plane. |
| 30000–32767 | TCP/UDP | NodePort services | All AIOS services are exposed on ports in this range. |
Label your nodes¶
Choose which nodes will store the registry database and the metrics database, and label them. Use several nodes for each, to match the number of replicas you want.
# Nodes that will hold the registry database
kubectl label node <registry-node-name> registry=yes --overwrite
# Nodes that will hold the metrics database
kubectl label node <metrics-node-name> metrics-db=yes --overwrite
Quick install with one script¶
The script k8s/install_mgmt_services.sh runs all five steps for you. It asks for confirmation before each step, so you can skip steps that are already done.
-
(Optional) Review the deployment files in
k8s/installer/and change anything you need. -
Set the database size and replication at the top of
k8s/install_mgmt_services.sh: -
Run the script from the repository root:
Answer
yesto each step you want to run.
When it finishes, go to Verify the installation.
Step-by-step installation¶
Use this path to understand what is installed, or to install only part of the system. Run all commands from the k8s/ directory.
Step 1: Install the registry database¶
For a minimal setup, a single database system backs all the registries.
# Usage: ./installer/db/install.sh <replicas> <size-of-each-replica>
# Example: 3 replicas of 10 GB each
./installer/db/install.sh 3 10Gi
When it succeeds, you will see these pods in the registries namespace:
NAME READY STATUS RESTARTS AGE
registry-0-647977bfd7-78v2f 1/1 Running 0 47s
registry-1-f7cb968f5-2bn4x 1/1 Running 0 41s
registry-2-5766f9b467-p896c 1/1 Running 0 34s
registry-init-client 0/1 Completed 0 29s
Inside the cluster, the database is reachable at registry-0.registry.svc.cluster.local.
Step 2: Install the registry services¶
This creates the following services:
| Service | Internal URL | External URL | Stores |
|---|---|---|---|
blocks-db-svc |
http://blocks-db-svc.registries.svc.cluster.local:3001 |
http://<NODE_PUBLIC_IP>:30100 |
Blocks running in the network |
clusters-db-service |
http://clusters-db-service.registries.svc.cluster.local:3000 |
http://<NODE_PUBLIC_IP>:30101 |
Clusters in the network |
policies-system |
http://policies-system.registries.svc.cluster.local:10000 |
http://<NODE_PUBLIC_IP>:30102 |
Policy definitions |
vdag-db-svc |
http://vdag-db-svc.registries.svc.cluster.local:10501 |
http://<NODE_PUBLIC_IP>:30103 |
vDAGs in the network |
assets-db-registry-svc |
http://assets-db-registry-svc.registries.svc.cluster.local:4000 |
http://<NODE_PUBLIC_IP>:30104 |
Available asset stores |
adhoc-server-registry-svc |
http://adhoc-serviers-registry-svc.registries.svc.cluster.local:6000 |
http://<NODE_PUBLIC_IP>:30105 |
Inference servers in the network |
templates-registry-svc |
http://templates-registry-svc.registries.svc.cluster.local:9000 |
http://<NODE_PUBLIC_IP>:30106 |
Templates for custom parsers |
spec-registry-svc |
http://spec-registry-svc.registries.svc.cluster.local:8000 |
http://<NODE_PUBLIC_IP>:30107 |
Reusable specifications |
tasks-db-svc |
http://tasks-db-svc.registries.svc.cluster.local:8000 |
http://<NODE_PUBLIC_IP>:30108 |
Asynchronous tasks and their status |
containers-registry-svc |
http://containers-registry-svc.registries.svc.cluster.local:8000 |
http://<NODE_PUBLIC_IP>:30109 |
Available container registries |
layers-registry-svc |
http://layers-registry-svc.registries.svc.cluster.local:8002 |
http://<NODE_PUBLIC_IP>:30110 |
Reusable LLM model layers |
splits-runners-registry-svc |
http://splits-runners-registry-svc.registries.svc.cluster.local:8001 |
http://<NODE_PUBLIC_IP>:30111 |
Available model-splitter servers |
components-registry-svc |
http://components-registry-svc.registries.svc.cluster.local:4000 |
http://<NODE_PUBLIC_IP>:30112 |
All registered components |
Step 3: Install the utility services¶
Utility services support the core services.
| Service | Internal URL | External URL | Purpose |
|---|---|---|---|
search-server |
http://search-server.utils.svc.cluster.local:12000 |
http://<NODE_PUBLIC_IP>:30150 |
Filtering and similarity search |
resource-allocator |
http://resource-allocator.utils.svc.cluster.local:8765 |
http://<NODE_PUBLIC_IP>:30151 |
Resource allocation |
resource-allocator (internal) |
http://resource-allocator.utils.svc.cluster.local:7777 |
Not exposed | Internal resource management interface |
vdag-processor |
http://vdag-processor.utils.svc.cluster.local:10500 |
Not exposed | Parses vDAG structures and fills the registries |
failure-notifier |
http://failure-notifier.utils.svc.cluster.local:5000 |
Not exposed | Notifies external services about failures, using policies |
Step 4: Install the metrics system¶
First install the metrics database. It is placed on the nodes you labelled metrics-db=yes.
# Usage: ./installer/metrics/install_db.sh <replicas> <size-of-each-replica>
./installer/metrics/install_db.sh 3 10Gi
When it succeeds, you will see:
NAME READY STATUS RESTARTS AGE
metrics-db-0-896c86859-t4d5b 1/1 Running 0 99s
metrics-db-1-7df48db4d7-8c4hb 1/1 Running 0 92s
metrics-db-2-77c6fd8b87-9kp5f 1/1 Running 0 85s
metrics-db-init-client 1/1 Running 0 80s
Then install the metrics services:
| Service | Internal URL | External URL | Purpose |
|---|---|---|---|
metrics-redis |
http://metrics-redis.metrics-system.svc.cluster.local:6379 |
http://<NODE_PUBLIC_IP>:30200 |
Clusters push all metrics through this Redis instance |
global-block-metrics |
http://global-block-metrics.metrics-system.svc.cluster.local:8889 |
http://<NODE_PUBLIC_IP>:30201 |
Stores block metrics and offers query APIs |
global-cluster-metrics |
http://global-cluster-metrics.metrics-system.svc.cluster.local:8888 |
http://<NODE_PUBLIC_IP>:30202 |
Stores cluster metrics and offers query APIs |
global-vdag-metrics |
http://global-vdag-metrics.metrics-system.svc.cluster.local:8890 |
http://<NODE_PUBLIC_IP>:30203 |
Stores vDAG metrics and offers query APIs |
Step 5: Install the core services¶
With the database, registries, utilities and metrics in place, install the Parser and the Cluster Controller Gateway:
| Service | Internal URL | External URL | Purpose |
|---|---|---|---|
parser |
http://parser.services.svc.cluster.local:8000 |
http://<NODE_PUBLIC_IP>:30500 |
Accepts specifications to create components, blocks, clusters and vDAGs |
cluster-controller-gateway |
http://cluster-controller-gateway.services.svc.cluster.local:5000 |
http://<NODE_PUBLIC_IP>:30600 |
Manages clusters and scheduling across the network |
Verify the installation¶
Check that all pods are running:
kubectl get pods -n registries
kubectl get pods -n utils
kubectl get pods -n metrics-system
kubectl get pods -n services
Your network's two most important addresses are:
| You'll share this | Address | Used for |
|---|---|---|
| Parser URL | http://<NODE_PUBLIC_IP>:30500 |
Joining clusters and nodes, and creating components, blocks and vDAGs |
| Cluster Controller Gateway URL | http://<NODE_PUBLIC_IP>:30600 |
Cluster infrastructure, vDAG controllers and management |
All services and ports¶
Full list of management cluster services (click to expand)
| Service | Internal URL | External URL | Purpose |
|---|---|---|---|
blocks-db-svc |
http://blocks-db-svc.registries.svc.cluster.local:3001 |
http://<NODE_PUBLIC_IP>:30100 |
Blocks running in the network |
clusters-db-service |
http://clusters-db-service.registries.svc.cluster.local:3000 |
http://<NODE_PUBLIC_IP>:30101 |
Clusters in the network |
policies-system |
http://policies-system.registries.svc.cluster.local:10000 |
http://<NODE_PUBLIC_IP>:30102 |
Policy definitions |
vdag-db-svc |
http://vdag-db-svc.registries.svc.cluster.local:10501 |
http://<NODE_PUBLIC_IP>:30103 |
vDAGs in the network |
assets-db-registry-svc |
http://assets-db-registry-svc.registries.svc.cluster.local:4000 |
http://<NODE_PUBLIC_IP>:30104 |
Available asset stores |
adhoc-server-registry-svc |
http://adhoc-serviers-registry-svc.registries.svc.cluster.local:6000 |
http://<NODE_PUBLIC_IP>:30105 |
Inference servers |
templates-registry-svc |
http://templates-registry-svc.registries.svc.cluster.local:9000 |
http://<NODE_PUBLIC_IP>:30106 |
Custom parser templates |
spec-registry-svc |
http://spec-registry-svc.registries.svc.cluster.local:8000 |
http://<NODE_PUBLIC_IP>:30107 |
Reusable specifications |
tasks-db-svc |
http://tasks-db-svc.registries.svc.cluster.local:8000 |
http://<NODE_PUBLIC_IP>:30108 |
Asynchronous tasks |
containers-registry-svc |
http://containers-registry-svc.registries.svc.cluster.local:8000 |
http://<NODE_PUBLIC_IP>:30109 |
Container registries |
layers-registry-svc |
http://layers-registry-svc.registries.svc.cluster.local:8002 |
http://<NODE_PUBLIC_IP>:30110 |
Reusable LLM model layers |
splits-runners-registry-svc |
http://splits-runners-registry-svc.registries.svc.cluster.local:8001 |
http://<NODE_PUBLIC_IP>:30111 |
Model-splitter servers |
components-registry-svc |
http://components-registry-svc.registries.svc.cluster.local:4000 |
http://<NODE_PUBLIC_IP>:30112 |
Registered components |
search-server |
http://search-server.utils.svc.cluster.local:12000 |
http://<NODE_PUBLIC_IP>:30150 |
Filtering and similarity search |
resource-allocator |
http://resource-allocator.utils.svc.cluster.local:8765 |
http://<NODE_PUBLIC_IP>:30151 |
Resource allocation |
resource-allocator (internal) |
http://resource-allocator.utils.svc.cluster.local:7777 |
Not exposed | Internal resource management |
vdag-processor |
http://vdag-processor.utils.svc.cluster.local:10500 |
Not exposed | vDAG processing |
failure-notifier |
http://failure-notifier.utils.svc.cluster.local:5000 |
Not exposed | Failure notifications |
metrics-redis |
http://metrics-redis.metrics-system.svc.cluster.local:6379 |
http://<NODE_PUBLIC_IP>:30200 |
Metrics ingestion |
global-block-metrics |
http://global-block-metrics.metrics-system.svc.cluster.local:8889 |
http://<NODE_PUBLIC_IP>:30201 |
Block metrics |
global-cluster-metrics |
http://global-cluster-metrics.metrics-system.svc.cluster.local:8888 |
http://<NODE_PUBLIC_IP>:30202 |
Cluster metrics |
global-vdag-metrics |
http://global-vdag-metrics.metrics-system.svc.cluster.local:8890 |
http://<NODE_PUBLIC_IP>:30203 |
vDAG metrics |
parser |
http://parser.services.svc.cluster.local:8000 |
http://<NODE_PUBLIC_IP>:30500 |
Parser |
cluster-controller-gateway |
http://cluster-controller-gateway.services.svc.cluster.local:5000 |
http://<NODE_PUBLIC_IP>:30600 |
Cluster Controller Gateway |
Next steps¶
- Set the network's rules: write governance policies. See the Network admin guide.
- Add compute: Join a cluster to your new network.