Cluster and node spec¶
These specifications describe the infrastructure that joins a network: a cluster (with its nodes, resources, configuration and contacts), or a single node joining an existing cluster. The Parser validates them and passes them to the Cluster Controller Gateway, which runs the governance (pre-check) policies before accepting them.
At a glance
Cluster: POST /api/createCluster. Step-by-step instructions are in Join a cluster.
Node: POST /api/addNode. Step-by-step instructions are in Join a node.
Units: all memory and storage sizes are in MB; bandwidth is in Mbps.
You rarely write these by hand
The onboarding scripts (k8s/onboarding/cluster/onboard.sh and k8s/onboarding/node/join-node.sh) read your hardware and fill in the resource fields for you. You mainly edit the identity, configuration and metadata fields.
Cluster specification¶
Before you register a cluster, prepare its Kubernetes infrastructure as described in Join a cluster.
Top-level fields¶
| Field | Type | Required | Description |
|---|---|---|---|
id |
string | Yes | Unique cluster ID. |
regionId |
string | Yes | Region the cluster is in. |
status |
string | No | Cluster status, for example live. |
nodes |
object | Yes | The cluster's nodes. See nodes. |
gpus |
object | Yes | Total GPU count and memory. |
vcpus |
object | Yes | Total vCPU count. |
memory |
number | Yes | Total RAM (MB). |
swap |
number | No | Total swap (MB). |
storage |
object | Yes | Total storage. |
network |
object | Yes | Network interfaces and bandwidth. |
config |
object | No | Runtime and policy configuration. See config. |
tags |
array | No | Labels, for example ml, gpu or production. Useful in search and filters. |
clusterMetadata |
object | No | Human-readable details and contacts. See clusterMetadata. |
reputation |
number | No | Reputation score (for example 0–100), set by the system. |
Nested fields¶
nodes¶
| Field | Type | Required | Description |
|---|---|---|---|
count |
number | Yes | Number of nodes. |
nodeData |
array | No | One entry per node, each with id, gpus, vcpus, memory, swap, storage and network (same format as the node spec). Each id must match that node's nodeID label. |
gpus, vcpus, storage, network¶
| Object | Field | Type | Description |
|---|---|---|---|
gpus |
count |
number | Total GPUs. |
memory |
number | Total GPU memory (MB). | |
vcpus |
count |
number | Total vCPU cores. |
storage |
disks |
number | Number of disks. |
size |
number | Total size (MB). | |
network |
interfaces |
number | Number of network interfaces. |
txBandwidth |
number | Transmit bandwidth (Mbps). | |
rxBandwidth |
number | Receive bandwidth (Mbps). |
config¶
| Field | Type | Description |
|---|---|---|
allowExternalNodes |
boolean | Whether machines outside your network may join over the internet. See Join a node. |
policyExecutorId |
string | The policy executor to use for remote policy execution. |
policyExecutionMode |
string | Where the cluster runs its policies, for example local. |
customPolicySystem |
object | Details of a custom policy runtime (name, version). |
publicHostname |
string | Public hostname used to expose the cluster's services. |
useGateway |
boolean | Whether services are exposed through the cluster gateway (Ambassador). |
actionsPolicyMap |
object | Maps actions to policies. See below. |
urlMap |
object | Service URLs. Filled in by the system. |
config.actionsPolicyMap¶
Optionally maps actions to policy rule URIs. The mapped policies run automatically when the action happens during the cluster's or a block's lifecycle.
| Action | What it is | Triggered by |
|---|---|---|
create_block |
A new block is created. | Cluster Controller Gateway |
remove_block |
A block is removed. | Cluster Controller Gateway |
parameter_update |
A block's parameters change. | Management command executor |
scale |
A block's instance count changes. | Auto-scaler, Cluster Controller Gateway |
dry_run |
A placement is simulated. | Cluster Controller Gateway |
remove_instance |
One instance is removed. | Cluster Controller Gateway |
init_create_status_update |
An init container reports status. | Cluster Controller Gateway (third-party systems) |
query_init_container_data |
Init container status is queried. | Cluster Controller Gateway (third-party systems) |
reassign-instances |
Instances move, for example after a failure. | Dynamic infrastructure scanner |
"actionsPolicyMap": {
"create_block": "policies.cluster.block-creation-policy:v1",
"scale": "policies.autoscaling.default-scaler-policy:v2",
"parameter_update": "policies.params.param-validator:v1"
}
clusterMetadata¶
| Field | Type | Description |
|---|---|---|
name |
string | Friendly name. |
description |
string | What the cluster is for. |
owner |
string | Owner or responsible team. |
email |
string | Contact email. |
countries |
array | Countries where it may be used. |
miscContactInfo |
object | Other contacts, for example Slack or PagerDuty. |
additionalInfo |
object | Free-form extra details. Nodes read kubeadmVersion and joinCommand from here. |
You can add any other fields, for example vendor. Placement filters can refer to them (clusterMetadata.vendor in the block spec example).
Example¶
{
"id": "cluster-west-vision-001",
"regionId": "us-west-2",
"status": "live",
"nodes": {
"count": 2,
"nodeData": [
{
"id": "node-1",
"gpus": {
"count": 2,
"memory": 32768,
"gpus": [
{ "modelName": "NVIDIA A100", "memory": 16384 },
{ "modelName": "NVIDIA A100", "memory": 16384 }
],
"features": ["fp16", "tensor_cores"],
"modelNames": ["NVIDIA A100"]
},
"vcpus": { "count": 32 },
"memory": 131072,
"swap": 8192,
"storage": {
"disks": 2,
"size": 1048576
},
"network": {
"interfaces": 2,
"txBandwidth": 10000,
"rxBandwidth": 10000
}
},
{
"id": "node-2",
"gpus": {
"count": 1,
"memory": 16384,
"gpus": [
{ "modelName": "NVIDIA V100", "memory": 16384 }
],
"features": ["fp16"],
"modelNames": ["NVIDIA V100"]
},
"vcpus": { "count": 16 },
"memory": 65536,
"swap": 4096,
"storage": {
"disks": 1,
"size": 524288
},
"network": {
"interfaces": 1,
"txBandwidth": 5000,
"rxBandwidth": 5000
}
}
]
},
"gpus": {
"count": 3,
"memory": 49152
},
"vcpus": {
"count": 48
},
"memory": 196608,
"swap": 12288,
"storage": {
"disks": 3,
"size": 1572864
},
"network": {
"interfaces": 3,
"txBandwidth": 15000,
"rxBandwidth": 15000
},
"config": {
"policyExecutorId": "policy-exec-007",
"policyExecutionMode": "local",
"customPolicySystem": {
"name": "AdvancedPolicyRunner",
"version": "2.1.0"
},
"publicHostname": "cluster-west-vision-001.company.net",
"useGateway": true,
"actionsPolicyMap": {
"onScaleUp": "evaluate-gpu-availability",
"onFailure": "notify-admin"
}
},
"tags": ["gpu", "production", "ml", "vision", "us-west"],
"clusterMetadata": {
"name": "Sample cluster",
"vendor": "dma-bangalore",
"description": "Dedicated to serving large-scale computer vision models in production.",
"owner": "AI Infrastructure Team",
"email": "ai-infra@company.net",
"countries": ["USA", "Canada"],
"miscContactInfo": {
"pagerDuty": "https://sample-website/ai-clusters",
"slack": "#ml-infra"
},
"additionalInfo": {}
},
"reputation": 94
}
Register a cluster¶
curl -X POST http://<parser-url>/api/createCluster \
-H "Content-Type: application/json" \
-d @cluster.json
The request must use the standard Parser format, with header and body.spec.values. See How the Parser works.
Pre-check policies¶
Adding clusters and nodes is governed by pre-check policies: Python rules that run before the action and can allow it, reject it or modify it. Both the network and each cluster can set them.
A pre-check policy returns:
# Allowed (input_data may be modified)
{
"allowed": True,
"input_data": input_data
}
# Not allowed
{
"allowed": False,
"input_data": <message or dict explaining why the action was not allowed>
}
class AIOSv1PolicyRule:
def __init__(self, rule_id, settings, parameters):
"""
Initializes an AIOSv1PolicyRule instance.
Args:
rule_id (str): Unique identifier for the rule.
settings (dict): Configuration settings for the rule.
parameters (dict): Parameters defining the rule's behavior.
"""
self.rule_id = rule_id
self.settings = settings
self.parameters = parameters
def eval(self, parameters, input_data, context):
"""
Evaluates the policy rule.
Args:
parameters (dict): The current parameters.
input_data (any): The input data to be evaluated.
context (dict): Context (external cache), which can be used to
store and access state across multiple runs.
"""
# The input_data dict can be modified by the policy.
# Make input_data modifications here.
return {
"allowed": True,
"input_data": input_data
}
What each action's policy receives is listed in Cluster Controller Gateway: pre-check policy inputs. How to attach policies is described in the Network admin guide and the Cluster admin guide.
Node onboarding specification¶
A node joins an existing cluster by submitting its hardware and system details to the Parser (addNode).
Fields¶
| Field | Type | Required | Description |
|---|---|---|---|
id |
string | Yes | Unique node ID. |
clusterId |
string | Yes | The cluster to join. |
gpus |
object | Yes | GPU details (see below). |
vcpus |
object | Yes | {"count": <logical CPU cores>} |
memory |
number | Yes | Total RAM (MB). |
swap |
number | No | Total swap (MB). |
storage |
object | Yes | {"disks": <count>, "size": <MB>} |
network |
object | Yes | {"interfaces": <count>, "txBandwidth": <Mbps>, "rxBandwidth": <Mbps>}. The bandwidth fields are optional and default to 0. |
tags |
array | No | Labels, for example ["gpu"]. |
nodeMetadata |
object | No | Free-form details, for example vendor or rack location. |
gpus
| Field | Type | Required | Description |
|---|---|---|---|
count |
number | Yes | Number of GPUs. |
memory |
number | Yes | Total GPU memory (MB). |
gpus |
array | Yes | One entry per GPU: {"modelName": "NVIDIA A100", "memory": 16384} |
modelNames |
array | No | Distinct GPU model names. |
features |
array | No | GPU features, for example tensor_cores. |
Example¶
{
"header": {
"templateUri": "Parser/V1",
"parameters": {}
},
"body": {
"spec": {
"values": {
"clusterId": "cluster-west-vision-001",
"id": "node-1",
"gpus": {
"count": 2,
"memory": 32768,
"gpus": [
{ "modelName": "NVIDIA A100", "memory": 16384 },
{ "modelName": "NVIDIA A100", "memory": 16384 }
],
"modelNames": ["NVIDIA A100"],
"features": ["tensor_cores"]
},
"vcpus": { "count": 32 },
"memory": 131072,
"swap": 8192,
"storage": { "disks": 2, "size": 1048576 },
"network": { "interfaces": 2, "txBandwidth": 0, "rxBandwidth": 0 },
"tags": ["gpu", "fp16", "production"],
"nodeMetadata": {
"vendor": "Supermicro",
"location": "Rack 2 - DC1",
"notes": "Installed 2024-12"
}
}
}
}
}