vDAG registry¶
The vDAG registry records every vDAG (workflow) in the network, and every vDAG controller deployed to serve them. Clients use it to find the controller for a vDAG they want to call.
At a glance
Runs on: the management cluster, as vdag-db-svc (NodePort 30103).
Written by: the system, when vDAGs are created through the Parser (vDAG spec) and when controllers are created through the Cluster Controller Gateway (vDAG creator guide).
Most common use: find a vDAG's controller.
vDAGs¶
Schema¶
@dataclass
class NodeObject:
# Label (name) of the step
nodeLabel: str = ''
# Type of the step ("block" or "vdag")
nodeType: str = ''
# URI of the nested vDAG, if nodeType is "vdag"
vdagURI: str = ''
# Policy that chooses the block for this step
assignmentPolicyRule: Dict[str, Any] = field(default_factory=dict)
# Policy run before the step's computation
preprocessingPolicyRule: Dict[str, Any] = field(default_factory=dict)
# Policy run after the step's computation
postprocessingPolicyRule: Dict[str, Any] = field(default_factory=dict)
# Model parameters for this step
modelParameters: Dict[str, Any] = field(default_factory=dict)
# Output format of this step
outputProtocol: Dict[str, Any] = field(default_factory=dict)
# Input format of this step
inputProtocol: Dict[str, Any] = field(default_factory=dict)
# Mapping of input/output fields between connected steps
IOMap: List[Dict[str, Any]] = field(default_factory=list)
# Block ID, if the step is pinned to a specific block
manualBlockId: str = field(default_factory=str)
@dataclass
class vDAGObject:
# Name of the vDAG
vdag_name: str = ''
# Version and release tag
vdag_version: Dict[str, str] = field(default_factory=lambda: {'version': '', 'release-tag': ''})
# Full URI (name:version-release)
vdagURI: str = ''
# Tags for discovery and grouping
discoveryTags: List[str] = field(default_factory=list)
# Controller configuration, including policies and inputs
controller: Dict[str, Any] = field(default_factory=dict)
# The steps
nodes: List[NodeObject] = field(default_factory=list)
# How the steps connect
graph: Dict[str, Any] = field(default_factory=dict)
# Which block was assigned to each step
assignment_info: Dict = field(default_factory=dict)
# Current status (e.g., pending, active, completed)
status: str = field(default="pending")
# Graph data compiled for use at runtime
compiled_graph_data: Dict[str, Any] = field(default_factory=dict)
# Free-form metadata
metadata: Dict[str, Any] = field(default_factory=dict)
APIs¶
| Endpoint | Method | What it does |
|---|---|---|
/vdag/:vdagURI |
GET |
Get a vDAG. |
/vdag/:vdagURI |
PUT |
Update a vDAG (MongoDB-style update). |
/vdag/:vdagURI |
DELETE |
Delete a vDAG's record. |
/vdags |
POST |
Search vDAGs with a MongoDB-style filter. |
# Get
curl -X GET http://<server-url>/vdag/sample-vdag:1.0-stable
# Update
curl -X PUT http://<server-url>/vdag/sample-vdag:1.0-stable \
-H "Content-Type: application/json" \
-d '{
"$set": {
"status": "active",
"metadata.owner": "team-ml"
}
}'
# Delete
curl -X DELETE http://<server-url>/vdag/sample-vdag:1.0-stable
# Search
curl -X POST http://<server-url>/vdags \
-H "Content-Type: application/json" \
-d '{
"status": "pending",
"metadata.owner": "team-ml"
}'
vDAG controllers¶
Every controller created to serve a vDAG is recorded here.
Schema¶
@dataclass
class vDAGController:
# Unique ID of the controller
vdag_controller_id: str = ''
# The vDAG this controller serves
vdag_uri: str = ''
# Public URL clients connect to
public_url: str = ''
# Cluster where the controller runs
cluster_id: str = ''
# Free-form metadata
metadata: Dict[str, Any] = field(default_factory=dict)
# Controller configuration
config: Dict[str, Any] = field(default_factory=dict)
# Tags for search and discovery
search_tags: List[str] = field(default_factory=list)
APIs¶
| Endpoint | Method | What it does |
|---|---|---|
/vdag-controllers/by-vdag-uri/:vdag_uri |
GET |
All controllers serving a vDAG. |
/vdag-controller/:controller_id |
GET |
Get a controller. |
/vdag-controller/:controller_id |
PUT |
Update a controller (MongoDB-style update). |
/vdag-controller/:controller_id |
DELETE |
Delete a controller's record. |
/vdag-controllers |
POST |
Search controllers with a MongoDB-style filter. |
Find the controllers for a vDAG¶
Use a returned controller's public_url as the gRPC address for sending tasks.
Other operations¶
# Get
curl -X GET http://<server-url>/vdag-controller/controller-123
# Update
curl -X PUT http://<server-url>/vdag-controller/controller-123 \
-H "Content-Type: application/json" \
-d '{
"$set": {
"metadata.owner": "team-alpha",
"public_url": "https://controller.example.com"
}
}'
# Delete
curl -X DELETE http://<server-url>/vdag-controller/controller-123
# Search
curl -X POST http://<server-url>/vdag-controllers \
-H "Content-Type: application/json" \
-d '{
"cluster_id": "cluster-west-1",
"metadata.owner": "team-alpha"
}'