Skip to content

vDAG registry

The vDAG registry records every vDAG (workflow) in the network, and every vDAG controller deployed to serve them. Clients use it to find the controller for a vDAG they want to call.

At a glance

Runs on: the management cluster, as vdag-db-svc (NodePort 30103).
Written by: the system, when vDAGs are created through the Parser (vDAG spec) and when controllers are created through the Cluster Controller Gateway (vDAG creator guide).
Most common use: find a vDAG's controller.


vDAGs

Schema

@dataclass
class NodeObject:
    # Label (name) of the step
    nodeLabel: str = ''
    # Type of the step ("block" or "vdag")
    nodeType: str = ''
    # URI of the nested vDAG, if nodeType is "vdag"
    vdagURI: str = ''
    # Policy that chooses the block for this step
    assignmentPolicyRule: Dict[str, Any] = field(default_factory=dict)
    # Policy run before the step's computation
    preprocessingPolicyRule: Dict[str, Any] = field(default_factory=dict)
    # Policy run after the step's computation
    postprocessingPolicyRule: Dict[str, Any] = field(default_factory=dict)
    # Model parameters for this step
    modelParameters: Dict[str, Any] = field(default_factory=dict)
    # Output format of this step
    outputProtocol: Dict[str, Any] = field(default_factory=dict)
    # Input format of this step
    inputProtocol: Dict[str, Any] = field(default_factory=dict)
    # Mapping of input/output fields between connected steps
    IOMap: List[Dict[str, Any]] = field(default_factory=list)
    # Block ID, if the step is pinned to a specific block
    manualBlockId: str = field(default_factory=str)


@dataclass
class vDAGObject:
    # Name of the vDAG
    vdag_name: str = ''
    # Version and release tag
    vdag_version: Dict[str, str] = field(default_factory=lambda: {'version': '', 'release-tag': ''})
    # Full URI (name:version-release)
    vdagURI: str = ''
    # Tags for discovery and grouping
    discoveryTags: List[str] = field(default_factory=list)
    # Controller configuration, including policies and inputs
    controller: Dict[str, Any] = field(default_factory=dict)
    # The steps
    nodes: List[NodeObject] = field(default_factory=list)
    # How the steps connect
    graph: Dict[str, Any] = field(default_factory=dict)
    # Which block was assigned to each step
    assignment_info: Dict = field(default_factory=dict)
    # Current status (e.g., pending, active, completed)
    status: str = field(default="pending")
    # Graph data compiled for use at runtime
    compiled_graph_data: Dict[str, Any] = field(default_factory=dict)
    # Free-form metadata
    metadata: Dict[str, Any] = field(default_factory=dict)

APIs

Endpoint Method What it does
/vdag/:vdagURI GET Get a vDAG.
/vdag/:vdagURI PUT Update a vDAG (MongoDB-style update).
/vdag/:vdagURI DELETE Delete a vDAG's record.
/vdags POST Search vDAGs with a MongoDB-style filter.
# Get
curl -X GET http://<server-url>/vdag/sample-vdag:1.0-stable

# Update
curl -X PUT http://<server-url>/vdag/sample-vdag:1.0-stable \
     -H "Content-Type: application/json" \
     -d '{
           "$set": {
             "status": "active",
             "metadata.owner": "team-ml"
           }
         }'

# Delete
curl -X DELETE http://<server-url>/vdag/sample-vdag:1.0-stable

# Search
curl -X POST http://<server-url>/vdags \
     -H "Content-Type: application/json" \
     -d '{
           "status": "pending",
           "metadata.owner": "team-ml"
         }'

vDAG controllers

Every controller created to serve a vDAG is recorded here.

Schema

@dataclass
class vDAGController:
    # Unique ID of the controller
    vdag_controller_id: str = ''
    # The vDAG this controller serves
    vdag_uri: str = ''
    # Public URL clients connect to
    public_url: str = ''
    # Cluster where the controller runs
    cluster_id: str = ''
    # Free-form metadata
    metadata: Dict[str, Any] = field(default_factory=dict)
    # Controller configuration
    config: Dict[str, Any] = field(default_factory=dict)
    # Tags for search and discovery
    search_tags: List[str] = field(default_factory=list)

APIs

Endpoint Method What it does
/vdag-controllers/by-vdag-uri/:vdag_uri GET All controllers serving a vDAG.
/vdag-controller/:controller_id GET Get a controller.
/vdag-controller/:controller_id PUT Update a controller (MongoDB-style update).
/vdag-controller/:controller_id DELETE Delete a controller's record.
/vdag-controllers POST Search controllers with a MongoDB-style filter.

Find the controllers for a vDAG

curl -X GET http://<server-url>/vdag-controllers/by-vdag-uri/sample-vdag:1.0-stable

Use a returned controller's public_url as the gRPC address for sending tasks.

Other operations

# Get
curl -X GET http://<server-url>/vdag-controller/controller-123

# Update
curl -X PUT http://<server-url>/vdag-controller/controller-123 \
     -H "Content-Type: application/json" \
     -d '{
           "$set": {
             "metadata.owner": "team-alpha",
             "public_url": "https://controller.example.com"
           }
         }'

# Delete
curl -X DELETE http://<server-url>/vdag-controller/controller-123

# Search
curl -X POST http://<server-url>/vdag-controllers \
     -H "Content-Type: application/json" \
     -d '{
           "cluster_id": "cluster-west-1",
           "metadata.owner": "team-alpha"
         }'