Skip to content

Tasks DB

Some operations take too long to finish within one HTTP request: installing a cluster's services, adding a node, placing a block, creating a vDAG, splitting a model. AIOS runs them in the background. The global Tasks DB lets you follow their progress.

At a glance

Runs on: the management cluster, as tasks-db-svc (NodePort 30108).
You'll use it to: check whether a long-running operation succeeded, using the task ID it returned.

How it works

sequenceDiagram
    participant U as You
    participant S as AIOS service
    participant T as Tasks DB
    U->>S: Start an operation (e.g. create-cluster-infra)
    S->>T: Create task (status: pending)
    S-->>U: Task ID (immediately)
    S->>S: Do the work in the background
    S->>T: Update status (completed / failed) and output
    U->>T: GET /task/<task_id>
    T-->>U: Status and result
  1. The service creates a task entry and returns its task ID straight away.
  2. It does the work in the background.
  3. When the work finishes or fails, it records the status and output in the Tasks DB.
  4. You look up the task by ID.

Tasks DB architecture

(Download the architecture diagram)

The writer and CRUD services can be scaled with load.


Task types

task_type Operation
vdag_creation Creating a vDAG
block_alloc Placing a block
cluster_infra_creation Installing a cluster's AIOS services
cluster_infra_remove Removing a cluster's AIOS services
node_addition Adding a node to a cluster
node_removal Removing a node from a cluster
model_splits Running a model splitting job

Task schema

@dataclass
class GlobalTask:
    task_id: str = field(default_factory=lambda: str(uuid.uuid4()))
    task_data: Dict[str, Any] = field(default_factory=dict)
    task_type: str = field(default_factory=str)
    task_update_timestamp: int = field(default_factory=lambda: int(datetime.utcnow().timestamp()))
    task_create_timestamp: int = field(default_factory=lambda: int(datetime.utcnow().timestamp()))
    task_status: str = field(default='pending')
    task_status_data: Dict[str, Any] = field(default_factory=dict)
Field Type Description
task_id str Unique ID (UUID).
task_data dict The task's inputs and configuration.
task_type str The kind of operation (see Task types).
task_create_timestamp int When the task was created (UTC epoch seconds).
task_update_timestamp int When it was last updated (UTC epoch seconds).
task_status str pending (default), in_progress, completed or failed.
task_status_data dict Details about the status: results, progress or error messages.

APIs

Endpoint Method What it does Typically used by
/task/<task_id> GET Get a task. You
/tasks?task_type=...&task_status=... GET List tasks by type and, optionally, status. You
/task POST Create a task. AIOS services
/task/<task_id> PUT Update a task. AIOS services
/task/<task_id> DELETE Delete a task. Admins
/task_update POST Push a status update. Applied asynchronously, so it reaches the DB shortly afterwards. AIOS services
# Get a task
curl -X GET http://<server-url>/task/<task_id>

# List pending vDAG creation tasks
curl -X GET "http://<server-url>/tasks?task_type=vdag_creation&task_status=pending"

# Create a task (used internally)
curl -X POST http://<server-url>/task \
     -H "Content-Type: application/json" \
     -d '{
           "task_data": {"key": "value"},
           "task_type": "example_task_type",
           "task_status": "pending",
           "task_status_data": {"info": "status info"}
         }'

# Update a task (used internally)
curl -X PUT http://<server-url>/task/<task_id> \
     -H "Content-Type: application/json" \
     -d '{
           "task_data": {"updated_key": "updated_value"},
           "task_status": "in_progress"
         }'

# Delete a task
curl -X DELETE http://<server-url>/task/<task_id>

# Push a status update (used internally)
curl -X POST http://<server-url>/task_update \
     -H "Content-Type: application/json" \
     -d '{
           "task_id": "12345",
           "status": "completed",
           "task_status_data": {}
         }'

/task_update fields

Field Type Required Description
task_id string Yes The task to update.
status string Yes The new status.
task_status_data object No Extra status details or output.