Assets registry
The assets registry stores the files that blocks and policies need, such as model weights, policy code archives, images and videos, together with metadata, versions and tags so they can be found and reused.
At a glance
Optional: any public storage with a downloadable URL works just as well. The assets registry adds versioning, search and a network-wide index.
Deploy your own: see Join a cluster: assets registry .
Two services
Service
What it does
Who runs it
Assets DB
Stores asset files and their metadata.
Anyone. Each one is backed by S3-compatible storage (Amazon S3, Ceph Object Storage, MinIO, ...) plus a database for metadata, which can be the network's global DB if you want assets to be searchable network-wide. It needs a public URL.
Assets DB registry
A network-wide index of assets DBs , with search and health checks. Register your assets DB here to make it discoverable.
The network (assets-db-registry-svc, NodePort 30104).
flowchart LR
U["You"] -->|"upload / register"| A1[("Assets DB A<br/><small>S3 / Ceph / MinIO</small>")]
U --> A2[("Assets DB B")]
A1 -->|"listed in"| R["Assets DB registry<br/><small>network-wide index</small>"]
A2 --> R
B["Blocks and policies"] -->|"download by URL"| A1
Assets DB
Asset schema
@dataclass
class AssetObject :
asset_uri : str = ''
asset_name : str = ''
asset_id : str = ''
assets_db_id : str = ''
asset_version : Dict [ str , str ] = field (
default_factory = lambda : { "version" : "" , "tag" : "" })
asset_metadata : Dict [ str , str ] = field ( default_factory = dict )
asset_public_url : str = ''
asset_file_info : Dict [ str , str ] = field ( default_factory = dict )
asset_tags : List [ str ] = field ( default_factory = list )
Field
Type
Description
asset_uri
str
Unique ID, generated as asset_name:version-tag, for example SampleAsset:1.0-initial. Empty if any part is missing.
asset_name
str
The asset's name.
asset_id
str
Internal storage ID, assigned when you upload the file to the assets DB (rather than giving your own public URL).
assets_db_id
str
Which assets DB holds the asset. Defaults to the DB the asset was uploaded to. Useful when several DBs share a global metadata store.
asset_version
dict
version and tag. Lets several versions of an asset share a name.
asset_metadata
dict
Free-form details: description, author, dates and so on.
asset_public_url
str
The asset's download URL . For uploaded files: http://<asset-db-url>/asset-object/<asset-id>.
asset_file_info
dict
File details, such as size, format or checksum.
asset_tags
list
Tags, for search.
APIs
Endpoint
Method
What it does
/upload_asset
POST
Upload a file with its metadata (multipart form).
/asset
POST
Register an asset that is already hosted elsewhere (JSON, with its asset_public_url).
/asset/<asset_uri>
GET
Get an asset's record.
/asset/<asset_uri>
PUT
Update an asset's record.
/asset/<asset_uri>
DELETE
Delete an asset.
/assets
POST
Search assets with a MongoDB-style filter.
/asset-object/<asset_id>
GET
Download an uploaded file.
/health
GET
Check that the service is running.
Upload a file Register an external file Get, update, delete Search Download, health
curl -X POST http://<server-url>/upload_asset \
-H "Content-Type: multipart/form-data" \
-F "asset=@/path/to/your/file.png" \
-F 'asset_metadata={
"asset_name": "SampleAsset",
"asset_version": { "version": "1.0", "tag": "initial" },
"asset_metadata": { "description": "Sample asset upload" },
"asset_tags": ["image", "sample"]
}'
curl -X POST http://<server-url>/asset \
-H "Content-Type: application/json" \
-d '{
"asset_name": "Sample Asset",
"asset_version": {"version": "1.0", "tag": "initial"},
"asset_metadata": {"description": "Sample asset for testing"},
"asset_public_url": "https://example.com/files/sample-asset.bin",
"asset_tags": ["test", "sample"]
}'
curl -X GET http://<server-url>/asset/SampleAsset:1.0-initial
curl -X PUT http://<server-url>/asset/SampleAsset:1.0-initial \
-H "Content-Type: application/json" \
-d '{
"asset_metadata": {"description": "Updated asset description"},
"asset_tags": ["updated", "sample"]
}'
curl -X DELETE http://<server-url>/asset/SampleAsset:1.0-initial
# Simple match
curl -X POST http://<server-url>/assets \
-H "Content-Type: application/json" \
-d '{
"asset_tags": ["sample"],
"asset_metadata": {"type": "image"}
}'
# MongoDB operators
curl -X POST http://<server-url>/assets \
-H "Content-Type: application/json" \
-d '{
"asset_tags": { "$in": ["sample", "image"] },
"asset_metadata.type": "image",
"asset_version.version": { "$gte": "1.0" }
}'
curl -X GET http://<server-url>/asset-object/<asset_id> -o downloaded_asset
curl -X GET http://<server-url>/health
Assets DB registry
Schema
@dataclass
class AssetsDBRegistry :
asset_registry_id : str = field ( default_factory = lambda : str ( uuid . uuid4 ()))
asset_registry_name : str = ''
asset_registry_metadata : Dict [ str , str ] = field ( default_factory = dict )
asset_registry_tags : List [ str ] = field ( default_factory = list )
asset_registry_public_url : str = ''
Field
Type
Description
asset_registry_id
str
Unique ID (UUID), generated automatically.
asset_registry_name
str
Human-readable name.
asset_registry_metadata
dict
Free-form details: version, owner, framework and so on.
asset_registry_tags
list
Tags, for search and grouping.
asset_registry_public_url
str
The assets DB's public URL.
APIs
Endpoint
Method
What it does
/asset_registry
POST
Register an assets DB.
/asset_registry/<asset_db_id>
GET
Get an entry.
/asset_registry/<asset_db_id>
PUT
Update an entry.
/asset_registry/<asset_db_id>
DELETE
Delete an entry.
/asset_registries
POST
Search entries with a MongoDB-style filter.
/check_health/<asset_db_id>
GET
Check that a registered assets DB is reachable and healthy.
# Register
curl -X POST "http://<server-url>/asset_registry" \
-H "Content-Type: application/json" \
-d '{
"asset_registry_name": "ResNet Models registry",
"asset_registry_metadata": {
"framework": "PyTorch",
"version": "1.0",
"author": "Example Org",
"license": "Apache-2.0"
},
"asset_registry_tags": ["image-classification", "deep-learning", "pytorch"],
"asset_registry_public_url": "https://example.com/models/resnet-models-registry"
}'
# Get
curl -X GET "http://<server-url>/asset_registry/123e4567-e89b-12d3-a456-426614174000"
# Update
curl -X PUT "http://<server-url>/asset_registry/123e4567-e89b-12d3-a456-426614174000" \
-H "Content-Type: application/json" \
-d '{
"asset_registry_metadata": {
"framework": "TensorFlow",
"version": "2.0"
},
"asset_registry_tags": ["neural-network", "tensorflow"]
}'
# Delete
curl -X DELETE "http://<server-url>/asset_registry/123e4567-e89b-12d3-a456-426614174000"
# Search
curl -X POST "http://<server-url>/asset_registries" \
-H "Content-Type: application/json" \
-d '{
"$and": [
{ "asset_registry_tags": { "$in": ["deep-learning"] } },
{ "asset_registry_metadata.framework": "PyTorch" }
]
}'
# Health check
curl -X GET "http://<server-url>/check_health/123e4567-e89b-12d3-a456-426614174000"