Prototypes

Prototypes are Docker Images that are pushed into the DTaaS Registry and then registered with DTaaS. Once registered, the Prototype becomes available as an object that can be used to create Servables.

Manifest

Prototypes are declared as part of a Manifest file read by the SDK to upload the image and register it. An example looks like:

{
    "example-pipeline": {
        "name": "example-pipeline",
        "tag": "0.0.1",
        "type": "pipeline",
        "dependencies": {
            "csv-dataset": "88799fd6f9cc47e6bfe94c078b8bfca7"
        },
        "arguments": {},
        "capabilities": {
            "csv-dataset": [
                "CREATE",
                "READ"
            ]
        },
        "config-capabilities": {
            "bucket_id": [
                "READ"
            ]
        }
    }
}

The name and tag refer to the Docker Image that this Prototype represents, and will also be able to be used to refer to this Prototype in Prototype APIs. Any Prototype API that takes an id can also take name:tag. The type defines the kind of Servable that this Prototype will create. Any dependencies specified define what other Prototypes this Prototype needs to function. For example, if example-pipeline wants to create a Dataset using the csv-dataset Prototype, then that Prototype needs to exist. The keys for dependencies are arbitrary and can have semantic meaning to the Prototype itself, for example, to be referred to in code to lookup the id of the Prototype it references. The values are Prototype ids of already existing prototypes, or the name of a prototype if multiple Prototypes are being declared in the same Manifest that are dependent. The arguments field is deprecated and can be ignored. If any capabilities are required on other Prototypes, they can be defined as shown. It is assumed that if Capabilities are required, then there is a dependency as well.

If any Capabilities are required on Servables or other objects passed into this Prototype when it is running as a Servable, the config-capabilities structure can be used to map required Capabilities onto values in your Servables config dict. For example, if the config dict for example-pipeline looks like this

{
    "bucket_id": "76eaaa2116f3472c8df1b9f5902d82f6",
}

Then the config-capabilities defined in the Manifest says that this Pipeline is going to require READ Capability on the object 76eaaa2116f3472c8df1b9f5902d82f6.