The Data Administrator represents in DITAS an actor (a single user, as well as, a representative of an entire organization) which will to expose some of the internal data using the facilities provided by DITAS. In fact, exposing the data to external users (which could be customers or even people interested on this data) requires to setup a proper infrastructure to make the user experience smooth and, at the same time, able to tailor the data accord to the level of privacy which must be preserved.
On this basis, the Data Administrator owns the data and it is responsible to create the DAL (Data Access Layer) which constitutes the middleware between the Data Administrator original data sources and the DITAS platform. The DAL hides the technicalities of the data sources which contain the data to be exposed and will communicate with the VDC (developed by the VDC Developer which has to be in contact and under the control of the Data Administrator). As a critical element of the DAL, the Data Administrator has also to configure the Privacy Enforcement Engine to make the data exposed compliant with the access rights of the application which will consume the data. Finally, the Data Administrator has to define the so-called CAF, a REST-API signature (which will be developed by the VDC Developer) that contains all the methods to access to the exposed data made available to the final user.
Once the DAL and the VDC components are ready, the Data Administrator is in charge of generating and publishing the related Abstract VDC Blueprint to the repository. To this aim, the BPGen tool is provided by DITAS to partially automate this process.
This part of the SDK contains all the guidelines in order to run the DAL Image with Privacy Enforcement Engine (if needed) using the deployment engine, as well as how to configure the abstract blueprint. At the end, there is Abstract Blueprint Schema Documentation.
Guidelines for DAL Configuration
Data administrator is responsible for defining the DAL and with its interface. The DAL exposes a protobuf API using gRPC.
Any DAL message, which is sent through this API and is defined as a protobuf, should include DalMessageProperties (https://github.com/DITAS-Project/ehealth-spark-vdc-with-dal/blob/master/VDC-DAL-grpc/src/main/protobuf/DalMessageProperties.proto) and DalPrivacyProperties (https://github.com/DITAS-Project/ehealth-spark-vdc-with-dal/blob/master/VDC-DAL-grpc/src/main/protobuf/DalPrivacyProperties.proto).
Dal Message Properties
syntax = "proto3";
package com.ditas.ehealth;
message DalMessageProperties {
string purpose = 1;
string requesterId = 2;
string authorization = 3;
}
Field | Type (JSON Format) | Description |
purpose | String | Access purpose |
requesterId | String | Requester identifier |
authorization | String | Access token received from IAM |
Dal Privacy Properties
syntax = "proto3";
package com.ditas.ehealth;
message DalPrivacyProperties {
enum PrivacyZone {
PUBLIC = 0;
PRIVATE = 1;
}
PrivacyZone privacyZone = 1;
}
Field | Type (JSON Format) | Description |
privacyZone | Enum | Privacy zone into which data is to be extracted -PRIVATE or PUBLIC. |
The DAL can be implemented in any language that has protobuf and gRPC implementations, e.g. Python, Java, Scala, Go. An example DAL definition as a service can be found in: https://github.com/DITAS-Project/ehealth-spark-vdc-with-dal/blob/master/VDC-DAL-grpc/src/main/protobuf/EHealthService.proto. The DAL uses the generated server implementation of the defined protobuf.
The DAL should be configured with access details of its corresponding data sources, with Policy Enforcement URL and IAM details. The Administrator is responsible of deploying the DAL with the Policy Enforcement, if it is needed. The policy enforcement engine is in https://github.com/DITAS-Project/policy-enforcement-engine. The configuration file inside conf/application.conf should be modified according to the enforcement engine implementation.
The DAL can be either deployed manually by the data administrator on his premises, or using the deployment engine on the DITAS infrastructure.
Abstract Blueprint Repository CRUD Operations
All the services, in order to help the Data Administrator to store, update and delete the abstract blueprint, are documented in the tables below.
Name of the Method/service: Save abstract VDC Blueprint(s) in the repository | ||||||||
Description/Usage | This method enables the data administrator to store his/her abstract VDC Blueprint(s) -that is(are) valid against the Blueprint schema- in the Blueprint Repository | |||||||
Input |
|
|||||||
Output\Response |
|
Name of the Method/service: Update an abstract VDC Blueprint in the repository | ||||||||
Description/Usage | This method enables the data administrator to update a specific abstract VDC Blueprint by its unique identifier | |||||||
Input |
|
|||||||
Output\Response |
|
Name of the Method/service: Retrieve a specific Section of an Abstract VDC Blueprint | ||||||||
Description/Usage | This method enables any DITAS component or role to retrieve a specific section of an abstract VDC Blueprint by its section number or name | |||||||
Input |
|
|||||||
Output\Response |
|
Services to Configure Abstract Blueprint
This section is dedicated to describing all the services that help the Data Administrator to configure the abstract VDC Blueprint
Semi-automated abstract blueprint generation and publication
DITAS provides a software component, named BPGen, to ease the data administrator in the creation of the abstract blueprint. In particular, BPGen automatically describes the API exposed by the VDC given an OpenAPI file, and populates the remaining fields of the abstract blueprint with default values. In this way, the data administrator is not required to remember the blueprint structure. Instead, (s)he only has to customize the default values and, if needed, remove unused attributes off the generated abstract blueprint. In addition, BPGen also helps the data administrator in publishing the abstract blueprint in the Repository Engine and keeping the published blueprint up-to-date. BPGen is available here
Prerequisites
In orded for BPGen to work properly, the following prerequisites are needed:
- Python 3.7
- GIT command-line client
- A GitHub account with read-write permissions on the desired project workspace
- A Repository Engine account with read-write permissions
Setup and configuration
Once downloaded from here, BPGen must be configured by executing the following command:
sh install.sh
In addition, GitHub access must be granted to BPGen. To this aim, the following steps must be followed:
- A GitHub personal access token must be created (instructions can be found here)
- File secrets.example inside BPGen must be renamed to secrets.json
- File secrets.json must be opened inside an editor, and the following strings must be replaced with the following values:
- GITHUB_TOKEN with the GitHub Token created in step 1
- RE_USERNAME and RE_PASSWORD with the Repository Engine credentials provided
Repository initialization
To create a new repository hosting all the support files to create the abstract blueprint, the repo-init command must be passed to BPGen:
sh sdk-bp.sh repo-init VDC myrepo
Once executed, this command will create a new GitHub repository named myrepo inside the project workspace, and it will add the following structure:
- bp_gen_vdc.cfg – repository configuration file: determines which portions of the abstract blueprint will be generated from source files
- cookbook/cookbook.json – infrastructure definitions file: specifies the infrastructures made available by the application developer, which will be inserted in the COOKBOOK_APPENDIX section of the abstract blueprint.
- output/blueprint.json contains the abstract blueprint produced by BPGen.
- api – folder hosting the OpenAPI file describing the API to be exposed by the VDC, which will be inserted in the EXPOSED_API section of the abstract blueprint.
- api_data_samples – folder containing data utility attributes for each exposed method, which will be inserted in the DATA_MANAGEMENT section of the abstract blueprint.
Default data utility attributes generation
To automatically generate data utility attributes for each method exposed by the VDC, the OpenAPI file must be put inside the api folder. In addition, file bp_gen_vdc.cfg must be edited and the following properties must be set:
- api to api/swagger.yaml, where swagger.yaml is the OpenAPI file name
- data-management to api_data_samples, if not already set
Once done so, the std-metrics command must be passed to BPGen:
sh sdk-bp.sh std-metrics -l myrepo
where myrepo is the name of the repository. Once executed, this command will create inside the api_data_samples as many JSON files as the methods exposed by the VDC. The data administrator will then have to edit these files and either change the values for each data utility metric, or delete the unneeded metrics.
Abstract blueprint generation
To generate the abstract blueprint for the first time, all files in the repository must be firstly pushed to the GitHub project workspace. Once done so, the create command must be passed to BPGen:
sh sdk-bp.sh create -p repo_url
where repo_url is the HTTP URL of the repository (e.g., https://github.com/DITAS-Project/myvdc.git). Once executed, this command will automatically generate the abstract blueprint inside the output folder, starting from the source files created and added in the previous steps. Note that, for the generated blueprint to pass schema validation, the OpenAPI file must contain for each method the x-data-sources property. For further details on this property, please refer to the EXPOSED_API description.
Abstract blueprint update
To update a previously created abstract blueprint (e.g., when the contents of one of the source files are changed), the update command must be passed to BPGen:
sh sdk-bp.sh update -p repo_url
The behavior of this command is analoguous to the one of create. To prevent sections manually altered in the abstract blueprint blueprint.json file to be overwritten with auto-generated data, file bp_gen_vdc.cfg must be edited and the corresponding section removed (e.g., if the data administrator has edited the EXPOSED_API section of the generated blueprint, to prevent such modifications from being lost (s)he must remove the api attribute in bp_gen_vdc.cfg before updating the blueprint.
Abstract blueprint publication
To publish the abstract blueprint to the Repository Engine, the publish command must be passed to BPGen:
sh sdk-bp.sh publish path/to/blueprint.json -server re_url
where path/to/blueprint.json is the path to the generated blueprint file (typically tmp/myvdc.git/output/blueprint.json), and re_url is the URL of the Repository Engine.
Services for Internal Structure
Data Administrator is responsible to define the internal Structure section that describes VDC as a product. Look at OSR blueprint and IDEKO blueprint for examples of such blueprints.
- Filling the metadata information about the DAL image(s): In order to tell the Deployment Engine where to take and how to serve the DAL image, the data administrator has to fill the DAL_images section of the blueprint with the de. The template of this section is as follows:
"DAL_Images": { "[DAL-NAME]": { "images": { "default": { "internal_port":[INTERNAL_PORT_OF_CONTAINER], "external_port": [EXTERNAL_PORT_OF_CONTAINER], "image": "[IMAGE_NAME:TAG_NAME]”, “environment”: { [environment variables] } } } }
- Filling the metadata information about the data sources. The data sources have to be described as following:
"id": "[DATA-SOURCE-ID]", "description": "[DESCRIPTION]", "class": "relational database/object storage/streaming", "type": "MySQL/Minio/InfluxDB", "parameters": { [Connection parameters of the specific data source] }, "schema": {}
-
"policy-enforcement-engine": { "internal_port": [INTERNAL_PORT_OF_CONTAINER], "image": "[IMAGE_NAME:TAG_NAME]", "environment": { [environment variables] } }
Identity Management
For direct control of the Identity Management you can login to KeyCloakand largely follow this documentation at https://www.keycloak.org/docs/latest/server_admin/index.html. Alternatively the config client (https://github.com/DITAS-Project/KeycloakConfigClient) can be used. A password must be requested from the DITAS operator. The URI of the Identity Management services is part of the internal structure of the blueprint under the “Identity_Access_Management” section. Furthermore, predefined roles and users can be provided in this section after they have been defined in KeyCloak.
Services for Data Management
- Data Administrator is responsible to define the initial values of the Data Management section. Fill the QoS and QoD parameters of Data Management
- Expose the DUE Service to get the initial values of the Data management section.
Services for Cookbook Appendix
KeyCloak Managment
he DITAs KeyCloak deployment provides a unique management API that can be used to create new users or groups for a VDC.
Keycloak Management API | ||||||||
Description | Keycloak Management API | |||||||
Input |
|
|||||||
Output\Response |
|
Abstract Blueprint Schema Documentation
In order to form the Blueprint that will reflect the structure and the function of a VDC, it is essential to create a format that will describe the body of this Blueprint, defining all the information and fields that it could possibly include. Based on this format, there are five core sections that a Blueprint contain: the Internal Structure, the Data Management, the Abstract Properties, the Cookbook Appendix and finally the Exposed API section. The complete Blueprint Schema can be found in Project Git Repository.
Internal Structure(Section 1)
The Internal Structure section records general information related to the VDC composition system, like a business description of the VDC, data sources that could be used, the category and scope of the VDC, etc.
Field |
Type (JSON Format) |
Description |
Role/Component |
Phase/Process |
Overview | Object | Information about the content of the data served by the VDC | Resolution Engine | Blueprint Selection Phase |
Overview.name | String | The name of the VDC Blueprint | N/A | N/A |
Overview.description | String | Textual description of the VDC Blueprint | Resolution Engine | Blueprint Selection Phase |
Overview.tags | Array | Each element of this array contains some keywords that describe the exposed data of each one VDC method | Resolution Engine | Blueprint Selection Phase |
Overview.tags.method_id | String | The id (operationId) of the method (as indicated in the EXPOSED_API.paths field) | Resolution Engine | Blueprint Selection Phase |
Overview.tags.tags | Array | Keywords that describe the exposed data of this specific VDC method | Resolution Engine | Blueprint Selection Phase |
Field |
Type (JSON Format) |
Description |
Role/Component |
Phase/Process |
Data_Sources | Array | Data sources used by this VDC | Deployment Engine | VDC deployment and movement |
Data_Sources.items.properties.id | String | A unique identifier of this data source | Deployment Engine | VDC deployment and movement |
Data_Sources.items.properties.description | String | Description | Deployment Engine | VDC deployment and movement |
Data_Sources.items.properties.location | Enum | Cloud/Edge | Deployment Engine | VDC deployment and movement |
Data_Sources.items.properties.class | Enum | “relational database”, “object storage”, “time-series database”, “api”,
“data stream” |
Deployment Engine | VDC deployment and movement |
Data_Sources.items.properties.type | Enum | “MySQL”, “Minio”, “InfluxDB”,
“rest”, “other” |
Deployment Engine | VDC deployment and movement |
Data_Sources.items.properties.parameters | Object | Connection parameters | Deployment Engine | VDC deployment and movement |
Data_Sources.items.properties.schema | Object | Schema | Deployment Engine | VDC deployment and movement |
Field |
Type (JSON Format) |
Description |
Role/Component |
Phase/Process |
Methods_Input | Object | This field contains the part of the data source that each method needs to be executed | DS4M | Data and computation movement, Monitoring |
Methods_Input.Methods | Array | The list of methods | DS4M | Data and computation movement, Monitoring |
Methods_Input.Methods
.method_id |
String | The id (operationId) of the method (as indicated in the EXPOSED_API.paths field) | DS4M | Data and computation movement, Monitoring |
Methods_Input.Methods
.dataSources |
Array | The list of data sources required by the method | DS4M | Data and computation movement, Monitoring |
Methods_Input.Methods.dataSources
.dataSource_id |
String | The id of the data sources (as indicated in the Data_Sources field) | DS4M | Data and computation movement, Monitoring |
Methods_Input.Methods.dataSources
.dataSource_type |
String | The type of the data sources (relational/not_relational/object) | DS4M | Data and computation movement, MonitoringData and computation movement, Monitoring |
Methods_Input.Methods
.dataSources.database |
Array | the list of databases required by a method in a data source | DS4M | Data and computation movement, Monitoring |
Methods_Input.Methods
.dataSources.database.database_id |
String | The id of the database | DS4M | Data and computation movement, Monitoring |
Methods_Input.Methods
.dataSources.database.tables |
Array | the list of tables/collections required by a method in a data source | DS4M | Data and computation movement, Monitoring |
Methods_Input.Methods
.dataSources.database.tables.table_id |
String | The id of the tables/collection | DS4M | Data and computation movement, Monitoring |
Methods_Input.Methods
.dataSources.database.tables.columns |
Array | The IDs of the column/field to be moved | DS4M | Data and computation movement, Monitoring |
Field |
Type (JSON Format) |
Description |
Role/Component |
Phase/Process |
Flow | Object | The data flow that implements the VDC | N/A | N/A |
Flow.platform | Enum | Spark or Node-RED | N/A | N/A |
Flow.parameters | Object | Platform details (for Spark) | N/A | N/A |
Flow.source_code | Any JSON structure | The flow JSON file (for Node-RED) | N/A | N/A |
Field | Type (JSON Format) | Description | Role/Component | Phase/Process |
DAL_Images | Object | Set of Docker images to include in the DALs associated to this VDC. The key is a unique DAL identifier while the values are the image information | Deployment Engine | VDC Deployment and movement |
DAL_Images.[dal_id].original_ip | string | IP where the original DAL has been deployed | Deployment Engine | VDC Deployment and movement |
DAL_Images.[dal_id].images | Object | Set of images to deploy in this DAL implementation. The key is a unique identifier and the values are the image information | ||
DAL_Images.[dal_id].images.[image_id].image | string | The Docker image name in standard format [repository]/[group]/<image_name>:[version] | Deployment Engine | VDC Deployment and movement |
DAL_Images.[dal_id].images.[image_id].internal_port | Int | The port in which the software of the image will be listening, if any. This port won’t be exposed outside but it will receive data through redirection. | Deployment Engine | VDC Deployment and movement |
DAL_Images.[dal_id].images.[image_id].external_port | Int | The port in which the image will be accessible. It will redirect any request to this port to the one specified in internal_port | Deployment Engine | VDC Deployment and movement |
DAL_Images.[dal_id].images.[image_id].environment | Object | Environment variables to pass to the image in key-value format. | Deployment Engine | VDC Deployment and movement |
Field | Type (JSON Format) | Description | Role/Component | Phase/Process |
VDC_Images | Object | Set of Docker images to include in the DAL. The key is a unique image identifier while the values are the image information | Deployment Engine | VDC Deployment and movement |
VDC_Images.[image_id].image | string | The Docker image name in standard format [repository]/[group]/<image_name>:[version] | Deployment Engine | VDC Deployment and movement |
VDC_Images.[image_id].internal_port | Int | The port in which the software of the image will be listening, if any. This port won’t be exposed outside but it will receive data through redirection. | Deployment Engine | VDC Deployment and movement |
VDC_Images.[image_id].internal_port | Int | The port in which the image will be accessible. It will redirect any request to this port to the one specified in internal_port | Deployment Engine | VDC Deployment and movement |
VDC_Images.[image_id].internal_port | Object | Environment variables to pass to the image in key-value format. | Deployment Engine | VDC Deployment and movement |
Field | Type (JSON Format) | Description | Role/Component | Phase/Process |
Identity_Access_Management | Object | Information about identity and access management of this VDC | Access Management, Request Monitor | Blueprint Selection Phase |
Identity_Access_Management.jwks_uri | String | The JWKS URL for getting verification keys | Application, Request Monitor, DAL, Any that needs to validate a token | Blueprint Selection Phase |
Identity_Access_Management.iam_endpoint | The endpoint of the IAM server | Application, Request Monitor, DAL, Any that needs to validate a token | Blueprint Selection Phase | |
Identity_Access_Management.roles | List of Strings | A set of roles that a user might have. | Request Monitor, PSES | Blueprint Selection Phase |
Identity_Access_Management.provider | List of Objects | A list of identity provider that can be used. Can be empty if only the DITAS internal one is used. | Blueprint Selection Phase | |
Identity_Access_Management.provider[i].name | String | Name of the provider | Request Monitor | Blueprint Selection Phase |
Identity_Access_Management.provider[i].type | String | Type of the provider to use. Only OAuth supported for now. | Request Monitor | Blueprint Selection Phase |
Identity_Access_Management.provider[i].uri | String | Address of the provider. | Request Monitor | Blueprint Selection Phase |
Identity_Access_Management.provider[i].portal | Login Portal for that provider. | Request Monitor | Blueprint Selection Phase |
Field | Type (JSON Format) | Description | Role/Component | Phase/Process |
Testing_Output_Data | Array | Sample dataset per VDC method | DUE | Blueprint Selection Phase |
Testing_Output_Data.method_id | String | The id of this exposed VDC method | DUE | Blueprint Selection Phase |
Testing_Output_Data.attributes | Array | The attributes of the output data returned by the method that is required by the end user | DUE | Blueprint Selection Phase |
Testing_Output_Data.zip_data | String | The URI to the zip sample data for this exposed VDC method | DUE | Blueprint Selection Phase |
Testing_Output_Data.history_time | Integer | The time interval, expressed in seconds before the current time, to compute the availability of the method. | SLA Manager | Monitoring |
Testing_Output_Data.history_invocations | Integer | The maximum number of invocations of the method, to compute its availability | SLA Manager | Monitoring |
Data Management(Section 2)
Data Management is a section dedicated to the VDC that plans on measuring the performance or monitoring the system for optimization and business purposes. Also, this section is responsible for enfolding information about data quality. Quality of service aspects along with data quality aspects constitutes the definition of data utility for the exposed data.
Field |
Type (JSON Format) |
Description |
Role/Component |
Phase/Process |
method_id | String | The id of this exposed VDC method | DURE, SLA Manager, DS4M | Blueprint Selection Phase,
Data and computation movement, Monitoring |
attributes | Object | Data utility, security and privacy attributes for this exposed VDC method | DURE, SLA Manager, DS4M | Blueprint Selection Phase,
Data and computation movement, Monitoring |
attributes.dataUtility | Array | A list of all the metrics related to data quality. The JSON schema for each one metric is presented in the next table | DURE, SLA Manager, DS4M | Blueprint Selection Phase,
Data and computation movement, Monitoring |
attributes.security | Array | A list of all the properties related to security. The JSON schema for each one property is identical to the schema used for the data utility metrics | DURE, SLA Manager, DS4M, PSE | Blueprint Selection Phase,
Data and computation movement, Monitoring |
attributes.privacy | Array | A list of all the properties related to privacy. The JSON schema for each one property is identical to the schema used for the data utility metrics | DURE, SLA Manager, DS4M, PSE | Blueprint Selection Phase,
Data and computation movement, Monitoring |
Abstract Properties(Section 3)
This section contains the goal model that specifies the non-functional application requirements that the blueprint is expected to fulfill once the concrete blueprint is instantiated. Such a goal model is used by the SLA Manager to detect violations, and by the DS4M to identify the best data and computation movement actions.This section remains empty in the abstract blueprint. Once VDC Blueprint resolution takes place, an intermediate blueprint is generated. In particular, the Data Utility Resolution Engine inserts in this section a subset of the goal model taken from the application designer requirements.
Cookbook Appendix(Section 4)
The Cookbook Appendix section contains all the necessary information for the orchestration and the deployment of each module/microservice (aka CookBook recipes). Also, it provides software configuration management, agnostic in nature, in order to support multiple tools such as Puppet and Chef.
Field |
Type (JSON Format) |
Description |
Role/Component |
Phase/Process |
Deployment | Object | Information about the available infrastructures and the software running on it | N/A | N/A |
Deployment.id | String | Identifier of the deployment | N/A | N/A |
Deployment.name | String | Human friendly name of the deployment | N/A | N/A |
Deployment.infrastructures | Object | Set of clusters deployed with this blueprint. The key is the infrastructure identifier and the value is the cluster nodes information | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.id | String | Identifier of the cluster infrastructure | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.name | String | Human readable name of the cluster infrastructure | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.type | String | Type of the cluster. For example: cloudsigma, aws or baremetal | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.nodes | Object | Nodes present in the cluster indexed by node role. The value is a list of nodes information | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.nodes.<role>.hostname | String | Hostname of the node | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.nodes.<role>.role | String | Role on the node in the cluster. In case of Kubernetes it will be master or slave | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.nodes.<role>.ip | String | External IP of the node | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.nodes.<role>.drive_size | Int | Size of the boot drive in bytes | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.nodes.<role>.data_drives | Array | List of data drives attached to the node | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.nodes.<role>.data_drives[i].name | String | Name of the data drive | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.nodes.<role>.data_drives[i].name | String | Size of the data drive | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.nodes.<role>.extra_properties | Object | Arbitrary properties associated to this node in key-value format whose key and values are strings. It can be used to set labels, for example. | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.status | String | Status of the cluster. A healthy cluster should be in running status. | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.vdcs | Object | Object containing information about the VDCs deployed in the cluster. The key is the VDC identifier while the value is the information relative to this particular VDC | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.vdcs.<vdc_id>.ports | Object | Information about which port is assigned to each image running in a VDC.The key is the image identifier in Docker format and the value the port in which it can be reached in any node of the cluster. | N/A | N/A |
Deployment.infrastructures.<infrastructure_id>.extra_properties | Object | Arbitrary properties associated to this cluster in key-value format whose key and values are strings. It can be used to set labels, for example. | N/A | N/A |
Deployment.extra_properties | Object | Arbitrary properties associated to this multi cluster deployment in key-value format whose key and values are strings. It can be used to set labels, for example. | N/A | N/A |
Deployment.status | String | General status of the whole multi cluster deployment. A healty deployment should be in the “running” state | N/A | N/A |
Field | Type (JSON Format) | Description | Role/Component | Phase/Process |
Resources | Object | Set of resources available to deploy VDCs. This resources are machines with attached disks grouped as clusters (infrastructures) that will form different Kubernetes clusters. | Deployment Engine | Deployment |
Resources.description | string | Optional description for the whole resource set | Deployment Engine | Deployment |
Resources.infrastructures | array | List of available clusters to create or use | Deployment Engine | Deployment |
Resources.infrastructures[i]
.description |
string | Optional description for the cluster | Deployment Engine | Deployment |
Resources.infrastructures[i]
.name |
string | Unique name for the cluster. It will be used to form the machines hostnames if it need to be created | Deployment Engine | Deployment |
Resources.infrastructures[i]
.type |
string | Type of infrastructure. A “Cloud” value means that the resources are not initialized and so the deployment engine needs to create them as Virtual Machines and initialize the Kubernetes cluster over them. “Edge” means that the machines are already configured as a cluster and the data in the “resources” section is just informative. | Deployment Engine | Deployment |
Resources.infrastructures[i]
.provider |
object | Information about the cloud or edge provider | Deployment Engine | Deployment |
Resources.infrastructures[i]
.provider.api_endpoint |
string | Endpoint to use in case of a cloud provider | Deployment Engine | Deployment |
Resources.infrastructures[i]
.provider.api_type |
string | The type of provider such as AWS, GCP, Cloudsigma, etc | Deployment Engine | Deployment |
Resources.infrastructures[i]
.provider.credentials |
Object | A key-value map with the credentials to access the cloud provider and be able to create the cluster. In case of an “Edge” cluster type, the existing k8s cluster credentials must be provided here. | Deployment Engine | Deployment |
Resources.infrastructures[i]
.provider.secret_id |
string | If the deployment engine is configured to use a vault, the credentials can be provided as a link to the vault to retrieve them | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources |
array | List of Virtual Machines to instantiate, in case of a “Cloud” deployment, or to use in case of an “Edge” one. | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].cores |
int | Number of cores of the VM | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].cpu |
int | CPU speed in MHz | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].disk |
int | Boot disk size in MB | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].image_id |
string | Identifier of the boot image to use | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].ip |
string | IP to assign to the VM. If not present, a random one will be chosen. The provider has to have enough free public IPs for all of the machines since they nee to have a fixed IP | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].name |
string | Unique name of the machine. Along with the infrastructure name, it will be used to compose the machine hostname | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].ram |
int | RAM size in MB | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].role |
string | Role in the kubernetes cluster. It can be “master” or “worker” | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].type |
string | Optionally the type of machine (i.e. n1-small) can be provided here instead of providing the individual features of RAM and CPU | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].drives |
array | Set of data drives attached to the machine | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].drives[i].name |
string | Unique name for the data drive | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].drives[i].size |
int | Size in MB of the data drive | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].drives[i].type |
string | Type of the data drive (HDD or SDD) | Deployment Engine | Deployment |
Resources.infrastructures[i]
.resources[i].extra_properties |
string | Key-value map to provide arbitrary tags to the machine. It can be used to provide information to the deployment engine about the features installed or configured in the boot disk or to mark a particular node with a particular tag. | Deployment Engine | Deployment |
Resources.infrastructures[i]
.extra_properties |
object | A key-value map that can be used to provide information to the deployment engine or components running inside a VDC. This is a place to put arbitrary tags that can be useful such as describe if a cluster must be treated as the default one to deploy VDCs, if it is a trustable or untrustable zone, etc. | Deployment Engine | Deployment |
Field |
Type (JSON Format) |
Description |
Role/Component |
Phase/Process |
Identity_Access_Management | Object | Information about identity and access management of this VDC | Access Management, Request Monitor | Usage Phase |
Identity_Access_Management.validation_keys | List of Strings | Set of Public keys that can be used to validate a key if the JWKS is unreachable. (optional) | Application, Request Monitor, DAL, Any that needs to validate a token | Usage Phase |
Identity_Access_Management.mapping | List of Objects | Set of automatic role mappings for a provider (optional)
In case more than one provider is used in the configuration, a component can use these rules to automatically associate roles based on the origin of a token. |
Request Monitor | Usage Phase |
Identity_Access_Management.mapping[i].provider | String | Name, needs to match one from the provider list | Request Monitor | Usage Phase |
Identity_Access_Management.mapping[i].roles | List of String | Roles that this mapping can apply | Request Monitor | Usage Phase |
Identity_Access_Management.mapping[i]
.role_map |
List of Object | Rules that can be evaluated based for each token. | Request Monitor | Usage Phase |
Identity_Access_Management.mapping[i]
.role_map[j].matcher |
String | Anko Script Rule | Request Monitor | Usage Phase |
Exposed API(Section 5)
This section of the abstract blueprint includes all the information about the methods, through which the administrator exposes totally or partially the data that are stored in the sources that he/she controls. The CAF RESTful API of the VDC is written according to the OpenAPI Specification (originally known as the Swagger specification) so that big vendors and also new providers are able to publish their services and components.
Field | Type (JSON Format) | Description | Comments |
summary | String | A short summary of what the operation does | mandatory field |
operationId | String | Unique string used to identify method | the same id is used to identify each one exposed VDC method throughout the whole abstract VDC blueprint,
mandatory field |
parameters | Array | A list of the input parameters for this method | optional field |
responses | Object | The list of possible responses as they are returned from calling this method | mandatory field |
responses.200(or 201).description | String | A short description of the response | mandatory field |
responses.200(or 201).content.application/json.schema | Object | the schema of the data, included in the response payload | mandatory field to enable the developer to conclude whether the method fits to his/her application |
x-data-sources | Array | A list of the data sources required for this method |
mandatory field |