If one could describe DITAS in a simple and intuitive way (from the perspective of an application developer), it can be said that is a tool that eases and optimizes data accessibility and provision, offering them timely, securely and accurately under a layer of abstraction. Moreover, it hides how the data -from the source- are exposed in the application. However, behind this abstract layer, lies an infrastructure capable of integrating different platforms, storage systems and network capabilities. Data need to be accessed, transformed and exposed in various ways according to storage and application requirements. Also, the data should be both secured and evaluated regarding the structure and performance that was originally agreed.
In terms of implementation, this “hidden” complex structure is represented in the VDC, a virtual container which aims to perform data cleansing, data correlation and data transformation as data moves from/towards the data sources. These actions could vary in both aim and nature, for example a data format transformation (e.g. from XML to JSON) and an encryption technique (e.g. connection over TLS) are inherently two different things. But, most of the tasks that have to be performed have common traits. Returning to our example where the developer needs the data in a certain format using them on top of a secure channel, the transformation will be performed first, followed by the transportation. So, choosing the right way to describe and implement the internal structure of a VDC, where the tasks are distinctive and follow a certain logical flow, is crucial.
That’s where the Node-RED tool comes into the equation. Node-RED is a programming tool used for wiring together hardware devices, APIs and online services by representing them as nodes. The advantages that render the uniqueness and innovation of Node-RED, as a programming tool, is its ability to amalgamate a widely-used programming language under a new flow-based programming model along with a large list of software and hardware components, retaining at the same time, an open-source character. These advantages can be well exploited by the DITAS platform offering a great deal of benefits that other tools can’t offer.
Furthermore, Node-RED’s programming model is particularly friendly and easy to use thanks to its browser-based flow editor and the intuitive approach it offers for “building” solutions. Node-RED “breaks” a task into a logical sequence of steps-nodes with specific functions, offering this way an abstract layer for building a solution and at the same time a very modular way for implementing it. So, considering the distinctiveness and modularity of the VDC tasks that ought to be performed (e.g. data transformation, data security, as mentioned above), the Node-RED’s programming model is proving to be quite useful. Each of these tasks can be represented as a node. Therefore, the VDC structure can be visualised as a flow that consists of separated jobs/nodes which can be wired with each other, making the benefits of using Node-RED crystal clear.
In addition, the Node-RED UI offers a user-friendly way for creating and inserting new nodes with drag and drop – like methods. As a result, adding and connecting new components becomes trivial. It’s obvious that an already implemented node can be reused in the same or another flow or even saved for future usage. Flows can be open-sourced or sold, enabling developers this way to focus on the innovative part of their task saving time on developing already implemented flows.
Regarding the VDC, as a product, portability and ease-of-deployment are essential. These traits already exist in Node-RED as both export/import and deployment of a flow are “single-click matter”. Given the fact that DITAS introduces the concept of moving data and computation between cloud and edge environments, this feature of Node-RED is very useful.
Apart from the general benefits that Node-RED offers to DITAS, it can be very useful when it comes to the objectives that DITAS, as a complete platform, tries to accomplish.
For example one of DITAS objectives is to improve the productivity when developing and deploying Data Intensive Applications. That’s why it is required to have the ability to be connected with the main SBC devices (i.e., Raspberry PI, Odroid), as well as the main operating systems (i.e., Android OS, Linux, Windows). Node-RED is designed as a way to visualize sending and receiving Message Queue Telemetry Transport (MQTT) messages to and from hardware devices. Connecting hardware (and their data streams) to applications is baked into Node-RED’s DNA. Also, the fact that Node-RED supports different types of IoT devices is beneficial. Platforms that require a proprietary gateway to connect IoT devices are dependent on the platform providers to respond to emerging technologies, thus limiting the reactivity of the platform to adopt new protocols and support an increasing number of heterogeneous IoT devices. That’s where Node-RED excels, as it can handle millions of devices and on the same time scales elastically and can integrate a plethora of hardware in a straightforward way.
The DITAS platform, through the VDC, will encapsulate all the mechanisms for connecting heterogeneous storage nodes. So, the integration with different data stores, coming from different worlds (SQL, NoSQL data stores, CEP systems) using the common definition of VDC that will provide the necessary abstraction is needed, can be easily achieved. Node-RED comes with a huge database of components, making this task easy. For example, you can use it to listen to events from HTTP, websockets, TCP, Twitter and more and store this data in databases without having to program much if at all.
Node-RED, as an IoT platform, cooperates smoothly with devices like Raspberry Pi which has one of the biggest communities on embedded devices. That cooperation is not limited to communication but also extends to the control of the device performance. Moreover, due to implicit capabilities of flow monitoring, the platform can instantaneously take decisions when using a diverse mode for data transmission and overcome bottlenecks.
As the independent deployments of different strategies are usually centrally controlled and more resources are consumed due to the monitoring overhead, the governance of the deployments through Node-RED is preferable because multiple subflows and alternative methodologies could be included and at the runtime the platform can choose the optimized one by sensing the actuating devices. That provides the ability to have multiple solutions for the same task, giving the flexibility to the platform to select different modes like bandwidth or latency optimization and enable the improvement of the observed latency when using a different deployment configuration.
Furthermore, DITAS should enable the execution of data-intensive applications in a mixed cloud/fog environment where different Data Management systems could be used. Node-RED has a plethora of pre-defined plugins that enable fast and efficient synergy between the Application, the Data Sources and the Management systems. That extensive list of library plugins can lead to faster production of enterprise-ready applications and automate and simplify any operations. Also, they can empower the collaboration between the VDC Architects, Data Intensive Application developers and Database administrators (DBAs), improve data access as well as speedup iterative testing.
Finally, they can facilitate the alignment, consistency and the overall governance between the Application, the VDC and the Data Sources. The big variety of libraries that Node-RED contains makes the development of different types of applications easy, using few lines of code and designed according to the DITAS platform.