Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - PrEstoCloud (PrEstoCloud - Proactive Cloud Resources Management at the Edge for Efficient Real-Time Big Data Processing)

Teaser

Recent trends in cloud computing go towards the development of new paradigms in the cloud (e.g. heterogeneous, federated, distributed multi-clouds and extending them beyond the fog computing) alleviating the tight interactions between the computing and networking...

Summary

Recent trends in cloud computing go towards the development of new paradigms in the cloud (e.g. heterogeneous, federated, distributed multi-clouds and extending them beyond the fog computing) alleviating the tight interactions between the computing and networking infrastructures, with the purpose of optimizing the use of cloud resources with respect to cost, flexibility and scalability. However, the ever increasing requirements for efficient and resilient data-intensive applications that are able to cope with the variety, volume and velocity of Big Data, lead to the big challenge of new agile architecture paradigms that enhance the dynamic processing even at the extreme edge of the network.
The main objective of PrEstoCloud project is to make substantial research contributions in the cloud computing and real-time data intensive applications domains, in order to provide a dynamic, distributed, self-adaptive and proactively configurable architecture for processing Big Data streams. In particular, PrEstoCloud aims to combine real-time Big Data, mobile processing and cloud computing research in a unique way that entails proactiveness of cloud resources use and extension of the fog computing paradigm to the extreme edge of the network. The envisioned PrEstoCloud solution is driven by the micro services paradigm and has been structured across five different conceptual layers: i) Metamanagement; ii) Control; iii) Cloud infrastructure; iv) Cloud/Edge communication and v) Devices, layers. This solution will address the challenge of cloud-based self-adaptive real-time Big Data processing, including mobile stream processing and will be demonstrated and assessed in several challenging, complementary and commercially-promising pilots. There are three PrEstoCloud pilots from the logistics, mobile journalism and security surveillance, application domains. The objective is to validate the PrEstoCloud solution, prove that it is domain agnostic and demonstrate its added-value for attracting early adopters, thus initialising the exploitation process early on showing concrete examples and benefits.

Work performed

Following the regular project processes and guidelines set up a requirement and State of the Art analysis was performed. A conceptual architecture was developed and concepts for each of the PrestoCloud components based on identified requirements were developed. The infrastructure has been set up, first prototypical implementations of components has been developed, deployed and integrated to a first demonstrator for the surveillance use. The platform architecture and components are shown in the figure. Beside the technical actions also the communication, dissemination and a first iteration of exploitation concept has been finished and communication successfully performed. Prestocloud and its progress is presented in all relevant social media, generates awareness on fares and publications out of PrestoCloud has been successfully accepted by scientific journals and conferences.

Final results

We expect to have a generic PrestoCloud platform supporting at least the 3 main use case scenarios and ready to run by one of the Project partners or been sold Big Data Solution and Service providers and/or cloud service and infrastructure providers.
In concrete we will go beyond state of the art regarding the main objectives:
Dynamic monitoring in real-time Big Data processing architectures – a detection of anomalous situations on the fly (not predefined anomalies): combined predictive capabilities with the ability to recommend cloud resource adaptations, a new semantic-based model for distributed real-time processing architectures, monitoring of the real-time processing architecture, a support of the meta-modelling of the adaptation process and we enable real-time changes of the processing pipeline.
Situation-aware and context-driven adaptation recommender systems – we combine predictive capabilities with the ability to recommend cloud resource adaptations by developing a big data situation meta model that can model situations relevant to cloud and edge resources topology, status and generic capabilities and propose algorithms for devising proactive adaptation actions.
Real-time mobile stream processing – we use conventional big data analytics and integrate promising concepts of edge computing to investigate the concept of bandwidth and medium utilization and decide on edge, cloud or hybrid computing, to investigate on acceleration concepts for a local analysis and global analytics control local analytics and business rules.
Pro-active cloud computing - autonomous cloud management platform („proactive cloud automation“) that: provides a workflow catalogue system for provisioning and deployment workflows, is using a scalable scheduler and has the ability to connect to a variety of cloud providers.
Network virtualization – we go for a deployment of a network overlay seamlessly interconnecting virtual networks on multiple sites, with reinforced security at the network level, formulated firewalls, access control, and monitoring data to provide meaningful input to the resource manager.
The project aims to improve the utilization in terms of cost / time efficiency of paid cloud resources as well as the resource consumption of own cloud and premises. By improving flexibility in resource usage through an on demand resources management it also will have impact on direct cost of operation, reducing costs for paid services/traffic, processing volume. The resource exploitation at extreme edge of network also should improve. Regarding services it is expected to improve infrastructure to enable new/better services as well as improvements regarding speed / volume. Also deployment of new services on the cloud, service performance, runtime and maintenance as well as the improvement of service monitoring can be taken into measurable account.

Website & more info

More info: http://www.prestocloud-project.eu.