#	Pagina
attuale pagina	/open-h2020/projects/199154/results.html

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 2 - dReDBox (Disaggregated Recursive Datacentre-in-a-Box)

Teaser

Summary

For all the superior features that low-power computing systems exhibit compared to conventional high-end server designs, there has been a common design axiom that both technological trends are based on: the main-board and its hardware components form the baseline, monolithic building block that the rest of the hardware/software stack design builds upon. In such conventional systems, the proportionality of IT resources is fixed during design time and remains static throughout machine lifetime, with known ramifications in terms of low system resource utilization, costly upgrade cycles and degraded energy proportionality.

dReDBox takes on the challenge of revolutionizing the low-power computing ecosystem by breaking once and for all server boundaries through materialization of the concept of disaggregation. Through a highly modular, rack-scale architecture, dReDBox specifies, designs and prototypes modular blocks of SoC-based microservers, memory and accelerators, interconnected via a high-speed, low-latency optical fabric and integrated into a single platform. In a dReDBox system, pool instances can be allocated in arbitrary sets, as driven by fit-for-purpose resource/power management software. dReDBox components allow for deployment in various integration form factors and target scenarios, as manifested by three target application use-cases, namely Video Surveillance, Network Analytics and Telecom.

Overall, dReDBox has the objective to design and develop a vertical approach for a flexible modular datacenter-in-a-box architecture. Its design aims at redefining the datacenter hardware building blocks by creating mainboard trays (dTRAYs) with homogeneous components that form resource pools (dBRICKs). dReDBox features a scalable hypervisor layer that abstracts all resource pools and orchestration software that manages and allocates resources to workloads. dReDBox delivers enhanced elasticity and improved virtual machine migration within the datacenter. The resource pools feature commodity, densely integrated, modular SoC-based compute blocks and high performance memory modules. The interconnect switching technology used features highly integrated, low-power, low-cost, low-footprint, and scalable technologies for module-to-module and tray-to-tray communication.

dReDBox has prototyped, notably for the first time, a fully integrated disaggregated datacenter, bringing together complementary assets from the hardware, network, operating system, orchestration and applications stacks. Using this prototype, the project has demonstrated the practical feasibility and economical viability of optically disaggregated datacenters, going well beyond previous state-of-the-art studies on disaggregation based on full/partial simulation of disaggregated infrastrucutre. dReDBox shows that disaggregated systems have the potential of improving the capital and operational costs thanks to tying the pay-as-you-grow model much tighter to actual hardware utilization and significant decrease of the required power budget.

Work performed

The main results achieved by dReDBox within its 36 months of activities are:

- Complete specification of co-designed software and hardware architecture and interfaces for a low-power, disaggregated and modular datacenter-in-a-box.

- Complete, vertically and horizontally integrated software and hardware platform prototype, demonstrating the feasibility and advantages of the approach in a TRL6 environment.

- Demonstration of the capabilities and benefit of the platform via three selected industrial application use-cases, all ported and demonstrated on the prototype platform.

- Complete design and implementation of hardware disaggregated memory controller for compute, memory and accelerator modules (dCOMPUBRICKs, dMEMBRICKs and dACCELBRICKs).

- Design of a scalable network architecture and components focusing on minimizing access latency and able to achieve low latency and capacity (i.e., < 1us latency at 80Gbps).

- Delivery of a full hypervisor supporting bare-metal and virtualized resource disaggregation, elastic scaling of memory, peripheral disaggregation and VM migration, and able to meet (and in some cases, i.e., VM migration largely exceed) performance targets.

- Delivery of a full scalable distributed orchestration stack, supporting dynamic resource allocation, virtual platform creation, and power-aware resource management, fully integrated with the rest of the hardware and software stack.

- Successful scientific and industrial dissemination of project achievements, with 40+ accepted peer-reviewed publications, 8 booth expositions, and more than 20 invited or workshop talks. Extensive press attention with more than 20 media pieces around the project.

- Mature joint and individual plans towards exploitation of project results after project ending, with 3 patents filed protecting relevant technology and contributions to Open Source ecosystems including the Linux kernel, Apache Zookeeper and OpenStack.

Final results

The consortium has delivered a first-of-its-kind modular datacenter rack architecture design, comprising components that are designed from the ground up to deliver enhanced elasticity compared to state of the art servers and datacenters. Based on this design, and for the first time, the project built a fully integrated disaggregated datacenter hardware and software prototype.

The architecture features a novel system and orchestration software stack, with operating system extensions for dynamic remote memory allocation, software-defined memory control and resource management, as well as a dedicated controller and API for bringing elasticity to requesting applications. Combined with the modular hardare, the software stack brings 2.5x faster memory elasticiy compared to state-of-the-art (scale-out systems), while only adding 3% overhead for VM creation compared to traditional setups. Building on disaggregation, the system is able to perform VM migration in worst-case constant time and with constant network utilization, compared to linear time in state-of-the-art system.

The dReDBox disaggregation controller and network architecture based on all-optical switches achieves low latency disaggregation (<1us end-to-end), high bandwidth (up to 80 Gbps per module), low footprint and low power consumption. Three ported applications have demonstrated that the elactricity of the system brings the flexibility needed to minimize deployment costs while keep performance stable in spite of load variability (video surveillance and mobile-edge use cases). Accelerator disaggregation and low-latency interconnect allow cost-effective and scalable real time analysis of 100Gbps network flows.

The anticipated benefit to target stakeholders and society is manifold. Integrators, vendors and users will be able to fully customize low-power server builds and deployments to match use-case needs. Additionally, they will be able to apply upgrade cycles at the granularity of module types, instead of having module upgrades lead to whole server upgrades. For cloud and IT service providers, disaggregation allow for unprecedented ability to match IT resources to service demand, leading to improved operational efficiency and lower power consumption. To service users - including government, enterprise, SME and consumers - disaggregation brings the potential of matching IT needs to true application/workload needs.