Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - DEEP-EST (DEEP - Extreme Scale Technologies)

Teaser

\"The DEEP – Extreme Scale Technologies (DEEP-EST) project started on 1 July 2017 and will last three years, plus a 9-month extension. The project will create a first incarnation of the Modular Supercomputer Architecture (MSA) and demonstrate its benefits. In the spirit of...

Summary

\"The DEEP – Extreme Scale Technologies (DEEP-EST) project started on 1 July 2017 and will last three years, plus a 9-month extension. The project will create a first incarnation of the Modular Supercomputer Architecture (MSA) and demonstrate its benefits. In the spirit of the predecessors DEEP and DEEP-ER projects, the MSA integrates compute modules with different performance characteristics into a single heterogeneous system. Each module is a parallel, clustered system of potentially large size. A federated network connects the module-specific interconnects. The MSA brings substantial benefits for heterogeneous applications and workflows: each part can be run on the module that fits best, improving time to solution and energy use. This is ideal for supercomputing centres running heterogeneous application mixes (higher throughput and energy efficiency). It also offers valuable flexibility to the compute providers, allowing the set of modules and their respective size to be tailored to actual usage.

The project will build a hardware prototype including three modules: a general-purpose Cluster Module and an Extreme Scale Booster respectively supporting the full range of HPC applications, and a Data Analytics Module specifically designed for High-Performance Data Analytics (HPDA) workloads. Proven programming models and APIs from HPC (combining MPI and OmpSs) and HPDA will be extended and combined with a significantly enhanced resource management and scheduling system to enable straightforward use of the new architecture and achieve high system utilisation and performance. Scalability projections will be given up to the Exascale performance class. The DEEP-EST prototype will be defined in close co-design between applications, system software and system component architects. Its implementation will employ European integration, network and software technologies. Six ambitious and highly relevant European applications from HPC and HPDA domains will drive the co-design, serving to evaluate the DEEP-EST prototype and demonstrate the benefits of its innovative Modular Supercomputer Architecture.

Objectives and actions proposed for the DEEP-EST project will support key parts of the European “Strategic Research Agenda” (SRA) in various aspects like \"\"Energy and Resiliency\"\" and \"\"Big data and HPC Usage Models\"\". The DEEP-EST technology addresses the needs of existing medium- and large-scale supercomputer centres, which all run a large variety of applications with widely differing resource requirements. Five of the six DEEP-EST applications combine simulations and advanced data analytics, and the co-design approach will ensure that DEEP-EST technology fits the needs of this market segment. In addition, very large-scale data analytics problems will require the use of clustered resources, for which the Data Analytics Module included in the MSA and the associated software stack will provide a good solution. The dissemination and exploitation of results achieved in DEEP-EST is one of the project’s main objectives and will be done in a concerted action by all partners. Measures will be taken to maximise the project’s impact in the European and global HPC community and beyond. In addition, the DEEP-EST project will highlight and address diversity in the project, within its user base, and in the wider community.

The main objectives of the DEEP-EST project are
• develop an energy efficient system architecture that fits HPC and HPDA workloads and satisfies the requirements of end-users and e-infrastructure operators
• build a fully working MSA system prototype (defined by co-design) implementing three modules: an Extreme Scale Booster, a Data Analytics and a Cluster
• foster European technologies
• build a resource management and scheduling system fully supporting the MSA
• enhance and optimise programming models for heterogeneous applications
• validate the full hardware (HW) / software (SW) stack with relevant HPC and extr\"

Work performed

These are the main achievements of the first 18 months of the DEEP-EST project:
• Extensive Co-Design discussions between application users and engineers lead to the final design-decisions for the Modular System Architecture (MSA).
• Hardware and Software specifications have been discussed and evaluated for the MSA that will lead to the three modules of the DEEP-EST hardware prototype: Cluster (CM), Booster (ESB), and Data Analytics Module (DAM).
• Definition of the benchmark suits that will help to monitor and analyze the systems performance.
• Establishing the project website (www .deep-projects.eu) and Social Media Accounts, which are regularly updated to include new events, job offers, and news.
• Development of a strategy for the six application use cases for the planned MSA.

Final results

The Modular system architecture is a novel approach to overcome the fundamental limitations of today’s supercomputer architectures with respect to flexibly and efficiently serving: heterogeneous applications like multi-physics simulations, combinations of classical HPC simulations with HPDA, and the large mixes of applications with different resource requirements typically run by a supercomputer centre.

DEEP-EST will build a fully working prototype using cutting-edge components and containing 3 modules, which together address the needs of both HPC and HPDA users. A thorough evaluation of the system will be conducted with six ambitious applications that combine HPC simulation and large scale HPDA.
DEEP-EST aims to improve the usability, flexibility and sustained performance of future supercomputers significantly beyond what can be provided by today’s monolithic systems.

Website & more info

More info: http://www.deep-projects.eu.