DEEP - Extreme Scale Technologies

ended

The DEEP-EST (“DEEP - Extreme Scale Technologies”) project will create a first incarnation of the Modular Supercomputer Architecture (MSA) and demonstrate its benefits. In the spirit of the DEEP and DEEP-ER projects, the MSA integrates compute modules with different performance characteristics into a single heterogeneous system. Each module is a parallel, clustered system of potentially large size. A federated network connects the module-specific interconnects. MSA brings substantial benefits for heterogeneous applications/workflows: each part can be run on an exactly matching system, improving time to solution and energy use. This is ideal for supercomputer centres running heterogeneous application mixes (higher throughput and energy efficiency). It also offers valuable flexibility to the compute providers, allowing the set of modules and their respective size to be tailored to actual usage. The DEEP-EST prototype will include three modules: general purpose Cluster Module and Extreme Scale Booster supporting the full range of HPC applications, and Data Analytics Module specifically designed for high-performance data analytics (HPDA) workloads. Proven programming models and APIs from HPC (combining MPI and OmpSs) and HPDA will be extended and combined with a significantly enhanced resource management and scheduling system to enable straightforward use of the new architecture and achieve highest system utilisation and performance. Scalability projections will be given up to the Exascale performance class. The DEEP-EST prototype will be defined in close co-design between applications, system software and system component architects. Its implementation will employ European integration, network and software technologies. Six ambitious and highly relevant European applications from HPC and HPDA domains will drive the co-design, serving to evaluate the DEEP EST prototype and demonstrate the benefits of its innovative Modular Supercomputer Architecture.