In the context of the French involvement in the PRACE European Research Infrastructure, led by GENCI, la Maison de la Simulation (MdlS) has helped optimise several simulation codes used by projects awarded by PRACE on the French Tier0 supercomputer Curie. Thus, between 2013 and 2015, 8 projects in various scientific fields (sciences of the Universe, materials, biology, geophysics, mathematics and computing science, chemistry and climatology) have benefited from its expertise. This work was coordinated and supervised by Julien Dérouillat, engineer at MdlS.
In link with the research teams of each project, experts from MdlS have worked to improve the simulation codes for taking full advantage of Curie’s performance. This work was focused on three main directions : Quality of parallelism (execution of multiple and independent tasks concurrently), parallel read-write of data and CPU performance.
From the instrumentation and diagnosis phases of computing codes to the definition and implementation of technical solutions, an average of 6 months is needed to perform this type of optimisation. With substantial gains, at the end: A more efficient execution of the computing code giving the opportunity to make better use of the architectures of massively parallel supercomputers and to save core hours for producing results.
It was, for example, around 4 million core hours that were saved by the SEDTRANS project (simulation of fluids with transportation of sediments), which had globally an allocation of nearly 11 million core hours on Curie, in the context of the 10th regular call of PRACE.
Thanks to the work carried out by Yacine Ould-Rouis for MdlS in link with Pedro Costa, from Delft University of Technology (The Netherlands) for the project, the diagnosis of the multi-physics simulation code used by SEDTRANS has highlighted the time lost between the modelling of fluids and that of particles, caused by the complexity of representations and of the coupling between them. Two successive optimisations, on communications first and then on the algorithm itself, finally allowed to gain 45% on the calculation loop. “This collaboration with la Maison de la Simulation helped us consider our computing code from a different point of view and improve it very significantly”, Pedro Costa said. All along the process, exchanges were plentiful and fruitful: “We would never have achieved this result without this quality of communication and reactivity on both sides”, Yacine Ould-Rouis added.
Efficiency of algorithms
Another example is coming from the GPU-Dynamo project, which had an allocation of 500,000 GPU hours on the hybrid partition of Curie, in the context of the 7th regular call of PRACE, for modelling Earth’s magnetic field and defining its origin. Conceived for being executed on graphic processors (or GPU meaning Graphics Processing Unit) made to boost calculations, the code was nevertheless confronted to scalability problems on the hybrid partition of Curie. With the help of Maxime Delorme (MdlS) and, for the project, of Alexandre Cameron, doctoral student at the Ecole Normale Supérieure under the supervision of Emmanuel Dormy, a first optimisation of the code permitted to increase the efficiency of algorithms by 15% but it was not sufficient regarding the high objectives of the project in terms of resolution. The second optimisation consisted in the implementation of a multi-grid solver, enabling the use of successive meshes of different sizes in order to get a detailed solution of the problem took into account. “The first tests showed that this new schema should allow us to reach the resolutions we are looking for”, Maxime Delorme explained. For Alexandre Cameron, « the collaboration with la Maison de la Simulation has been effective and useful”. Moreover, two papers are under preparation concerning this last optimisation.
As a result, this multigrid solver allowed to reduce by a factor of up to 2.5 when resolving Poisson’s equation for large size domains - compared with the results of the SOR solver initially used.
GPU-Dynamo project: impact on the overall performance of the new Poisson multi-grid solver versus the previous SOR solver initially used. © MdlS