Open-Source GPU Technology for Supercomputers: Researchers Navigate Advantages and Disadvantages
Researchers from the HSE International Laboratory for Supercomputer Atomistic Modelling and Multi-scale Analysis, JIHT RAS and MIPT have compared the performance of popular molecular modelling programs on GPU accelerators produced by AMD and Nvidia. In a paper published by the International Journal of High Performance Computing Applications, the scholars ported LAMMPS on the new open-source GPU technology, AMD HIP, for the first time.
The scholars thoroughly analysed the performance of three molecular modelling programs – LAMMPS, Gromacs and OpenMM – on GPU accelerators Nvidia and AMD with comparable peak parameters. For the tests, they used the model of ApoA1 (Apolipoprotein A1) — apolipoprotein in blood plasma, the main carrier protein of ‘good cholesterol’. They found that the performance of research calculations is influenced not only by hardware parameters, but also by software environment. It turned out that ineffective performance of AMD drivers in complicated scenarios of parallel launch of computing kernels can lead to considerable delays. Open-source solutions still have their disadvantages.
In the recently published paper, the researchers were the first to port LAMMPS on a new open-source GPU technology, AMD HIP. This developing technology looks very promising since it helps effectively use one code both on Nvidia accelerators and on new GPUs by AMD. The developed LAMMPS modification has been published as an open source and is available in the official repository: users from all over the world can use it to accelerate their calculations.
Vsevolod Nikolskiy, HSE University doctoral student and one of the paper’s authors
We thoroughly analysed and compared the GPU accelerator memory sub-systems of Nvidia Volta and AMD Vega20 architectures. I found a difference in the logics of parallel launch of GPU kernels and demonstrated it by visualizing the program profiles. Both the memory bandwidth and the latencies of different levels of GPU memory hierarchy as well as the effective parallel execution of GPU kernels — all these aspects have a major impact on the real performance of GPU programs.
The paper’s authors argue that participation in the technological race of the contemporary microelectronics giants demonstrates an obvious trend toward greater variety of GPU acceleration technologies.
Vladimir Stegailov, HSE University professor
On the one hand, this fact is positive for end users, since it stimulates competition, growing effectiveness and the decreasing cost of supercomputers. On the other hand, it will be even more difficult to develop effective programs due to the need to consider the availability of several different types of GPU architectures and programming technologies. Even supporting program portability for ordinary processors on different architectures (x86, Arm, POWER) is often complicated. Portability of programs between different GPU platforms is a much more complicated issue. The open-source paradigm eliminates many barriers and helps the developers of big and complicated supercomputer software.
In 2020, the market for graphic accelerators experienced a growing deficit. The popular areas of their use are well-known: cryptocurrency mining and machine learning tasks. Meanwhile, scientific research also requires GPU accelerators for mathematical modelling of new materials and biological molecules.
Nikolay Kondratyuk, researcher at HSE University and one of the paper’s authors
Creating powerful supercomputers and developing fast and effective programs is how tools are prepared for solving the most complex global challenges, such as the COVID-19 pandemic. Computation tools for molecular modelling are used globally today to search for ways to fight the virus.
The most important programs for mathematical modelling are developed by international teams and scholars from dozens of institutions. Development is carried out within the open-source paradigm and under free licenses. The competition of two contemporary microelectronics giants, Nvidia and AMD, has led to the emergence of a new open-source infrastructure for GPU accelerators’ programming, AMD ROCm. The open-source character of this platform gives hope for maximum portability of codes developed with its use, to supercomputers of various types. Such AMD strategy is different from Nvidia’s approach, whose CUDA technology is a closed standard.
It did not take long to see the response from the academic community. Projects of the largest new supercomputers based on AMD GPU accelerators are close to completion. The Lumi supercomputer in Finland with 0.5 exaFLOPS of performance (which is similar to performance of 1,500,000 laptops!) is quickly being built. This year, a more powerful supercomputer, Frontier, is expected in the USA (1.5 exaFLOPS), and in 2023 – an even more powerful El Capitan (2 exaFLOPS) is expected.
Nikolay Kondratyuk
Research Fellow, International Laboratory for Supercomputer Atomistic Modelling and Multi-scale Analysis
Vsevolod Nikolskiy
Research Fellow, International Laboratory for Supercomputer Atomistic Modelling and Multi-scale Analysis
Vladimir Stegailov
Leading Research Fellow, International Laboratory for Supercomputer Atomistic Modelling and Multi-scale Analysis
See also:
Scientists Model Turbulence at Atomic Level
Scientists at HSE University and MIPT have developed a supercomputer-based method to model fluid flows at atomistic scales making it possible to describe the emergence of turbulence. The researchers used the supercomputers cHARISMa and Desmos to compute the flow of a fluid consisting of several hundred million atoms. This method is already being used to simulate the flow of liquid-metal lead coolant in a nuclear reactor. The paper has been published in The International Journal of High Performance Computing Applications.
HSE cHARISMa Supercomputer Completes One Million Tasks
Since 2019, the cHARISMa supercomputer has been helping staff, teachers and students of HSE university to solve research tasks. In February 2023, it completed its millionth task—a computational experiment dedicated to studying the phenomenon of multiparticle localisation in quasi-one-dimensional quantum systems.
From Covid-19 to Risk Appetite: How HSE University’s Supercomputer Helps Researchers
Whether researching how the human brain works, identifying the source of COVID-19, running complex calculations or testing scientific hypotheses, supercomputers can help us solve the most complex tasks. One of the most powerful supercomputers in the CIS is cHARISMa, which is now in its third year of operation at HSE University. Pavel Kostenetskiy, Head of the HSE University Supercomputer Modeling Unit, talks about how the supercomputer works and what kind of projects it works on.
HSE Supercomputer Doubles Its Performance
The peak performance of the HSE cHARISMa supercomputer has doubled, reaching 2 petaflops (2 quadrillion floating-point operations per second). HSE University now outperforms the Kurchatov Institute in terms of computing power. The only more powerful university computers are MSU’s Lomonosov-2 and SPbPU’s Polytechnic. Thanks to the timely upgrade, cHARISMa has retained its respectable 6th position among the Top 50 most powerful computer systems in the CIS for three years.
HSE Supercomputer Is Named cHARISMa
In July this year, there was an open vote to name the HSE’s supercomputer. Two names - Corvus (‘crow’ in Latin; the crow is HSE's mascot) and cHARISMa (Computer of HSE for Artificial Intelligence and Supercomputer Modelling) – received the most votes. The latter won by a narrow margin, with 441 people (one in three of those who took part in the vote) choosing this name.
Supercomputer Set Up at HSE University
A new supercomputer, which has been recently set up at MIEM, will allow the university to carry out high quality research in deep learning and mathematical modeling. The computer was ranked sixth in the April Top-50 ranking of supercomputers in Russia.