CP2K : High Performance Quantum Chemistry on HECToR
Iain Bethune, EPCC
Dr. Joost VandeVondele, University of Zurich
CP2K is a freely available, open source application which uses Density Functional Theory (DFT) to perform ab-initio quantum mechanical calculations on a variety of physical systems. It can be used to predict highly accurate molecular and crystalline structures and properties, and determine the evolution of systems using molecular dynamics, amongst many other features. Since its release in 2000, CP2K usage has grown dramatically, and this is expected to continue with the recent addition of support for hybrid functionals for DFT. CP2K is heavily used on HECToR - Dr. Ben Slater (co-proposer of the project) and his research group at University College London are using CP2K to study the properties of exotic phases of water ice. Dr. Carole Morrison of the University of Edinburgh is using CP2K to study proton transport in biological systems. In all, around 50,000 CPU hours are used each month on HECToR by CP2K.
During the year from August 2008 to July 2009, Iain Bethune, an Applications Consultant at EPCC, undertook detailed profiling and analysis of the code. Several optimisations and improvements were made, including restructuring communications in the key 'realspace to planewave' transformation which is central to CP2K's implementation of the Gaussian and Plane Waves (GPW) method. The computationally expensive parallel 3D Fourier transform was also modified to reduce parallelisation overheads, and to allow the used of advanced performance planning of the Fastest Fourier Transform in the West ( FFTW) library. As described earlier, CP2K can be used to study non-homogenous systems including biological molecules, and solid/liquid/gas boundaries, which exhibit poor load balancing on massively parallel supercomputers such as HECToR. Modifications were made to the existing load balancing algorithm to allow a better spread of work across all available processors. These improvements were able to be contributed back into the CP2K CVS repository immediately, after thorough testing with an automated suite of regression tests, which ensure the continued accuracy of the code.
At the end of the project, substantial performance improvements had been achieved - up to 30% on 256 cores for a small benchmark system of liquid water, and up to 300% on 1024 cores for a larger, non-homogenous system. These improvements are already available to users of HECToR and HPCx via the centrally installed versions of CP2K, and are available to CP2K users worldwide through the CVS repository on the CP2K website.
Following on from this project, another dCSE grant has been obtained to improve the scalability of the code further using a hybrid MPI/OpenMP approach. This will allow CP2K to take greater advantage of the increasingly multi-core nature of modern supercomputers, such as the recently installed quad-core AMD Opteron processors of HECToR.
Acknowledgements
This work was funded under a HECToR Distributed Computational Science and Engineering (dCSE) grant from NAG Ltd. We are grateful for the in-depth technical support and direction provided by Dr. Joost VandeVondele (University of Zurich), who contributed greatly to the success of the project.
Figure 1 (F. Schiffmann and J. VandeVondele). A snapshot from an ab-initio molecular dynamics simulation of an accurate model of the functionalized solid/liquid interface in dye sensitized solar cells. For this system (1300 atoms) 1 BO MD steps takes 1.2 min on 512 cores.
