The HECToR Service is now closed and has been superceded by ARCHER.

Improving CONQUEST to allow ab initio molecular dynamics calculation on 100,000+ atoms

This is the fourth distributed Computational Science and Engineering (dCSE) project concerned with developing the ab initio Density Functional Theory (DFT) code Conquest on HECToR. Previous dCSE projects for developing Conquest are described here, here and here.

The overall aim of the project was to improve the linear scaling in Conquest by developing more efficient dynamic load balancing. At the begining of this project, the partitioning of the unit cell in Conquest was fixed, therefore each atom would remain assigned to the same partition during dynamics (whether molecular dynamics or geometry optimisation) regardless of its physical location. Therefore, this project will also address those situations when an atom moves out of its bundle, or occasionally its partition. In particular, the project will:

  • Improve the automated initial load assignment algorithm by developing a flexible non-cubic compact 3D Hilbert space-filling-curve for distributing the partitions of the simulation cell to processors.
  • Develop mechanisms for dynamic reassignment of simulation data, enable stable molecular dynamics (MD) simulations on Conquest; and to allow the reuse of simulation data from previous MD steps in the effort to reduce computational cost.

This work will have the following benefits to the scientific community:

  • It will open up new capabilities in Conquest, which already has excellent scaling properties, and will potentially allow the code to perform ab initio DFT molecular dynamics on systems exceeding 10,000 atoms or beyond for 2~5 ps on HECToR.
  • The ability to perform large scale ab initio quantum mechanical MD will potentially open up many exciting areas of research, especially in the study of biological processes, where the influence of the environment of different chemical units is important and cannot be modelled ab initio at present.
  • The facility to perform calculations on large numbers of atoms will also allow detailed comparisons to be performed between fully quantum mechanical and MD simulations on biochemical systems and combined quantum mechanics/molecular mechanics simulations which are the present route to understanding dynamical quantum effects in these systems.
  • For the wider computational community, the non-cubic compact space-filling-curve developed in this work can be easily transferred to other codes for efficient domain decomposition. Space filling curves are used by other codes (including ONETEP), and the non-cubic generalisation will be generally useful.

The individual achievements of the project are summarised below:

  • An algorithm for generating generalised non-cubic Hilbert space filling curve has been developed.
  • Improved automatic partitioning and load balancing algorithms based on the non-cubic Hilbert curve have been implemented in Conquest.
  • Tests on systems of various sizes (30,000+ atoms running on 1024+ nodes) and shapes (bulk, slab, columns and multiple-slabs etc.) have been demonstrated, and the new automatic partitioner gives superior performance in almost all aspects compared with the old partitioning system which was based on standard 3D Hilbert curves.
  • The ability for the user to manually control the partitioning process has been added. External utilities may no longer be required to define the manual partitioning maps.
  • The ability for Conquest to dynamically reassign simulation data has been added, this allows the code to perform stable MD simulations and reuse simulation data.
  • Extended Lagrangian Born-Oppenheimer MD has been implemented in Conquest.
  • The reuse of simulation data during MD simulations improved computation time by as much as 40%-90% (depending on the type of system), while the implementation of extended Lagrangian Born-Oppenheimer MD demonstrated the ability to perform stable simulations with loose energy tolerances (computationally cheap), paving the way for potential large scale quantum mechanical MD simulations in the future.
  • The developments from this project have been incorporated into the main Conquest source repository.

Please see PDF or HTML for a report which summarises this project.