The HECToR Service is now closed and has been superceded by ARCHER.

Optimising the performance of the VASP code on HECToR

The VASP code is used for performing ab initio quantum mechanical total energy calculations and molecular dynamics (MD) simulations with pseudopotentials and a plane wave basis set. VASP employs a finite-temperature local-density approximation and an exact evaluation of the instantaneous electronic ground state at each MD-step. VASP is one of the most widely used applications on HECToR. This dCSE project focussed on two separate areas, namely optimisation on Phase 2a/b of HECToR for VASP 5.2.2 and the introduction of a new level of parallelism to the code, namely parallelisation over k-points.

The key aims of the project are:

1. Optimise the parallel performance of VASP

  • Determine the best values of the parameter NPAR.
  • Optimise the shared memory collective communication.
  • Introduce ScaLAPACK for the key linear algebra routines.
  • Validate against a set of representative test cases.

2. Implement k-point parallelism within VASP

  • Develop code for a single energy point calculation on non spin polarized systems.
  • Parallelisation over spin polarized systems will be added and then implemented within the force calculation.
  • Validate against a set of representative test cases which are suitable to exploit k-point parallelism.

The individual achievements of the project are summarised below:

1. For the optimised parallel performance of VASP 5.2.2

  • Recommended settings for NPAR were documented.
  • Preprocessing directives have been developed for those users who wish to compile their own version of VASP 5.2.2. These include improved collective communications and the elimination of unnecessary zeroing of certain arrays. Furthermore, a shared memory approach to aggregate intra node messages for communication is also available via these directives.
  • These performance improvements now enable scalability to more than 128 processing cores for certain systems.

For more information about these developments for VASP 5.2.2 on HECToR, please see PDF or HTML for a report which summarises this part of the project.

2. K-point parallelism was added to VASP 5.2.2

  • An extra level of parallelism was implemented by the addition of k-point parallelisation whereby a group of cores may now be configured to perform calculations for a subset of k-points.
  • The number of such groups is specified by the new KPAR tag, which is set in the standard INCAR input file.
  • The k-point parallelism was implemented for hybrid functionals rather than the originally planned spin polarized systems as, after consultation with the UK's HPC Materials Chemistry Consortium (MCC), this was considered more useful to HECToR users.
  • K-point parallelism can be used to give a speed up of at least 2 times relative to the original code on the same core count with good scalability on the XE6. In particular for a unit cell of Litharge (α-PbO) with a total of 4 atoms using 108 k-points, the new code runs more than 5 time more quickly than the original when using more than 1600 cores.
  • Although k-point parallelism is not applicable to all systems, there are many cases where parallelisation over k-points is a useful technique. This is especially true when the total energy calculation is only one part of a larger calculation, for instance in a geometry optimisation.
  • The k-point parallel version of VASP 5.2.2 is available on HECToR via module load vasp5/5.2.2_kpoint
  • Work is in progress to release the source code to the MCC, and also to feed it back to the original authors of the code with the hope that the new feature will be included in future releases of VASP.

For more information about the k-point parallelism which is now available in VASP 5.2.2 on HECToR, please see PDF or HTML for a report which summarises this part of the project.