The HECToR Service is now closed and has been superceded by ARCHER.

Scalable coupling of Molecular Dynamics and Direct Numerical Simulation of multi-scale Flows Part 2

This project concerns two distinct computer codes on HECToR and their combined abilities to solve fluid flow problems across a wide range of physical scales. The first algorithm is a continuum-level, Direct Numerical Simulation (DNS) technique, which evolves the large-scale, macroscopic features in the flow by solving the non-linear Navier-Stokes equations (Transflow). The second algorithm is a Molecular Dynamics (MD) description of the atomic-level behaviour near flow boundaries (StreamMD). This work follows on from the previous project which resulted in a successful implementation of the Continuum-to-Molecular Coupling and Embedding (CMCe) module which now interfaces independent applications for simulations over the molecular and continuum scales.

The main objective of this project is to improve the performance of this interface to enable highly-scalable simulations on HECToR. This will be achieved by:

  • Developing improved load balancing of the coupled DNS and MD application, by an appropriate division of compute resources between the various algorithms.
  • Implementing both non-blocking MPI communication and mixed-mode operation (OpenMP + MPI) in the coupler.
  • Developing the facility for "MD farming", where MD instances are generated at different locations in the continuum.

The overall outcome of this work may be summarised as follows:

  • The original CMCe module was re-engineered such that the CFD (Transflow) and MD (StreamMD) applications may be run in Multiple Program Multiple Data (MPMD) mode. During simulations, data exchange is now performed via calls to the CMCe module from Transflow and StreamMD.
  • Both Transflow and StreamMD were optimised for more efficient load balancing and non-blocking MPI calls were also implemented.
  • The weak scalability of TransFlow for a test case with a million grid points per MPI task can now achieve 90% parallel efficiency on HECToR and StreamMD with 3.3 million molecules can achieve 94% parallel efficiency. Therefore, overall scalability with the CMCe module is excellent for coupled DNS and MD simulations with millions of grid points and molecules, using several thousand of cores on HECToR.
  • The re-engineered CMCe module also ensures that any number of MD instances can interface to the CFD (continuum) solver, facilitating "MD farming".

Please see PDF or HTML for a report which summarises this project.