The HECToR Service is now closed and has been superceded by ARCHER.


This Distributed Computational Science and Engineering (dCSE) project was to improve the scalability of turbulence applications (principally the EBL code) from hundreds of cores to thousands of cores. This was necessary in order to exploit HECToR properly and to allow advances in the scientific investigations that use these applications.

EBL (named after the Ekman Boundary Layer) is used to investigate pressure driven flow over a single surface, located in the x-z plane. For this problem trigonometric functions form a good spectral basis with respect to the x and z direction which have periodic boundary conditions imposed on them. The y direction is normal to the fixed surface and Jacobi polynomials offer a good spectral basis with respect to the boundary conditions imposed in that direction. The application calculates the forcing terms in position space and performs the Runge-Kutta steps for the time evolution in the spectral space. The Reynolds number of a flow determines the size of the smallest domain that may be used to study it. This means that when the EBL code is run there is a minimum size of lattice that may be used and that the lattice whilst cuboidal is not a cube.

  • The aim of this project was to demonstrate that, by employing two dimensional decompositions of the problem domain and despite the increased communication overhead, three dimensional spectral codes can be made to scale from hundreds of cores to thousands of cores. The spectral code EBL was used to implement the improved decomposition and results demonstrate that near perfect scalability can now be achieved on up to 14,000 cores whereas before this project this was only 365 cores.

The individual achievements of the project are summarised below:

  • The existing EBL code has been re-engineered such that for a problem size that previously would have used a maximum of 365 cores, good scaling can now be acheived for up to 14,000 cores.
  • EBL can now run with much larger problem sizes and with higher Reynolds numbers due to the two-dimensional domain decomposition.
  • This work is transferable to other 3 dimensional spectral codes, e.g. SWT and SS3F.
  • The new version of the EBL code has been tested on the XT6 for HECToR Phase 2b and it is available as the module “ebl”.

Please see PDF or HTML for a report which summarises this project.