Optimisation of the Input and Output (I/O) in DL_POLY_3
This dCSE concerned the optimisation and parallelisation of the I/O in the classical molecular dynamics (MD) package DL_POLY_3. This work now allows the code to exploit efficiently thousands of cores and to simulate systems with many millions of particles, both of which were previously not possible due to the overheads incurred by I/O.
DL_POLY_3 is a general purpose package for classical molecular dynamics simulations developed by I.T. Todorov and W. Smith at STFC Daresbury Laboratory. The main purpose of this software is to enable the exploitation of large scale MD simulations on multi-processor platforms.
The aim of the project was to improve the I/O performance so that real science could be performed on many hundreds of cores when simulating large systems. This optimisation involved large scale data rearrangements to enable both good use of the Lustre file system and also an efficient parallelisation of the I/O. The modular design also allowed it to be adapted very easily to use either MPI-I/O or NetCDF.
The outcomes of the project are:
- For sufficiently large systems the code does now scale to many thousands of cores, even when performing realistic amounts of I/O.
- All large I/O transactions are now performed in parallel.
- The writing of files is now up to 3 orders of magnitude faster than the original.
- The reading is up to two orders of magnitude faster.
- A good set of default parameters for the method have been found, and so most users need make no change to their input files to get good I/O performance (though there are options for expert users to tune it for their own needs).
- I/O based on both MPI-I/O (the default) and netCDF have been implemented.
- If using MPI-I/O the files are in exactly the same format as those written/read by the older method.
Please see PDF or HTML for a report which summarises this project.
