Conclusion and future work

The parallel scaling of the diagonalisation routines is not particularly impressive at about 40%, however on a machine with a lower CPU/network performance ratio than an Intel Xeon E5 with a QuickPath Interconnect, it would be expected to be better. But, what is most important to researchers is the wall clock time and this will not improve. The greatest speedup in this project was obtained from using the sparsity of the Hamiltonian, H, to enable a selective density matrix calculation with an efficient parallelisation, turning the O(N³) problem into a linear one. The other significant benefit was from the changes in storage and parallelisation of the structure constants generation. The size of the matrix of structure dependent constants, B, was previously a significant barrier to the amount of atoms simulated. This is now fully distributed and is therefore significantly smaller, particularly if there are only few heavy atoms in a system of predominantly light ones.

Even though the GPU platform results were far off the set target it is likely, as the phenomenal performance of the cublasDgemm shows, that in future versions the libraries could be improved. The exploration of the iterative methods for obtaining the density matrix led to plenty of experimentation and ideas for extentions to metals which are worth exploring in the future. During this project, contact was also made with two international groups, who have each developed their own $H\rightarrow \rho$ sparse libraries, one of them with a novel algorithm different than the classical approach implemented here. In view of the intrinsic limits of dense diagonalisation and the inefficiencies of its parallel implementations it may turn out that the direct iterative procedures are a better solution for simulating larger numbers of atoms in the tight binding framework.

In closing, the CUDA development was an interesting experience and one definitely worth having.