Implementation of k-point parallelism

In the original code the tag NPAR is used to create intra-band and inter-band communicators. MPI processes are bound to an intra-band communicator to work on specific bands and when information is required about the other bands, MPI processes communicate through the inter-band communicators. To exploit k-point parallelism, I constructed an extra layer of communication using the value of KPAR. KPAR intra-k-point communicators (named COMM) and MPI processes/KPAR inter-k-point communicators (named COMM_CHAIN_K) are created. Each COMM communicator has all the information needed for the k-points it is assigned. Subsequently, as before the MPI processes in the COMM communicator are divided into MPI processes bounded to inter- and intra-band communicators. When a loop over k-points is encountered, the processes in each COMM communicator iterate through all the k-points they are responsible for and as soon the loop exits the COMM_CHAIN_K communicators are used to gather together the results. This is the general design followed, although the main difficulties at the development process arose at places where information from all k-points was required e.g., for implementing the linear tetrahedron method in the evaluation of the Fermi level etc.

Also extra care was needed when a full, non-symmetrized k-grid was to be employed, as some particular quantities have to be stored in all communicators, even when they are referring to k-points the communicator is not responsible for. The cases where the Hartee-Fock exchange is used were further complicated as they involve nested k-loops. Extra communication is needed there (compared to the non-Hartee Fock) cases, but as seen in the test cases () and () this does not hinder the scaling of the code, at least for the cases studied.

In regards with I/O, depending on the case I have chosen either to communicate all relevant information on the master node, or have a `leader' in each COMM communicator to do the writing (using a loop so that information was written in orderly form).