Implementation

The implementation makes use of the Zoltan functions for load balancing and for data migration; Zoltan_LB_Balance and Zoltan_Migrate. There are also some calls to the auxiliary function for inverting send/receive lists, Zoltan_Invert_Lists. In this section an outline of the implementation will be given as well as some more details on how the implementation was structured.

The implementation follows the basic outline below:

• Setup module variables
• Set Zoltan parameters
• Register Zoltan callback functions
• Load balance the nodes making up the mesh
• Migrate owned nodes
• Construct list of halo nodes still needed
• Invert the list of halo nodes needed to get send list
• Migrate the halo nodes
• Construct new mesh and other data structures
• Allocate fields on the new mesh
• Transfer the fields data for each element on the new mesh
• Clean up module variables

All of these steps were implemented in three Fortran module files; Zoltan_integration,
Zoltan_callbacks and Zoltan_global_variables. The subroutine zoltan_drive from
Zoltan_integration implements the process outlined above and it is this routine which is called from elsewhere in Fluidity. Zoltan_callbacks contains all of the callback functions that are provided to Zoltan. Zoltan_global_variables is setup to contain various Fluidity variables and data structures which are shared between Zoltan_integration and Zoltan_callbacks. It was unfortunate that global variables were necessary but due to the callbacks required by Zoltan having a fixed interface this was the only way to allow access to the required Fluidity data structures in the callback functions.

The auxiliary function Zoltan_Invert_Lists is very useful as it allows each process to just build a list of nodes or elements it needs. The list each process creates is the receive list required by the Zoltan_Migrate function and through using Zoltan_Invert_Lists the send list which is also required for Zoltan_Migrate can be obtained.

It is not feasible to give details in this report of how each of these steps was implemented as many require understanding of the Fluidity data structures. Further details can be found in the well documented source code .