Parallel Adaptivity

The adaptivity method used in Fluidity as described in the previous section (2.2) is inherently a serial process. Parallelising the method itself across multiple MPI processes would lead to a large amount of complexity and interprocess communication. A review of methods is given in [Farrell(2009)]. Instead the following approach is taken (Fig 3):

This approach uses the serial adaptivity method previously described to adapt all the elements which are not in a halo. Those elements in a halo are also resident on another process so if they were adapted the same adaption would be needed on the remote process requiring communication. By locking the halo elements this communication is avoided but after the local adapt step the locked elements will not have been adapted and hence may be of poor quality. By re-partitioning with high edge-weights applied to the poor quality elements the partitioner should attempt to not cut those edges and hence the poor quality elements will be in the centre of a domain and can be adapted during the next local adapt iteration. The current load re-balancing is carried out by bespoke code, Sam [Gorman(2006)]. Sam only works with piecewise linear elements and does not allow detectors or periodicity within the domain. It is a hard-coded solution, such that any new element pairs would need bespoke code written in order to perform the migration of data. However, it does have optimisations that are not present in Zoltan, such as the migration of element and field data simultaneously.

Figure 3: 2D mesh example where the graph partition is indicated by the broken wavy line: (right) sub-optimal restricted elements (hatched) forms a polyline; (centre) after repartitioning only isolated elements remain locked; (left) all elements have been visited.

Jon Hill 2012-03-02