5. Conclusions

This chapter concludes the work undertaken under the dCSE project ``Multigrid Improvements to CITCOM'' presented in this report. A brief summary of the work undertaken in each of the three phases of this project is described in section 5.1 followed by achievements in section 5.2. This chapter is concluded with some recommendations aimed at further improvements to the CITCOM package.

5.1 Summary

After the introduction of this dCSE project in Chapter 1 describing the project duration, work plan and background of the CITCOM package, Chapter 2 describes the initial study undertaken towards understanding the CITCOM package and learning. Chapter 3 is dedicated to the description of Multigrid methods, a model problem, test problems, computational results and analysis of the results for representative test problems in two and three dimensions. These results are obtained using four multigrid schemes showing excellent scaling for each multigrid scheme. This is followed by a description of post processing tools used in preparing these results. Local mesh refinement strategy, setup and implementation followed by the outcome of this implementation is presented in Chapter 4. Chapter 5 concludes this project and report.

5.2 Achievements

The work carried out as part of this dCSE project enabled CITCOM to achieve faster convergence. For the best cases CITCOM performs over 31% faster for the V-cycle multigrid scheme, over 38% faster for the W-cycle multigrid scheme in comparison to the corresponding FMG(V) and FMG(W) schemes respectively for the simple 2D test problem and over 12% faster for the W-cycle multigrid scheme in comparison to the corresponding FMG(W) scheme for the complex 3D test problem.

Other observations based on the analysis of the four multigrid schemes along with tests problems and their results, in two and three dimensions, given in chapter 3 are summarised below. These observations account for the four multigrid schemes, namely, Multigrid V-cycle, Multigrid W-cycle, FMG(V) and FMG(W).

• For relatively simple and fast converging problems the Multigrid V-cycle scheme is the fastest scheme.

• Full Multigrid or FMG schemes perform well in contrast to the corresponding basic Multigrid V-cycle and Multigrid W-cycle schemes for complex and hard to solve problems. In these cases, basic schemes, namely, the Multigrid V-cycle and Multigrid W-cycle might fail to converge.

• V-cycle based multigrid schemes generally perform better than the corresponding W-cycle based multigrid schemes.

• The Multigrid V-cycle scheme offers optimal choice for relatively simple and easy to solve problems and the FMG(V) scheme offers optimal choice for relatively complex and hard to solve problems.

• Scaling of all multigrid schemes is generally excellent. This is particularly true if the sub problem size per MPI process is not reduced to the extremely smallest possible size, that is, just two elements per MPI process in any direction.

• The use of one or two cores per node instead of all four cores per node may slightly affect scaling, that is, the use of all cores per node gives best scaling. This is encouraging in the context of the efficient usage of multi-core configurations.

Local mesh refinement, within the existing framework of CITCOM, is a difficult option to try. In the presence of the high level of complexity involved in the refinement of a regularly structured grid in two dimensions generally, and in three dimensions particularly, together with the complex structure of CITCOM itself, we have managed to achieve partial success under these hard circumstances.

CITCOM solves time dependent problems but can be restricted to a 0 (zero) time step only. In the later case, it can achieve a solution for a few small test problems (smaller versions of the test problems used in phase 2). We have tried this approach but it takes more time than expected after local mesh refinement. It is thought to be the case as a result of the local mesh refinement. Here, there are elements of two different sizes which interface with each other without any smooth transition. In the former case of time dependence, computations start deteriorating after only a few time steps. This behaviour is not understood but suspicion is that the advection-diffusion related computation may be the influencing factor. On the other hand, it is also possible that the non-multigrid part of the code, which performs advection-diffusion and tracers related computations may be in need of some more modifications to adjust for the local mesh refinement.

5.3 Recommendations

This package has a potential for further improvements and a few of numerous possible ways in which it can be improved and extended are suggested here.

• Local mesh refinement needs to be addressed in the context of temperature field computations addressing advection-diffusion and convection related issues.

• Implementation of tracers needs to be re-visited in order of address the changes in the number of nodes and elements in the z-direction due to local mesh refinement in comparison to the global refinement to achieve the nested hierarchy of mesh levels.

• To get acceptable results for larger problems CITCOM requires a longer run time than the maximum (12 hrs) allowed on HECToR. The current option is to restart the CITCOM computation at the point of termination from last run where it had written information to the output files essential for restarting. However, a vast amount of computation is performed again after restarting in some cases this may take up to one-third of the total run time after restart. If this can be modified to write the full information to the output files required for restarting without the need of repeating part of the computations, it could save a big chunk of time which is otherwise wasted in performing the same computation after each restart.