Introduction to the nested model problem

WP2 of the dCSE project involves investigating the performance and ease of implementation of nested models within NEMO. The first step is therefore to compile and run a nested version of NEMO where a finer model is run inside the main (coarser) ocean model - e.g. a 1$^{\circ}$ region nested inside a 2$^{\circ}$ model. Any number of nested regions can be used and within each region multiple levels of nesting are also possible. Figure 10 illustrates this with an example showing the main model, A, containing two nested regions B, and C with region C having one level of nesting and region B having two levels of nesting.

Figure 10: Schematic showing a possible NEMO grid, A, containing two nested regions B and C. Nested region B contains a second level of nesting. Note that the schematic is not drawn to scale, in practice the B, C and D regions would be smaller in physical size and grid spacing than region A. However, their increased resolution may mean they actually contain similar number of actual grid points.

Nested models in NEMO use the Adaptive Grid Refinement in Fortran (AGRIF) pre-processor code called, conv, to generate the code for the nested regions. By default, the NEMO code uses arrays of dimension (jpi, jpj) where jpi and jpj are set as parameters within the code. The pre-processor completely re-structures the code by inserting interface routines which pass array information from a special dynamically allocated AGRIF data type the size of which is determined based on the desired size of the nested region. Essentially the AGRIF pre-processor allows the same ocean model to be run on grids with different resolutions in space and/or time. Further details on running and setting up nested models within NEMO can be found at [10].

Two test models are supplied by the NOCS team, BASIC and MERGED. BASIC is a largely unmodified version of the NEMO source code which attempts to use the AGRIF pre-processor to set up a nested model. The BASIC model is a 2 degree model inside which a 1 degree model is run. Only 1 nested region is used in the BASIC model. The MERGED model is an extension of BASIC model including the NOCS specific code changes. Unlike the BASIC model is has two nested regions. It also runs at a higher resolution with the outer model resolution being 1 degree and the nested regions being $\frac{1}{4}$ of a degree.

The nested models produce similar output files to the un-nested version of NEMO (see section 5 for details) with the name of the files pre-pended by 1_ for the nested version. For example the normal, un-nested run output file is called ocean.output and the corresponding run output from the nested region is called 1_ocean.output if one nested region is used and 2_ocean.output if two nested regions are used.

We have been unable to get the nested models fully running on HECToR. We have resolved the issues with the BASIC model but as yet the MERGED model does not run successfully on HECToR. We have worked extensively with Steven Alderson from NOCS to resolve this problem. Currently (and since NERC projects left HPCx) the NOCS team internal SGI cluster is the only system on which their version of the NEMO AGRIF MERGED model is running. This system uses the Intel compiler suite. Getting the MERGED model to run is a crucial part of WP2.