Using CDFTOOLS for verification of NEMO output

Once a single netCDF file has been created as described in Section  5.2 the data can be compared against vanilla (i.e. before any changes have been made to the code) output using the CDFTOOLS package. CDFTOOLS is package of Fortran 90 programs and libraries for performing diagnostic tests on NEMO output. Further details on CDFTOOLS and the individual tools can be found at [5]. One field from each of the 5 output grids (restart files are ignored) is compared against the vanilla output. The grids and fields compared are given below:

Grid        Field       Physical property
GRID T      votemper  = temperature (C)
GRID U      vozocrtx  = zonal current (m/s)
GRID V      vomecrty  = meridional current (m/s)
GRID W      vovecrtz  = vertical velocity (m/s)
GRID icemod isssalin  = sea surface salinity (PSU)

Comparing a global single field from each of the main output datasets should be sufficient verification that the modified code gives correct results. It would be excessive to compare every single field, besides which the length of time required to run the verification would become prohibitive.

It should be noted that the NEMO output files need to contain some actual data. Running with the default namelist settings specifies that data will be output every 300 time steps which is fine for production runs. However, as our test run is only for 60 time steps this means that no real data is actually written out and the CDFTOOLS codes will crash. Sample error output for such a crash is below:

 ~/CDFTOOLS/bin/cdfmeanvar votemper_ncdf3.nc votemper T
 npiglo=         1442
 npjglo=         1021
 npk   =           64
 nvpk  =           64
  Problem in getvar for votemper
 ERROR in NETCDF routine, status=          -40
 NetCDF: Index exceeds dimension bound                                           
FORTRAN STOP

Thus, to ensure that data is written out the output frequency must be changed from 300 to 30 time steps. This is achieved by changing the value of nwrite in the namelist file.

Then, the procedure for verifying that the NEMO output is sensible is as follows:-

1. For each dataset (i.e. vanilla and the one to be tested) combine the processor specific output files into a single netCDF file using nocscombine as described in Section  5.2.
2. Inside the particular experiment directory, create soft links to several map files mask.nc, new_maskglo.nc, mesh_zgr.nc and mesh_hgr.nc which are required by the CDFTOOLS, e.g.

   ln -s /work/n01/n01/acc/DATA/ORCA025/mask.nc        mask.nc
   ln -s /work/n01/n01/acc/DATA/ORCA025/new_maskglo.nc new_maskglo.nc
   ln -s /work/n01/n01/acc/DATA/ORCA025/mesh_zgr.nc    mesh_zgr.nc
   ln -s /work/n01/n01/acc/DATA/ORCA025/mesh_hgr.nc    mesh_hgr.nc
   

   If you forget to create these soft links the following somewhat cryptic error is obtained:-

   fionanem@nid15875:
   /work/n01/n01/fionanem/NEMO_V3.0/ORCA025/EXP_V3.0_005> 
   ~/CDFTOOLS/bin/cdfmeanvar votemper_ncdf3.nc votemper T
    npiglo=         1442
    npjglo=         1021
    npk   =           64
    nvpk  =           64
    ERROR in NETCDF routine, status=            2
    No such file or directory                                                       
   FORTRAN STOP
   

3. For each dataset compute the spatial 3D mean (i.e. over all depth levels) temperature and variance using the cdfmeanvar tool as follows:-

     cdfmeanvar vanilla_inputfile.nc votemper T > vanilla_output_votemper_T.txt
     cdfmeanvar test_inputfile.nc votemper T > test_output_votemper_T.txt
   

4. Compare the two output files using diff, xxdiff or similar. Ideally they should be identical or any differences should be explainable.
5. Repeat steps 1-4 for each grid and field to be tested.

As this verification will be necessary following any changes to the code it is sensible to have some scripts to perform it. Such scripts can be found in Appendix A. Essentially the verification scripts compare the NEMO output to vanilla output and note any differences. The verification takes around 20-40 minutes depending on the number of parameters being tested and thus must be run on the serial queue to avoid excessive use of the login node resources.