H5HOME = /work/n01/n01/fionanem/local/noparallel ZLIBHOME = /work/n01/n01/fionanem/local NCDF_INC = /work/n01/n01/fionanem/local/noparallel/include \ -I$(H5HOME)/include -I$(ZLIBHOME)/include NCDF_LIB = -L/work/n01/n01/fionanem/local/noparallel/lib \ -lnetcdf -L$(H5HOME)/lib -lhdf5 -lhdf5_hl -L$(ZLIBHOME)/lib \ -lz -lsz -L$(H5HOME)/lib -lhdf5After re-compilation the code is executed and the output verified as being correct using the method described in section 5.3.
The procedure above uses netCDF 4.0 in classic mode and is the first step in converting NEMO to use netCDF 4.0. However, what we actually want to do is to convert NEMO to use netCDF 4.0 with compression and chunking enabled. This is best tackled as a two stage process. The first step is to generate netCDF 4.0 format output files without any compression/chunking. The second step is to add chunking and/or compression and thus take advantage of the full functionality of netCDF 4.0.
For step one we need to tell netCDF that we want to take advantage of the new features. This can be achieved by making some minor modifications to the NEMO code. In particular, all subroutine calls to NF90_CREATE need to be modified such that each instance of NF90_NOCLOBBER is replaced with NF90_HDF5. The two source files which require alteration are:-
i_rc = NF90_CREATE(f_nw,m_c,f_e)with
m_c = NF90_HDF5 i_rc = NF90_CREATE(f_nw,m_c,f_e)With these modifications the code is then recompiled and tested. The main output files should now be in netCDF 4.0 format which can be verified by attempting to read one of the output files with versions of ncdump 1which have been compiled with netCDF 3.6.2 and with netCDF 4.0, e.g.
The modifications described above allow netCDF 4.0 format files to be output from NEMO. However, as yet no compression or chunking has been included. To use the chunking/compression features of netCDF 4.0 additional modifications must be made to the code. These modifications are outlined below:
INTEGER, dimension(4) :: chunksizes INTEGER :: chunkalg, shuffle, deflate, deflate_levelchunksizes is an array containing the chunksize to be used for each dimension of the dataset.
chunksizes(1) = 10 chunksizes(2) = 10 chunksizes(3) = 10 chunksizes(4) = 1 deflate_level = 1 ! Turn compression on deflate = 1 ! Level of compression shuffle = 1 ! Allow shuffling chunkalg = 0 ! Turn chunking onHere chunksize is chosen such that it is less than the number of data points within that dimension. It should be noted that for three-dimensional fields such as the ice data (i.e. *icemod*.nc files) this may not be ideal.
iret = NF90_DEF_VAR (ncid,lon_name,NF90_FLOAT,dims(1:ndim),nlonid)with the modified code example including the following two lines immediately after the call to NF90_DEF_VAR, e.g.
iret = NF90_DEF_VAR_CHUNKING(ncid,nlonid,chunkalg,chunksizes) iret = NF90_DEF_VAR_DEFLATE(ncid,nlonid,shuffle,deflate, deflate_level)
The code was then re-tested and the size of the output files compared with those created without and chunking/compression. For the original code some sample output file sizes were as follows:
ls -l *0100.nc -rw-r--r-- 1 fionanem n01 47364120 Apr 17 09:40 O25-V3_00000060_restart_0100.nc -rw-r--r-- 1 fionanem n01 2286160 Apr 17 09:40 O25-V3_00000060_restart_ice_0100.nc -rw-r--r-- 1 fionanem n01 7102636 Apr 17 09:40 O25-V3_CU30_19580101_19580101_grid_T_0100.nc -rw-r--r-- 1 fionanem n01 3246008 Apr 17 09:40 O25-V3_CU30_19580101_19580101_grid_U_0100.nc -rw-r--r-- 1 fionanem n01 3246013 Apr 17 09:40 O25-V3_CU30_19580101_19580101_grid_V_0100.nc -rw-r--r-- 1 fionanem n01 15709982 Apr 17 09:40 O25-V3_CU30_19580101_19580101_grid_W_0100.nc -rw-r--r-- 1 fionanem n01 1118067 Apr 17 09:40 O25-V3_CU30_19580101_19580101_icemod_0100.ncFor the modified version the corresponding files sizes were:
ls -l *0100.nc -rw-r--r-- 1 fionanem n01 47364120 Apr 21 13:01 O25-V3_00000060_restart_0100.nc -rw-r--r-- 1 fionanem n01 2286160 Apr 21 13:01 O25-V3_00000060_restart_ice_0100.nc -rw-r--r-- 1 fionanem n01 1765659 Apr 21 13:01 O25-V3_CU30_19580101_19580101_grid_T_0100.nc -rw-r--r-- 1 fionanem n01 932602 Apr 21 13:01 O25-V3_CU30_19580101_19580101_grid_U_0100.nc -rw-r--r-- 1 fionanem n01 939433 Apr 21 13:01 O25-V3_CU30_19580101_19580101_grid_V_0100.nc -rw-r--r-- 1 fionanem n01 2569717 Apr 21 13:01 O25-V3_CU30_19580101_19580101_grid_W_0100.nc -rw-r--r-- 1 fionanem n01 536500 Apr 21 13:01 O25-V3_CU30_19580101_19580101_icemod_0100.ncComparing the output file sizes we see that the four *grid*.nc and *icemod*.nc files have reduced in size by up to 3.55 times relative to the original data. The two restart files are not affected as no compression/chunking has been applied to these. The overall effect of the chunking/compression for the test model is summarised by table 11.
The results presented in table 11 demonstrate that a
significant reduction in disk usage (
4 Gbytes for this example) can be
achieved by using the chunking and or compression features of netCDF 4.0
within NEMO. For the test model, no significant improvement in the runtime
is observed however this is to
be expected as the restart files dominate in terms of their size. In our
test model we run for 60 time steps, outputting data every 30 time steps
with a restart model written output after 60 time steps. However, for a
production run of NEMO the model would typically run for in excess of
10,000 time steps with output every 300 steps and a restart file written
every 1800 time steps. Therefore an improvement in runtime would be
expected due to the reduction in output file size. The actual size of
any improvement will depend on the output frequency chosen, which in
turn depends on the parameters being modelled and the particular science
studied.
Conversion of the restart files to netCDF 4.0 and a full investigation of the optimal chunksize/compression parameters were not possible due to a shift in focus requested by the researchers where we were asked to concentrate on the nested AGRIF models.