HDF5 Filters

There are a number of different filters which can be applied to an HDF5 dataset:

• N-bit filter - essentially compresses the data by removing the unused bits before storing the data on output. The data is then unpacked on input restoring the missing bits. Quite complex to use but may save diskspace. Checks would be required such that no information is lost from the data.
• Scale-offset filter - performs a scale and offset on each data value truncating the resulting value to a fixed number of bits before storing. E.g the operation performed is $d_{new} = a*d_{old}(i) + b$ where $d_{old}$ is the original data value, $d_{new}$ is the new data value, $a$ is the scale and $b$ is the offset. minimum-bits determines the minimum number of bits that will be used. For integer data the filter is lossless (unless too small a value for minimum-bits is selected. However, for floating point data, the filter translates the floating point data to integer data (the filter is lossy - information is lost due to its action) and so is not useful for NEMO.
• Szip filter - the Szip compression