Compilation

We employed the following serial batch script to compile

#!/bin/bash --login
#PBS -q serial
#PBS -N compile_OF
#PBS -l walltime=05:00:00
#PBS -A y07
cat $0
. /opt/modules/3.1.6/init/bash
module swap PrgEnv-pgi PrgEnv-gnu
#following line makes serial codes may break on front end dual core nodes.
#module load xtpe-barcelona
module swap gcc gcc/4.3.3
module swap xt-mpt xt-mpt/3.2.0
cd /work/y07/y07/openfoam/dual-core/OpenFOAM/OpenFOAM-1.5
source etc/bashrc
export LD_LIBRARY_PATH=/work/y07/y07/openfoam/dual-core/OpenFOAM/
                        OpenFOAM-1.5/mylib:$LD_LIBRARY_PATH
./Allwmake >& make.out

This fails after around four hours with

make[3]: *** [../OpenFOAM/OpenFOAM-1.5/lib/crayxtDPOpt/libuserd-foam.so] Error 1

This is resolved by editing

../OpenFOAM/OpenFOAM-1.5/wmake/rules/crayxt/general

and replacing

PROJECT_LIBS = -l$(WM_PROJECT) -Wl,--whole-archive -L$(SE_DIR)/lib/snos64 //
               -lportals -Wl,--no-whole-archive -liberty -ldl

with

PROJECT_LIBS = -l$(WM_PROJECT) -liberty -ldl

then remaking. We then reinstate this edited line in general and remake once more.

This workaround is necessary due to the OS Cray XT4 CSE 2.1 (as it is now) does not ship the necessary dynamic libraries. This will be fixed in CSE 2.2.

Note, that there are some `portals' warnings (Cray builds its MPI using `portals'), but we can ignore them, i.e. they appear as mylib is not in LD_LIBRARY_PATH. They are, however, not required as we are link in the static portals code. There are a number of dummy libraries in mylib is just to satisfy runtime requirements, including a dummy portal library stub.