Compiling Gromacs 4.6.x on HECToR Phase 3
GROMACS compilation consists of two steps:
- Compiling the serial tools (for the service nodes).
- Compiling the parallel MD simulation program (for the compute nodes).
Build the serial tools for the frontend nodes
Set up your environment for the serial build
Switch to the login node architecture:
module swap craype-interlagos craype-istanbul
Switch to the GNU programming environment:
module swap PrgEnv-cray PrgEnv-gnu
and load the FFTW 3 and CMake modules:
module add fftw module add CMake
Module list for centrally installed serial executables:
Currently Loaded Modulefiles: 1) modules/3.2.6.7 2) nodestat/2.2-1.0400.31264.2.5.gem 3) sdb/1.0-1.0400.32124.7.19.gem 4) MySQL/5.0.64-1.0000.5053.22.1 5) lustre-cray_gem_s/1.8.6_2.6.32.45_0.3.2_1.0400.6453.7.1-1.0400.39662.3.1 6) udreg/2.3.1-1.0400.4264.3.1.gem 7) ugni/2.3-1.0400.4374.4.88.gem 8) gni-headers/2.1-1.0400.4351.3.1.gem 9) dmapp/3.2.1-1.0400.4255.2.159.gem 10) xpmem/0.1-2.0400.31280.3.1.gem 11) hss-llm/6.0.0 12) Base-opts/1.0.2-1.0400.31284.2.2.gem 13) xtpe-network-gemini 14) PrgEnv-gnu/4.0.46 15) atp/1.6.2 16) xt-asyncpe/5.20 17) pmi/4.0.0-1.0000.9282.69.4.gem 18) xt-libsci/12.0.02 19) xt-totalview/8.11.0 20) totalview-support/1.1.4 21) gcc/4.7.2 22) pbs/10.2.2.113537 23) packages-phase3 24) bolt/0.5 25) budgets/1.0 26) craype-istanbul 27) cray-mpich2/5.6.5 28) CMake/2.8.2 29) fftw/3.3.0.2
Configure the single-precision build
Create a build directory in the source tree
cd gromacs-4.6.1 mkdir build cd build
Set the environment variables for the CMake build (note use of native compiler commands):
export CXX=CC export CC=cc export CMAKE_PREFIX_PATH="/opt/fftw/3.3.0.2/x86_64:/opt/gcc/4.7.2/snos/lib" export FLAGS="-static -O3 -ftree-vectorize -ffast-math -funroll-loops"
Use CMake to configure the build:
cmake ../ -DCMAKE_SKIP_RPATH=YES -DBUILD_SHARED_LIBS=OFF \ -DGMX_MPI=OFF -DGMX_OPENMP=ON -DGMX_GPU=OFF -DGMX_X11=OFF -DGMX_DOUBLE=OFF \ -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \ -DCMAKE_INSTALL_PREFIX=/usr/local/packages/gmx/4.6.1-phase3
Build the serial, single-precision code
make install
Configure and build the serial, double-precision code
Create a build directory in the source tree
cd gromacs-4.6.1 mkdir build_d cd build_d
Set the environment variables for the CMake build (note use of native compiler commands):
export CXX=CC export CC=cc export CMAKE_PREFIX_PATH="/opt/fftw/3.3.0.2/x86_64:/opt/gcc/4.7.2/snos/lib" export FLAGS="-static -O3 -ftree-vectorize -ffast-math -funroll-loops"
Use CMake to configure the build:
cmake ../ -DCMAKE_SKIP_RPATH=YES -DBUILD_SHARED_LIBS=OFF \ -DGMX_MPI=OFF -DGMX_OPENMP=ON -DGMX_GPU=OFF -DGMX_X11=OFF -DGMX_DOUBLE=ON \ -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \ -DCMAKE_INSTALL_PREFIX=/usr/local/packages/gmx/4.6.1-phase3
Build the serial, double-precision code
make install
Build the parallel simulation code for the compute nodes
Set up your environment for the parallel build
Switch to the compute node architecture:
module swap craype-istanbul craype-interlagos
Switch to the GNU programming environment:
module swap PrgEnv-cray PrgEnv-gnu
and load the FFTW 3 and CMake modules:
module add fftw module add CMake
Module list for centrally installed parallel executables:
Currently Loaded Modulefiles: 1) modules/3.2.6.7 2) nodestat/2.2-1.0400.31264.2.5.gem 3) sdb/1.0-1.0400.32124.7.19.gem 4) MySQL/5.0.64-1.0000.5053.22.1 5) lustre-cray_gem_s/1.8.6_2.6.32.45_0.3.2_1.0400.6453.7.1-1.0400.39662.3.1 6) udreg/2.3.1-1.0400.4264.3.1.gem 7) ugni/2.3-1.0400.4374.4.88.gem 8) gni-headers/2.1-1.0400.4351.3.1.gem 9) dmapp/3.2.1-1.0400.4255.2.159.gem 10) xpmem/0.1-2.0400.31280.3.1.gem 11) hss-llm/6.0.0 12) Base-opts/1.0.2-1.0400.31284.2.2.gem 13) xtpe-network-gemini 14) PrgEnv-gnu/4.0.46 15) atp/1.6.2 16) xt-asyncpe/5.20 17) pmi/4.0.0-1.0000.9282.69.4.gem 18) xt-libsci/12.0.02 19) xt-totalview/8.11.0 20) totalview-support/1.1.4 21) gcc/4.7.2 22) pbs/10.2.2.113537 23) packages-phase3 24) bolt/0.5 25) budgets/1.0 26) craype-interlagos 27) cray-mpich2/5.6.5 28) CMake/2.8.2 29) fftw/3.3.0.2
Configure and build the parallel, single-precision build
Create a build directory in the source tree
cd gromacs-4.6.1 mkdir build_mpi cd build_mpi
Set the environment variables for the CMake build (note use of compiler wrapper scripts):
export CXX=CC export CC=cc export CMAKE_PREFIX_PATH=/opt/fftw/3.3.0.2/interlagos export FLAGS="-static -O3 -march=bdver1 -ftree-vectorize -ffast-math -funroll-loops"
Use CMake to configure the build and then build and install:
cmake ../ -DCMAKE_SKIP_RPATH=YES -DBUILD_SHARED_LIBS=OFF \ -DGMX_MPI=ON -DGMX_OPENMP=ON -DGMX_GPU=OFF -DGMX_X11=OFF -DGMX_DOUBLE=OFF \ -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \ -DCMAKE_INSTALL_PREFIX=/usr/local/packages/gmx/4.6.1-phase3 \ -DGMX_CPU_ACCELERATION="AVX_128_FMA" make install-mdrun
Configure and build the parallel, double-precision build
Create a build directory in the source tree
cd gromacs-4.6.1 mkdir build_mpi_d cd build_mpi_d
Set the environment variables for the CMake build (note use of compiler wrapper scripts):
export CXX=CC export CC=cc export CMAKE_PREFIX_PATH=/opt/fftw/3.3.0.2/interlagos export FLAGS="-static -O3 -march=bdver1 -ftree-vectorize -ffast-math -funroll-loops"
Use CMake to configure the build and then build and install:
cmake ../ -DCMAKE_SKIP_RPATH=YES -DBUILD_SHARED_LIBS=OFF \ -DGMX_MPI=ON -DGMX_OPENMP=ON -DGMX_GPU=OFF -DGMX_X11=OFF -DGMX_DOUBLE=ON \ -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \ -DCMAKE_INSTALL_PREFIX=/usr/local/packages/gmx/4.6.1-phase3 \ -DGMX_CPU_ACCELERATION="AVX_128_FMA" make install-mdrun