The HECToR Service is now closed and has been superceded by ARCHER.

Compiling Gromacs 4.6.x on HECToR Phase 3

GROMACS compilation consists of two steps:

  • Compiling the serial tools (for the service nodes).
  • Compiling the parallel MD simulation program (for the compute nodes).

Build the serial tools for the frontend nodes

Set up your environment for the serial build

Switch to the login node architecture:

module swap craype-interlagos craype-istanbul

Switch to the GNU programming environment:

module swap PrgEnv-cray PrgEnv-gnu

and load the FFTW 3 and CMake modules:

module add fftw
module add CMake

Module list for centrally installed serial executables:

Currently Loaded Modulefiles:
  1) modules/3.2.6.7
  2) nodestat/2.2-1.0400.31264.2.5.gem
  3) sdb/1.0-1.0400.32124.7.19.gem
  4) MySQL/5.0.64-1.0000.5053.22.1
  5) lustre-cray_gem_s/1.8.6_2.6.32.45_0.3.2_1.0400.6453.7.1-1.0400.39662.3.1
  6) udreg/2.3.1-1.0400.4264.3.1.gem
  7) ugni/2.3-1.0400.4374.4.88.gem
  8) gni-headers/2.1-1.0400.4351.3.1.gem
  9) dmapp/3.2.1-1.0400.4255.2.159.gem
 10) xpmem/0.1-2.0400.31280.3.1.gem
 11) hss-llm/6.0.0
 12) Base-opts/1.0.2-1.0400.31284.2.2.gem
 13) xtpe-network-gemini
 14) PrgEnv-gnu/4.0.46
 15) atp/1.6.2
 16) xt-asyncpe/5.20
 17) pmi/4.0.0-1.0000.9282.69.4.gem
 18) xt-libsci/12.0.02
 19) xt-totalview/8.11.0
 20) totalview-support/1.1.4
 21) gcc/4.7.2
 22) pbs/10.2.2.113537
 23) packages-phase3
 24) bolt/0.5
 25) budgets/1.0
 26) craype-istanbul
 27) cray-mpich2/5.6.5
 28) CMake/2.8.2
 29) fftw/3.3.0.2

Configure the single-precision build

Create a build directory in the source tree

cd gromacs-4.6.1
mkdir build
cd build

Set the environment variables for the CMake build (note use of native compiler commands):

export CXX=CC
export CC=cc
export CMAKE_PREFIX_PATH="/opt/fftw/3.3.0.2/x86_64:/opt/gcc/4.7.2/snos/lib"
export FLAGS="-static -O3 -ftree-vectorize -ffast-math -funroll-loops"

Use CMake to configure the build:

cmake ../ -DCMAKE_SKIP_RPATH=YES -DBUILD_SHARED_LIBS=OFF \
          -DGMX_MPI=OFF -DGMX_OPENMP=ON -DGMX_GPU=OFF -DGMX_X11=OFF -DGMX_DOUBLE=OFF \
          -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \
          -DCMAKE_INSTALL_PREFIX=/usr/local/packages/gmx/4.6.1-phase3

Build the serial, single-precision code

make install

Configure and build the serial, double-precision code

Create a build directory in the source tree

cd gromacs-4.6.1
mkdir build_d
cd build_d

Set the environment variables for the CMake build (note use of native compiler commands):

export CXX=CC
export CC=cc
export CMAKE_PREFIX_PATH="/opt/fftw/3.3.0.2/x86_64:/opt/gcc/4.7.2/snos/lib"
export FLAGS="-static -O3 -ftree-vectorize -ffast-math -funroll-loops"

Use CMake to configure the build:

cmake ../ -DCMAKE_SKIP_RPATH=YES -DBUILD_SHARED_LIBS=OFF \
          -DGMX_MPI=OFF -DGMX_OPENMP=ON -DGMX_GPU=OFF -DGMX_X11=OFF -DGMX_DOUBLE=ON \
          -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \
          -DCMAKE_INSTALL_PREFIX=/usr/local/packages/gmx/4.6.1-phase3

Build the serial, double-precision code

make install

Build the parallel simulation code for the compute nodes

Set up your environment for the parallel build

Switch to the compute node architecture:

module swap craype-istanbul craype-interlagos

Switch to the GNU programming environment:

module swap PrgEnv-cray PrgEnv-gnu

and load the FFTW 3 and CMake modules:

module add fftw
module add CMake

Module list for centrally installed parallel executables:

Currently Loaded Modulefiles:
  1) modules/3.2.6.7
  2) nodestat/2.2-1.0400.31264.2.5.gem
  3) sdb/1.0-1.0400.32124.7.19.gem
  4) MySQL/5.0.64-1.0000.5053.22.1
  5) lustre-cray_gem_s/1.8.6_2.6.32.45_0.3.2_1.0400.6453.7.1-1.0400.39662.3.1
  6) udreg/2.3.1-1.0400.4264.3.1.gem
  7) ugni/2.3-1.0400.4374.4.88.gem
  8) gni-headers/2.1-1.0400.4351.3.1.gem
  9) dmapp/3.2.1-1.0400.4255.2.159.gem
 10) xpmem/0.1-2.0400.31280.3.1.gem
 11) hss-llm/6.0.0
 12) Base-opts/1.0.2-1.0400.31284.2.2.gem
 13) xtpe-network-gemini
 14) PrgEnv-gnu/4.0.46
 15) atp/1.6.2
 16) xt-asyncpe/5.20
 17) pmi/4.0.0-1.0000.9282.69.4.gem
 18) xt-libsci/12.0.02
 19) xt-totalview/8.11.0
 20) totalview-support/1.1.4
 21) gcc/4.7.2
 22) pbs/10.2.2.113537
 23) packages-phase3
 24) bolt/0.5
 25) budgets/1.0
 26) craype-interlagos
 27) cray-mpich2/5.6.5
 28) CMake/2.8.2
 29) fftw/3.3.0.2

Configure and build the parallel, single-precision build

Create a build directory in the source tree

cd gromacs-4.6.1
mkdir build_mpi
cd build_mpi

Set the environment variables for the CMake build (note use of compiler wrapper scripts):

export CXX=CC
export CC=cc
export CMAKE_PREFIX_PATH=/opt/fftw/3.3.0.2/interlagos
export FLAGS="-static -O3 -march=bdver1 -ftree-vectorize -ffast-math -funroll-loops"

Use CMake to configure the build and then build and install:

cmake ../ -DCMAKE_SKIP_RPATH=YES -DBUILD_SHARED_LIBS=OFF \
          -DGMX_MPI=ON -DGMX_OPENMP=ON -DGMX_GPU=OFF -DGMX_X11=OFF -DGMX_DOUBLE=OFF \
          -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \
          -DCMAKE_INSTALL_PREFIX=/usr/local/packages/gmx/4.6.1-phase3 \
          -DGMX_CPU_ACCELERATION="AVX_128_FMA"

make install-mdrun

Configure and build the parallel, double-precision build

Create a build directory in the source tree

cd gromacs-4.6.1
mkdir build_mpi_d
cd build_mpi_d

Set the environment variables for the CMake build (note use of compiler wrapper scripts):

export CXX=CC
export CC=cc
export CMAKE_PREFIX_PATH=/opt/fftw/3.3.0.2/interlagos
export FLAGS="-static -O3 -march=bdver1 -ftree-vectorize -ffast-math -funroll-loops"

Use CMake to configure the build and then build and install:

cmake ../ -DCMAKE_SKIP_RPATH=YES -DBUILD_SHARED_LIBS=OFF \
          -DGMX_MPI=ON -DGMX_OPENMP=ON -DGMX_GPU=OFF -DGMX_X11=OFF -DGMX_DOUBLE=ON \
          -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \
          -DCMAKE_INSTALL_PREFIX=/usr/local/packages/gmx/4.6.1-phase3 \
          -DGMX_CPU_ACCELERATION="AVX_128_FMA"

make install-mdrun