Compiling NWChem 6.1 on HECToR Phase 3

All built using PGI compiler.

Get Cray Gemini Interconnect version of Global Arrays Toolkit

Download the Gemini GA 4.3 source from:

or HECToR at:

/usr/local/packages/nwchem/6.1-phase3/libga-crayxt

Compiling

Unpack the source

tar -xzvf Nwchem-6.1.tar.gz

Include Gemini GA Directories

Copy:

  • armci-gemini/
  • global/
  • tcgmsg-mpi/

from the Gemini GA directory to the src/tools directory in the NWChem 6.1 distribution.

Modify the NWChem Makefiles for Gemini

In the GNUmakefile files in: src/, src/tools/ change all references to

armci

to

armci-gemini

Set up your environment

Load the modules:

module swap PrgEnv-cray PrgEnv-pgi
module add onesided/1.5.0
module add ntk/1.5.0

and use the following environment options.

export NWCHEM_TOP=/path/to/top/level/NWChem/directory
export NWCHEM_MODULES="all"
export USE_MPI=y
export LIBMPI=" "
export LARGE_FILES=y
export ARMCI_NETWORK=LIBONESIDED
export NWCHEM_TARGET=LINUX64
export TARGET=LINUX64
export USE_64TO32=y
export HAS_BLAS=yes
export USE_SCALAPACK=y
export TARGET=LINUX64
export BLASOPT=" "
export BLAS_OPT=" "
export LIBONESIDED_DIR="/opt/cray/onesided/1.5.0/gem/lib"
export LIBNUMATOOLKIT_DIR="/opt/cray/ntk/1.5.0/gem/lib"
export OLD_GA=y
alias gmake="make FC=ftn _FC=pgf90 CC=cc COPT='-O0 -g'"

# qa options
export NWCHEM_EXECUTABLE=$NWCHEM_TOP/bin/$NWCHEM_TARGET/nwchem
export MPIRUN_PATH=aprun
export MPIRUN_NPOPT="-n"

Module list for centrally installed executable:

Currently Loaded Modulefiles:
  1) modules/3.2.6.6
  2) nodestat/2.2-1.0400.29866.4.3.gem
  3) sdb/1.0-1.0400.30000.6.18.gem
  4) MySQL/5.0.64-1.0000.4667.20.1
  5) lustre-cray_gem_s/1.8.4_2.6.32.45_0.3.2_1.0400.6221.1.1-1.0400.30252.1.29
  6) udreg/2.3.1-1.0400.3911.5.6.gem
  7) ugni/2.3-1.0400.3912.4.29.gem
  8) gni-headers/2.1-1.0400.3906.5.1.gem
  9) dmapp/3.2.1-1.0400.3965.10.12.gem
 10) xpmem/0.1-2.0400.29883.4.6.gem
 11) hss-llm/6.0.0
 12) Base-opts/1.0.2-1.0400.29823.8.1.gem
 13) xtpe-network-gemini
 14) PrgEnv-pgi/4.0.30
 15) xt-mpich2/5.4.2
 16) atp/1.3.0
 17) xt-asyncpe/5.06
 18) pmi/3.0.0-1.0000.8661.28.2807.gem
 19) xt-libsci/11.0.04
 20) xt-totalview/8.9.2
 21) totalview-support/1.1.2
 22) pgi/11.10.0
 23) pbs/10.2.2.113537
 24) packages-phase3
 25) budgets/1.0
 26) xtpe-interlagos
 27) ntk/1.5.0
 28) onesided/1.5.0

Build NWChem

Ensure BLAS and LAPACK will work:

gmake 64_to_32

Build the program:

gmake realclean
gmake