The HECToR Service is now closed and has been superceded by ARCHER.

9. Tools

9.1 Cray Performance Analysis Tool (CrayPAT)

The Cray Performance Analysis Tool (CrayPAT) helps you analyse the performance of programs running on HECToR systems.

CrayPAT can generate tracing experiments, in which every event specified in the pat_build is logged, or asynchronous sampling experiments, in which the program is periodically polled for general statistics. Tracing experiments typically give more detailed information, but involve more overheads to constantly monitor the requested events compared with sampling experiments. The recommended way to use CrayPAT is to run a sampling experiment initially and use the statistics generated to better inform which events to follow in a tracing experiment. This process is automated with the APA (Automatic Profiling Analysis) option.

For detailed instructions on using CrayPAT and information on interpreting the results please see the Performance Analysis chapter of the HECToR Optimisation Guide. The guide contains a worked example of using CrayPAT to analyse the performance of a program and discusses examples of CrayPAT output.

9.2 Cray Apprentice2

Note: Users can obtain a linux desktop copy of Cray Apprentice2 for use on their local machine, rather than on HECToR, by copying from the /usr/local/packages/dtappr/ directory on HECToR. This desktop version is available on a convenience basis only and is not supported by the HECToR service.

Cray Apprentice2 is a performance data visualisation tool. After you have used pat_build to instrument a program for a performance analysis experiment, executed the instrumented program, and used pat_report to convert the resulting data file to a Cray Apprentice2 data format, you can use Cray Apprentice2 to explore the experiment data file and generate a variety of interactive graphical reports.

To run Cray Apprentice2:

load the perftools module (Cray Apprentice2 is part of this)

module add perftools

run pat_report

pat_report -f ap2 patfile

enter the app2 command to launch Cray Apprentice2

app2 [$--$limit tag_count $\Vert$ $--$limit_per_pe tag_count] [data_files]

Example 1. Cray Apprentice2 basics

This example shows how to use Cray Apprentice2 to create a graphical representation of a CrayPAT report.

Using experiment file program1+pat+2511td from above example, generate a report in XML format (note the inclusion of the -f ap2 option):

module add perftools
pat_report -f ap2 program1+pat+2511td
  Output redirected to: program1+pat+2511td.ap2

Run Cray Apprentice2:

 app2 program1+pat+2511td.ap2

Cray Apprentice2 will then display pat_report data in graphical form.

9.3 Totalview debugger

HECToR supports a special implementation of the RogueWave Totalview debugger. Totalview provides source-level debugging of Fortran, C, and C++ code compiled by the Cray, PGI and GNU compilers using between 1 and 2048 compute processes. (for details please see the Cray documentation):


  • Provides both a command line interface (with command line help) and a Motif-based graphical user interface
  • Supports Fortran, C, and C++ code compiled by the Cray, PGI, and GNU compilers
  • Supports programs written in mixed languages
  • HECToR licence supports debugging of up to 2048 compute processes
  • Supports MPI message queue display
  • Supports watchpoints

Please see the Debugging chapter of the HECToR Optimisation Guide for detailed information on running Totalview on HECToR. The Optimisation Guide contains a step-by-step example of using Totalview and example job submission scripts.

9.4 Paraview

9.4.1 Using ParaView 3.12.0 on HECToR

9.4.2 The instructions for versions previous to 3.12.0

9.5 VampirTrace

VampirTrace consists of a tool set and a runtime library for instrumentation and tracing of software applications. During a program run, VampirTrace generates an OTF trace file which can be analysed and visualised by Vampir (commercial software not available on HECToR).


To use VampirTrace on HECToR you should load the appropriate VampirTrace and PAPI modules. First load the papi module. If you are using the PGI compilers for your code, then load the VampirTrace module, if you are using the GNU compilers load the VampirTraceGnu module.


To perform measurements with VampirTrace, the user’s application program needs to be instrumented. In order to enable the instrumentation, the user needs to replace the compiler and linker commands with VampirTrace’s wrappers.

Compiler Wrappers

All the necessary instrumentation of user functions, MPI and OpenMP events is handled by VampirTrace’s compiler wrappers (vtcc, vtcxx, vtf90 and vtf77).

In the makefile used to build the application, all compile and link commands should be replaced by the VampirTrace compiler wrapper. The wrappers perform the necessary instrumentation of the program and link the suitable VampirTrace library.

For instance, use the following wrappers to trace MPI functions: :

C: vtcc -vt:cc cc -vt:mpi
C++: vtcxx -vt:cxx CC -vt:mpi
F90: vtf90 -vt:f90 ftn -vt:mpi
F77: vtf77 -vt:f77 ftn -vt:mpi
Runtime Measurement

Before running your code, you will need to export in your batch script the following environment variable. For example:

  • VT_PFORM_LDIR = < name of local directory which can be used to store temporary trace files >

After running the code, if not already done, unify the trace files with:

vtunify < number of cores > < name of executable >

For more information

8. Software libraries | Contents | 10. Software on HECToR

9.6 DDT Debugger

Please see the Debugging chapter of the HECToR Optimisation Guide for detailed information on running DDT on HECToR.