The HECToR Service is now closed and has been superceded by ARCHER.

3. Connecting to HECToR

3.1 Interactive access

To log into HECToR you should use the "login.hector.ac.uk" address and connect using SSH. On Apple Mac and Linux systems SSH should be installed by default and you should only need to open a terminal and use the SSH command:

ssh [userID]@login.hector.ac.uk

Windows systems do not generally have SSH available by default but you can install the free PuTTY software and use this to access HECToR:

The first time you log onto the system you will need to retrieve your initial password from the SAFE web interface. As soon as you log in you will be prompted to change this to a password of your choosing. The screencast below demonstrates the process of retrieving your password and logging on for the first time. Note: your login password will sometimes be referred to as your "LDAP" password by the system.

3.2 Transferring data to and from HECToR

The HECToR systems are connected to the outside world via the UK academic SuperJANET5 network.

The simplest way of transferring data to and from HECToR is using the scp command. Here are some examples:

Example 1: The command:

scp ./source.f [username]@login.hector.ac.uk:

on your local system, will transfer the file source.f to your home directory on the HECToR system.

Example 2: The command:

scp ./input_data.tar.gz [username]@login.hector.ac.uk:/work/[project]/[group]/[username]/RUN5

will copy the file input_data.tar.gz from your local system to the RUN5 sub-directory on your work directory.

Example 3: The command:

scp -r [username]@login.hector.ac.uk:/work/[project>]/[group]/[username]/RESULTS ./

will copy the sub-directory RESULTS from your work directory to the current directory on your local system (note the use of the -r option).

Example 4: If your local system supports ssh logins, you may also run the scp commands from the HECToR system. For example, the same transfer of the RESULTS sub-directory on HECToR to the home directory of your local system could also be accomplished by running the following command on the HECToR system:

scp -r /work/[project]/[group]/[username]/RESULTS local_username@machine_name.institution.ac.uk:~

3.2.1 Performance considerations

HECToR is capable of generating data at a rate far greater than the rate at which this can be downloaded over SuperJANET5. In practice, it is expected that only a portion of data generated on HECToR will be required to be transfered back to users' institutions - the rest will be, for example, intermediate or checkpoint files required for subsequent runs. However, it is still essential that all users try to transfer data to and from HECToR as efficiently as possible. The most obvious ways to do this are:

  1. Only transfer those files that are required for subsequent analysis, visualisation and/or archiving. Do you really need to download those intermediate result or checkpointing files? Probably not.
  2. Combine lots of small files into a single tar file, to reduce the overheads associated in initiating data transfers.
  3. Compress data before sending it, e.g. using gzip or bzip2.
  4. Consider doing any pre- or post-processing calculations on HECToR. Long running pre- or post-processing calculations should be run via the batch queue system, rather than on the login nodes. Such pre- or post-processing codes could be serial or OpenMP parallel applications running on a single node, though if the amount of data to be processed is large, a MPI application able to use multiple nodes may be preferable.

Note that the performance of data transfers between HECToR and your local institution may differ depending upon whether the transfer command is run on HECToR or on your local system.

Though the scp command is the simplest mechanism for file transfer it is not the most efficient. This is mainly becasue scp uses an encrypted connection which adds a significant overhead to the transfer. For large data sets significantly faster transfer times can be obtained using GridFTP or bbFTP.

3.2.2 GridFTP

This program requires the use of an eScience certificate. GridFTP transfers can be initiated using the globus-url-copy program with gsiftp URLs. For example:

 
globus-url-copy \
   gsiftp://my.host.ac.uk/home/user/data.dat \
   gsiftp://login11b.hector.ac.uk/work/[project]/[group]/[username]/data.dat

Currently the gridftp daemon is available on login11b.hector.ac.uk. The RDF file-systems can be accessed via dtn01.hector.ac.uk, dtn02.hector.ac.uk or dtn03.hector.ac.uk. In many cases improved performance can be achieved by increasing the number of parallel streams using the -p flag.

3.2.3 bbFTP

bbFTP allows users to transfer data using multiple data streams in the same way as GridFTP but without the need for a Grid certificate. Please see the separate HECToR bbFTP Guide for more information on using bbFTP.

2. Hardware | Contents | 4. Resource Management