HECToRNews 7, December 2009Welcome to the HECToR newsletter.
- Training Course Timetable
- HECToR Phase2B Upgrade
- Access to HECToR
- Closure of HECToR CSE service over Christmas and the New Year
- Programming Environment issues
- Distributed Support
Training Course Timetable
The current schedule is below:
- January 4-6, 2010 Imperial College London - Parallel Programming with MPI
- January 7-8, 2010 Imperial College London - OpenMP
- January 11-15, 2010 NAG Oxford - Best Practice in HPC Software Development
- January 26-28, 2010 University of Bath - Parallel Programming with MPI
- February 3, 2010 NAG Oxford - Introduction to HECToR
- February 4-5, 2010 NAG Oxford - Quad Core Training
- April 1, 2010 NAG Manchester - Introduction to HECToR
- April 12-14, 2010 University of Nottingham - Parallel Programming with MPI
- April 15-16, 2010 University of Nottingham - OpenMP
These training courses run by NAG are provided free of charge to HECToR users and UK academics whose work is covered by the remit of one of the participating research councils (EPSRC, NERC and BBSRC).
HECToR Phase2B Upgrade
In 2010 HECToR will be upgraded giving the user 24 processing cores on each node. More details are available online. The Phase2B upgrade will take place in two stages:
- A 20 cabinet XT6 (or 'Baker') system will be installed, with an early user service available around April 2010. The system will comprise 44,544 cores, delivering an estimated peak performance of around 340 Tflops. Approximately 30 cabinets of the XT4 will also be retained. The XT4 will continue to provide the main service at this stage.
- This will be followed by an upgrade to the current Seastar 2+ interconnect by replacing it with the newer Gemini interconnect by the end of 2010.
Further details of the upgrade will be confirmed nearer the time. The transition to 24 cores is an exciting prospect and if a user suspects that their code may need performance investigation prior to the upgrade, then they are encouraged to seek the help of the NAG CSE team by contacting the HECToR helpdesk. Help and advice can then be given on how the code might be able to get full benefit from the hardware upgrade.
Access to HECToR
HECToR Capability Challenge 2010 - Call for Proposals
This call ties in with the Phase 2a and 2b upgrades of the HECToR service. It provides an opportunity for UK investigators to request access to a large amount of compute time in order to explore the capability dimension of the HECToR service. It also gives access to applications support from the HECToR service in order to undertake multi-and many-core development on their applications codes. The closing date for submissions is January 7th 2010, for more details please see here.
HECToR Direct Access
A reminder that this call remains open. EPSRC are piloting a new access class for the HECToR service called 'Direct Access' (Class 1B) where users can request significant amounts ( greater than 1,000,000 AUs) of computing resource only for a period of up to six months. The Direct Access mechanism will provide successful applicants with discretionary access to the HECToR system in order to provide resource to cover activities such as:
- Bridging access between grant applications,
- Resource to trial application developments at scale,
- Provide preliminary results in aid of grant applications via Responsive Mode or to a Call for Proposals;
There is no deadline for submission, however: the proposals will be assessed by a Resource Allocation Panel (RAP) which will meet every four months (February 2010, June 2010 and October 2010), the deadline for applications being submitted to the Panel will be three weeks before the Panel meeting. Full details are available online here.
Low Priority Access
A low priority queue is now available on HECToR. The key facts are as follows:
- A single low priority queue is available.
- The low priority queue applies to the XT4 only. It is not available on the X2.
- Class 1a projects will not be charged for low priority jobs. All other projects will be charged.
- The minimum job size supported in the queue is 512 cores.
- The maximum run time supported is 3 hours.
- Jobs can be submitted to the queue at any time.
- The low priority queue will only be enabled when the backlog of 'normal' jobs is below 3 hours. The backlog is currently re-calculated hourly.
- Jobs in the low priority queue will continue to run right up to the start time of planned maintenance slots. The low priority queue will not be drained. We would therefore encourage you to use check-pointing.
- Any low priority jobs which fail to complete as a result of planned maintenance or a node/system failure will not be refunded.
- The execution queue is called 'low'. To submit a low priority job, users should add the following to their PBS job submission script
#PBS -q lowpriority
The low priority access initiative will run for an initial trial period from 16 Dec to 1 March 2010. We will assess the impact on utilisation and review as necessary.
Closure of HECToR CSE service over Christmas and the New Year
Please be advised that the HECToR CSE service will close at 17:30 on December 22nd and re-open at 09:30 on January 4th.
If you are submitting a proposal to the Capability Challenge call which closes on January 7th, please note that to guarantee that your Technical Assessment is completed by the submission date we need to receive it before December 15th, to allow ten working dates to process it. The CSE team will do their best to carry out Technical Assessments received after that date but cannot promise that they will be completed in time.
The HECToR helpdesk will be operating as normal 08:30-18:00 Mon-Fri during the festive season with the exception of the three UK Bank Holidays (Fri 25th Dec, Mon 28th Dec and Fri 1st Jan). Merry Christmas and a Happy New Year from The HECToR CSE Team.
Programming Environment issues
Compiling for quad core
Please remember when compiling your code for quad core you should now load the module xtpe-barcelona. This will add the appropriate compiler flags for targeting quad core to the ftn and cc wrappers. It will also link in the quad core enabled version of libsci. These instructions along with the full explanation of what the quad core upgrade entails are available here. Please note that module load xtpe-barcelona has superceded the use of module load xtpe-quadcore. Existing compilation scripts may need to be updated to include this change. But any codes which were originally compiled using "xtpe-quadcore" will still continue to run.
HECToR Archive Solution
The ability to archive data is now available on HECToR. Usage is via a simple command line interface and a scheduled nightly archive from pre-defined directories can be set up. Instructions on how to use the archive are here.
Any PIs who require access to the archive should contact the HECToR Helpdesk .
bbFTP on HECToR
bbFTP is an alternative to standard FTP and shows better performance when used to efficiently transfer large amounts of data to/from HECToR. For more information on the use of bbFTP on HECToR please see here.
If you have large amounts of data to transfer to or from HECToR, we recommend the use of bbFTP. bbFTP has been installed on HECToR (and also on HPCx). This is particularly useful if you need to transfer files from HPCx to HECToR.
The latest version of CASTEP (5.0) is now available on HECToR (for the XT system) and you can select this version by using:
module add castep/5.0/xt
There is also a newly installed version of CASTEP 5.0.1 with shared memory enhancements implemented by the NAG HECToR CSE team which improves the scaling and performance of the code, for more details please see here.
The shared memory optimizations for multi-core which were implemented in CASTEP are well understood by the CSE team and they are widely applicable to other codes so if you think this very effective optimization may be beneficial to your code please contact the NAG CSE team.
You can select the shared memory version of CASTEP 5.0 with:
module load castep/5.0.1/shm-xt
More information on running CASTEP jobs (along with sample job submission scripts) is available on the HECToR User Wiki. (You will need your HECToR SAFE login details to access the Wiki).
To gain access to the CASTEP code on HECToR you must have a valid UKCP licence, available from http://www.castep.org/. Once you have a valid licence please contact the HECToR helpdesk to enable access to the CASTEP code for your account.
Apprentice2 Desktop Version 5.0.1
Apprentice2 desktop version 5.0.1 is now available for download on HECToR.The tar file and readme are available in
This is also referred to as dCSE support and funding is available to provide extended help with improving the performance of existing HECToR codes and developing high-performance algorithmic improvements for them. Support is also available to port new codes from other systems to HECToR. Awards to support proposed projects are assessed via an independent panel review which is held roughly every three months.
There are twenty eight current projects that are underway. In addition, five new projects are scheduled to begin early 2010. These new projects will be carried out by personnel with direct experience from the individual scientific areas as well as HPC experts from the NAG CSE team and EPCC. The reports from the completed dCSE projects are available here along with examples of dCSE success stories arising from application codes whose performance has been dramatically improved through dCSE support.
Further information on the dCSE support service can be found here. The next application deadline is the 4th January 2010. Priority will be given to projects that propose specialist support to address any improvements for the phase 2a/b multi-core transitions and also those which can justify a reasonable AU saving impact for the wider HECToR community. Please also note that the current dCSE deadline is also relevant to applicants who wish to request support to run alongside a Capability Challenge submission.
Applicants are advised to contact firstname.lastname@example.org with a brief description of their proposed work well before the application deadline. All applicants for the current round will be informed of the outcome of their proposals early February 2010. NAG staff are available to visit institutions to talk about this service. If you are interested in a visit please contact us at email@example.com