next up previous contents
Next: Operating systems Up: HECToR Previous: Modes of operation   Contents

Communication and I/O networks

The nodes communicate via a high bandwidth interconnect which uses the Cray SeaStar2 communication chips. Each dual core processor has its own private SeaStar2 chip which is connected directly into its HyperTransport link. The SeaStar chips are arranged on a 3-dimensional torus with dimensions 20 x 12 x 24. Each SeaStar2 chip provides high speed links to its 6 neighbours (see figure 1). Each link is capable of delivering a peak bi-directional bandwidth of 7.6 GB/s.

Figure 1: Diagram illustrating the directionality of the 6 links coming from each SeaStar2 chip.
Image network

In addition to the compute nodes there are also dedicated I/O nodes, login nodes and nodes set aside for serial compute jobs. The login nodes can be used for editing, compilation, profiling, de-bugging, job submission etc. When a user connects to HECToR via ssh the least loaded login node is selected to ensure that users are evenly loaded across the system and that no single login node ends up with an excessive load.

HECToR has 12 I/O nodes which are directly connected to the toroidal communications network described above. These I/O nodes are also connected to the data storage system (i.e. physical disks). The data storage on HECToR consists of 576 TB of high performance RAID disks. The service uses the Lustre distributed parallel file system to access these disks.


next up previous contents
Next: Operating systems Up: HECToR Previous: Modes of operation   Contents