HECToR Secondary External Filesystem Upgrade: Timeline
Users may recall that data was migrated from an internal lustre filesystem to an external one back in autumn 2010. The original internal disk has remained unused until now. The original 600TB of disk has now been reconfigured to be part of the external filesystem.
Rather than extend the existing external filesystem, a secondary external filesystem has been configured. The current external filesystem has a capacity of 594TB. The creation of the second filesystem with an additional 600TB brings the overall capacity to over 1PB.
The diagram below shows the future HECToR filesystem layout with two external /work filesystems:
Data Migration Plan
All I/O is currently being handled by one esFS filesystem, comprising half the total available storage. Migrating data between filesystems will not only balance the volume of data on each filesystem, but spread the load between filesystems. An analysis has been performed on the current volume of data held by each project, and the timings from the original esFS migration. A decision has been taken to migrate all non-NERC projects to the secondary filesystem.
In order to avoid an extensive amount of downtime for all users, this work will be phased project by project depending on the volume of data on /work. The data will be migrated to the second external filesystem in nine phases. These nine phases are detailed as below
- Tue 28 June : All projects prefixed d and y
- Wed 29th June: All projects prefixed z, u, b, j, p
- Thu 30 June: All projects prefixed c, x, t
- Fri 1 July -> Mon 4 July: e24, e85, e68
- Tue 5 July: e102, e10
- Thu 7 July: e01, e124, e141
- Fri 8 July: e42, e05, e149
- Sat 9 July: e89, e63, e71
- Sun 10 July: ALL other projects prefixed e
If the migration date proposed above causes a particular issue for your group, please contact the Helpdesk.
How will the migration affect me?
During the migration period no new projects, groups or users will be created on the system. Updates on quotas will not be passed back to the SAFE.
During the migration of each project, jobs for users within that project will be placed on HOLD and will not run. You will not need to change any scripts as part of this change - your data will still be available in /work as before.
The Message of the Day (MOTD) will be updated as the data migration progresses. This will show which projects have been migrated, and which are still to be migrated. The MOTD is available on the System Status page on the web.
Secondary External Filesystem
Users and Principle Investigators will see a difference in the SAFE and when checking quotas. Once the data has been migrated, it will be reported in a filesystem identified as /work2. If your project is not listed above, there is no change and it will appear in /work as before.
