Backups and Snapshots


Backups

Tape backups of the data storage are performed for the purposes of disaster recovery and do not provide the ability to recover old or lost files. However, we utilise GPFS snapshots, and if the file exists on an overnight snapshot, you can recover them as shown in the snapshots section

What is backed up:

  • User home directories /data/home
  • Shared project storage /data/

What is NOT backed up

  • Scratch Space /data/scratch
  • auto deleting scratch space/data/autoScratch

The backup schedule consists of daily incremental backups which are retained for a month. This is currently handled by Spectrum Protect (previously TSM) to build a full backup. The tapes are kept geographically separate to our cluster.


Snapshots

Snapshots are taken daily and are kept for a couple of days, they hold a copy of the entire storage at the point they were taken. All files exist in the snapshots with exactly the same permissions as they did in the original, which means you can recover files from snapshots yourself.

Snapshot Structure

Snapshots can be found under /data/.snapshots which contains one directory for each available snapshots.

These directories are named by date (any additional ones are there for administration purposes, such as taking tape backups). Below these directories is the same directory structure as the rest of the GPFS file store. Hence home directories are under /data/.snapshots/YYYY-M-D/home/abc123 etc.

These directories are read-only; they can not be changed.

A full list of available snapshots can be gathered with mmlssnapshot gpfsWork this will tell you exactly when the snapshot was taken:

$ mmlssnapshot gpfsWork
Snapshots in file system gpfsWork:
Directory                SnapId    Status  Created
2016-1-22                1001      Valid   Fri Jan 22 23:00:20 2016
2016-1-23                1002      Valid   Sat Jan 23 23:00:13 2016
2016-1-24                1004      Valid   Sun Jan 24 23:00:20 2016

Recovering files from snapshots

To recover a file from the snapshot simply copy it somewhere outside the .snapshots directory.

cp /data/.snapshots/2016-1-24/home/abc123/lost_file /data/home/abc123/recovered_data/