Quotas

Quotas are used to keep track of storage space that is being used on per-user or per-group basis. Quotas are measured both in terms of number of files used and total size of files. File sizes are measured in terms of the number of blocks used.


Finding out current usage

When you are logged in to Apocrita, you can see which storage you have access to, and how much you have used, using the qmquota utility.

This will list your home directory, scratch storage and any shared project storage you have access to.

$ qmquota -s
Disk quotas for abc123
                           Space                                                Files
Location                   Use%    Used    Free   Quota   Limit   Use%   Used   Free    Quota  Limit
/data/Shared-Lab-Storage   0.00%   32K     1T     1T      1.1T    -      3      -       -      -
/data/home/abc123          19%     92.1G   40G    50G     55G     -      20667  -       -      -
/data/scratch/abc123       0.00%   0       300G   300G    500G    0.00%  4      999996  -      1000000

Several options are available to modify the output similar to those for the quota command:

$ qmquota -h
Usage: qmquota [OPTIONS]
-s, --human-readable : display numbers in human friendly units (MB, GB...)
-c, --colour         : Highlights storage which is over the quota
-h, --help           : This small usage guide
-A, --nfs-all        : display quota for all NFS mountpoints
-v, --verbose        : Include where no storage is allocated
-l, --local-only     : Do not query network filesystems

Reporting

You will receive emails when you exceed scratch or home directory quotas.

You will also receive emails if you are the designated group contact or data owner for shared storage.

Group Contacts

Group contacts are recorded in the README file in the top directory. If this needs to be altered, please contact its-research-support@qmul.ac.uk


Good practice for managing your storage

Storage space is counted in blocks of 128KB, the smallest unit of allocation in GPFS. As a result, space usage may seem larger than the sum of files used.

  • Tar large directories of files when not in use. You can de-compress and re-compress when you need them, even within your jobs. Use bzip2 for maximum compression.
  • We provide a database server running PostgreSQL and MariaDB for accessing/writing small amounts of data. This is more efficient than writing lots of small files.
  • Try to avoid having multiple processes writing to the same file. e.g. collecting logs from an application that's running on lots of nodes. Store each with a different name e.g. labelled by their process id (PID) or $SGE_TASK_ID.