File storage

Files are stored on Roar in a central filesystem. Users have access to four directories: home, work, scratch, and group. These have different purposes:

• home – for configuration files, and links to work, group, and scratch.
• work – for your own work; only you have read-write access to your home and work directories.
• scratch – for temporary storage of large files. Scratch is not backed up, and files with a time stamp older than 30 days will be automatically deleted.
• group – for collaborative work. Group space can be purchased by PIs in 5 TB increments. By default, group members have read access to all files in group.

Files in home, work, and group are backed up by a sequence of daily "snapshots", which are kept for 90 days. To have files restored from a snapshot, email Client Support at icds@psu.edu.

Quotas

home, work, group, and scratch directories are subject to limits, both on the amount of storage space and on the number of inodes. Files, directories, and symlinks all count towards inode limits.

Checking usage

Exceeding quotas on home or work directories can cause errors when running progrms, writing files, or even logging in.

There are two tools to check on your disk usage:

• check_storage_quotas reports your total usage;
• du reports the sizes of files and directories.

du -sh * gives "human-readable" sizes (in MB, GB, TB) for each item in the current directory.

In managing storage, we often want to know where the big files are. du -sh * | sort -h -r lists directory sizes in order from large to small (the output of du is "piped" to sort).

Quota issues in home

Many user configuration files and packages are stored by default in home. If these become too large, they can exceed the quota and cause errors. This commonly occurs with directories such as

• .local - used by Python
• .comsol - used by Comsol

These dot files are hidden by default, but you can view them with ls -la.

If the size of one of these directories becomes a problem, it can be moved to work, and a link placed in your home directory. To make such a link, in your home directory execute (e.g., for .local)

ln -s .local work/.local

Archive storage

To store infrequently-used files, low-cost archive storage can be purchased. The Globus web interface is the only way to access these files, so access is neither quick nor convenient.

If you store a directory that contains many files, you should pack the directory into a single file with tar (see Packing files) before transferring.