// File System on Discover Cluster

The Discover cluster provides several different types of file systems: home, nobackup, and temporary/scratch. See the showquota documentation for information on how to monitor your storage usage.

File System Type Variable on Discover cluster Default Quota Backup Cycles
Home Directory IBM GPFS $HOME 1GB Daily
Scratch IBM GPFS $NOBACKUP 5Tb/300k inodes No Backups
Scratch local $LOCAL_TMPDIR node specific No backups, data is deleted at the end of the job or session
Scratch GPFS $TMPDIR Same as NOBACKUP No backups, data is deleted at the end of the job or session
Fast Access Scratch GPFS $TSE_TMPDIR 250GB, 200k inodes No backups, data is deleted at the end of the job or session

Best Practices


  • Scratch filesystems $LOCAL_TMPDIR and $TMPDIR are node specific file systems, users may want to be careful using them as they may store files/data on them and lose it when they log onto the Discover cluster, but on a different node.

  • Data in $TSE_TMPDIR is automatically deleted when your job ends because storage on this resource is extremely limited.

  • ALWAYS use the symlinks, e.g, $HOME, $NOBACKUP, /home, or /discover/nobackup, in your scripts to specify paths. NEVER use the real path, /gpfsm/dnbxx, because it could be changed due to disk augmentations or system events.

  • Adding packages to a Python or Conda environment in $HOME tends to use a lot of temporary data in the .conda, .cache, and .local directories which can quickly exceed the user $HOME quota. To resolve this, move these directories to $NOBACKUP and then create a soft link to them from $HOME. Example:
    mv $HOME/.conda $NOBACKUP
    ln -s $NOBACKUP/.conda $HOME/.conda     Repeat these commands for the .cache and .local directories as well.

  • For users of the GEOS software, please read the GMAO's guidance on how to control GEOS output, both to learn how to reduce the number of data collections saved and how to create a custom data collection to eliminate the need to write unneeded data to disk or tape.

In addition to user $NOBACKUP space, there are some project $NOBACKUP spaces. These are driven by specific project requirements and are located under /discover/nobackup/projects/<project_name>.

Accessing CSS through Slurm

CSS read-only access on Discover is provided to a subset of Discover's Slurm-managed compute nodes.

Discover CSS Access