// Storage Analysis Tools

There are a couple utilities embedded within the Discover cluster to allow users to understand their storage on the system to prevent themselves from overwriting and halting their programs from running, due to storage shortages.

1. To get a brief overview of your storage usage on discover, users are encourage to use the showquota command.
2. Personal Storage Utilization Analytics Tools

If you would like to expand your storage space, please contact NCCS Support with your username, project, and a reason on why you require more storage.

Personal Storage Utilization Analytic

This is a suite of light-weight tools (applications) that help users assess the storage used by the code they run on NCCS systems. It reports disk and archival usage by directory on a job-specific basis rather than total disk usage, which is reported by the showquota command.

Note: This version reports the total usage in the identified directories. To get the usage generated by a given job, either use new directories for output data or run trax before and after your job. The second option only works after you have determined the directories to be monitored. Alternatively, run du -s on likely output directories before running strace, partr, and trax.

USAGE

On discover, add /usr/local/other/primertools/bin to your $PATH variable. The programs are partr and trax. For bash: export PATH="/usr/local/other/primertools/bin:$PATH"

And for tsch/csh, use: setenv PATH $PATH\:/usr/local/other/primertools/bin

DISCOVERY

Run strace on your program (e.g. usercode.exe) $ strace -f -o tracefile.trc ./usercode.exe

If your workflow runs multiple programs, you can create a shell script (for the sake of this example, "wrapper.sh") that calls all your other program(s). Then run the wrapper like this: $ strace -f -o tracefile.trc ./wrapper.sh

A file called tracefile.trc will be created. The partr program will reduce this to a list of "relevant" directories where files were created or written and the number of times these directories are referenced: $ partr tracefile.trc -d
[.] 3 (the working directory)
[/discover/nobackup/userid/fakedata] 1
[/home/userid/opt/PSUA] 1


MONITORING

Make a text file with the list of directories, one entry per line (e.g.: dirlist.txt). The program trax provides a summary of watched directories:

To calculate the amount of storage stored by your program, run trax before and after your program is run. Then, calculate the difference between the two results. You may use the output from partr (list of affected directories) as the input for the list of directories for trax.