I've run out of storage space (inodes)

In addition to a file size limit, your filesystems are also limited by number of files (inodes). Trying to write to a filesystem over it's inode quota will cause an error (and probably kill your job).

Current file-count (and space) can be found using nn_storage_quota.

Filesystem         Available      Used     Use%     Inodes     IUsed     IUse%
home_user123 20G 1.957G 9.79% 92160 21052 22.84%
project_nesi99999 2T 798G 38.96% 100000 66951 66.95%
nobackup_nesi99999 6.833T 10000000 2691383 26.91%

Note

There is a delay between making changes to a filesystem and seeing the change in nn_storage_quota, immediate file count can be found using du --inodes.

The command find . -printf "%h\n" | cut -d/ -f-2 | sort | uniq -c | sort -rn > filecount.txt will print an ordered list to the file 'filecount.txt'

There are a few ways to deal with file count problems

  • Use /nesi/nobackup
    The nobackup directory has a significantly higher inode count and no disk space limits.
  • Delete Unnecessary files
    Some applications will generate a large number of files during runtime, using the command du --inodes -d 1 you can drill down into the directories with the largest file count deleting files as viable.  
  • Tarball
    Many files can be compressed into a single 'tarball' 
    tar -cf name.tar /path/containing/files/
    Then when files need to be accessed again they can be un-tarred using,
    tar -xf tarname.tar
  • Contact Support
    If you are following the recommendations here yet are still concerned about indoes, open a support ticket and we can raise the limit for you.
Labels: disk quota exceeded
Was this article helpful?
1 out of 1 found this helpful