In addition to a file size limit, your filesystems are also limited by number of files (inodes). Trying to write to a filesystem over it's inode quota will cause an error (and probably kill your job).
Current file-count (and space) can be found using
Filesystem Available Used Use% Inodes IUsed IUse%
home_user123 20G 1.957G 9.79% 92160 21052 22.84%
project_nesi99999 2T 798G 38.96% 100000 66951 66.95%
nobackup_nesi99999 6.833T 10000000 2691383 26.91%
There is a delay between making changes to a filesystem and seeing the change in
nn_storage_quota, immediate file count can be found using
find . -printf "%h\n" | cut -d/ -f-2 | sort | uniq -c | sort -rn > filecount.txtwill print an ordered list to the file 'filecount.txt'
There are a few ways to deal with file count problems
The nobackup directory has a significantly higher inode count and no disk space limits.
- Delete Unnecessary files
Some applications will generate a large number of files during runtime, using the command
du --inodes -d 1you can drill down into the directories with the largest file count deleting files as viable.
Many files can be compressed into a single 'tarball'
Then when files need to be accessed again they can be un-tarred using,
tar -cf name.tar /path/containing/files/
tar -xf tarname.tar
- Contact Support
If you are following the recommendations here yet are still concerned about indoes, open a support ticket and we can raise the limit for you.