There are two tracked resources in the NeSI filesystem, disk space and inodes (number of files) .
Trying to write to a filesystem over it's inode or disk quota will cause an error (and probably kill your job).
Current file-count and disk space can be found using
Filesystem Available Used Use% Inodes IUsed IUse%
home_user123 20G 1.957G 9.79% 92160 21052 22.84%
project_nesi99999 2T 798G 38.96% 100000 66951 66.95%
nobackup_nesi99999 6.833T 10000000 2691383 26.91%
There is a delay between making changes to a filesystem and seeing the change in
nn_storage_quota, immediate file count and disk space can be found using the commands
There are a few ways to deal with file count problems
The nobackup directory has a significantly higher inode count and no disk space limits. Files here are not backed up, so best used for intermediary or replaceable data.
- Delete Unnecessary files
Some applications will generate a large number of files during runtime, using the command
du --inodes -d 1 | sort -hr(for inodes) or
du -h -d 1 | sort -hrfor disk space. You can then drill down into the directories with the largest file count deleting files as viable.
Many files can be compressed into a single 'tarball'
Then when files need to be accessed again they can be un-tarred using,
tar -czf name.tar /path/containing/files/
tar -xzf tarname.tar
- Contact Support
If you are following the recommendations here yet are still concerned about indoes or disk space, open a support ticket and we can raise the limit for you.