Note: after the system upgrade of June-July 2018, some of the globus endpoints or data-transfer nodes (DTN) which are named in this documentation will no longer be in service. Please check back for updated information once the new filesystems are in use.
In Linux, your "current working directory" designates the path from "/" to the directory where you are currently located. Use the
pwd command ( "print working directory") to see this:
username@login-01:~$ pwd /home/username
Globus session - default location
When you ssh in to a NeSI cluster, your terminal (shell) session opens in your home directory, designated by the path
/home/<your_username> aliased to
Similarly, if you point Globus to a filesystem endpoint, it will also open a session in your home-directory there.
There's just one catch: Your cluster home-directory has a 2GB allocation so is not ideal for large data-transfers. So before you start transferring files, navigate to a suitable path for the transfer.
The path to your project folder is /projects/
Note: When you select a new path, make sure to press either "enter" or the "Go" button to activate the "change-directory" command.
FYI: A simple way to verify that you're in the right path is to put a small file in your projects directory and make sure Globus can see it when that path is selected.
How much storage space do you have?
On the cluster, get a disk-space usage summary with the command
[username@login01 ~]$ fs_my_quota_usage Your disk space usage summary ============================= /home/username Disk space: quota = 2.00 GB, usage = 1.28 MB (0.06%) File count: usage = 32 (no quota set) /projects/project_code Disk space: quota = 30.00 GB, usage = 9.59 MB (0.03%) File count: quota = 1000000, usage = 7 (0.00%) Note: These figures may be up to 15 minutes out of date.
Storage allocation - the scratch filesystem
For transfer of files which exceed this capacity, designate the
/scratch directory on the cluster.
df -h /scratch will show you how much space is available there at any time:
$ df -h /scratch Filesystem Size Used Avail Use% Mounted on /dev/gpfsscratch 13T 6.9T 5.8T 55% /scratch
If you are bulk-transferring data onto the scratch directory of cluster with Globus,
- transfer the data to somewhere in the subdirectory
- Note: Please move your data out of this scratch space after a transfer from another Globus DTN. The
/scratchfilesystem is not backed up.
/scratch/jobs/abcd123/data_transfer. It shows an scp session to copy the data off the filesystem, and then deletion of the data.
[abcd123@login-01 ~]$ cd /scratch/jobs/abcd123 [abcd123@login-01 scratch] $ mkdir data_transfer (perform data transfer into this directory) [abcd123@login-01 scratch/jobs/abcd123 ] $ tar czvf data_transfer.tar.gz data_transfer/ [abcd123@login-01 scratch/jobs/abcd123] $ scp -r data_transfer.tar.gz destination_filesystem/destination_directory [abcd123@login-01 scratch/jobs/abcd123] $ rm -rf data_transfer [abcd123@login-01 scratch/jobs/abcd123] $ cd