Globus - Basic Use within NeSI

Note: after the system upgrade of July 2018, some of the Globus endpoints or data-transfer nodes (DTN) which are named in this documentation will no longer be in service. Please check back for updated information once the new filesystems are in use.

Preliminaries:

1)  register with Globus for an ID

To use Globus from within NeSI, you need a globus ID. If you do not have one, sign up at https://www.globusid.org , following the instructions at Register for a Globus ID.

2) Identify NeSI Data Transfer Nodes

The NeSI cluster machines have been configured to act as Globus Data Transfer Nodes (or endpoints). Associate your cluster filesystem with the following Globus DTN names:

Globus DTN NeSI Filesystem
nesi#pan_auckland Pan cluster fileystem
nesi#otago-dtn01 High Capacity Research Storage Cluster, U of Otago
nesi#otago-dtn02 High Capacity Research Storage Cluster, U of Otago
nesi#otago-dtn03 High Capacity Research Storage Cluster, U of Otago
nesi#otago-dtn04 Otago AWS Endpoint
nesi#otago-dtn-chc01 High Capacity Research Storage Cluster, Christchurch Campus
nesi#otago-dtn-wlg01 High Capacity Research Storage Cluster, Wellington Campus
Plant and Food Research iRODS instance Plant and Food Research iRODS instance

 

3) Be aware of Globus paths (see Globus-Paths-Permissions-Storage-Allocation  if necessary).

4)  Note:  If you are accessing data on the Otago HCS, use the endpoint nesi#otago-dtn01  but in order to access this filesystem, you will need to register for access to nesi#otago-dtn01

 

Transfer Data between cluster nodes

  1. Sign in to https://www.globus.org with your id. You will be taken to the transfer page https://www.globus.org/app/transfer
  2. Select the endpoints you wish to move files between (start typing "nesi#" to see the list of NeSI DTNs to select from).  Authenticate via Tuakiri.    

  3. Globus defaults to your home directory (maximum storage 2GB) . Navigate your path to an appropriate directory (see Globus Paths, Permissions, Storage Allocation ).
  4. Transfer the files.
  5. Check your email for confirmation the job has succeeded.
NOTE: Until mid-2018, your cluster may be on an older filesystem: check with support@nesi.org.nz about when to move more than 0.5TB of data if using Pan.

 

Transfer Data between your personal computer and cluster nodes

Install a personal globus DTN on your personal computer (instructions at Personal Globus Endpoint ), and use globus to transfer files to/from there.

At Otago and Auckland Universities, you cannot currently transfer files between your personal globus endpoint and the local NeSI cluster endpoint from within the University (behind firewall).

Labels: globus dtn
Was this article helpful?
0 out of 0 found this helpful