NeSI is planing to provide a service for working on Jupyter Notebooks. As a first step JupyterLab is provided. This is a single-user web-based Notebook server, running in the user space. JupyterLab server should be started on a login or virtual lab node. The server then can be reached from your local browser using port forwarding. The procedure is described in the following 3 steps:
The following JupyterLab kernel are installed:
- R (requires
module load IRkernel)
Launch the JupyterLab server
JupyterLab is provided as part of Anaconda3. If you want to use additional kernels you may need additional modules, e.g. IRkernel for R kernel:
module load Anaconda3
module load IRkernel # optional
The JupyterLab server can be started using:
jupyter lab --port 15051 --no-browser
--no-browser option prevents JupyterLab from trying to open a browser on the compute/login node side. The --port option specify a number related to this process or network service. This port ID need to be unique to the process on the present machine. The default port for JupyterLab is 8888. If this port is already in use, JupyterLab will automatically choose another port.
To avoid the need to modify the procedure below, we suggest to (once) select a unique number (between 2000 and 65000). Here we choose 15051 (please select another number). This port number will also be required in the next steps.
For computational or memory intensive tasks please run your notebook on a compute node. Therefore, we provide a short script which takes care of the port forwarding to the compute node and launches jupyter lab:
srun -n 1 -t 60 jupyter-compute 15051 # please change port number
After initial notes Jupyter outputs a URL with a unique key and the network port number where the web-server is listening, similar to:
... [C 14:03:19.911 LabApp] To access the notebook, open this file in a browser: file:///scale_wlg_persistent/filesets/project/nesi99996/.local/share/jupyter/runtime/nbserver-503-open.html Or copy and paste one of these URLs: http://localhost:15051/?token=d122855ebf4d029f2bfabb0da03ae01263972d7d830d79c4
- A non-default port can be selected using the
--portoption. Thus you could avoid getting varying ports And can use you port forwarding configuration without modifications.Please select a different port number than this.
jupyter lab --port 15051 --no-browser
Setup SSH port forwarding
An additional ssh session with port forwarding needs to be established from your local machine. In the command the local and the remote port need to be specified. For simplicity reason we keep them the same (here 8888) as specified by JupyterLab server (see above)
Linux/MacOS and Ubuntu Terminal (Windows10)
The ssh command need to be called with following arguments (-N optional), e.g. for Mahuika:
ssh -N -L 15051:localhost:15051 mahuika
- In the following we assume you already configured your
.ssh/configas described in the Standard Terminal Setup.
- For Maui_Ancil, e.g. w-mauivlab01 you may want to add the following to your
.ssh/configto avoid multi hop access.<username> needs to be changed. Hostnames can be adapted for other nodes, e.g.
Host maui_vlab User <username> Hostname login.mahuika.nesi.org.nz ProxyCommand ssh -W %h:%p maui ForwardX11 yes ForwardX11Trusted yes ServerAliveInterval 300 ServerAliveCountMax 2
- I you get a error messagetry creating the following directory:
bind: No such file or directory unix_listener: cannot bind to path:
mkdir -P ~/.ssh/sockets
- MobaXterm has a internal terminal which acts like a linux terminal and can be configured as described in the Standard Terminal Setup. Therewith the command above can be used.
MobaXterm has a GUI to setup and launch sessions with port forwarding, click 'Tools > MobaSSH Thunnel (port forwarding)':
- specify the lander02.nesi.org.nz as SSH server address (right, lower box, first line)
- specify your user name (right, lower box, second line)
- specify the remote server address, e.g. login.mahuika.nesi.org.nz (right, upper box first line)
- specify the JupyterLab port number on the local side (left) and at the remote server (right upper box, second line)
JupyterLab in your local browser
Finally, you need to open your local web browser and copy and paste the URL specified by the JupyterLab server into the address bar. After initializing Jupyter Lab you should see a page similar to: