Description

Python is a programming language that lets you work more quickly and integrate your systems more effectively.

The Python home page is at http://python.org.

Available modules

Packages with modules

Module	NeSI Cluster
python/2.7.10	fitzroy
python/2.7.4	fitzroy
python/2.7.5	fitzroy
python/2.7.8	fitzroy
Python/3.6.0-gimkl-2017a	pan
Python/2.7.3-gmpolf-1.4.8	pan
Python/2.7.3-goolf-1.4.10	pan
Python/2.7.3-goolf-1.5.14-QIIME	pan
Python/2.7.3-goolf-1.5.14	pan
Python/2.7.3-ictce-5.4.0	pan
Python/2.7.6-goolf-1.5.14	pan
Python/2.7.8-goolf-1.5.14	pan
Python/2.7.8-ictce-5.4.0	pan
Python/2.7.8-intel-2015a	pan
Python/2.7.9-foss-2015a	pan
Python/2.7.9-intel-2015a	pan
Python/2.7.11-foss-2015a	pan
Python/2.7.11-intel-2015a	pan
Python/2.7.13-bare	pan
Python/2.7.13-gimkl-2017a-GEOS-3.5.1	pan
Python/2.7.13-gimkl-2017a-GEOS-3.6.1	pan
Python/2.7.13-gimkl-2017a	pan
Python/3.4.1-goolf-1.5.14	pan
Python/3.4.3-intel-2015a	pan
Python/3.5.0-intel-2015a	pan
Python/3.5.1-foss-2015a	pan
Python/3.5.1-intel-2015a	pan
Python/3.6.0-gimkl-2017a-GEOS-3.5.1	pan
Python/3.6.0-gimkl-2017a-GEOS-3.6.1	pan

Additional supported packages

Version	NeSI Cluster
Python/2.6.7	fitzroy
Python/2.6.6	pan

Licensing requirements

All versions of Python available on NeSI platforms are owned and licensed by the Python Software Foundation. Each version is released under a specific open-source licence. The licences are available on the Python documentation server.

Example scripts

Example script for the Pan cluster

#!/bin/bash -e

#SBATCH --job-name    MyPythonJob
#SBATCH --account     nesi99999
#SBATCH --time        01:00:00
#SBATCH --mem-per-cpu 4G
#SBATCH --output      MyPythonJob.%j.out # Include the job ID in the names of
#SBATCH --error       MyPythonJob.%j.err # the output and error files

module load Python/3.5.1-intel-2015a

python MyPythonScript.py

Example script for the Fitzroy cluster

#!/bin/bash -e

#@ job_name         = MyPythonJob
#@ account_no       = nesi99999
#@ class            = General
#@ wall_clock_limit = 01:00:00
#@ initialdir       = /hpcf/working/nesi99999/MyPythonJob
#@ output           = $(job_name).$(jobid).out
#@ error            = $(job_name).$(jobid).err
#@ queue

# LoadLeveler has an annoying habit of transferring parts of the user's
# environment as it existed at the time of submission to the job. Clear any
# loaded modules.
module purge

# Load a virtual environment if one is needed
source ~/mypython/bin/activate

python MyPythonScript.py

Further notes

Third-party modules

Programmers around the world have written and released many packages for Python, which are not included with the core Python distribution and must be installed separately. Each Python environment module comes with its own particular suite of packages, and the system Python has its own installed packages.

In addition to relying on the packages installed with Python by our staff, you can install packages under your home directory. As an example,

[jblo123@build-wm ~]$ module load Python/3.5.1-intel-2015a
[jblo123@build-wm ~]$ pip install --user cogent

will install the "cogent" package in directory ~/.local/lib/python3.5/site-packages, which is included in Python's sys.path by default.

iPython

iPython (interactive Python) is an enhanced tool for accessing a Python command line. It is available in many NeSI Python modules.

Starting iPython

To open an iPython console, simply run the ipython command:

[jblo123@build-wm ~]$ module load Python/2.7.9-intel-2015a
[jblo123@build-wm ~]$ ipython

Listing available functions

You can use iPython to list the functions available that start with a given string. Please note that if the string denotes a module (i.e., it has a full stop somewhere in it), that module (or the function you want from it) must first be imported, using either an "import X" statement or a "from X import Y" statement.

import os
os.<TAB>   # List all functions in the os module
os.O_<TAB> # List functions starting with "O_" from the os module
len<TAB>   # List functions starting with "len"

Here, denotes a press of the TAB key on your keyboard. Note that you can not see inside a module from outside it, so that for example you can't do

o<TAB>

or even

os<TAB>

and expect to see the methods and values provided by the os module - you have to put the full stop after the "os" if you want to do that.

Getting information about an object

In iPython, you can query any object by typing the object name followed by a question mark (?), then hitting Enter. For instance:

In [1]: x = 5
In [2]: x?
Type:        int
String form: 5
Docstring:
int(x=0) -> int or long
int(x, base=10) -> int or long

Convert a number or string to an integer, or return 0 if no arguments
are given.  If x is floating point, the conversion truncates towards zero.
If x is outside the integer range, the function returns a long instead.

If x is not a number or if base is given, then x must be a string or
Unicode object representing an integer literal in the given base.  The
literal can be preceded by '+' or '-' and be surrounded by whitespace.
The base defaults to 10.  Valid bases are 0 and 2-36.  Base 0 means to
interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4

You can also do this on functions (len?), methods (os.mkdir?) and modules (os.path?). If you try to do it on something that isn't defined yet, Python will tell you that the object in question couldn't be found.

Quitting iPython

Just enter the quit command at the iPython prompt.