Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

THIS PAGE CONTAINS OLD INFORMATION - HPC staff will work on updating it soon.

This page is intended as a quick introduction for new users submitting their first job to the HPRC Cluster. A few things new users should be aware of:

...

.

...

 toc

...

In

...

The first step in using the HPRC Cluster is to log in to the login node - the typical work flow the user:

  1. Logs into zodiac.hpc.jcu.edu.au

...

titleLogging into the Cluster

...

Software Packages

The HPRC Cluster uses environment modules to manage the available software packages. This allows multiple versions of the same software to be installed without interfearing with each other. To enable the environment module systemthe following command needs to be executed on the command line:

No Format
-bash-4.1$ source /etc/profile.d/modules.sh

 

The software that is available on the HPRC clusted is listed here: HPRC User Software.  Alternately you can query the software available on the cluster with the following commands:

Command

Result

module avail

A list of available software is displayed

module help <software>

Version number and brief synopsis is displayed for <software>

Expand
titleExample "module avail" run on the Thu Mar 6 11:19:21 EST 2014
No Format
-bash-4.1$ module avail
--------------------------------------------------------------------------------------- /usr/share/Modules/modulefiles ----------------------------------------------------
MPInside/3.5.1     compiler/gcc-4.4.5 module-cvs         modules            mpich2-x86_64      null               perfcatcher
chkfeature         dot                module-info        mpi/intel-4.0      mpt/2.05           perfboost          use.own
---------------------------------------------------------------------------------------------- /etc/modulefiles -----------------------------------------------------------
compat-openmpi-x86_64 openmpi-x86_64
------------------------------------------------------------------------------------------------- /sw/modules -------------------------------------------------------------
4ti2                      blast/2.2.23              crimap_Monsanto           hdf5                      migrate/3.6(default)      picard-tools              tmap/1.1
BEDTools                  blast/2.2.29(default)     dx                        hmmer                     mira                      proj                      tmhmm
EMBOSS                    bowtie                    elph                      ima2                      modeltest                 pvm                       topali
GMT                       bwa/0.7.4(default)        enmtools                  jags                      molphy                    r8s                       towhee
Macaulay2                 caftools                  fasta                     java                      mpich2                    rainbowcrack              towhee-openmpi
Python/2.7                cap3                      fastme                    jcusmart                  mrbayes                   rpfits                    trans-abyss
R/2.15.1(default)         carthagene/1.2.2(default) ffmpeg                    jmodeltest                mrmodeltest               ruby/1.9.3                tree-puzzle
R/3.0.0                   carthagene/1.3.beta       fftw2                     lagan                     msbayes                   ruby/2.0.0                trinityrnaseq
abyss                     casacore                  fftw3                     lamarc                    ncar                      samtools                  udunits
ariadne                   cernlib                   garli                     lapack                    netcdf                    scalapack                 udunits2
arlequin                  cfitsio                   gdal                      libyaml/0.1.4             netphos                   scipy                     velvet
asap                      chlorop                   glimmer                   matlab/2008b              numpy                     seadas/6.2                wcslib
atlas                     clipper                   glpk                      matlab/2012a              oases                     seg                       wise2
bayesass                  clustalw                  gmp                       matlab/2012b              octave                    signalp                   wwatch3
beagle                    cluster                   gnu/4.1.2                 matlab/2013a(default)     openbugs                  sprng                     yasm
beast                     cns                       gnu/4.4.0                 maxent                    openjdk                   ssaha2                    zonation
beast-1.5.4               coils                     gnuplot                   maxima                    openmpi                   stacks
bfast                     colony2                   grass                     merlin                    pari                      structure
blacs                     consel                    gromacs                   migrate/3.2.15            paup                      targetp
blas                      crimap                    hdf                       migrate/3.5.1             phyml                     tclreadline/2.1.0

 

Running Jobs

A common misconception for users new to the HPRC Cluster  HPRC Cluster Explained

A simple way to run a job on the cluster is to create a shell script containing with embedded PBS Directives containing the information required by the scheduler to schedule the job.

Example: paup witth the ML_analysis.nex sample file

In this example we will run paup with the ML_analysis.nex sample file provided on the paup sample nexus files page. After logging into the cluster download the example file with the command:

No Format
-bash-4.1$ wget http://paup.csit.fsu.edu/data/ML_analysis.nex
--2014-03-11 13:08:16--  http://paup.csit.fsu.edu/data/ML_analysis.nex
Resolving paup.csit.fsu.edu... 144.174.50.3
Connecting to paup.csit.fsu.edu|144.174.50.3|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2990 (2.9K) [text/plain]
Saving to: “ML_analysis.nex”

100%[=====================================================================================================================================================================>] 2,990       --.-K/s   in 0s

2014-03-11 13:08:17 (70.7 MB/s) - “ML_analysis.nex” saved [2990/2990]

 

Creating the job script

Using a text editor – examples include vim and nano – create your shell script with the filename: ML_analysis.sh and the following contents (the colours are only used for illistration purposes below):

...

Legand:

  1. The very first line of the script file is the Shebang line and must be on the first line.
  2. The second section contains the PBS directives. For more information on PBS directives please see the HPRC PBS script files page.
    PBS Directive 
    #PBS -c sCheckpointing is to be done on a job at pbs_mom shutdown.
    #PBS -j oe
    Merge standard output and standard error streams into the named file.
    #PBS -N ML_analysis 
    #PBS -l pmem=5gb
     
    #PBS -l walltime=500:00:00
     
    #PBS -m ae
    #PBS -M your.name@jcu.edu.au

     
     

     

  3. The third section outputs information about the job, and is only included as an example of what can be done.
  4. The fourth section containf the commands that are actually run in the job. In this case we are using a bash shell.

 

Submitting the Job - qsub

The final step is to submit the job to the job scheduler:

No Format
-bash-4.1$ qsub ML_analysis.sh
148122.jobmgr.hpc.jcu.edu.au

Monitoring the Job - qstat

Once the job has been submitted you can monitor its progress by using the qstat command.

When you first submit your job it is placed into the job queue, and its status column contains Q, meaning the job is in the queue:

No Format
-bash-4.1$ qstat 148122.jobmgr.hpc.jcu.edu.au
Job ID                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
148122.jobmgr              ML_analysis      jcxxxxxxx             0 Q normal

 

Once your job starts running its status changes to R:

No Format
-bash-4.1$ qstat 148122.jobmgr.hpc.jcu.edu.au
Job ID                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
148122.jobmgr              ML_analysis      jcxxxxx               0 R normal
Expand
titleMore qstat examples

More qstat examples

You can display all of the jobs you are running

No Format
-bash-4.1$ qstat -u jcxxxxx -1n
jobmgr.hpc.jcu.edu.au:
                                                                                  Req'd    Req'd       Elap
Job ID                  Username    Queue    Jobname          SessID  NDS   TSK   Memory   Time    S   Time
----------------------- ----------- -------- ---------------- ------ ----- ------ ------ --------- - ---------
148185.jobmgr.hpc.jcu.  jcxxxxx     normal   ML_analysis           0     1      1    5gb 500:00:00 R  00:00:17   n026/19

 

Deleting a job - qdel

If you need to your job you can use the qdel command

No Format
-bash-4.1$ qdel 148122.jobmgr.hpc.jcu.edu.au

Your Jobs Output

Different programs have different ways of outputting their data. If they output data directly to a file then your results will be in whatever file you specified. If, however, the results are printed out to the standard out (as is the case for this example) then PBS captures them into a file for you.

-bash-4.1$ cat  ML_analysis.o148122

...

Job Resources

It is important to match resources requested with the PBS Directives in your script and the resource usage of your job.

 

...

  1. Prepares a submission script for their jobs
  2. Submits their jobs to the Job Scheduler
  3. Monitors their jobs
  4. Collects the output of their jobs.

A few things that new users should be aware of:

  • Typically, jobs are not run in an interactive manner, except when:
    • users are running small one off jobs
    • evaluating the resources required for bigger jobs
    • using graphical applications like MATLAB
    • Examples of interactive jobs:
      Content by Label
      showLabelsfalse
      showSpacefalse
      cqllabel = "interactive-cluster-job"
      labelsinteractive-cluster-job
  • HPRC Cluster software is not run in a window on their desktop, neither is it launched by clicking on it in a network drive.
  • Users need to log into the cluster and inform the job scheduler about their job and it will run it when it can.

Quick Start

Excerpt Include
Guide
Guide
nopaneltrue