THIS PAGE CONTAINS OLD INFORMATION - HPC staff will work on updating it soon.
This page is intended as a quick introduction for new users submitting their first job to the HPRC Cluster.
In the typical work flow the user:
- Logs into zodiac.hpc.jcu.edu.au
- Prepares a submission script for their jobs
- Submits their jobs to the Job Scheduler
- Monitors their jobs
- Collects the output of their jobs.
A few things that new users should be aware of:
- Typically, jobs are not run in an interactive manner, except when:
- users are running small one off jobs
- evaluating the resources required for bigger jobs
- using graphical applications like MATLAB
- Examples of interactive jobs:
Content by label
There is no content with the specified labels
- HPRC Cluster software is not run in a window on their desktop, neither is it launched by clicking on it in a network drive.
- Users need to log into the cluster and inform the job scheduler about their job and it will run it when it can.
Quick Start
The first step in using the HPRC Cluster is to log in to a login node - Starting PuTTY will show this window: Enter the hostname (zodiac.hpc.jcu.edu.au) and the port 8822 The HPRC Cluster uses environment modules to manage the available software packages. This allows multiple versions of the same software to be installed without interfearing with each other. To enable the The software that is available on the HPRC clusted is listed here: HPRC User Software. Alternately you can query the software available on the cluster with the following commands: Command Result A list of available software is displayed Version number and brief synopsis is displayed for To run a job on the cluster create a shell script containing the PBS Directives containing the information required by the scheduler to schedule the job, and the job commands. After logging into the cluster download the example file with the command: Using a text editor – examples include vim and nano – create your shell script with the filename: # Legand: The second section contains the PBS directives. For more information on PBS directives please see the HPC PBSPro script files page. The final step is to submit the job to the job scheduler: Once the job has been submitted you can monitor its progress by using the qstat command. When you first submit your job it is placed into the job queue, and its status column contains Once your job starts running its status changes to If you need to your job you can use the qdel command Different programs have different ways of outputting their data. If they output data directly to a file then your results will be in whatever file you specified. If, however, the results are printed out to the standard out (as is the case for this example) then PBS captures them into a file for you. It is important to match resources requested with the PBS Directives in your script and the resource usage of your job. There can be consequences for incorrectly specifiying these resource requirements Logging In
ssh1.hpc.jcu.edu.au or ssh2.hpc.jcu.edu.au (zodiac.hpc.jcu.edu.au for people still wanting to access the old cluster).
The default port for ssh is port 22, which you can use if you are accessing the cluster from on campus, however if you are accessing it from off campus you need to use port 8822.
ssh <username>@zodiac.jcu.edu.au
(add -p 8822 if you are connecting from outside the JCU network) and you will be asked for your password
Software Packages
environment module system the following command needs to be executed on the command line:-bash-4.1$ source /etc/profile.d/modules.sh
module availmodule help <software><software>Running Jobs
Example: paup witth the ML_analysis.nex sample file
-bash-4.1$ wget http://paup.csit.fsu.edu/data/ML_analysis.nex
--2014-03-11 13:08:16-- http://paup.csit.fsu.edu/data/ML_analysis.nex
Resolving paup.csit.fsu.edu... 144.174.50.3
Connecting to paup.csit.fsu.edu|144.174.50.3|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2990 (2.9K) [text/plain]
Saving to: “ML_analysis.nex”
100%[==============================================================>] 2,990 --.-K/s in 0s
2014-03-11 13:08:17 (70.7 MB/s) - “ML_analysis.nex” saved [2990/2990]
Creating the job script
ML_analysis.sh and the following contents (the colours are only used for illistration purposes below):#!/bin/bash
#
# Checkpointing is to be done on a job at pbs_mom shutdown.
#
#PBS -c s
#
# Merge standard output and standard error streams into the named file.
#
#PBS -j oe
#
# Set the name of the job
#
#PBS -N ML_analysis
#
# Advise the scheduler that the job requires one cpu (ppn) from one node.
#
#PBS -l nodes=1:ppn=1
# Advise the scheduler about the amount of physical memory required.
# kb for kilobytes, mb for megabytes, gb for gigabytes.
#
#PBS -l pmem=5gb
#
# Advise the scheduler that this job will have completed within 10 minutes.
#
#PBS -l walltime=00:10:00
#
# Send mail at batch job abort/exit to the Email address provided.
#
#PBS -m ae
#PBS -M your.name@jcu.edu.auncpu=`wc -l $PBS_NODEFILE | awk '{print $1}'`echo "------------------------------------------------------"echo " This job is allocated "$ncpu" CPU cores on "cat $PBS_NODEFILE | uniqecho "------------------------------------------------------"echo "PBS: Submitted to $PBS_QUEUE@$PBS_O_HOST"echo "PBS: Working directory is $PBS_O_WORKDIR"echo "PBS: Job identifier is $PBS_JOBID"echo "PBS: Job name is $PBS_JOBNAME"echo "------------------------------------------------------" cd $PBS_O_WORKDIRsource /etc/profile.d/modules.shmodule load pauppaup -n ML_analysis.nexSubmitting the Job - qsub
-bash-4.1$ qsub ML_analysis.sh
148122.jobmgr.hpc.jcu.edu.au
Monitoring the Job - qstat
Q, meaning the job is in the queue:-bash-4.1$ qstat 148122.jobmgr.hpc.jcu.edu.au
Job ID Name User Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
148122.jobmgr ML_analysis jcxxxxxxx 0 Q normal
R:-bash-4.1$ qstat 148122.jobmgr.hpc.jcu.edu.au
Job ID Name User Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
148122.jobmgr ML_analysis jcxxxxx 0 R normal
Deleting a job - qdel
-bash-4.1$ qdel 148122.jobmgr.hpc.jcu.edu.au
Your job's Output
Other Examples
Job Resources
Furthor Reading