Child pages
  • HPRC Software Environment (RHEL 7 Linux Cluster)
Skip to end of metadata
Go to start of metadata

Consult HPRC Desktop Software for information on some ways to connect to HPRC resources.

You will need to submit a service-now request to have software installed onto the HPC cluster.

IMPORTANT:  Installing your own software

  • Unless you wrote the software, you should not be installing it on HPC infrastructure.
  • HPC staff may choose to provide assistance with software you install, but the task will be given low priority.
  • There is a possibilty that software installed into your home directory will not be runnable in future (for security/risk reasons).

As a general rule, you can view a list of software installed on the HPC cluster using

module avail

To list all versions of a particular piece of software, use

module available <software name>

For example, "module avail R" will list all installed versions of R.

To load the environment that enables use of commands provided by a particular piece of software, use one of the commands

module load <software name>/<version>
module load <software name>

As a general rule, it is best to specify a version.  If you don't specify a version, the default version (which will change over time) will be used.


 Operating System (RHEL7) Software

The software listed below is installed as part of the operating system (a "module load ..." command isn't required).  RedHat determine the which version of software will be available.

As a general rule, it is better to use versions of software available under environment modules control.  HPC cluster systems are set to automatically install updates provided by RedHat.

Note: Much of the software mentioned below contain more than a single executable - e.g., gcc is an acronym for Gnu Compiler Collection (includes C++ compiler, Fortran compiler, and more).


Linux Shell (default):  bash

Linux Shells (other):  kshtcshzsh

Text Editor (easiest for most):  nano

Text Editors (other):  emacsvim (the graphical version of an editor may be installed - e.g., gvim )

Command/Scripting Languages:  perl, python, ruby, tcl, tclx, tix, tk, tkinter

Vesion Control (most popular):  git

Version Control (other):  cvs,  mercurial, subversion 

File Compression & Archiving:  bzip2 gzipp7zip, tar, unzipxzzip

  • I do not recommend use of the Linux version of zip as it may produced files that cannot be extracted with the Linux version of unzip.
  • The tar command supports inline compression, using bzip2gzip, & xz (possibly more).
  • In my experience, xz  provides the best compression.

Remote System Access:  filezilla, openssh

  • OpenSSH is the core software enabling access to HPC infrastruture (not just at JCU).  Common commands used: scp, sftp, slogin, ssh 

Software Build Tools:  autoconf, automake, bakefile, cmake, gcc, gdl, ghc, imake, libtool, make

Software Libraries:

armadillo, arpackatlas, blas, blas64, boost, cairo, cfitsio, clipper, cogl, fftw, fftw2, freexl, gd, gdal, giflib, glibcglpkgmpgslhdfhdf5imlib, imlib2, lapack, lapack64libccp4libexiflibffilibgeotifflibjpeg-turbolibmnglibmpclibpnglibquadmathlibrsvg2libtifflzo, suitesparseSuperLU

Other Useful Software/Commands:

bc, bltbwidgetcurlgawkgrassjasperlesstif, gnuplot, GraphicsMagickgraphvizImageMagicsingularity

Singularity is the only current container engine that researchers have sufficient privilege to use.

In order to minimize costs associated with porting BioInformatics & related software to RedHat Enterprise Linux, a lot of software will only be available as a conda environment.

 Anaconda2 Environments

IMPORTANT:  As python 2.7 is now end of life, only environments that cannot be created under anaconda3 will be created under anaconda2.

The world's most popular python/R data science platform.  Running the command "conda-env list" is the best way to get the current list of environments that have been created.

Environments available:  base , dadi, phyluce

Example: 

module load anaconda2
conda activate phyluce
...
conda deactivate phyluce

In the above example, "conda ..." may be replaced by "source ...".

Some people may need to run the command

. ~/.bashrc

in order to run other conda commands.

 Anaconda3 Environments

The world's most popular python/R data science platform.  

Environments available:  Running the command "conda-env list" is the best way to get the current list of environments that have been created.

  • abyss, admixtools, admixture, angsd, assembly-scan, assembly-stats, astral, augustus,
  • bamm, bamtools, base, bayesan, bazel, bcftools, beast, beast2, bedtools, bismark, blast, boost, bowtie2, busco, bwa,
  • canu, cap3, cdhit, cloog, corset, cufflinks, cutadapt,
  • dadi, diamond, dlcpar, dmd,
  • emboss,
  • fasta3, fastme, fastp, fastqc, faststructure, fastx, flye, ffmpeg, freebayes, freexl,
  • gatk, gatk4, gblocks, gdal, gdc, glpk, gmap, gmsh, gnuplot, googlesparsehash, gradle,
  • hdf4, hdf5, hisat2, hmmer, hmmer2, iqtree, itsx,
  • jags, jellyfish, julia, kraken, kraken2, lastz, libnetcdf,
  • mafft, mapdamage2, MaxBin2, mcl, MetaBAT2, metis, migrate, minimap2, mirdeep2, mrbayes, multiqc, muscle,
  • netcdf-fortran, netcdf4, nettle, nlopt, 
  • orthofinder,
  • pal2nal, paml, pandoc, parallel, pcre, pcre2, pear, petsc, picard, pipits, plink, pplacer, prodigal, proj4, proteinortho, purge_haplotigs, pyfastx,
  • qgis, quast,
  • raxml, rcorrector, rdpclassifier, rsem, ruby,
  • salmon, samtools, scalpel, seqtk, sga, shannon, shovill, slim, spades, sparsehash, sratools, stacks, star, stringtie, structure,
  • tophat, tpp, transdecoder, transrate, trimal, trimgalore, trimmomatic, trinity, trinotate,
  • vcflib, vcftools, velvet, vsearch,

Example: 

module load anaconda3
conda activate bamtools
...
conda deactivate bamtools

In the above example, "conda ..." may be replaced by "source ...".

Some people may need to run the command

. ~/.bashrc

in order to run other conda commands.

 MATLAB Toolboxes

A high-level technical computing lanaguage and interactive environment for algorithm development, data vizualization, and numerical analysis.

MATLAB is commercial software that JCU has purchased varying degrees of capacity to use.  One (1) User License is consumed via "one user, one computer using no more than 2 CPU cores".

The JCU HPC license has the following licensing limits:

  • 50 User License (core):  MATLAB, MATLAB Coder, Simulink, Simulink Coder, Simulink Control Design
  • 50 User License (toolboxes):  Control System, Optimizatiom, Signal Processing, Symbolic Math, System Identification
  • 11 User License (toolboxes):  Statistics & Machine Learning
  • 10 User License (toolboxes):  Global Optimization, Parallel Computing
  • 5 User License (toolboxes):  Mapping, Neural Network
  • 4 User License (core):  MATLAB Compiler
  • 4 User License (toolboxes):  Fuzzy Logic, Image Processing, Wavelet
  • 2 User License (toolboxes):  Audio System, DSP System, Curve Fitting, Instrument Control, Partial Differential Equation, Simscape, Simscape Power Systems
  • 1 User License (toolboxes):  Aerospace

Example: 

module load matlab/2019b
matlab ...
 Python (2.7.14) Packages

IMPORTANT:  Python 2.7 has hit end of life and should only be used if functionality isn't available in Python 3.

A programming language that lets you work quickly and integrate systems more effectively.

Packages Available:  To get an up to date list of installed packages use pip list or pip list --format=columns

  • absl-py , alignment , antiorm , astor
  • backports-abc , backports.functools-lru-cache , backports.shutil-get-terminal-size , backports.weakref , biom-format , biopython , bleach , BUSCO , bx-python , bz2file
  • certifi , chardet , clepy , click , cloudpickle , composition , configparser , cutadapt , cvxopt , cycler , Cython
  • dadi , dask , db , db-sqlite3 , decorator , DendroPy , distributed
  • entrypoints , enum34
  • forbiddenfruit , funcsigs , functools32 , future , futures
  • gast , grpcio
  • h5py , hachoir-core , hachoir-metadata , hachoir-parser , HeapDict , html5lib , HTSeq
  • idna , illumiprocessor , ipykernel , ipython , ipython-genutils , ipywidgets , itero
  • jax , Jinja2 , jsonschema , jupyter , jupyter-client , jupyter-console , jupyter-core
  • locket
  • Markdown , MarkupSafe , matplotlib , metadata , mistune , mock , mpi4py , mpmath , msgpack
  • nbconvert , nbformat , nose , notebook , numpy
  • opt-einsum
  • pandas , pandocfilters , partd , pathlib2 , pbr , pexpect , phyluce , pickleshare , Pillow , pip , pitz , pp , prompt-toolkit , protobuf , psutil , ptyprocess , Pygments , pyparsing , pyproj , pyqi , pysam , python-dateutil , python-lzo , pytz , PyVCF , PyYAML , pyzmq
  • qiime2 , qtconsole
  • requests
  • scandir , schwimmbad , scikit-learn , scipy , Send2Trash , setuptools , shannon , simplegeneric , singledispatch , six , sortedcontainers , subprocess32 , sympy
  • tblib , Tempita , tensorboard , tensorflow , termcolor , terminado , testpath , toolz , tornado , traitlets
  • urllib3
  • virtualenv
  • wcwidth , webencodings , Werkzeug , wheel , widgetsnbextension
  • xopen
  • zict
 Python (3.6.5) Packages

A programming language that lets you work quickly and integrate systems more effectively.

Packages Available:  To get an up to date list of installed packages use pip3 list or pip3 list --format=columns

  • absl-py , alabaster , alignment , antiorm , appdirs , astor , atomicwrites , attrs
  • Babel , backcall , biom-format , biopython , bleach , blur , bx-python , bz2file
  • CacheControl , certifi , cftime , chardet , click , click-plugins , cligj , cloudpickle , colormath , composition , coverage , cutadapt , cvxopt , cycler , Cython
  • dadi , dammit , dask , db , db-sqlite3 , deblur , decorator , DendroPy , distributed , docutils , doit
  • entrypoints
  • ficus , filelock , Fiona , forbiddenfruit , future
  • gast , GDAL , geopandas , gneiss , graphsurgeon , grpcio
  • h5py , HeapDict , html5lib , HTSeq
  • idna , illumiprocessor , imagesize , ipykernel , ipython , ipython-genutils , ipywidgets , itero
  • jedi , Jinja2 , jsonschema , jupyter , jupyter-client , jupyter-console , jupyter-core
  • khmer , kiwisolver
  • locket , lockfile , lzstring
  • Mako , Markdown , MarkupSafe , matplotlib , meson , mistune , more-itertools , mpi4py , mpmath , msgpack , msgpack-python , multiqc , munch
  • natsort , nbconvert , nbformat , netCDF4 , networkx , nose , notebook , numexpr , numpy
  • packaging , pandas , pandocfilters , parso , partd , patsy , pexpect , pickleshare , Pillow , pip , pluggy , pockets , prompt-toolkit , protobuf , psutil , ptyprocess , py , pycuda , Pygments , pyinotify , pyparsing , pyproj , pysam , pytest , pytest-cov , python-dateutil , python-lzo , pytools , pytz , PyVCF , PyYAML , pyzmq
  • qcli , qiime2 , qtconsole
  • requests
  • schwimmbad , scikit-bio , scikit-learn , scipy , screed , seaborn , Send2Trash , setuptools , shannon , Shapely , shmlast , simplegeneric , simplejson , six , snowballstemmer , sortedcontainers , spectra , Sphinx , sphinx-rtd-theme , sphinxcontrib-napoleon , sphinxcontrib-websupport , statsmodels , sympy
  • tblib , tensorboard , tensorflow , tensorrt , termcolor , terminado , testpath , toolz , tornado , traitlets
  • uff, urllib3,
  • virtualenv,
  • wcwidth, webencodings, Werkzeug, wheel, widgetsnbextension,
  • xopen,
  • zict,
 R (3.6.1) Packages

A free software environment for statistical computing and graphics.

Packages Available:  One way to get an accurate listing of R packages available (for every R version installed) is to load the relevant environment module and then run the command: ls $R_HOME/lib64/R/library 

  • abc , abc.data , abind , acepack , actuar , ade4 , ade4TkGUI , adegenet , adegraphics , adehabitatHR , adehabitatLT , adehabitatMA , AER , affy , affyio , agricolae , AICcmodavg , airway , akima , AlgDesign , alr3 , amap , Amelia , annotate , AnnotationDbi , AnnotationFilter , AnnotationForge , AnnotationHub , ape , apeglm , arm , aroma.light , askpass , assertthat , automap
  • backports , base , base64enc , BB , bbmle , BDgraph , bdsmatrix , beeswarm , bestglm , betapart , betareg , BH , BiasedUrn , bibtex , biglm , Biobase , BiocFileCache , BiocGenerics , BiocManager , BiocParallel , BiocStyle , BiocVersion , BiodiversityR , biomaRt , biomod2 , Biostrings , biovizBase , bit , bit64 , bitops , blob , bookdown , boot , bootstrap , brew , Brobdingnag , broom , BSgenome , BSgenome.Hsapiens.UCSC.hg19 , BufferedMatrix
  • calibrator , callr , candisc , car , carData , caret , Category , caTools , catspec , CCA , cellranger , checkmate , chron , CircStats , class , classInt , cli , clipr , cluster , cmprsk , coda , codetools , coin , colorspace , combinat , compiler , CompQuadForm , contfrac , corpcor , corrplot , cowplot , coxme , crayon , crosstalk , ctc , cubature , curl , cwhmisc
  • d3Network , DAAG , datasets , data.table , date , DBI , dbplyr , degreenet , DelayedArray , deldir , dendextend , denstrip , DEoptim , DEoptimR , DESeq , DESeq2 , deSolve , dfoptim , DiagrammeR , dichromat , digest , diptest , dismo , diveRsity , DMwR , doMC , doParallel , doSNOW , dotCall64 , downloader , dplyr , DynDoc , dynlm
  • e1071 , earth , EasyABC , Ecdat , Ecfun , EDASeq , edgeR , effects , ellipse , ellipsis , elliptic , emdbook , emmeans , EMT , emulator , ensembldb , ergm , estimability , evaluate , expint , expm
  • fansi , faraway , fAsianOptions , fastmatch , fBasics , FD , fda , fdrtool , ff , fields , filehash , fission , FKF , flexclust , flexmix , FNN , fOptions , forcats , foreach , foreign , formatR , Formula , fpc , fts , futile.logger , futile.options
  • gam , gamlss , gamlss.data , gamlss.dist , gamm4 , gbm , gbRd , gclus , gdata , gdistance , gdsfmt , gdtools , gee , geepack , geiger , genefilter , geneLenDataBase , geneplotter , generics , genetics , GenomeInfoDb , GenomeInfoDbData , GenomicAlignments , GenomicFeatures , GenomicRanges , geodist , geometry , geoR , geoRglm , getopt , GGally , ggbeeswarm , ggbio , ggdendro , ggm , ggplot2 , ggplot2movies , ggvis , glasso , glcm , Glimma , glmnet, glmmTMB, glue , gmodels , gmp , GO.db , goftest , goseq , GOstats , gower , GPArotation , gpclib , gplots , graph , graphics , grDevices , grid , gridExtra , grpreg , GSEABase , gsl , gss , gstat , gsubfn , gtable , gtools , Gviz
  • haven , heplots , hergm , hexbin , hierfstat , hier.part , highlight , highr , Hmisc , hms , hpar , HSAUR , HSAUR2 , htmlTable , htmltools , htmlwidgets , httpuv , httr , huge , hwriter , hydroGOF , hydroPSO , hydroTSM , hypergeo
  • igraph , igraphdata , impute , influence.ME , influenceR , inline , interactiveDisplayBase , intergraph , intervals , inum , ipred , IRanges , irlba , ISwR , iterators , itertools
  • jomo , jpeg , jsonlite
  • Kendall , kernlab , KernSmooth , klaR , knitr , ks
  • labeling , labelled , laeken , Lahman , lambda.r , LaplacesDemon , latentnet , later , lattice , latticeExtra , lava , lavaan , lazyeval , leaps , LearnBayes , lgcp , lhs , libcoin , lifecycle , limma , linprog , lme4 , lmtest , locfit , logspline , lokern , lpSolve , lsmeans , ltm , lubridate
  • magic , magrittr , MALDIquant , manipulateWidget , mapdata , mapproj , maps , maptools , maptree , mAr , markdown , marray , MASS , Matching , MatchIt , Matrix , matrixcalc , MatrixModels , matrixStats , maxLik , maxlike , MBESS , mboost , mcgibbsit , mclust , mcmc , MCMCpack , mda , mediation , memoise , MEMSS , methods , mgcv , mi , mice , microbenchmark , mime , miniUI , minpack.lm , minqa , misc3d , miscTools , mitml , mitools , mix , mixtools , mlbench , mlergm , MLInterfaces , mlmRev , mlogit , mnormt , ModelMetrics , modeltools , moments , mondate , MPV , mratios , MSGFplus , msm , MSnbase , MSnID , multcomp , multcompView , multicool , MuMIn , munsell , Mus.musculus , MVA , mvabund , mvna , mvtnorm , mzID , mzR
  • ncbit , ncdf4 , network , networksis , nlme , nloptr , nnet , nnls , nor1mix , nortest , np , numDeriv , nws
  • OpenMx , openssl , openxlsx , optimx , ordinal , OrganismDbi , org.Hs.eg.db , org.Mm.eg.db , outliers , oz
  • pan , parallel , partitions , party , partykit , pastecs , pbapply , pbivnorm , pbkrtest , PBSmapping , pcaMethods , pegas , permute , PFAM.db , phangorn , pheatmap , picante , pillar , pixmap , pkgconfig , pkgKitten , PKPDmodels , plm , plogr , plotMCMC , plotmo , plotrix , pls , plyr , PMCMR , png , PoiClaClu , poilog , polspline , polyclip , polycor , polynom , polysat , prabclus , praise , preprocessCore , PresenceAbsence , prettyunits , pROC , processx , prodlim , progress , proj4 , pRoloc , pRolocdata , promises , proteomics , ProtGenerics , proto , proxy , ps , pscl , psych , purrr
  • qgraph , qtl , quadprog , quantmod , quantreg , questionr , qvalue
  • R2admb , R2OpenBUGS , R2WinBUGS , R6 , random , RandomFields , RandomFieldsUtils , randomForest , ranger , RANN , rappdirs , RArcInfo , raster , rasterVis , rbenchmark , RBGL , R.cache , rcdd , Rcmdr , RcmdrMisc , RColorBrewer , Rcpp , RcppArmadillo , RcppEigen , RcppGSL , RcppNumerical , RcppProgress , Rcsdp , RCurl , rda , Rdpack , readr , readstata13 , readxl , recipes , relaimpo , relevent , relimp , rematch , ReportingTools , reshape , reshape2 , rgdal , rgeos , rgexf , rgl , Rgraphviz , rhdf5 , Rhdf5lib , Rhtslib , rio , rjson , RJSONIO , rlang , rlecuyer , RLRsim , rmarkdown , rmeta , R.methodsS3 , rms , RMySQL , RNAseq123 , RNAseqData.HNRNPC.bam.chr14 , rnaseqGene , robustbase , ROCR , RODBC , rols , R.oo , Rook , rpanel , rpart , rpf , rpx , RSAGA , Rsamtools , RSQLite , rstudioapi , RSVGTipsDevice , rtiff , rtracklayer , RUnit , R.utils , RUVSeq
  • S4Vectors , sampleSelection , sampling , sandwich , scales , scatterplot3d , SDMTools , segmented , sem , semTools , sendplot , seqinr , sequencing , setRNG , sf , sfsmisc , sgeostat , shape , shapefiles , shapes , shiny , ShortRead , SimComp , SimRAD , sm , sna , snow , snowfall , snowFT , SOAR , som , sourcetools , sp , spacetime , spam , sparr , SparseM , spatial , SpatialTools , spatstat , spatstat.data , spatstat.utils , spBayes , spData , spdep , spgrass6 , splancs , splines , spThin , sqldf , SQUAREM , stabledist , stabs , StanHeaders , statmod , statnet.common , stats , stats4 , stringdist , stringi , stringr , strucchange , subplex , subselect , SummarizedExperiment , survey , survival , sva , svGUI , svUnit , sys , systemfit , systemfonts
  • tables , tcltk , tcltk2 , TeachingDemos , tensor , tensorA , testthat , tgp , TH.data , threejs , tibble , tidyr , tidyselect , timeDate , timeSeries , tinytex , tis , tkrplot , tools , translations , tree , trip , tripack , truncreg , trust , TSA , TTR , tweedie , TxDb.Mmusculus.UCSC.mm10.knownGene
  • ucminf , udunits2 , units , unmarked , untb , urca , utf8 , utils
  • VariantAnnotation , vcd , vctrs , vegan , vegan3d , VGAM , VIM , vioplot , vipor , viridis , viridisLite , visNetwork , vsn
  • waveslim , wavethresh , webshot , whisker , widgetTools , withr
  • xfun , XML , xml2 , xtable , xts , XVector
  • yaml
  • zeallot , Zelig , zip , zipcode , zlibbioc , zoeppritz , zoo ,
 Other software available by load environment module(s)

An up-to-date list of available software can be obtained by running the command "module avail".

A "module load ..." command, where you replace ... with the name of an installed piece of software, will be required in order to use software listed below.

Note:  Multiple versions may exist for some of the software listed here.  The list below was accurate on 28-Apr-2020.

  • abyssadmbadmixtureangsdaugustusautomake,
  • bamaddrg, bamm, bamtools, bayesass, BayeScan, bazel, bcftools, beast, bedtools, BEDTools, bedtools2, bfast, blast, blat, bowtie, bowtie2, bwa,

  • cap3, cdbfasta, cd-hit, chimeraslayer, clearcutcloogclustaloclustalwcolony2corsetcufflinks,
  • diamond, dmd,
  • emboss, exiftool,
  • fasta, fastahack, fastaqfastqcfastStructurefastxffmpegfpart,
  • garli, Gblocksgdgdcglassfishgmapgmshgradle,
  • hgdownload, hmmer, htslib,
  • infernal, iqtree, isl, ITSx,
  • jags, jbigkit, jellyfish, jpeg,
  • kraken,
  • lastz, libcerf, libgd, libgtextutils, libjpeg, lzip,
  • maespa, mafft, matlab,
  • ncurses, netbeans, netcdf, netcdf-fortran, netphos, nettle, NGSadmix, nlopt, 
  • octave, ogdi, openjpeg2,
  • p11-kit, p7zip, pandoc, parallel, paup, pbmpi, pbzip2, pcre, pcre2, pear, perl, pftools, pgdspider, phylip, picard, picard-tools, pipits, plplot, proj, psort,
  • qhull, qrupdate, qt,
  • r8s, raxml, rdp_classifier, readline, rnammer, rsem, ruby,
  • salmon, samtools, SDL, secretomep, sed, SEG, seqtk, sga, singalp, slim, SLiM, snappy, sortmerna, spades, sparsehash, sqlite, sratoolkit, stacks, star, strauto, structure, subversion, sumaclust, swarm,
  • TensorRT, termcap, tiff, tmhmm, tophat, TransDecoder, trimal, trimgalore, trimmomatic, trinityrnaseq, Trinotate,
  • udunits2, units, unzip,
  • vcflib, vcftools, velvet, vim, vsearch,
  • wavpack, wolfpsort, wxWidgets,
  • xfig, xz,
  • yasm,
  • zeromq, zlib,



Future Plans 

HPC users may be given access to a small number of docker/podman commands (cp, images, info, ps, & run) to allow use of supplied container images.  Docker doesn't have suitable controls to safely run/store containers on an NFS share - where your home directory resides, so HPC staff will need to set them up.  As a general rule, we recommend using anaconda or singularity instead.

An example of a command to run a job/workflow using a docker image (salmon) is:

sudo docker run -v `pwd`:/scr -w /scr -u $(id -u):$(id -g) combinelab/salmon salmon index -t athal.fa.gz -i mus.rna.index

Of course, you will need to place this command in a PBS script (see HPRC PBSPro script files).

  • No labels