Child pages
  • HPRC Best Practices
Skip to end of metadata
Go to start of metadata

There is a significant risk to relying on your desktop/laptop computer and/or USB disks.  Example situations to think about:  Disk failure, corruption of data, power outages.  HPC staff have been alerted to 5 cases where PhD students nearing the end of their candidature have "lost everything" because a disk in their laptop failed.  HPC staff have also been asked to try and recover files from a faulty USB disk - not always possible.  JCU perform regular RCD testing - expect to lose power from time to time.

HPC Cluster (Linux)

The longer your computational research jobs take, the more you should be aware of the following.

  1. Submitting jobs (using qsub) to compute nodes is recommended.
  2. If you must work interactively, learn how to use the screen command in Linux.
  3. Be aware of how much resource your jobs require - particularly the amount of memory (RAM) required.  Overconsumption of memory on a server can result in failiure - all running jobs on that server will need to be re-run (not just yours).

Research requiring Microsoft Windows

If your research computations can only be completed on Micosoft Windows, the following points are worth noting.

  1. Identify the minimum resource requirement (CPU, memory, and disk space) for your work.
  2. Request a resource (virtual machine) - HPC will be able to cater for most requirements.  Use Remote Desktop Connection to work on that resource.

Research Data Protection

If your research produces data/information that can never be recaptured (e.g., photos/videos of "events"), make every effort to protect it - you cannot travel back in time.

  1. Keep one copy of all research data on HPC.  Feel free to have other copies elsewhere (e.g., on you desktop/laptop computer).
  2. Establishing a regular (e.g., daily) synchronisation of data is best.  This process should only update changes.
  3. Avoid deleting your files, or think long and hard before doing so.  HPC storage is not backed up.

All data placed on HPC is replicated to an off-site location, but this does not protect against accidental deletion.

Backups

Unfortunately, HPC does not have sufficient funding to provide a regular backup target for research data.

For small amounts of data, you could backup your data to OneDrive, DropBox, or similar.

There are many external options in this space with varying levels of cost.  Be aware of data transfer costs prior to signing up - many offer free uploads but downloading (recovery) is quite/very expensive.

  • No labels