There is a significant risk to relying on your desktop/laptop computer and/or USB disks. Example situations to think about: Disk failure, corruption of data, power outages. I know of about 5 cases where PhD students nearing the end of their candidature have "lost everything" because a disk in their laptop failed. I have been asked to try and recover files from a faulty USB disk - not always possible. JCU perform regular RCD testing - expect to lose power from time to time.
The longer your computational research jobs take, the more you should be aware of the following.
- Submitting jobs (using
qsub) to compute nudes is recommended.
- If you must work interactively, learn how to use the
screencommand in Linux.
- Be aware of how much resource your jobs require - particularly the amount of memory (RAM) required. Overconsumption of memory on a server can result in failiure - all running jobs on that server will need to be re-run (not just yours).
- Identify the minimum resource requirement (CPU, memory, and disk space) for your work.
- Request a resource (virtual machine) - HPC will be able to cater for most requirements. Use Remote Desktop software to work on that resource.
Research Data Protection
If your research produces data/information that can never be recaptured (e.g., photos/videos of "events"), make every effort to protect it - you cannot travel back in time.
- Keep one copy of all research data on HPC. Feel free to have other copies elsewhere (e.g., on you desktop/laptop computer).
- Establishing a regular (e.g., daily) synchronisation of data is best. This process should only update changes.
- Avoid deleting your files, or think long and hard before doing so. HPC storage is not backed up.
All data placed on HPC is replicated to an off-site location, but this does not protect against accidental deletion.
Unfortunately, HPC does not have sufficient funding to provide a regular backup target for research data.
For small amounts of data, you could backup your data to OneDrive.
There are many external options in this space with varying levels of cost. Be aware of data transfer costs prior to signing up - many offer free uploads but downloading (recovery) is quite/very expensive.