Child pages
  • Selecting a storage resource that best fits my requirements
Skip to end of metadata
Go to start of metadata

Public cloud resources (AWS, Azure, etc.) are excluded from this document.  They may represent the best option in almost all cases, if you have ongoing budget to cover the costs.

Note:  Research data may not be held ad-infinitum - timeframe for archiving or removal of files varies.

JCU Storage Recommendations (primary copy of data)





HPC Cluster
PC/MacVirtual Machine/tmp/fast/home/scratch/gpfs01
Resiliency: Disk failure(s)noyesyesnoyesyesyes
Resiliency: Disasters (e.g., fire)noyesnonopossibly+noyes
Resiliency: Silent data corruptionnoyesnonoyesyesyes
Long-term data retentionnopossibly*nonoyesnoyes
Intensive I/O worknonoyesyesnonoyes#
Senstive datanopossibly*nonononono

IMPORTANT:  Personal computing devices are not considered to be a responsible location for housing the primary copy of your research data.

# The /gpfs01 filesystem has performance to do intensive I/O work, but it is only a cache to data being house in SE Qld.

* Suitability is dependent on configuration of the environment.  Consultation with TS staff required.

+ Due to size, several weeks of data could be lost in the event of a disaster.

Storage Sizings


Capacity (TB)

Space QuotasFile quotaNotes

CloudStor (AARNet)

10@1TB

Your quota can be increased by HPC staff.

OneDrive (Office365)

unknown5TB

Full sync to your device(s) may not be possible.

HPC Cluster: /home512

5TB

250,000

Temporary quota extensions are possible.

HPC Cluster: /scratch

80

5TB

1,000,000

Temporary quota extensions are possible.  Old data will be deleted.

HPC Cluster: /fast7

In-server SSDs.  Automatic deletion of files >=30 days old
HPC Cluster: /tmp0.3

In-server SSDs.  Automatic deletion of files >=14 days old

ARDC Allocation (QCIF)

516TBvaries

Merit based approval. 90% usage target for all collections.

Allocation may be reduced/removed if not used.

 Best choice for post-JCU access.

@ JCU is likely to purchase significantly more capacity from AARNet (CloudStor) for the housing of research data that isn't likely to be part of a computational research workflow.
Notes:
  After leaving JCU, you may lose access to your research data.  In time, it will be archived to an offsite location prior to local deletion.  Current government guidelines only require research data to be retained for 7 years.

Sensitive Data Requirements

Requirements relating to working with sensitive or classified data can incur an extremely high cost.  In 2020, JCU discussions around such requirements resulted in a decision to look at out-sourcing of infrastructure to support research in such areas.  Key points to note:

  • The JCU HPC cluster has not be architected/configured for work on senstive or classified data.
  • The JCU network has not been architected/configured for work on sensitive or classified data.
  • If you are working with sensitive of classified data, you should notify HPC/eResearch staff.
  • Researchers are responsible for ensuring that appropriate controls are in place for any work on sensitive or classified data.
  • If you are working with sensitive data it is important that you keep up to date with changing requirements (e.g., if an encryption algorithm is no longer safe).
  • No labels