...
- HPC storage has been purchased to deliver expected capacity requirements - evaluation of vendor responses is dominated by $/TB.
- Our current (2021) general use storage platform is very much entry level capabilities when it comes to performance.
- Researchers should avoid running parallel/concurrent
tar,gzip,etc. commands. Utilities such as pbzip2 should also be avoided.- If your computational research is I/O intensive, ensure that it is configured to use local scratch space (
/fast/tmp) - Consider other researchers and create a separate, sequential operations job for all your post-job I/O transactions.
- If your computational research is I/O intensive, ensure that it is configured to use local scratch space (
- Compression tools don't provide much benefit when working on binary data files - it's generally better to leave binary files uncompressed
...