This page has been created for researchers using zip for compression. In the past, HPC staff are aware of people creating zip files (>2GiB) which cannot be unzipped on HPC - presumably due to unzip limitations at the time. The current version of unzip supports extraction of content that is >2GiB in size If you want to use zip, be sure to check that the created file can be unzipped before removing original file(s). If you encounter this problem, one solution that has worked for me is to perform the unzip on platform with a 64-bit version of unzip. I cannot guarantee this work-around is robust.
Generally, I'd recommend using
tar for grouping multiple files into a single file. The GNU
tar command is capable of performing compression inline. In my experience,
xz provides the best compression rates (comparison with gzip & bzip2). An example that creates a compressed tarball of everything with a directory (
tar cfJ proj1.tar.xz proj1/ && echo "Success"
Note: Processes can be interrupted/killed - e.g., SSH connection timeout. If you don't see a "Success" message, assume that the tarball is incomplete/corrupted - try again.