IMPORTANT: Officially, JCU's OneDrive for Business platform was not purchase to house research data, however JCU staff/students may choose to place research data on OneDrive.
Extensive documentation on the linux onedrive connector capabilities can be found at https://github.com/abraunegg/onedrive/blob/master/docs/USAGE.md - the information provided below is a very brief introduction for getting started. At the time of writing this document, HPC staff had only completed very limited testing of the Linux interface to OneDrive. The onedrive command can also be used to connect to SharePoint libraries (untested).
A command line based interface to OneDrive has been installed on HPC cluster servers. There is some configuration required before you start using the command(s) provided.
Create a "onedrive" directory somewhere within your scratch space directory.
mkdir -p -m 700 /scratch/jc012345/onedrive
Modify your environment by loading the onedrive module.
module load onedrive
Authenticate with OneDrive:
Issue the command
You should get a response something like:
[jc012345@ln01 ~]$ onedrive Authorize this app visiting: https://..... Enter the response uri:
Enter the response uri:prompt. You should receive a message that indicates successful authentication.
Now it's time to configure your use of
onedrive on HPC. The following is very similar to a configuration that has been tested by HPC staff for use of OneDrive as an archive for files on HPC.
mkdir -p ~/.config/onedrive cat <<EOF >~/.config/onedrive/config sync_dir = "/scratch/jc012345/onedrive" skip_file = "~*|.~*|*.tmp" skip_dotfiles = "true" skip_symlinks = "true" upload_only = "true" no_remote_delete = "true" remove_source_files = "false" EOF
To confirm your configuration (may be larger than the contents of your configuration file), execute
sync_dir" you configured - e.g., to move all *.dat files in your current working directory, use
mv *.dat /scratch/jc012345/onedrive.
Two aliases have been created for upload/download convenience. To upload files from your "
sync_dir", simply use the command:
At this stage, it looks like download capabilities are tied to sync'ing everything from OneDrive. Files you have uploaded to OneDrive from another personal/work computing device may not be permitted on HPC (research files only). Therefore, you should not be sync'ing (on HPC servers) data with OneDrive. Point in time downloads from OneDrive to HPC can be done, providing you delete all content that isn't related to your research.
The linux onedrive client installed on HPC systems is capable of working with Teams/Sharepoint libraries. However, due to JCU's O365 storage limit, files on HPC should not be sync'ed with Teams/Sharepoint document libraries. Such libraries can be synced to other computing devices (e.g., your desktop computer).
The future architecture of the JCU HPC cluster is under review - based on a government & university drive to utilize 'cloud' infrastructure where practical & sustainable. Technology solutions (including HPC) staff are investigation options for a future HPC cluster where >=95% of storage is likely to be scratch space, with routine deletion of old files. The
no_remote_delete = "true"
configuration entry (in
~/.config/onedrive/config) will be essential if this future state becomes reality.