Network Interconnect
| ID | Timeline | Switch(es) Information | SSH (Gb/s) | MPI/NFS (Gb/s) |
|---|
| 01 | 2005 – 2011 | Ethernet: Nortel 5510 (x5) | 1 | 1 |
| 02 | 2011 – 2016 | Ethernet: SMC 8848M (x2) | 1 |
|
| 2011 – 2016 | Infiniband: Voltaire 4036 (x2) |
| 40 |
| 2013 – 2017 | Ethernet: Edge-corE 4610-50T (x1) | 1 |
|
| 2014 – 2017 | Infiniband: Mellanox SX6036 (x1) |
| 40/56 |
| 2016 – 2017 | Infiniband: Mellanox SX6036 (x2) | 56 | 40/56 |
| 03 | 2017 – present | Ethernet: Ubiquiti EL48s (x3) | 1 |
|
| 2017 – 2019 | Ethernet: HPE FF5950 (x1) |
| 25/100 |
| 2018 – present | Ethernet: Juniper EX4650 (x1) |
| 25/100 |
| 2019 – present | Ethernet: Juniper EX4300 (x2) | 1 |
|
| 2019 – present | Ethernet: Juniper QFX5100 (x1) |
| 10/40 |
Storage Interconnect
| ID | Timeline | Switch(es) Information | Port speeds (Gb/s) |
|---|
| 11 | 2000 – 2007 | FC (shared) | 1Gb/s |
| 12 | 2005 – 2013 | Brocade FC (shared) | up to 8Gb/s |
| 13 | 2013 – present | Two (2) Brocade 6510 FC | up to 16Gb/s |
Storage Platforms (dates related to HPC use only)
| ID | Timeline | Description | Raw size | Spindle count and speed(s) | Other notes |
|---|
| 21 | 2000 – 2007 | SGI TP9100 | 2.5TB | 54 (10K RPM) |
|
| 22 | 2005 – 2007 | SUN FLX380^ | 6.2TB | 28 (10K & 7200 RPM) - shared |
|
| 23 | 2008 – 2011 | Compellent* | 8.4TB | 16 (15K & 7200 RPM) |
|
| 2009 – 2011 | Compellent (expansion)* | 32.8TB | 48 (15K & 7200 RPM) - shared |
|
| 24 | 2010-2012 | IBM XIV Storage Arrays | ?? | ?? (7200 RPM with very large cache) - shared. | For research VMs |
| 25 | 2011-2016 | SGI IS5000 (scratch) | 48TB | 24 spindles (7200 RPM) | 4x8Gb/s FC |
| 2011-2016 | SGI IS5000+ | 120TB | 60 spindles (7200 RPM) | 4x8Gb/s FC |
| 26 | 2013 – present | Compellent | ?? | ?? (15K & 7200 RPM ) - shared | For research VMs |
| 27 | 2013 – 2018 | SGI IS5500 | 540TB | 180 spindles (7200 RPM) | 8x8Gb/s FC |
| 28 | 2014 – 2020 | DDN SFA12K40 | 2240TB | 12 SSDs, 560 spindles (7200 RPM) | 4x56Gb/s IB |
| 29 | 2016 – 2021 | DELL SC4020 Storage | 1070TB | 14 SSDs, 160 spindles (7200 RPM) |
|
| 30 | 2019 – present | DDN SFA7990E | 768TB | 12 SSDs, 64 spindles (7200 RPM) |
|
^ Controllers weren't strictly compatible with HPC NAS/HSM server(s) at the time.
* Controllers weren't strictly compatible with HPC NAS/HSM server(s) at the time. Long-term performance problems - found to be due to dirty fibres. Thin provisioning, safety features, and funding issues led to outages.
+ Platform was upgraded by the addition of 20x3TB disks purchased by the VECNet project.
Tape Libraries (near-line storage for HSM)
| ID | Timeline | Description | Extra Information |
|---|
| 41 | 2000 – 2007 | StorageTek L180 | 4 DLT4 tape drives (until 2005)
4 LTO2 tape drives (2005 – 2007) |
| 42 | 2007 – 2011 | Sun SL500 | 4 LTO4 tape drives, up to189 LTO4 cartridges were installed |
| 43 | 2011 – 2016 | SpectraLogic T950 | 8 LTO5 tape drives, 300 LTO5 cartriges initially, increased to 600 over time |
Servers
Cray (1992 – 1994)
Supercomputing came to JCU with the arrival of a CRAY EL-92 system.
The details of this system (long since decommissioned) aren't known.
IBM (1994 – 1997)
Handwritten notes appear to indicate that an IBM RS6000 system (grunter) was installed in 1994.
The details of this system (long since decommissioned) aren't known.
SGI (1997 – 2002)
An SGI PowerChallenge (octopus) system was installed in 1997. The details of this system (long since decommissioned) aren't fully known. The information below comes from an archive of an old HPC web page. Given the name of this system, it is likely that it was purchased with only 8 CPUs and was later upgraded to 16 CPUs.
Component | Available Details |
|---|
CPUs | 16x(MIPS R10000 195MHz) |
ECC memory | 4GB |
SCSI disks | 6x(4.3GB), 10x(9GB), 3x(18GB) |
Networking | 100Mb/s (FDDI and ethernet) |
SGI (2000 – 2005)
An SGI Origin 3200 (hydra) system was installed in 2000. An SGI 1200 server was also installed for administration of the Origin 3200 (reused, later, for Altix 350).
The details of the SGI Origin system (long since decommissioned) aren't fully known. The information below comes from archives of old HPC web pages and some from human memory. Several upgrades were made to FC storage capacity and the TP9100 also used for ITR mail and web services storage. Near the end of it's life hydra likely presented out closer to 2TB of FC storage.
| Initial | 2002 Upgrade |
|---|
CPUs | 16x(MIPS R14000 500MHz) | 16x(MIPS R14000 500MHz) |
CPUs | 4x(MIPS R12000 400MHz) | 68x(MIPS R12000 400MHz) |
ECC memory | 10GB | 42GB |
SCSI storage | 2x(18GB), RAID-1 | 2x(18GB), RAID-1 |
FC storage | 1.1TB from TP9100 | 1.1TB from TP9100 |
Networking | 2x(1Gb/s) | 2x(1Gb/s) |
FC HBAs | 2x(1Gb/s) | 2x(1Gb/s) |
SGI (2005 – 2008)
An SGI Altix 350 (dmf) was installed in 2005 to serve as a NAS+HSM server for the HPC compute cluster. The details of this system aren't fully known. The information below comes from packing slips and human memory. I believe this system was upgraded at some stage in its life, but haven't found any documentation associated with an upgrade yet.
CPUs | 2x(Itanium2 1.5GHz) |
ECC memory | 8GB |
SCSI storage | 2x(146GB 10000 RPM), RAID-1 |
FC storage | 4.5TB from FLX380 |
Networking | 4x(1Gb/s) |
FC HBAs | 2x(2Gb/s) |
SGI (2005 – 2011)
The first cluster configured for JCU researcher use consisted of 75 SUN V20z servers, 3 SUN V40z servers, and 1 DELL PowerEdge 2850 server. The latter was used to host APAC National Grid VMs. A reconfiguration of V20z and V40z servers was performed in 2010 to double the memory per core ratio.
Original Server Builds
| V20z(s) | V40z(s) | DELL 2850 |
|---|
| CPUs | 2x(2.6GHz AMD Opeteron 252) | 2x(2.4GHz AMD Opteron 852) | 2x(2.8GHz Intel Xeon) |
| RAM | 4GB (400MHz) | 4GB (333MHz) | 4GB (400MHz) |
| Disks | 2x(73GB, 10K RPM), RAID-1 | 2x(73GB, 10K RPM), RAID-1 | 2x(146GB, 10K RPM), RAID-1 |
| Networking | 2x1Gb/s | 4x1Gb/s | 4x1Gb/s |
| FC HBAs |
| 2x1Gb/s |
|
SGI (2008 -- 2011)
An SGI Altix 450 (dmf1) was installed as a NAS+HSM server for the new HPC cluster, which consisted of 33 SUN X2200M2 servers (C1), 2 SUN X2200M2 servers (C2), and 3 SUN X4600M2 servers. A second Altix 450 (dmf2) was purchased in 2009 so that a highly available NAS+HSM service could be configured (by SGI professional services staff). In 2010, the memory in X2200M2 and X4600M2 servers saw a doubling of installed RAM.
Original Server Builds
| X2200M2 (C1) | X2200M2 (C2) | X4600M2 | Altix 450 |
|---|
| CPUs | 2x(2.3GHz quad-core AMD Opteron 2356) | 8x(2.8GHz dual-core AMD Opteron 8220) | 4x(1.6GHz dual-core Itanium 9140M) |
| RAM | 8GB (667MHz) | 32GB (667MHz) | 32GB (667MHz) |
| Disks | 1x(250GB, 7200RPM) | 2x(146GB, 15K RPM) | 2x(146GB, 15K RPM) | 2x(300GB 15000RPM), RAID-1 |
| Networking | 4x1Gb/s | 4x1Gb/s | 10x1Gb/s |
| FC HBAs |
|
|
| 4x4Gb/s |
SGI (2011--2018)
32xC2112-RP4 (CN)
2xC2112 (BM)
4xC2108 (VECNET)
1xC2018 (admin)
2xC3106 (nashsm?)
Workstations
In the timeframe 2000-2005, the following workstation equipment was received and used by HPC staff and select JCU researchers.
Qty | Description | Processor(s) | Memory | Storage | Network |
|---|
6 | SGI O2 | 1x(MIPS) |
|
|
|
2 | SGI Octane | 1x(195MHz MIPS R10000 - 1MB cache) |
|
|
|
2 | SGI 750 | 2x(733MHz Intel Itanium - 2MB L2 cache) | 1GB | 1x(18GB Ultra-3 SCSI) | 10Mb/s |
1 | SUN Netra N1 | 1x(500MHz UltraSPARC IIe - 512KB L2 cache) | 1GB | 2x(40GB IDE) | 100Mb/s |
8 | SUN Netra N1 | 1x(500MHz UltraSPARC IIe - 512KB L2 cache) | 1GB | 1x(40GB IDE) | 100Mb/s |
1 | SUNblade 2000 | 2x(900MHz UltraSPARC III - 2MB L2 cache) | 4GB |
|
|
20 | SGI 330L | 2x(933MHz Intel Pentium III - 256KB L2 cache) |
| 1x(18GB Ultra160 SCSI) |
|
One SUN Netra N1 provided the QPSF website service. Another SUN Netra N1 provided the Tarantella service (X terminal access from a web browser and more).