Child pages
  • HPRC Infrastructure History
Skip to end of metadata
Go to start of metadata

IP Networks

TimelineIDDescriptionPorts speedPorts available
2005 – 09/201101Five (5) Nortel 5510 IP switches1Gb/s240 (BMC, admin, MPI, NFS)
10/2011 – 06/201202a

One (1) SMC 8848M IP switch (BMC)

One (1) SMC 8848M IP switch (cluster)

1Gb/s

1Gb/s

48 (BMC)

48 (admin)

06/2012 – 08/201302bOne (1) Nortel 5510 IP switch added1Gb/s96 (BMC)
08/2013 – present02cOne (1) Edge-corE 4610-50T IP switch added1Gb/s144 (BMC)

IB Networks

Purchase DateIDDescriptionPortsDecommission
10/201103Two (2) Voltaire IB switches72 @ 40Gb/s 
10/201403aOne (1) Mellanox IB switch36 @ 56Gb/s 
01/201703bTwo (2) Mellanox IB switches72 @ 56Gb/s 
    Timeline ID Description Ports speed Ports available
    2000 – 2007 04 Two (2) FC switches 1Gb/s Unknown
    2005 – present 05 ITR FC switches (shared) 2Gb/s, 4Gb/s, 8Gb/s Varied
    08/2013 – present 06 Two (2) Brocade 6510 FC switches (RDSI) 8Gb/s, 16Gb/s 48
    Timeline ID Description FC speed Raw size Spindle count and speed(s)
    2000 – 2007 07 SGI TP9100 Storage Array ? x 1Gb/s 2.5TB

    54 spindles, 10K RPM

    2005 – 2007 08

    SUN FLX380 Storage Expansion*

    ? x 2Gb/s 6.2TB

    28 spindles, 10K RPM and 7200 RPM

    2008 – 2009 09a

    Compellent Storage Array* (SPoF)

    ? x 2Gb/s

    ? x 4Gb/s

    8.4TB

    16 spindles, 15K RPM and 7200 RPM

    2009 – 2011 09b

    Compellent Storage Expansion*

    1Gb/s iSCSI ports

    ? x 2Gb/s

    ? x 4Gb/s

    32.8TB

    48 spindles,15K RPM and 7200 RPM (shared)

    Long-term performance problems - found to be due to dirty fibres.

    Thin provisioning, safety features, and funding issues led to outages.

    2010-2012 10

    IBM XIV Storage Arrays

    4 Gb/s FC ports

      N/A Varied, 7200RPM (shared).  Used for research VM provisioning only.
    10/2011-present 11

    SGI IS5000 (scratch)

    SGI IS5000 (primary)

          after upgrade

    4 x 8Gb/s

    4 x 8Gb/s

     

    48TB

    120TB

    180TB

    24 spindles, 7200RPM

    60 spindles, 7200RPM

    80 spindles, 7200RPM

    2013 – present 09c Compellent Storage Upgrade   Unknown

    Varied, 15K RPM and 7200 RPM (shared)

    Used for research VM provisioning only.

    08/2013 – present 12 SGI IS5500 (RDSI stage 1) 16 x 8Gb/s 540TB 180 spindles, 7200RPM

    *  Controllers weren't strictly compatible with HPC NAS server(s).

     

     

      In Oct-2011, newly purchased SGI infrastructure was installed, consisting of:
      One IS5000 storage array - dual active controllers, 3GB cache, 60 2TB SATA disks, 8x8Gb/s FC ports (4 in use)
      One IS5000 storage array - dual active controllers, 3GB cache, 24 2TB SATA disks, 8x8Gb/s FC ports (4 in use)

      In Feb 2013, HPC staff moved VMs from XIV to a new DELL Compellent storage arrays purchased by ITR. This move was done due to lack of space on the very expensive IBM XIV storage systems. Four 500GB and one 1TB volume have been provided to serve as back-end storage for VMs managed/installed by HPRC staff.

      Tape Libraries

        This hardware was dedicated to HPC.
        Library and tape drives were SCSI connected to server.
        4 DLT-4 tape drives were housed in the L180 (2000-2005).
        L180 was upgraded to house 4 LTO-2 tape drives (2005-2007).
        The number of tape cartridges held at a given point in time is unknown.

        This unit was shared by corporate ITR (backups) and HPC (HSM).
        Five 4Gb/s FC HBAs used for connection of library and tape drives to server(s).
        Approximately 200 cartridge slots licensed and reserved for HPC use.
        Housed 4 LTO-4 tape drives for exclusive HPC use.
        In Feb-2011, 189 LTO-4 tape cartridges were held in the SL500 for HPC use.

        In Oct-2011, a SpectraLogic T950 tape library was installed for HPC use (dedicated).
        The tape library has one (1) 2Gb/s FC connection for library (robot) operations.
        The tape library houses eight (8) LTO-5 tape drives, each connected via 8Gb/s FC.
        950 tape catridge slots were licensed at the time of purchase. 300 LTO-5 tapes were present in the tape library immediately after installation/configuration. Another 100 LTO-5 tapes were added to this library in mid-2012.

        Servers

          Supercomputing came to JCU with the arrival of a CRAY EL-92 system.
          The details of this system (long since decommissioned) aren't known.

          Handwritten notes appear to indicate that an IBM RS6000 system (grunter) was installed in 1994.
          The details of this system (long since decommissioned) aren't known.

          An SGI PowerChallenge (octopus) system was installed in 1997.
          The details of this system (long since decommissioned) aren't fully known. The information below comes from an archive of an old HPC web page. Given the name of this system, it is likely that it was purchased with only 8 CPUs and was later upgraded to 16 CPUs.

          Component

          Available Details

          CPUs

          16x(MIPS R10000 195MHz)

          ECC memory

          4GB

          SCSI disks

          6x(4.3GB), 10x(9GB), 3x(18GB)

          Networking

          100Mb/s (FDDI and ethernet)

          An SGI Origin 3200 (hydra) system was installed in 2000.
          An SGI 1200 server was also installed for administration of the Origin 3200 (reused, later, for Altix 350).
          The details of the SGI Origin system (long since decommissioned) aren't fully known. The information below comes from archives of old HPC web pages and some from human memory. Several upgrades were made to FC storage capacity and the TP9100 also used for ITR mail and web services storage. Near the end of it's life hydra likely presented out closer to 2TB of FC storage.

           

          Initial

          2002 Upgrade

          CPUs

          16x(MIPS R14000 500MHz)

          16x(MIPS R14000 500MHz)

          CPUs

          4x(MIPS R12000 400MHz)

          68x(MIPS R12000 400MHz)

          ECC memory

          10GB

          42GB

          SCSI storage

          2x(18GB), RAID-1

          2x(18GB), RAID-1

          FC storage

          1.1TB from TP9100

          1.1TB from TP9100

          Networking

          2x(1Gb/s)

          2x(1Gb/s)

          FC HBAs

          2x(1Gb/s)

          2x(1Gb/s)

          This system featured NUMAlink interconnect between C-bricks (housing CPUs and memory). DMF software, facilitating HSM for research data, was installed on this system.

          SGI Altix 350 (dmf) installed - NAS+HSM server for a compute cluster.

          Details (dmf)

          The details of this system aren't fully known. The information below comes from packing slips and human memory. I believe this system was upgraded at some stage in its life, but haven't found any documentation associated with an upgrade yet.

          CPUs

          2x(Itanium2 1.5GHz)

          ECC memory

          8GB

          SCSI storage

          2x(146GB 10000 RPM), RAID-1

          FC storage

          4.5TB from FLX380

          Networking

          4x(1Gb/s)

          FC HBAs

          2x(2Gb/s)

          75 SUN V20Z servers (Config 1) installed.
          3 SUN V40Z servers (Config 1) installed.
          1 DELL PowerEdge 2850 server (ng0) installed - hosting APAC National Grid VMs.
          Reconfiguration (2010-2011)
          36 V20Z servers were decommissioned in 2010, memory was removed into remaining V20Z compute nodes, resulting in Config 2.
          Additional V20Z/V40Z memory permitted an memory upgrade of V40Z servers. Also, 9 disks from decommissioned V20Z servers were removed into V40Z servers, resulting in Config 2. Each V40Z server also had its dual-port FC HBA and dual-port gigabit ethernet cards removed. These V40Z servers were decommissioned in early 2011.

          Details (V20Z servers, Config 1)

          CPUs

          2x(2.6GHz AMD Opteron 252)

          ECC memory

          4GB (DDR 400MHz)

          SCSI disks

          2x(73GB 10000RPM), RAID-1

          Networking

          2x(1Gb/s)

          Details (V40Z servers, Config 1)

          CPUs

          2x(2.4GHz AMD Opteron 852)

          ECC memory

          4GB (DDR 333MHz)

          SCSI disks

          2x(73GB 10000RPM), RAID-1

          Networking

          4x(1Gb/s)

          FC HBAs

          2x(1Gb/s)

          Details (ng0)

          CPUs

          2x(2.8GHz Intel Xeon)

          ECC memory

          4GB (DDR2 400MHz)

          SCSI disks

          2x(146GB 10000RPM), RAID-1

          Networking

          4x(1Gb/s)

          Details (V20Z servers, Config 2)

          CPUs

          2x(2.6GHz AMD Opteron 252)

          ECC memory

          8GB (DDR 400MHz)

          SCSI disks

          2x(73GB 10000RPM)

          Networking

          2x(1Gb/s)

          Details (V40Z servers, Config 2)

          CPUs

          2x(2.4GHz AMD Opteron 852)

          ECC memory

          8GB (DDR 333MHz)

          SCSI disks

          5x(73GB 10000RPM)

          Networking

          2x(1Gb/s)

          SGI Altix 450 (dmf1) installed as a NAS+HSM server for a compute cluster.
          33 SUN X2200M2 servers (Config 1) installed.
          2 SUN X2200M2 servers (Config 2) installed.
          3 SUN X4600M2 servers installed.
          Reconfigurations (2009)
          In 2009, a second Altix 450 (dmf2) was purchased so that a highly available NAS+HSM service could be setup (by SGI professional services).
          Reconfigurations (2010)
          In Q1 2010, an upgrade was made to all X2200M2 servers (see Config 1A and Config 2A below).
          In Q3 2010, one of the X4600M2 servers also had a PCI-X dual-port (2Gb/s) FC HBA card installed. This server hosted VMs with storage residing on ITRs XIV storage appliances. No separate configuration item is listed below for this one-off upgrade.

          Details (dmf1)

          CPUs

          4x(1.6GHz dual-core Itanium 9140M)

          ECC memory

          32GB (DDR2 667MHz)

          SAS disks

          2x(300GB 15000RPM), RAID-1

          Networking

          10x(1Gb/s)

          FC HBAs

          4x(4Gb/s)

          Details (X2200M2, Config 1)

          CPUs

          2x(2.3GHz quad-core AMD Opteron 2356)

          ECC memory

          8GB (DDR2 666MHz)

          SATA disks

          1x(250GB 7200RPM)

          Networking

          4x(1Gb/s)

          Details (X2200M2, Config 2)

          CPUs

          2x(2.3GHz quad-core AMD Opteron 2356)

          ECC memory

          8GB (DDR2 666MHz)

          SAS disks

          2x(146GB 15000RPM), RAID-1

          Networking

          4x(1Gb/s)

          Details (X4600M2)

          CPUs

          8x(2.8GHz dual-core AMD Opteron 8220)

          ECC memory

          32GB (DDR2 667MHz)

          SAS disks

          2x(146GB 15000RPM)

          Networking

          4x(1Gb/s)

          Details (dmf2)

          CPUs

          4x(1.6GHz dual-core Itanium 9140M)

          ECC memory

          32GB (DDR2 667MHz)

          SAS disks

          2x(146GB 15000RPM)

          Networking

          10x(1Gb/s)

          FC HBAs

          4x(4Gb/s)

          Details (X2200M2, Config 1A)

          CPUs

          2x(2.3GHz quad-core AMD Opteron 2356)

          ECC memory

          16GB (DDR2 666MHz)

          SATA disks

          1x(250GB 7200RPM)

          Networking

          4x(1Gb/s)

          Details (X2200M2, Config 2A)

          CPUs

          2x(2.3GHz quad-core AMD Opteron 2356)

          ECC memory

          16GB (DDR2 666MHz)

          SAS disks

          2x(146GB 15000RPM), RAID-1

          Networking

          4x(1Gb/s)

          FC HBAs

          2x(4Gb/s)

          Workstations

          In the timeframe 2000-2005, the following workstation equipment was received and used by HPC staff and select JCU researchers.

          Qty

          Description

          Processor(s)

          Memory

          Storage

          Network

          6

          SGI O2

          1x(MIPS)

           

           

           

          2

          SGI Octane

          1x(195MHz MIPS R10000 - 1MB cache)

           

           

           

          2

          SGI 750

          2x(733MHz Intel Itanium - 2MB L2 cache)

          1GB

          1x(18GB Ultra-3 SCSI)

          10Mb/s

          1

          SUN Netra N1

          1x(500MHz UltraSPARC IIe - 512KB L2 cache)

          1GB

          2x(40GB IDE)

          100Mb/s

          8

          SUN Netra N1

          1x(500MHz UltraSPARC IIe - 512KB L2 cache)

          1GB

          1x(40GB IDE)

          100Mb/s

          1

          SUNblade 2000

          2x(900MHz UltraSPARC III - 2MB L2 cache)

          4GB

           

           

          20

          SGI 330L

          2x(933MHz Intel Pentium III - 256KB L2 cache)

           

          1x(18GB Ultra160 SCSI)

           

          One SUN Netra N1 provided the QPSF website service. Another SUN Netra N1 provided the Tarantella service (X terminal access from a web browser and more).

          • No labels