PrintPrint

Discover Supercomputer: NCCS’s Primary Computing Platform

The centerpiece of the NCCS is the 90,000-core “Discover” supercomputing cluster, an assembly of multiple Linux scalable units built upon commodity components capable of nearly 3.5 petaflops, or 3,500 trillion floating-point operations per second.

Discover is particularly suited for large, complex, communications-intensive problems employing large matrices and science applications, which benefit from its ecosystem of system software and tools. Example applications include fine-scale, high-fidelity simulations of the atmosphere and oceans that span time scales from days (weather prediction), to seasons and years (short-term climate prediction), to decades and centuries (climate change projection).

Discover derives its name from the NASA adage of "Explore. Discover. Understand.” The NCCS installed the first scalable unit in the fall of 2006 and continues to expand this highly successful computing platform.

Discover System Details

System Architecture

As mentioned above, the system architecture is made up of multiple scalable units. The list below describes the total aggregate components of the system and its individual scalable units.

Aggregate

  • 89 Racks (compute, storage, switches, and more)
  • 3.478 Pflop/s
  • 90,336 Total Cores

File System and Storage

  • IBM GPFS
  • 45 PB Storage

Operating Environment

  • Operating System: SLES
  • Job Scheduler: Slurm
  • Compilers: C, C++, Fortran (Intel and PGI)
  • Analysis applications include IDL, Matlab, and more

Individual Scalable Units

    Scalable Unit 9

    • Manufacturer: IBM
    • 91 Tflop/s
    • 4,480 Intel Xeon Sandy Bridge processor cores
    • IBM iDataPlex Compute Nodes
    • 2 8-core processors per node
    • 64 GB of memory per node (4GB per core)
    • No Swap Space
    • 2.6 GHz Intel Xeon Sandy Bridge
    • Interconnect: Infiniband FDR
    • Production: 3Q 2013

    Scalable Unit 10

    • Manufacturer: SGI
    • 1,229 Tflop/s
    • 30,240 Intel Xeon Haswell processor cores
    • SGI C2112 Compute Nodes
    • 2 14-core processors per node
    • 128 GB of memory per node
    • No Swap Space
    • 2.6 GHz Intel Xeon Haswell
    • Interconnect: Infiniband FDR
    • Production: 1Q 2015

    Scalable Unit 11

    • Manufacturer: SGI
    • 683 Tflop/s
    • 17,136 Intel Xeon Haswell processor cores
    • SGI C2112 Compute Nodes
    • 2 14-core processors per node
    • 128 GB of memory per node
    • No Swap Space
    • 2.6 GHz Intel Xeon Haswell
    • Interconnect: Infiniband FDR
    • Production: 1Q 2015

      Scalable Unit 12

      • Manufacturer: SGI
      • 683 Tflop/s
      • 17,136 Intel Xeon Haswell processor cores
      • SGI C2112 Compute Nodes
      • 2 14-core processors per node
      • 128 GB of memory per node
      • No Swap Space
      • 2.6 GHz Intel Xeon Haswell
      • Interconnect: Infiniband FDR
      • Production: 2Q 2015

      Scalable Unit 13

      • Manufacturer: SGI
      • 723 Tflop/s
      • 18,144 Intel Xeon Haswell processor cores
      • SGI C2112 Compute Nodes
      • 2 14-core processors per node
      • 128 GB of memory per node
      • No Swap Space
      • 2.6 GHz Intel Xeon Haswell
      • Interconnect: Infiniband FDR
      • Production: 2Q 2016

      Scalable Unit 14

      • Manufacturer: Supermicro
      • 1,560 Tflop/s
      • 20,800 Intel Xeon Skylake processor cores
      • 520 Supermicro FatTwin nodes
      • 2 20-core processors per node
      • 192 GB of memory per node
      • No Swap Space
      • 2.4 GHz Intel Xeon Skylake
      • Interconnect: Intel OmniPath
      • Target: Q2 2018