University:
Boston University
Course:
MET CS 201 | Introduction to Programming
Academic year:

2023
Views:

175

Pages:

13
Author:

eishuma5iuk

Introduction to High Performance Computing Outline • What is HPC? Why computer cluster? • Basic structure of a computer cluster • Computer performance and the top 500 list • HPC for scientific research and parallel computing • National-wide HPC resources: XSEDE What is HPC? • High Performance Computing (HPC) refers to the practice of aggregating computing power in order to solve large problems in science, engineering, or business. • Purpose of HPC: accelerates computer programs, and thus accelerates work process. • Computer cluster: A set of connected computers that work together. They can be viewed as a single system. • Similar terminologies: supercomputing, parallel computing. • Parallel computing: many computations are carried out simultaneously, typically computed on a computer cluster. • Related terminologies: grid computing, cloud computing. Computing power of a single CPU chip • Moore‘s law is the observation that the computing power of CPU doubles approximately every two years. • Nowadays the multi-core technique is the key to keep up with Moore's law. Why computer cluster? • Drawbacks of increasing CPU clock frequency: --- Electric power consumption is proportional to the cubic of CPU clock frequency (ν3). --- Generates more heat. • A drawback of increasing the number of cores within one CPU chip: --- Difficult for heat dissipation. • Computer cluster: connect many computers with highspeed networks. • Currently computer cluster is the best solution to scale up computer power. • Consequently software/programs need to be designed in the manner of parallel computing. Basic structure of a computer cluster • Cluster – a collection of many computers/nodes. • Rack – a closet to hold a bunch of nodes. • Node – a computer (with processors, memory, hard disk, etc.) • Socket/processor – one multi-core processor. • Core/processor – one actual processing unit. • Network switch • Storage system • Power supply system • Cooling system  Figure: IBM Blue Gene supercomputer Inside a node 1. Network device --- Infiniband card: to transfers data between nodes. 2. CPU --- Xeon multi-core processors: to carry out the instructions of programs. 3. Memory: fast and temporal storage, to store data for immediate use. 4. Hard disk: slow and permanent storage to store data permanently. 5. Space for possible upgrade 6. Accelorator --- Intel Xeon Phi Coprocessor (Knights Corner): to accelerate programs.  Figure: A node of the supercomputer Stampede at TACC. Accelerators  NVIDA GPU (Tesla P100):  Intel Xeon Phi MIC processor (Knights Landing): • Multiprocessors: 56 • Cores: 68; Threads: 272 • CUDA cores: 3584 • Frequency: 1.4 GHz; Two 512-bit VPUs • Memory: 12 GB • Memory: 16 GB MCDRAM + external RAM • PCI connection to host CPU • Self-hosted • Peak DP compute: 4036‒4670 GFLOPS • Peak DP compute: 3046 GFLOPS What resources does an HPC system provide? • A large number of compute nodes and cores. • Large-size (~ TB) and high-bandwidth memory. • Large-size (~ PB) and fast-speed storage system; storage for parallel I/O. • High-speed network: high-bandwidth Ethernet, Infiniband, Omni Path, etc. • Graphic Processor Unit (GPU) • Xeon Phi many-integrated-core (MIC) processor/coprocessor. • A stable and efficient operation system. • A large number of software or applications. • User services. How to measure computer performance? • Floating-point operations per second (FLOPS): 𝑐𝑜𝑟𝑒𝑠 𝑐𝑦𝑐𝑙𝑒𝑠 𝐹𝐿𝑂𝑃𝑠 𝐹𝐿𝑂𝑃𝑆 = 𝑛𝑜𝑑𝑒𝑠 × × × 𝑛𝑜𝑑𝑒𝑠 𝑠𝑒𝑐𝑜𝑛𝑑 𝑐𝑦𝑐𝑙𝑒 • The 3rd term clock cycles per second is known as the clock frequency, typically 2 ~ 3 GHz. • The 4th term FLOPs per cycle is how many floating-point operations are done in one clock cycle. Typical values for Intel Xeon CPUs are: --- Sandy Bridge and Ivy Bridge: 8 DP FLOPs/cycle, 16 SP FLOPs/cycle. --- Haswell and Broadwell : 16 DP FLOPs/cycle, 32 SP FLOPs/cycle. • GigaFLOPS – 109 FLOPS; TeraFLOPS – 1012 FLOPS; PetaFLOPS – 1015 FLOPS; ExaFLOPS – 1018 FLOPS. Computer power grows rapidly • Iphone 4 vs. 1985 Cray-2 supercomputer • Rapid growth of the power of the top-500 supercomputers (logarithmic y-axis, in GFLOPS) Top 500 Supercomputers • The list of June 2017 Statistics of the Top 500