University:
Boston University
Course:
GRS CS 910 | Directed Study: Systems
Academic year:

2022
Views:

324

Pages:

24
Author:

vielay3og

GPU Computing with CUDA CUDA Memories 1 Outline of lecture ‣ Warp scheduling ‣ CUDA Memory hierarchy ‣ Introduction to the Finite Difference Method 2 Recap ‣ GPU - A real alternative in scientific computing - Massively parallel commodity hardware ‣ Market driven development ‣ Cheap! ‣ CUDA - NVIDIA technology capable of programming massively parallel processors with general purpose - C/C++ with extensions 3 Recap ‣ GPU chip - Designed for high throughput tasks - Many simple cores - Fermi: ‣ 16 SMs - 32 SPs (cores) SP - Shared, registers, cache - SFU, CU ‣ 1TFLOP peak throughput ‣ 144GB/s peak bandwidth SM 4 Recap ‣ Programming model - Based on threads - Thread hierarchy: grouped in thread blocks - Threads in a thread block can communicate ‣ Challenge: parallel thinking - Data parallelism - Load balancing - Conflicts - Latency hiding 5 CUDA - Programming model ‣ Connecting the hardware and the software Hardware Software representation 6 CUDA - Programming model ‣ Connecting the hardware and the software Thread Thread block Hardware Software representation 6 CUDA - Programming model ‣ Connecting the hardware and the software Hardware Software representation 6 CUDA - Programming model ‣ Connecting the hardware and the software Hardware Software representation 6 CUDA - Programming model ‣ Connecting the hardware and the software Hardware Software representation 6 How a Streaming Multiprocessor (SM) works ‣ CUDA Threads are grouped in thread blocks - All threads of the same thread block are executed in the same SM at the same time ‣ SMs have shared memory, then threads within a thread block can communicate - The entirety of the threads of a thread block must be executed before there is space to schedule another thread block 7 How a Streaming Multiprocessor (SM) works ‣ Hardware schedules thread blocks onto available SMs - No guarantee of order of execution - If a SM has more resources the hardware will schedule more blocks Block 100 Block 15 Block 7 SM Block 1 8 How a Streaming Multiprocessor (SM) works ‣ Hardware schedules thread blocks onto available SMs - No guarantee of order of execution - If a SM has more resources the hardware will schedule more blocks Block 100 Block 15 Block 1 SM Block 7 8 How a Streaming Multiprocessor (SM) works ‣ Hardware schedules thread blocks onto available SMs - No guarantee of order of execution - If a SM has more resources the hardware will schedule more blocks Block 100 Block 15 Block 7 Block 1 SM 8 How a Streaming Multiprocessor (SM) works ‣ Hardware schedules thread blocks onto available SMs - No guarantee of order of execution - If a SM has more resources the hardware will schedule more blocks Block 100 Block 15 Block 7 Block 1 SM 8 How a Streaming Multiprocessor (SM) works ‣ Hardware schedules thread blocks onto available SMs - No guarantee of order of execution - If a SM has more resources the hardware will schedule more blocks Block 100 Block 15 Block 7 SM 8 How a Streaming Multiprocessor (SM) works ‣ Hardware schedules thread blocks onto available SMs - No guarantee of order of execution - If a SM has more resources the hardware will schedule more blocks Block 15 Block 7 Block 100 SM 8 Warps ‣ Inside the SM, threads are launched in groups of 32 called warps - Warps share the control part (warp scheduler) - At any time, only one warp is executed per SM - Threads in a warp will be executing the same instruction - Half warps for compute capability 1.X - Fermi: ‣ Maximum number of active threads 1024*8*32 = 262144 9 Warps ‣ Inside the SM, threads are launched in groups of 32 called warps - Warps share the control part (warp scheduler) - At any time, only one warp is executed per SM - Threads in a warp will be executing the same instruction - Half warps for compute capability 1.X - Fermi: Max threads per block ‣ Maximum number of active threads 1024*8*32 = 262144 9 Warps ‣ Inside the SM, threads are launched in groups of 32 called warps - Warps share the control part (warp scheduler) - At any time, only one warp is executed per SM - Threads in a warp will be executing the same instruction - Half warps for compute capability 1.X - Fermi: Max threads per block Max blocks per SM ‣ Maximum number of active threads 1024*8*32 = 262144 9 Warps ‣ Inside the SM, threads are launched in groups of 32 called warps - Warps share the control part (warp scheduler) - At any time, only one warp is executed per SM - Threads in a warp will be executing the same instruction - Half warps for compute capability 1.X - Fermi: Max threads per block Max blocks per SM SMs ‣ Maximum number of active threads 1024*8*32 = 262144 9 Warp designation ‣ Hardware separates threads of a block into warps - All threads in a warp correspond to the same thread block - Threads are placed in a warp sequentially - Threads are scheduled in warps Block N, Warp 1 Block N, Warp 2 Block N 16x8 TB Block N, Warp 3 Block N, Warp 4 Warp scheduling ‣ The SM implements a zero-overhead warp scheduling - Warps whose next instruction has its operands ready for consumption are eligible for execution - Eligible warps are selected for execution on a prioritized scheduling policy - All threads in a warp execute the same instruction 11 Warp scheduling ‣ Fermi - Double warp scheduler - Each SM has two warp schedulers and two instruction units 12

GPU Computing with CUDA

Related Documents

Recommended Documents

Description

Related Documents

Free up your schedule!

Take 5 seconds to unlock