University:
Boston University
Course:
GRS CS 910 | Directed Study: Systems
Academic year:

2022
Views:

206

Pages:

23
Author:

Malik Morrison

Generic CL Reduction: Preparation #define GROUP SIZE ${group size} #define READ AND MAP(i) (${map expr}) #define REDUCE(a, b) (${reduce expr}) % if double support : #pragma OPENCL EXTENSION cl khr fp64: enable % endif typedef ${out type} out type ; ${preamble} CL Reduction: Sequential Part kernel void ${name}( global out type ∗out, ${arguments}, unsigned int seq count, unsigned int n) { local out type ldata [GROUP SIZE]; unsigned int lid = get local id (0); unsigned int i = get group id(0)∗GROUP SIZE∗seq count + lid; out type acc = ${neutral}; for (unsigned s = 0; s < seq count; ++s) { if ( i >= n) break; acc = REDUCE(acc, READ AND MAP(i)); i += GROUP SIZE; } CL Reduction: Explicitly Synchronized Part ldata [ lid ] = acc; <% cur size = group size %> % while cur size > no sync size : barrier (CLK LOCAL MEM FENCE); <% new size = cur size // 2 assert new size ∗ 2 == cur size %> if ( lid < ${new size}) { ldata [ lid ] = REDUCE( ldata [ lid ], ldata [ lid + ${new size}]); } <% cur size = new size %> % endwhile CL Reduction: Implicitly Synchronized Part % if cur size > 1: barrier (CLK LOCAL MEM FENCE); if ( lid < ${no sync size}) { local volatile out type ∗lvdata = ldata; % while cur size > 1: <% new size = cur size // 2 assert new size ∗ 2 == cur size %> lvdata [ lid ] = REDUCE( lvdata [ lid ], lvdata [ lid + ${new size}]); <% cur size = new size %> % endwhile } % endif if ( lid == 0) out[get group id (0)] = ldata [0]; } Outline 1 Leftovers 2 Code writes Code 3 Case Study: Generic OpenCL Reduction 4 Reasoning about Generated Code 5 Automatic GPU Programming Judging Code Quality Possible information sources for judging code quality/desirability: Heuristics (e.g. Occupancy, Flops/Byte, . . . ?) OpenCL Event profiling Makes comp. synchronous on Nvidia! Wall time (!) Compiler build log Vendor Profiler Search Strategies Possible search strategies: Exhaustive Exhaustive + Heuristics Grouped Orthogonal Search Genetic Algorithms (your invention here) Compiler cache makes repeated searches fast. Grouped Orthogonal Search Grouped Orthogonal Search Define groups Group 1 Group 2 Group 3 Grouped Orthogonal Search Choose group Grouped Orthogonal Search Map admissible options Grouped Orthogonal Search Group-wide exhaustive search Grouped Orthogonal Search Start over with best result → pick new group. . . Using the Nvidia profiler in-process 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # enable profiler import os os. environ [”COMPUTE PROFILE”] = ”1” with open(”/tmp/myprg−prof−config”, ”w”) as prof config: prof config . write (”\n”. join (events )) os. environ [”COMPUTE PROFILE CONFIG”] = ”/tmp/myprg−prof−config” # obtain timing data prof f = open(” opencl profile 0 . log”, ”r”) gain count = 0 while gain count < 2: # run kernel here prof output = prof f . readlines () if prof output : print ”gained %d lines” % len(prof output ) gain count += 1 if gain count == 2: print ””. join ( l for l in prof output [1:−1] if kernel name in l ) Using the Nvidia profiler in-process 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # enable profiler import os os. environ [”COMPUTE PROFILE”] = ”1” with open(”/tmp/myprg−prof−config”, ”w”) as prof config: prof config . write (”\n”. join (events )) os. environ [”COMPUTE PROFILE CONFIG”] = ”/tmp/myprg−prof−config” # obtain timing data prof f = open(” opencl profile 0 . log”, ”r”) gain count = 0 while gain count < 2: # run kernel here prof outputSample = prof f output: . readlines () if prof output : print ”gained %d lines” % len(prof output ) gain count += 1 if Nvidia GPU Profiler: Events gld request : Number of executed global load instructions per warp in a SM gst request : Number of executed global store instructions per warp in a SM divergent branch : Number of unique branches that diverge instructions : Instructions executed warp serialized : Number of SIMD groups that serialize on address conflicts to local memory And many more: see (root of CUDA toolkit)/(doc/Compute Profiler VERSION.txt (Careful: CUDA terminology) Outline 1 Leftovers 2 Code writes Code 3 Case Study: Generic OpenCL Reduction 4 Reasoning about Generated Code 5 Automatic GPU Programming Automating GPU Programming GPU programming can be time-consuming, unintuitive and error-prone. Obvious idea: Let the computer do it. One way: Smart compilers Automating GPU Programming GPU programming can be time-consuming, unintuitive and error-prone. Obvious idea: Let the computer do it. One way: Smart compilers GPU programming requires complex tradeoffs Tradeoffs require heuristics Heuristics are fragile Automating GPU Programming GPU programming can be time-consuming, unintuitive and error-prone. Obvious idea: Let the computer do it. One way: Smart compilers GPU programming requires complex tradeoffs Tradeoffs require heuristics Heuristics are fragile Another way: Dumb enumeration Enumerate loop slicings Enumerate prefetch options Choose by running resulting code on actual hardware Loo.py Example Empirical GPU loop optimization: a, b, c, i , j , k = [var(s) for s in ”abcijk ”] n = 500 k = make loop kernel([ LoopDimension(”i”, n), LoopDimension(”j”, n), LoopDimension(”k”, n), ], [ (c[ i +n∗j], a[ i +n∗k]∗b[k+n∗j]) ]) gen kwargs = { ”min threads”: 128, ”min blocks”: 32, } → Ideal case: Finds 160 GF/s kernel without human intervention. Loo.py Status Limited scope: Require input/output separation Kernels must be expressible using “loopy” model (i.e. indices decompose into “output” and “reduction”) Enough for DG, LA, FD, . . . Loo.py Status Limited scope: Require input/output separation Kernels must be expressible using “loopy” model (i.e. indices decompose into “output” and “reduction”) Enough for DG, LA, FD, . . . Kernel compilation limits trial rate Non-Goal: Peak performance Good results currently for dense linear algebra and (some) DG subkernels

Easy, Effective, Efficient - GPU Programming in Python with PyOpenCL and PyCUDA Part 4

Related Documents

Recommended Documents

Description

Related Documents

Free up your schedule!

Take 5 seconds to unlock