University:
Boston University
Course:
MET CS 201 | Introduction to Programming
Academic year:

2023
Views:

54

Pages:

19
Author:

eishuma5iuk

Easy, Effective, Efficient: GPU Programming in Python with PyOpenCL and PyCUDA Motivation 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 import pyopencl as cl , numpy a = numpy.random.rand(256∗∗3).astype(numpy.float32) ctx = cl. create some context () queue = cl.CommandQueue(ctx) a dev = cl. Buffer (ctx , cl .mem flags.READ WRITE, size=a.nbytes) cl . enqueue write buffer (queue, a dev, a) prg = cl.Program(ctx, ””” kernel void twice( global float ∗a) { a[ get global id (0)] ∗= 2; } ”””). build () prg. twice(queue, a.shape, (1,), a dev) OpenCL: Runtime and Device CL consists of two parts: Host-side “runtime”: C: #include "CL/cl.h", libOpenCL.so Compute Device 0 (Platform 0) ··· Python: import pyopencl as cl Device-side programming language A dialect of C99 (also when using PyOpenCL) Today: A better look at both of those, with an emphasis on PyOpenCL ··· Memory ··· Device 1 Compute Host (CPU) Memory ··· ··· ··· (Platform 0) Memory Compute Device 0 (Platform 1) ··· ··· Memory ··· Compute Device 1 ··· ··· ··· (Platform 1) Memory Outline 1 The OpenCL Runtime 2 Device Language 3 OpenCL implementations Outline 1 The OpenCL Runtime 2 Device Language 3 OpenCL implementations OpenCL Object Diagram CL “Platform” “Platform”: a collection of devices, all from the same vendor. All devices in a platform use same CL driver/implementation. Multiple platforms can be used from one program → ICD. libOpenCL.so: ICD loader /etc/OpenCL/vendors/somename.icd: Plain text file with name of .so containing CL implementation. CL “Compute Device” CL Compute Devices: CPUs, GPUs, accelerators, . . . Anything that fits the programming model. A processor die with an interface to off-chip memory Can get list of devices from platform. Contexts context = cl.Context(devices=None | [dev1, dev2 ], dev type=None) context = cl. create some context( interactive =True) Spans one or more Devices Create from device type or list of devices See docs for cl.Platform, cl.Device dev type: DEFAULT , ALL, CPU, GPU Needed to. . . . . . allocate Memory Objects . . . create and build Programs . . . host Command Queues . . . execute Grids Computations Memory Transfers Sync primitives ... Host can wait for drained queue Profiling Host Queue 2 Host and Device run asynchronously Host submits to queue: Queue 1 OpenCL: Command Queues Device ... Command Queues and Events queue = cl.CommandQueue(context, device=None, properties =None | [(prop, value ),...]) Attached to single device cl.command queue properties. . . OUT OF ORDER EXEC MODE ENABLE: Do not force sequential execution PROFILING ENABLE: Gather timing info Building Blocks in Action import pyopencl as cl platforms = cl. get platforms () my platform = platforms[0] print my platform.vendor devices = my platform.get devices () my device = devices [0] print my device.name ctx = cl.Context([my device]) cpq = cl.command queue properties queue = cl.CommandQueue(ctx, my device, cpq.PROFILING ENABLE) Simple version: ctx2 = cl. create some context () queue2 = cl.CommandQueue(ctx) Command Queues and Events event = cl.enqueue XXX(queue, ..., wait for =[evt1, evt2 ]) Every enqueue operation returns an Event. Also possible: Operation-less events (“Markers”) Wait (evt.wait()), polling Specify dependencies Every enqueue operation takes a list arg wait for of dependencies. Profile event.profile.. . . QUEUED, SUBMIT START, END (time stamp in ns) Profiling example start event = cl.enqueue marker(queue) # enqueue some commands stop event = cl.enqueue marker(queue) stop event . wait() elapsed seconds = 1e−9∗( start event . profile .END − start event. profile .END) # −−− OR −−− op event = knl(queue, global size , grp size , args ...) op event. wait() elapsed seconds = 1e−9∗( op event. profile .END − start event. profile .START) Capturing Dependencies A f B = f(A) C = g(B) E = f(C) F = h(C) G = g(E,F) P = p(B) Q = q(B) R = r(G,P,Q) B g p C P q f h E Q F g g r r G r R Capturing Dependencies A Switch queue to out-of-order mode! B = f(A) Specify as list of events using C = g(B) wait for= optional keyword to E = f(C) enqueue XXX. F = h(C) Can also enqueue barrier. G = g(E,F) Common use case: P = p(B) E Transmit/receive from other MPI Q = q(B) ranks. R = r(G,P,Q) Possible on Nv Fermi: Submit parallel work to increase machine use. f B g p C P q f h Q F g g r r G r R Memory Objects: Buffers buf = cl. Buffer (context , flags , size =0, hostbuf=None) Chunk of device memory No type information: “Bag of bytes” Observe: Not tied to device. → no fixed memory address → pointers do not survive kernel launches → movable between devices flags: READ ONLY/WRITE ONLY/READ WRITE {ALLOC,COPY,USE} HOST PTR Memory Objects: Buffers buf = cl. Buffer (context , flags , size =0, hostbuf=None) COPY HOST PTR: Use hostbuf as initial content of buffer USE HOST PTR: hostbuf is the buffer. Caching in device memory is allowed. ALLOC HOST PTR: New host memory (unrelated to hostbuf) is visible from device and host. Memory Objects: Buffers buf = cl. Buffer (context , flags , size =0, hostbuf=None) Specify hostbuf or size (or both) hostbuf: Needs Python Buffer Interface e.g. numpy.ndarray, str. Important: Memory layout matters Passed to device code as pointers (e.g. float *, int *) enqueue {read,write} buffer( queue, buf, hostbuf) Can be mapped into host address space: cl.MemoryMap.