University:
Boston University
Course:
MET CS 201 | Introduction to Programming
Academic year:

2023
Views:

37

Pages:

52
Author:

Bekrtqyp

Introduction to OpenMP Research Computing Services Information Services and Technology Outline • Brief overview of parallel computing and OpenMP • OpenMP Programming Parallel constructs Work-sharing constructs Data clauses Data race condition Synchronization constructs More clauses Parallel Computing  Parallel computing is a type of computation in which many calculations are carried out simultaneously.  Speedup of a parallel program, p: number of processors (or cores), α: fraction of the program that is serial. Two types of parallel parallelism • Shared memory system • Distributed memory system • For example, a single node on a cluster • For example, mutli nodes on a cluster • Open Multi-processing (OpenMP) • Message Passing Interface (MPI) OpenMP  OpenMP (Open Multi-Processing) is an API (application programming interface) that supports multi-platform shared memory multiprocessing programming.  Supporting languages: C, C++, and Fortran.  Consists of a set of compiler directives, library routines, and environment variables that influence run-time behavior.  OpenMP 4.0 supports accelerators. Parallelism of OpenMP • Multithreading: a master thread forks a specified number of slave threads and the system divides a task among them. The threads then run concurrently, with the runtime environment allocating threads to different processors (or cores). • Fork-join model: The first OpenMP program: Hello world! • Hello world in C language #include int main() { int id; #pragma omp parallel private(id) { id = omp_get_thread_num(); if (id%2==1) printf("Hello world from thread %d, I am odd\n", id); else printf("Hello world from thread %d, I am even\n", id); } } • Hello world in Fortran language program hello use omp_lib implicit none integer i !$omp parallel private(i) i = omp_get_thread_num() if (mod(i,2).eq.1) then print *,'Hello from thread',i,', I am odd!' else print *,'Hello from thread',i,', I am even!' endif !$omp end parallel end program hello OpenMP directive syntax • In C/C++ programs #pragma omp directive-name [clause[[,] clause]. . . ] • In Fortran programs !$omp directive-name [clause[[,] clause]. . . ] • Directive-name is a specific keyword, for example parallel, that defines and controls the action(s) taken. • Clauses, for example private, can be used to further specify the behavior. Compile and run OpenMP programs Compile C/C++/Fortran codes > icc/icpc/ifort -openmp name.c/name.f90 -o name > gcc/g++/gfortran -fopenmp name.c/name.f90 -o name > pgcc/pgc++/pgf90 -mp name.c/name.f90 -o name Run OpenMP programs > export OMP_NUM_THREADS=4 # set number of threads > ./name > time ./name # run and measure the time. OpenMP programming • Parallel Construct • Work-Sharing Constructs • Synchronization constructs Barrier Construct Loop Construct Critical Construct Sections Construct Atomic Construct Single Construct Workshare Construct (Fortran only) • More clauses: reduction, num_thread, schedule • Data clauses shared, private, lastprivate, firstprivate, default • Construct : An OpenMP executable directive and the associated statement, loop, or structured block, not including the code in any called routines. Parallel construct • Syntax in C/C++ programs #pragma omp parallel [clause[[,] clause]. . . ] …… code block ...... • Syntax in Fortran programs !$omp parallel [clause[[,] clause]. . . ] …… code block ...... !$omp end parallel • Parallel construct is used to specify the computations that should be executed in parallel. • A team of threads is created to execute the associated parallel region. • The work of the region is replicated for every thread. • At the end of a parallel region, there is an implied barrier that forces all threads to wait until the computation inside the region has been completed. Work-sharing constructs Functionality Syntax in C/C++ Syntax in Fortran Distribute iterations #pragma omp for !$omp do Distribute independent works #pragma omp sections !$omp sections Use only one thread #pragma omp single !$omp single Parallelize array syntax N/A !$omp workshare • Many applications can be parallelized by using just a parallel region and one or more of work-sharing constructs, possibly with clauses. • The parallel and work-sharing (except single) constructs can be combined. • Following is the syntax for combined parallel and work-sharing constructs, Combine parallel construct with … Syntax in C/C++ Syntax in Fortran Loop construct #pragma omp parallel for !$omp parallel do Sections construct #pragma omp parallel sections !$omp parallel sections Workshare construct N/A !$omp parallel workshare Loop construct • The loop construct causes the iterations of the loop immediately following it to be executed in parallel. • Syntax in C/C++ programs #pragma omp for [clause[[,] clause]. . . ] …… for loop ...... • Syntax in Fortran programs !$omp do [clause[[,] clause]. . . ] …… do loop ...... [!$omp end do] • The terminating !$omp end do directive in Fortran is optional but recommended. • Distribute iterations in a parallel region #pragma omp parallel for shared(n,a) private(i) for (i=0; i>. +, *, -, /, .AND., .OR., .EQV., .NEQV. . • The first try to solve the data-race problem: use atomic (correct but slow) • The atomic construct avoids the data racing condition. Therefore the result is correct. • But all elements are added sequentially, and there is performance penalty for using atomic, because the system coordinates all threads. • This code is even slower than a serial code! sum = 0; #pragma omp parallel for shared(n,a,sum) private(i) // Optimization: use reduction instead of atomic for (i=0; i 5) num_threads(n) default(none) shared(n) { #pragma omp single { printf("Number of threads in parallel region: %d\n", omp_get_num_threads()); } printf("Print statement executed by thread %d\n", omp_get_thread_num()); } /*-- End of parallel region --*/ Schedule clause • Specifies how iterations of the loop are assigned to the threads in the team. • Supported on the loop construct only. • The iteration space is divided into chunks. A chunk is a contiguous nonempty subset of the iteration space. It represents the granularity of workload distribution. • Syntax schedule(kind [,chunk_size] ) • The static schedule works best for regular workloads. It is the default schedule on many OpenMP compilers. • The dynamic and guided schedules are useful for handling poorly balanced and unpredictable workloads. There is a performance penalty for using dynamic or guided schedules. • Schedule type type description static The chunks are assigned to the threads statically in a round-robin manner, in the order of the thread number. dynamic The chunks are assigned to threads as the threads request them. When a thread finishes, it will be assigned the next chunk that hasn’t been executed yet. guided Similar to the dynamic schedule, except that the chunk size changes at run time. It begins with big chunks, but then adjusts to smaller chunk sizes if the workload is imbalanced. runtime The schedule and (optional) chunk size are set through the OMP_SCHEDULE environment variable. • An example of schedule clause: In this code, the workload in the inner loop depends on the value of the outer loop iteration variable i. Therefore, the workload is not balanced, and the static schedule is not the best choice. Dynamic or guided schedules are required. #pragma omp parallel for default(none) schedule(runtime) private(i,j) shared(n) for (i=0; i