MPI Beginning Lecture - Introduction Pt 4

University:
Boston University
Course:
MET CS 201 | Introduction to Programming
Academic year:

2023
Views:

268

Pages:

20
Author:

Kolstrup96gf

Speedup Ratio and Parallel Efficiency is ratio of over workers. is fraction of due sections of code not parallelizable. T1 S   TN 1 T1  as N   1 f f (f  )T1 N Amdahl’s Law above states that a code with its parallelizable component comprising 90% of total computation time can at best achieve a 10X speedup with lots of workers. A code that is 50% parallelizable speeds up two-fold with lots of workers. The parallel efficiency is Program that scales linearly ( has parallel efficiency A task-parallel program is usually more efficient than a dataparallel program. Data-parallel codes can sometimes achieve super-linear behavior due to efficient cache usage per worker. 49 Speedup Ratio & Parallel Efficiency 50 How MPI_Reduce Works On 7 x  i i 0 Processor 0 with corresponding Sendbuf content p0 p1 0 Sendbuf x7  7 p2 1 p3 2 p0 p4 3 4 p2 1 p5 5 6 7 p6 9 MPI_Reduce intermediate steps p7 6 p4 5 p1 p6 13 67 p5 22 p3 28 “Root” process with Recvbuf content pR 28 Collective Functions Process 0 Process 1* Process 2 Process 3 b Operation Process 0 Process 1* Process 2 Process 3 MPI_Bcast b b b b a b c d MPI_Gather a b c d MPI_Allgather a,b,c,d a,b,c,d a,b,c,d a,b,c,d MPI_Scatter a b c d MPI_Alltoall a,e,i,m b,f,j,n c,g,k,o d,h,l,p ReceiveBuff ReceiveBuff ReceiveBuff ReceiveBuff a,b,c,d a,b,c,d e,f,g,h i,j,k,l m,n,o,p SendBuff SendBuff SendBuff SendBuff a,b,c,d • This example uses 4 processes • Rank 1 is, arbitrarily, designated data gather/scatter process • a, b, c, d are scalars or arrays of any data type • Data are gathered/scattered according to rank order Collectives Example Code program collectives_example implicit none integer p, ierr, i, myid, root include "mpif.h" ! This brings in pre-defined MPI constants, ... character*1 x(0:3), y(0:3), alphabets(0:15) data alphabets/'a','b','c','d','e','f','g','h','i','j','k','l', & 'm','n','o','p'/ data root/1/ ! process 1 is the data sender/receiver c**Starts MPI processes ... call MPI_Init(ierr) ! starts MPI call MPI_Comm_rank(MPI_COMM_WORLD, myid, ierr) ! current pid call MPI_Comm_size(MPI_COMM_WORLD, p, ierr) ! # of procs Collectives Example (cont’d) if (myid .eq. 0) then write(*,*) write(*,*)'* This program demonstrates the use of collective', & ' MPI functions' write(*,*)'* Four processors are to be used for the demo' write(*,*)'* Process 1 (of 0,1,2,3) is the designated root' write(*,*) write(*,*) write(*,*)' Function Proc Sendbuf Recvbuf' write(*,*)' -------- ---- ------- -------' endif Gather Operation alphabets(0) = ‘a’ alphabets(1) = ‘b’ c**Performs a gather operation ... x(0) = alphabets(myid) alphabets(14) = ‘o’ do i=0,p-1 alphabets(15) = ‘p’ y(i) = ' ' enddo call MPI_Gather(x,1,MPI_CHARACTER, ! Send-buf,count,type, & y,1,MPI_CHARACTER, ! Recv-buf,count?,type?, & root, ! Data destination & MPI_COMM_WORLD,ierr) ! Comm, flag write(*,"('MPI_Gather:',t20,i2,(3x,a1),t40,4(3x,a1))")myid,x(0),y Recv-buf according to rank order All-gather Operation c**Performs an all-gather operation x(0) = alphabets(myid) do i=0,p-1 y(i) = ' ' enddo call MPI_Allgather(x,1,MPI_CHARACTER, ! send buf,count,type & y,1,MPI_CHARACTER, ! recv buf,count,type & MPI_COMM_WORLD,ierr) ! comm,flag write(*,"('MPI_Allgather:',t20,i2,(3x,a1),t40,4(3x,a1))")myid,x(0),y Scatter Operation c**Perform a scatter operation if (myid .eq. root) then do i=0, p-1 x(i) = alphabets(i) y(i) = ' ‘ enddo else do i=0,p-1 x(i) = ‘ ‘ y(i) = ‘ ‘ enddo endif call MPI_scatter(x,1,MPI_CHARACTER, ! Send-buf,count,type & y,1,MPI_CHARACTER, ! Recv-buf,count,type & root, ! data origin & MPI_COMM_WORLD,ierr) ! comm,flag write(*,"('MPI_scatter:',t20,i2,4(3x,a1),t40,4(3x,a1))")myid,x,y Alltoall Operation c**Perform an all-to-all operation do i=0,p-1 x(i) = alphabets(i+myid*p) y(i) = ' ' enddo call MPI_Alltoall(x,1,MPI_CHARACTER, ! send buf,count,type & y,1,MPI_CHARACTER, ! recv buf,count,type & MPI_COMM_WORLD,ierr) ! comm,flag write(*,"('MPI_Alltoall:',t20,i2,4(3x,a1),t40,4(3x,a1))")myid,x,y Broadcast Operation c**Performs a broadcast operation do i=0, p-1 x(i) = ‘ ‘ y(i) = ‘ ‘ enddo if(myid .eq. root) then x(0) = ‘b’ y(0) = ‘b’ endif call MPI_Bcast(y,1,MPI_CHARACTER, ! buf,count,type & root,MPI_COMM_WORLD,ierr) ! root,comm,flag write(*,"('MPI_Bcast:',t20,i2,4(3x,a1),t40,4(3x,a1))")myid,x,y call MPI_Finalize(ierr) end ! let MPI finish up ... Example 1.6 Integration (modified) float fct(float x) { return cos(x); } /* Prototype */ float integral(float a, int i, float h, int n); int main(int argc, char* argv[]) { int n, p, myid, i; float h, integral_sum, a, b, pi, my_int; float buf[50], tmp; Example 1.6 (cont’d) MPI_Init(&argc,&argv); /* starts MPI */ MPI_Comm_rank(MPI_COMM_WORLD, &myid); /* current proc id */ MPI_Comm_size(MPI_COMM_WORLD, &p); /* num of procs */ pi = acos(-1.0); a = 0.; b = pi*1./2.; n = 500; h = (b-a)/n/p; /* = 3.14159... */ /* lower limit of integration */ /* upper limit of integration */ /* number of increment within each process */ /* length of increment */ my_int = integral(a,myid,h,n); printf("Process %d has the partial sum of %f\n", myid,my_int); MPI_Gather(&my_int, 1, MPI_FLOAT, buf, 1, MPI_FLOAT, 0, MPI_COMM_WORLD); Example 1.6 (cont’d) MPI_Scatter(buf, 1, MPI_FLOAT, &tmp, 1, MPI_FLOAT, 0, MPI_COMM_WORLD); printf("Result sent back from buf = %f\n", tmp); if(myid == 0) { integral_sum = 0.0; for (i=0; ip; i++) { integral_sum += buf[i]; } printf("The Integral =%f\n", integral_sum); } } MPI_Finalize(); /* let MPI finish up ... */ MPI_Probe, MPI_Wtime (f90) This example demonstrates dynamic memory allocation and parallel timer. Program dma_example implicit none include "mpif.h" integer, parameter :: real_kind = selected_real_kind(8,30) real(real_kind), dimension(55) :: sdata real(real_kind), dimension(:), allocatable :: rdata real(real_kind) :: start_time, end_time integer :: p, i, count, myid, n, status(MPI_STATUS_SIZE), ierr !* Starts MPI processes ... call MPI_Init(ierr) !* starts MPI call MPI_Comm_rank(MPI_COMM_WORLD, myid, ierr) ! myid call MPI_Comm_size(MPI_COMM_WORLD, p, ierr) ! Num. proc MPI_Probe, MPI_Wtime (f90 cont’d) start_time = MPI_Wtime() ! start timer, measured in seconds if (myid == 0) then sdata(1:50)= (/ (i, i=1,50) /) call MPI_Send(sdata, 50, MPI_DOUBLE_PRECISION, 1, 123, & MPI_COMM_WORLD, ierr) else call MPI_Probe(0, MPI_ANY_TAG, MPI_COMM_WORLD, status, ierr) call MPI_Get_count(status, MPI_DOUBLE_PRECISION, count, ierr) allocate( rdata(count) ) call MPI_Recv(rdata, count, MPI_DOUBLE_PRECISION, 0, & MPI_ANY_TAG, MPI_COMM_WORLD, status, ierr) write(*,'(5f10.2)')rdata(1:count:10) endif end_time = MPI_Wtime() ! stop timer MPI_Probe, MPI_Wtime (f90 cont’d) if (myid .eq. 1) then WRITE(*,"(' Total cpu time =',f10.5,' x ',i3)") end_time start_time,p endif call MPI_Finalize(ierr) end program dma_example !* let MPI finish up ... MPI_Probe, MPI_Wtime (C ) int main(int argc, char* argv[]) { double sdata[55], *rdata, start_time, end_time; int p, i, count, myid, n; MPI_Status status; /* Starts MPI processes ... */ MPI_Init(&argc, &argv); /* starts MPI */ MPI_Comm_rank(MPI_COMM_WORLD, &myid); /* get current process id */ MPI_Comm_size(MPI_COMM_WORLD, &p); /* get number of processes */ MPI_Probe, MPI_Wtime (C cont’d) start_time = MPI_Wtime(); /* starts timer */ if (myid == 0) { for(i=0;i50;++i) { sdata[i]=(double)i; } MPI_Send(sdata,50,MPI_DOUBLE,1,123,MPI_COMM_WORLD); } else { MPI_Probe(0,MPI_ANY_TAG,MPI_COMM_WORLD,&status); MPI_Get_count(&status,MPI_DOUBLE,&count); MPI_Type_size(MPI_DOUBLE,&n); /* sizeof */ rdata= (double*) calloc(count,n); MPI_Recv(rdata,count,MPI_DOUBLE,0,MPI_ANY_TAG, MPI_COMM_WORLD, &status); for(i=0;icount;i+=10) { printf("rdata element %d is %f\n",i,rdata[i]);} } end_time = MPI_Wtime(); /* ends timer */ MPI_Probe, MPI_Wtime (C cont’d) if (myid == 1) { printf("Total time is %f x %d\n", end_time-start_time, p); } MPI_Finalize(); /* let MPI finish up ... */ }