University:
Boston University
Course:
MET CS 201 | Introduction to Programming
Academic year:

2022
Views:

233

Pages:

17
Author:

gotejatsgrr7

Communicators communicator = process group + communication context I Predefined instances I I I I Accessors I I I I COMM WORLD COMM SELF COMM NULL rank = comm.Get rank() # or comm.rank size = comm.Get size() # or comm.size group = comm.Get group() Constructors I I I newcomm = comm.Dup() newcomm = comm.Create(group) newcomm = comm.Split(color, key) Communicators – Create() 1 from mpi4py import MPI 2 3 4 comm = MPI.COMM_WORLD group = comm.Get_group() 5 6 7 newgroup = group.Excl([0]) newcomm = comm.Create(newgroup) 8 9 10 11 12 13 if comm.rank == 0: assert newcomm == MPI.COMM_NULL else: assert newcomm.size == comm.size - 1 assert newcomm.rank == comm.rank - 1 14 15 16 group.Free(); newgroup.Free() if newcomm: newcomm.Free() Communicators – Split() 1 from mpi4py import MPI 2 3 4 world_rank = MPI.COMM_WORLD.Get_rank() world_size = MPI.COMM_WORLD.Get_size() 5 6 7 8 9 10 11 if world_rank < world_size//2: color = 55 key = -world_rank else: color = 77 key = +world_rank 12 13 14 15 newcomm = MPI.COMM_WORLD.Split(color, key) # ... newcomm.Free() Exercise #1 a) Create a new process group containing the processes in the group of COMM WORLD with even rank. Use the new group to create a new communicator. Tip: use Group.Incl() or Group.Range incl() b) Use Comm.Split() to split COMM WORLD in two halves. I I The first half contains the processes with even rank in COMM WORLD. The process rank ordering in the new communication is ascending. The second half contains the processes with odd rank in COMM WORLD. The process rank ordering in the new communication is descending. I Blocking communication I Python objects comm.send(obj, dest=0, tag=0) obj = comm.recv(None, src=0, tag=0) I Array data comm.Send([array, count, datatype], dest=0, tag=0) comm.Recv([array, count, datatype], src=0, tag=0) I Nonblocking communication I Python objects request = comm.isend(object, dest=0, tag=0)} request.Wait() I Array data req1 = comm.Isend([array, count, datatype], dest=0, tag=0) req2 = comm.Irecv([array, count, datatype], src=0, tag=0) MPI.Request.Waitall([req1, req2])} PingPong 1 2 3 from mpi4py import MPI comm = MPI.COMM_WORLD assert comm.size == 2 4 5 6 7 8 9 10 11 12 if comm.rank == 0: sendmsg = 777 comm.send(sendmsg, dest=1, tag=55) recvmsg = comm.recv(source=1, tag=77) else: recvmsg = comm.recv(source=0, tag=55) sendmsg = "abc" comm.send(sendmsg, dest=0, tag=77) PingPing 1 2 3 from mpi4py import MPI comm = MPI.COMM_WORLD assert comm.size == 2 4 5 6 7 8 9 10 if comm.rank == 0: sendmsg = 777 target = 1 else: sendmsg = "abc" target = 0 11 12 13 14 request = comm.isend(sendmsg, dest=target, tag=77) recvmsg = comm.recv(source=target, tag=77) request.Wait() Exchange 1 2 from mpi4py import MPI comm = MPI.COMM_WORLD 3 4 5 6 sendmsg = [comm.rank]*3 right = (comm.rank + 1) % comm.size left = (comm.rank - 1) % comm.size 7 8 9 10 11 req1 req2 lmsg rmsg = = = = comm.isend(sendmsg, dest=right) comm.isend(sendmsg, dest=left) comm.recv(source=left) comm.recv(source=right) 12 13 14 15 MPI.Request.Waitall([req1, req2]) assert lmsg == [left] * 3 assert rmsg == [right] * 3 PingPing with NumPy arrays 1 2 3 4 from mpi4py import MPI import numpy comm = MPI.COMM_WORLD assert comm.size == 2 5 6 7 8 9 10 11 12 13 if comm.rank array1 = array2 = target = else: array1 = array2 = target = == 0: numpy.arange(10000, dtype=’f’) numpy.empty(10000, dtype=’f’) 1 numpy.ones(10000, dtype=’f’) numpy.empty(10000, dtype=’f’) 0 14 15 16 17 request = comm.Isend([array1, MPI.FLOAT], dest=target) comm.Recv([array2, MPI.FLOAT], source=target) request.Wait() Exercise #2 a) Modify PingPong example to communicate NumPy arrays. Tip: use Comm.Send() and Comm.Recv() b) Modify Exchange example to communicate NumPy arrays. Use nonblocking communication for both sending and receiving. Tip: use Comm.Isend() and Comm.Irecv() 2 C H A P T E R 4 . C O L L processes data A A 0 broadcast A A A A A 0 0 0 0 0 0 data processes 9 A 0 A 1 A 2 A 3 A 4 A 5 scatter A A A gather A A A 0 1 2 3 4 5 E C T I V E C O M M U N I C A T I O N 2 C H A P T E R 4 . C O L L E C T I V E C O M M U N I processes data A B C D E F A 0 A 0 allgather 0 A A 0 A 0 A 0 0 0 0 0 0 0 B B B B B B 0 0 0 0 0 0 C C C C C C 0 0 0 0 0 0 D D D D D D 0 0 0 0 0 0 E E E E E E 0 0 0 0 0 0 F F F F F F 0 0 0 0 0 0 data processes 9 A B C D E F 0 0 0 0 0 0 A B C D E F 1 1 1 1 1 1 A B C D E F 2 2 2 2 2 2 A B C D E F 3 3 3 3 3 3 A B C D E F 4 4 4 4 4 4 A B C D E F A 5 5 5 5 5 5 alltoall A A A A A 0 1 2 3 4 5 B B B B B B 0 1 2 3 4 5 C C C C C C 0 1 2 3 4 5 D D D D D D 0 1 2 3 4 5 E E E E E E 0 1 2 3 4 5 F F F F F F 0 1 2 3 4 5 C A T I O N Broadcast 1 2 from mpi4py import MPI comm = MPI.COMM_WORLD 3 4 5 6 7 if comm.rank == 0: sendmsg = (7, "abc", [1.0,2+3j], {3:4}) else: sendmsg = None 8 9 recvmsg = comm.bcast(sendmsg, root=0) Scatter 1 2 from mpi4py import MPI comm = MPI.COMM_WORLD 3 4 5 6 7 if comm.rank == 0: sendmsg = [i**2 for i in range(comm.size)] else: sendmsg = None 8 9 recvmsg = comm.scatter(sendmsg, root=0) Gather & Gather to All 1 2 from mpi4py import MPI comm = MPI.COMM_WORLD 3 4 sendmsg = comm.rank**2 5 6 recvmsg1 = comm.gather(sendmsg, root=0) 7 8 recvmsg2 = comm.allgather(sendmsg) Reduce & Reduce to All 1 2 from mpi4py import MPI comm = MPI.COMM_WORLD 3 4 sendmsg = comm.rank 5 6 recvmsg1 = comm.reduce(sendmsg, op=MPI.SUM, root=0) 7 8 recvmsg2 = comm.allreduce(sendmsg) Exercise #3 a) Modify Broadcast, Scatter, and Gather example to communicate NumPy arrays. b) Write a routine implementing parallel matrix–vector product y = matvec(comm,A,x). I I I the global matrix is dense and square. matrix rows and vector entries have matching block distribution. all processes own the same number of matrix rows. Tip: use Comm.Allgather() and numpy.dot()