University:
Boston University
Course:
CAS CS 320 | Concepts of Programming Languages
Academic year:

2021
Views:

268

Pages:

21
Author:

Killian Mata

Results on Multi-node Computing Tsubame 2.0 100000 6.4 hours Runtime (s) 10000 5 minutes 1000 GPUs CPUs 100 10 1 1 10 Number of CPU / GPU 100 1000 Multi-GPU Scalability Tsubame 2.0 CPU‐GPU Scalability Tsubame 2.0 10000 Speedup (x) 1000 100 CPUs GPUs GPU/SocketCPU 10 1 1 10 Number of CPU/GPU 100 Two-Phase Flow Simulation Mesh Method (Surface Capture) Particle Method ex. SPH Low accuracy < 106 particles ■ Navier-Stokes solver：Fractional Step ■ Time integration：3rd TVD Runge-Kutta ■ Advection term：5th WENO ■ Diffusion term：4th FD ■ Poisson：MG-BiCGstab ■ Surface tension：CSF model ■ Surface capture：CLSVOF(THINC + Level-Set) High accuracy > 108 mesh points not splash Numerical noise and unphysical oscillation EQUATIONs for Two-Phase Flow Time Integration : 3rd-order TVD Runge-Kutta 5th-upFD or 5th HJ-WENO diffusion : 4th FD Surface Tension CSF 5th HJ-WENO Advection VOF THINC/WLIC Re-initialization 5th HJ-WENO 3D Advection Computation Advection equation f  u    f  0 t 3 stencils 19 input values/cell 259 flop/cell (5th-WENO) 49 flop/cell (5th-up FD) Discretization: Space：5th-WENO Time ：3rd TVD Runge-Kutta 312 GFlops (1GPU:GTX285) Stencil Computation ■ Example: 2-dimensional diffusion Equation by FDM f i ,nj1  f i ,nj t  f i n1, j  2 f i ,nj  f i n1, j f i ,nj 1  2 f i ,nj  f i ,nj 1      2 2     x y   f i , j 1 f i ,nj1  c0 f i ,nj  c1 f i n1, j  c2 f i n1, j  c3 f i ,nj 1  c4 f i ,nj 1 f i 1, j fi, j FLOP = 9 Byte = 4*6 = 24 byte : read 5, write 1 FLOP/Byte = 9/24 = 0.375 f i , j 1 f i 1, j Arithmetic INTENSITY: FLOP/Byte FLOP = number of FP operation for applications Byte = Byte number of memory access for applications F = Peak Performance of floating point operation B = Peak Memory Bandwidth FLOP Performance  FLOP/F  Byte/B  α FLOP/Byte  F FLOP/Byte  F/B  α Application Performances Roofline model: Williams, Patterson 2008 Communication s of the ACM FLOP/Byte = F/B 103 0 10 -2 10 10-1 100 FLOP/Byte 101 MD (N log N) N-Body (N log N) 101 SGEMM FFT (NUKADA) 102 LBM Riken BMT GFLOPS 708 GFlops -D diffusion Performance [GFlops] GeForce GTX 285 Core i7 Extreme Performance of Advection Computation flop/byte Scheme GFlops no-SMem SMem Tesla S1070 1st-up FD 0.29 0.60 37.61 2nd-c FD 0.34 0.71 42.46 3rd-up FD 0.38 1.06 75.49 4th-c FD 0.34 0.96 66.64 5th-up FD 0.45 1.55 94.58 6th-c FD 0.4 1.36 89.62 5th-WENO 2.40 8.22 289.66 3D advection 416x416x416 cells Time integration：3rd-order TVD Runge-Kutta Level-Set method (LSM) The Level-Set methods (LSM) use the signed distance function to capture the interface. The interface is represented by the zero-level set (zero-contour). : Level-Set function(distance function) : Heaviside function Re-initialization for Level-Set function Advantage : Curvature calculation, Interface boundary Drawback : Volume conservation Fig. Takehiro Himeno, et. Al., JSME, 65-635,B(1999),pp2333-2340 Continuous Surface Force (CSF) model by Brackbill, Kothe and Zemach (1991) The interfacial surface force is transformed to a volume force in the region near the interface via a delta function Curvature Surface tension force Normal vector Surface tension represented by volume force Approximate delta function Anti-diffusive Interface Capture THINC (tangent of hyperbola for interface capturing) Scheme [ Xiao, etal, Int. J. Numer. Meth. Fluid. 48(2005)1023 ] ・VOF(volume of fluid) type interface capturing method ・Flux from tangent of hyperbola function ・Semi-Lagrangian time integration ・1D implementation can be applied to 2D & 3D → Simple ・Finite Volume like usage ＊ THINC is the method how to compute flux → 3 krenel (x, y, z) can be fused to 1 kernel. Merit in memory R/W Sparse Matrix Solver Ax = b 1  u for    p   t   Krylov sub-space methods: CG, BiCGStab, GMRes, , , Pre-conditioner: Incomplete Cholesky, ILU, MG, AMG, Block Diagonal Jacobi Non-zero Packing: CRS → ELL, JDL ×× × × × ×× × × × × ×× × × × ×× × × × ×× × × × × ×× × × × × ×× × × × × ×× × × × × × ×× × × × × ×× × × × × ×× × × × ×× × × × × × × × × × × × × × × × × × × × × × × × × × × ×× × × × ×× × × × × ×× × × × ×× × × BiCGStab + MG r0  p0  M 1 b  Ax0  Collaboration with Mizuho Information & Research Institute Set k 0 for k = 0; k < N; k++; 1  q , M Aqk   r0 , rk  1 k qk  rk   k M Apk k  k  1 M 1 Aqk , M 1 Aqk  r0 , M Apk xk 1  xk   k pk  k qk   rk 1  qk  k M 1 Aqk if rk 1, rk 1    2 b, b k  r0 , rk 1  exit; k r0 , M 1 Apk  pk 1  rk 1   k  pk  k M 1 Apk  loop end MG V-Cycle Smoother : Red & Black ILU n Step G 0 L0 f 0  S 0 Poisosn Eq. Restriction G 1 L1v1  R1 AF AC R P ：： : : Correction Eq. Fine Matrix Corse Matrix Restriction Prolongation G AC = RAFP G 3 L3v 3  R 3 Restriction G4 Prolongation L2 v 2  R 2 L2 v 2  R 2 Restriction Prolongation L1v1  R1 Restriction 2 n+1 Step Prolongation L3v 3  R 3 Prolongation L4 v 4  R 4 Multi-Dimensional Domain Decomposition ・3D domain decomposition ・1 GPU is assigned to each domain ・Communication buffer for each face ・Host buffer & Device buffer Milk Crown Formation 4.0 m/sec impact speed Experiment Gunji, ishii, Saito, Sakai Simulation Rayleigh-Taylor Instability with Surface Tension Force When a heavy fluid is supported against gravity by a light fluid, a Rayleigh-Taylor instability develops in which perturbations of the interface grow exponentially in time as exp(nt) for small amplitudes. Bellman, R., Pennington, R.H.: Effect of surface tension and viscosity on Taylor instability. Q. Appl. Methods 12, 12, 151 (1954) Drazin, P.G., Reid, W.H.: Hydrodynamic Stability. Cambridge University Press, Cambridge (1967) Daly, B.J.: Numerical study of the effect of surface tension on interface instability. Phys. Fluids 12, 1340 (1969) Snapshots of the R-T Instability 1024x192 cells Milk Crown Drop on dry floor

Tsunami Simulation on GPUs - Results on Multi-Node Computing

Related Documents

Recommended Documents

Description

Related Documents

Free up your schedule!

Take 5 seconds to unlock