Binary32 (a hi ) 8 bits 23 bits Binary32 (a lo ) 8 bits 23 bits Double-Float (a=a hi +a lo, a lo 0.5ulp(a hi ) ) 8 bits 46 bits Binary64 11 bits sign

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "Binary32 (a hi ) 8 bits 23 bits Binary32 (a lo ) 8 bits 23 bits Double-Float (a=a hi +a lo, a lo 0.5ulp(a hi ) ) 8 bits 46 bits Binary64 11 bits sign"

Χλόη Μπουκουβαλαίοι
5 χρόνια πριν
Προβολές:

1 Maxwell GPU DGEMM 1,a) 1,b) NVIDIA 2014 Maxwell GM107 GM204 GPU : =1:32 GM204 GeForce GTX double-float DF BLAS DGEMM DGEMM DF DGEMM 2 1. IEEE [1] binary32 binary64 NVIDIA GPU 2010 Fermi : 1: Kepler 1: Maxwell GPU GM107 GM204 : 1:32 GM107 GM204 Fermi Kepler : 1:12 1:24 *1 GPU CUDA GPU 1 a) daichi.mukunoki@riken.jp b) imamura.toshiyuki@riken.jp *1 GPU : =1:24 Kepler GK104 Tesla K10 GPU GPU Cell Broadband Engine Cell/B.E. PLAYSTATION3 : 1:14 Dekker [2] 2 4 Double-Double DD Double-Float DF GPU Fused Multiply-Add FMA 2 a b + c 16 GM107 GM204 Maxwell GPU DF GM204 Maxwell 1

2 Binary32 (a hi ) 8 bits 23 bits Binary32 (a lo ) 8 bits 23 bits Double-Float (a=a hi +a lo, a lo 0.5ulp(a hi ) ) 8 bits 46 bits Binary64 11 bits sign (1 bit) exponent significand 52 bits 1 DF binary64 GPU GeForce GTX 980[3] C = αab + βc BLAS DGEMM DF GEMM O(N 2 ) O(N 3 ) GPU GPU DF [4] [5] DF DGEMM DD 4 GEMM [6][7] 2. DF 2.1 DF DF 1 a 2 binary32 a hi a lo a = a hi + a lo 1 a lo 0.5ulp(a hi )) a hi a lo Binary DF 24 2 = 48 binary binary32 8 DF float 2 cudfreal cudfreal 8 GPU binary64 8 DF Array-of-Structure 2.2 DF 2 2 DF QD [8] Sloppy 2 [9] 3 48 DF FMA FMA FMA DF a b + c Binary64 DF Binary64 DF 1 2 SPLIT [2] binary64 d t d 53bit 29bit t d d 24bit 29bit 29bit binary32 5bit DF binary64 DF binary64 3. GEMM GEMM C = αab + βc BLAS BLAS 2

3 device forceinline cudfreal DFadd (const cudfreal a, const cudfreal b){ } cudfreal t, c; float v; t.x = v = fadd rn (a.x, b.x); fsub rn (t.x, a,x); t.y = fadd rn ( fsub rn (a.x, fsub rn (t,x, v)), fsub rn (b.x, v)); t.y = fadd rn (t.y, fadd rn (a.y, b.y) ); c.x = fadd rn (t.x, t.y); c.y = fsub rn (t.y, fsub rn (c.x, t.x)); return c; 2 DF device forceinline cudfreal DFmul (const cudfreal a, const cudfreal b){ } cudfreal c; float t, v, e; v = t = c.x = e = c.y = return c; fmul rn (a.x, b.y); fmaf rn (a.y, b.x, v); fmaf rn (a.x, b.x, t); fmaf rn (a.x, b.x, -c.x); fadd rn (e, t); 3 DF device forceinline cudfreal double to cudfreal (const double d){ } double t = dmul rn ( , d); double h = dsub rn (t, dsub rn (t, d)); double l = dsub rn (d, h); return make cudfreal ((float)h, (float)l); 4 Binary64 DF 3 DGEMM binary64 binary64 DGEMM DGEMM DF DF DGEMM DGEMM-DF(DF) binary64 DF DGEMM DGEMM-D(DF) GPU binary64 DF DGEMM-DF(DF) DGEMM binary64 DGEMM-D(DF) DGEMM-D(DF) DGEMM DGEMM-D(DF) double to cudfreal 4 binary64 DF N N O(N 2 ) O(N 3 ) O(N 3 ) CUDA GEMM [10][11][12] C = AB A M K B K N C M N 2MNK A B C BM BK BK BN BM BN MNK(1/BM + 1/BN) M = N = K BM = BN 2N 3 2N 3 /BN 1/BN 3.1 GeForce GTX 980 GeForce GTX 980 SMM MHz = GFlops = GFlops 2 a b + c 2 DF DF 1 16 DF /16 = GFlops DF 1 1Flops 3.2 DGEMM a b + c 2Flops 3 64bit (3 8)/2 = 12Bytes/Flop 3

4 N BN=48 Sub-mat B K Shared-memory BK =16 Matrix B Sub-mat A Sub-mat B M Sub-mat A Sub-mat C BN =48 Shared-memory Matrix A K Matrix C BK =16 Sub-mat C 5 DF DGEMM 180GB/s 156GFlops = 1872GB/s 180GB/s BN 1872/180 = 10.4 BN 11 GPU 128Bytes 64bit 16 BN BK BK =16 A B BN 16 BN BK = (BN/16) (BN/16) (BN/16) (BN/16) BN 1 CUDA GPU 1 96KB 1 48KB 1 32bit = /256/4 =64 BN DF DGEMM DGEMM-DF(DF) DGEMM-D(DF) DGEMM DF 311GFlops = 3732GB/s 180GB/s BN 3732/ DGEMM BK =16 4 BN BN 48 DGEMM DGEMM DF DGEMM-D(DF) binary64 DF DGEMM-DF(DF) 5 DF DGEMM DGEMM-DF(DF) DGEMM-D(DF) GeForce GTX 980 CPU Intel Core i (3.60GHz 4 ) 16 GB OS CentOS el7.x86 64 GeForce GTX 980 4

5 CUDA Version 6.5 *2 nvcc V gcc nvcc -arch=sm 52 -O3 -arch=sm 52 GM204 C = αab + βc N N N α β rand 5 Flops Flops DF 1 1Flop CUDA BLAS CUBLAS 6.5[13] MAGMA 1.5.0[14] SGEMM DGEMM MAGMA Maxwell Kepler GFlops CUBLAS 6.5 MAGMA SGEMM (GeForce GTX 980) N SGEMM DGEMM (GeForce GTX 980) 4.2 SGEMM 6 DGEMM 7 SGEMM DGEMM 1 DGEMM DGEMM-DF(DF) 2 CUBLAS SGEMM DGEMM 89% DGEMM DGEMM-DF(DF) DGEMM-D(DF) 71.6% DGEMM-D(DF) DGEMM- DF(DF) D-DF DGEMM-D(DF) binary64 DF Binary64 DF 1/32 6 DF DGEMM 1 NVIDIA Management Library NVML 80 CUBLAS SGEMM 80 *2 CUDA 6.5 GeForce GTX 970/980 Version GFlops CUBLAS 6.5 MAGMA This work This work (DGEMM-D(DF)) This work (DGEMM-DF(DF)) N DGEMM : 1:32 Maxwell GPU DF BLAS DGEMM DF 1/16 GeForce GTX 980 DGEMM DF DGEMM 2 DF binary64 GEMM 1/16 GeForce GTX 980 GPU DF CUDA GPU Cell/B.E. 5

6 1 GeForce GTX 980 GEMM SGEMM (CUBLAS) 4981 GFlops 4448 GFlops (N=3712) 89.3 % SGEMM (MAGMA) 4981 GFlops 2699 GFlops (N=3840) 54.2 % DGEMM (CUBLAS) GFlops GFlops (N=2944) 89.6 % DGEMM (MAGMA) GFlops GFlops (N=2944) 88.8 % DGEMM GFlops GFlops (N=2944) 88.7 % DGEMM-D(DF) GFlops GFlops (N=2304) 71.6 % DGEMM-DF(DF) GFlops GFlops (N=1536) 89.0 % DF JSPS [1] IEEE Computer Society: IEEE Standard for Floating- Point Arithmetic, IEEE Std , pp (2008). [2] Dekker, T.: A Floating-Point Technique for Extending the Available Precision, Numerische Mathematik, Vol. 18, pp (1971). [3] NVIDIA Corporation: Whitepaper NVIDIA GeForce GTX 980 Featuring Maxwell, GTX 980 Whitepaper FINAL.PDF (2014). [4] Graça, G. D. and Defour, D.: Implementation of floatfloat operators on graphics hardware, Proc. 7th Conference on Real Numbers and Computers (RNC7), pp (2006). [5] Thall, A.: Extended-Precision Floating-Point Numbers for GPU Computation, ACM SIGGRAPH 2006 Research Posters (2006). [6] Nakata, M., Takao, Y., Noda, S. and Himeno, R.: A Fast Implementation of Matrix-matrix Product in Double-double Precision on NVIDIA C2050 and Application to Semidefinite Programming, Proc. 3rd International Conference on Networking and Computing (ICNC 2012), pp (2012). [7] GPU 3 4. Vol. 6, No. 41, pp (2013). [8] Hida, Y., Li, X. S. and Bailey, D. H.: QD (C++/Fortran- 90 double-double and quad-double package), dhbailey/mpdist/. [9] Nagai, T., Yoshida, H., Kuroda, H. and Kanada, Y.: Fast Quadruple Precision Arithmetic Library on Parallel Computer SR11000/J2, Proc. 8th International Conference on Computational Science, Part I, ICCS 08, pp (2008). [10] Nath, R., Tomov, S. and Dongarra, J.: An Improved MAGMA GEMM for Fermi GPUs, University of Tennessee Computer Science Technical Report, No. UT-CS (2010). [11] Tan, G., Li, L., Triechle, S., Phillips, E., Bao, Y. and Sun, N.: Fast Implementation of DGEMM on Fermi GPU, Proc International Conference for High Performance Computing, Networking, Storage and Analysis, SC 11, No. 35, pp (2011). [12] Lai, J. and Seznec, A.: Performance Upper Bound Analysis and Optimization of SGEMM on Fermi and Kepler GPUs, Proceedings of the 2013 IEEE/ACM International Symposium on Code Generation and Optimization (CGO), CGO 13, pp (2013). [13] NVIDIA Corporation: CUBLAS Library (included in CUDA Toolkit), [14] University of Tennessee: Matrix Algebra on GPU and Multicore Architectures (MAGMA), 6

Σχετικά έγγραφα

GPU DD Double-Double 3 4 BLAS Basic Linear Algebra Subprograms [3] 2

GPU DD Double-Double 3 4 BLAS Basic Linear Algebra Subprograms [3] 2 GPU 4 1,a) 2,b) 1 GPU Tesla M2050 Double-Double DD 4 BiCGStab GPU 4 BiCGStab 1 1.0 2.2 4 GPU 4 1. IEEE754-2008[1] 128bit binary128 CG Conjugate Gradient [2] 1 1 2 a) mukunoki@hpcs.cs.tsukuba.ac.jp b) daisuke@cs.tsukuba.ac.jp