CUDA FFT. High Performance 3-D FFT in CUDA Environment. Akira Nukada, 1, 2 Yasuhiko Ogata, 1, 2 Toshio Endo 1, 2 and Satoshi Matsuoka 1, 2, 3

Transcript

1 Vol. 1 No (Aug. 2008) CUDA 3 FFT 1, 2 1, 2 1, 2 1, 2, 3 NVIDIA GPU CUDA CUDA 3 FFT GeForce 8 GPU 3 FFT CUFFT GFLOPS High Performance 3-D FFT in CUDA Environment Akira Nukada, 1, 2 Yasuhiko Ogata, 1, 2 Toshio Endo 1, 2 and Satoshi Matsuoka 1, 2, 3 CUDA environment, which is supported in latest NVIDIA GPUs, allows data sharing between threads using shared memory, and also provides more flexible memory accesses. We propose a high performance 3-D FFT algorithm for the CUDA environment. Using GeForce 8 series GPUs, we achieved a high performance up to 79.5 GFLOPS at 3-D FFT, which is from 3.1 to 3.3 times the performance compared with the performance of CUFFT library Graphics Processing Unit GPU 3 1 Tokyo Institute of Technology 2 Japan Science and Technology Agency, Core Research for Evolutional Science and Technology 3 National Institute of Informatics GPU CPU CPU GPU GPU GPU General-Purpose computation on Graphics Processing Unit GPGPU 1) GPU NVIDIA Cg 2) Microsoft High Level Shader Language HLSL C BrookGPU 3) GPU NVIDIA GPU CUDA Compute Unified Device Architecture 4) C/C++ GPU N 5) 6) GPU GPU FFT 7) 3 3 FFT CUDA FFT O(N log N) O(N) CPU GPU FFT GPU 8),9) 512 MB GPU FFT 2. NVIDIA CUDA GPU CUDA 231 c 2008 Information Processing Society of Japan

2 232 CUDA 3 FFT CUDA GPU G80 GeForce 8800 GTX GeForce 8800 GTX 16 MP Streaming Processor SP 8 16 kb 8, GTX 128 SP CUDA GPU SP SIMD Single Instruction Multiple Data CUDA Warp 24 Warp 1 Warp SP 1 CUDA GPU PCI-Express CUDA GPU GeForce GTS G MB 8800 GTS 512 G92 65 nm 90 nm G80 3 GPU 8800 GT 8800 GTX SP 8800 GTS 1 GFLOPS FFT SP AMD Phenom GHz GFLOPS 35.2 GFLOPS 8800 GTS 6 GPU multirow FFT 10),11) multirow FFT FFT Table 1 1 NVIDIA GeForce 8 Specifications of NVIDIA GeForce 8 series GPUs. SP Memory Model MP # clock GFLOPS Capacity Interface Clock Bandwidth 8800 GT GHz MB 256-bit 1,800 MHz 57.6 GB/s 8800 GTS GHz MB 256-bit 1,940 MHz 62.0 GB/s 8800 GTX GHz MB 384-bit 1,800 MHz 86.4 GB/s Table 2 2 Number of streams and achieved memory bandwidth. Bandwidth (GB/s) # of streams 8800 GT 8800 GTX FFT 1 FFT GPU SP SIMD multirow FFT GT GTX MB 2 float float2 STREAMS SIMD

3 233 CUDA 3 FFT global void cuda_memcpy(float *s, float *d) { int index = mul24(blockidx.x,blockdim.x) + threadidx.x; int step = mul24(griddim.x, blockdim.x); int n = 256 * 256 * (256 / STREAMS); float2 src1 = (float2 *)s, dst1 = (float2 *)d; float2 src2 = src1 + n, dst2 = dst1 + n; float2 src3 = src2 + n, dst3 = dst2 + n;... for (int i = index; i < n; i += step) { dst1[i] = src1[i]; dst2[i] = src2[i];... dststreams[i] = srcstreams[i]; } } 10 GB/s Coalesced Memory Access SP Half-Warp byte 8byte 16 byte 64 byte 128 byte 256 byte GPU GDDR Coalesced Memory Access Warp Coalesced Memory Access CUDA 3. CUDA 3 FFT 3 FFT 3 1 FFT CPU 3 FFT 3 X Y Z X Y Z CUDA GPU Coalesced Memory Access Half-Warp bit float 4byte 16 = 64 byte kb FFT X CPU 1 FFT Y Z 1 16 kb

4 234 CUDA 3 FFT multirow FFT multirow FFT 256 FFT multirow FFT α 1,024 8 Coalescing Memory Access 256 FFT 2 16 FFT 16 FFT multirow FFT FFT 16 FFT GT 8800 GTX 256 FFT multirow FFT 10 GB/s 16 FFT multirow FFT 38 GB/s Y Z 1 FFT 16 FFT 3 FFT FFT Z 256 FFT 2 16 FFT Z 256 FFT 3 1 Y 4 2 Y 5 X 1 FFT Y Z V(256,16,16,16,16) WORK Fortran 5 for Y Z COMPLEX V(256,16,16,16,16),WORK(256,16,16,16,16) for Z1,Y2,Y1,X WORK(X,*,Y1,Y2,Z1)=FFT256_1(V(X,Y1,Y2,Z1,*)) for Y2,Y1,Z2,X V(X,Z2,*,Y1,Y2)=FFT256_2(WORK(X,Z2,Y1,Y2,*)) for Y1,Z1,Z2,X WORK(X,*,Z2,Z1,Y1)=FFT256_1(V(X,Z2,Z1,Y1,*)) for Y2,Y1,Z2,X V(X,Y2,*,Z2,Z1)=FFT256_2(WORK(X,Y2,Z2,Z1,*)) for Z1,Z2,Y1,Y2 V(*,Y2,Y1,Z2,Z1)=FFT256(V(*,Y2,Y1,Z2,Z1)) FFT256() 256 FFT FFT256 1() FFT256 2() 256 FFT 16 FFT for Z1,Y2,Y1,X Z1 Y2 Y1 X 4 4 (Z1*Y2*Y1*X) 1 Coalesced Memory Access FFT 16 FFT FFT 16 FFT 2 V(X,Y1,Y2,Z1,Z2) V(X,Y2,Y1,Z2,Z1) Stockham 11) Y Z 16 FFT 3 A D 4 Table 3 3 Definition of memory access patterns. A (256,*,16,16,16) B (256,16,*,16,16) C (256,16,16,*,16) D (256,16,16,16,*)

5 235 CUDA 3 FFT 4 GB/s 8800 GT Table 4 Achieved memory bandwidth (GB/s) with each memory access pattern for input and output on 8800 GT. The number of thread blocks is 42, and the number of threads per thread block is 64. Output Input A B C D A B C D Table 6 6 The configuration of the system used for performance evaluations. CPU AMD Phenom 9500, 2.2 GHz, Quad-Core Chipset AMD 790FX RAM DDR2-800 SDRAM 1 GB 4 OS Fedora Core 8, linux Driver NVIDIA Linux driver Software CUDA SDK 1.1 CUDA Toolkit 1.1 GCC GB/s 8800 GTX Table 5 Achieved memory bandwidth (GB/s) with each memory access pattern for input and output on 8800 GTX. The number of thread blocks is 48, and the number of threads per thread block is 64. Output Input A B C D A B C D C D A B 1 A B 1 C D FFT SP 5 X 256 FFT SP FFT FFT SP SP 100% 64 8 SP GeForce 8800 GT/GTS/GTX 3 FFT 6 790FX 8800 GT/GTS PCI-Express GTX PCI-Express GPU FFT GFLOPS N 3 3 FFT 15N 3 log 2 N CUFFT3D

6 236 CUDA 3 FFT Table 8 8 The elapsed time and achieved bandwidth in each step of our implementation. Step 1&3 Step 2&4 Step 5 Total Model Time (ms) GB/s Time (ms) GB/s Time (ms) GB/s Time (ms) GB/s 8800 GT GTS GTX CUFFT FFT Table 7 Performance of 3-D FFT of size FFT 65, FFT Table 9 The performance of 65,536 sets of 256-point 1-D FFTs. Our implementation CUFFT3D Model Time (ms) GFLOPS GFLOPS Our Implementation CUFFT1D Model Time (ms) GFLOPS Time (ms) GFLOPS 8800 GT GTS GTX GT GTS GTX NVIDIA CUDA Toolkit 1.1 FFT 3 FFT GeForce 8 3 CUFFT FFT &3 2& FFT FFT FFT FFT FFT 5 SP 8800 GT 8800 GTS GTX 1 4 SP SP 8800 GTS 8800 GTX , FFT CUFFT 1 FFT 9 CUFFT SP 8800 GTX 5 GFLOPS 30% SP 32%

7 237 CUDA 3 FFT Table FFT The performance of 3-D FFT of size including the data transfer time between host and device. Host-to-Device 3D FFT on Device Device-to-Host Total Model PCI-Express Time (ms) GB/s Time (s) GFLOPS Time (ms) GB/s Time (ms) GFLOPS 8800 GT 2.0 x GTS 2.0 x GTX 1.1 x FFTW 3.2alpha2 CPU FFT Table 12 The performance of 3-D FFT of size in single precision using FFTW library 3.2alpha2 on CPUs. Processor Clock Socket/Core Compiler Time (ms) GFLOPS AMD Phenom GHz 1/4 gcc Intel Core 2 Quad Q GHz 1/4 gcc AMD Opteron GHz 8/16 Intel Compiler FFT GFLOPS Table 10 The performance (GFLOPS) of 3-D FFT of size 64 3, and CUFFT3D Our implementation Model GT GTS GTX GFLOPS CUFFT 11 FFT GPU PCI-Express 1 GPU 8800 GTX GPU PCI-Express 2.0 FFT GPU GPU 3 FFT GPU GPU FFT 2 FFT X FFT CPU CPU FFTW 12) 3.2alpha2 OpenMP SSE CPU OpenMP CPU GPU 11 CUDA GPU

8 238 CUDA 3 FFT 5. GPU CPU CUDA GPU CUDA 3 FFT X Y Z multirow FFT GeForce 8 GPU FFT NVIDIA CUFFT GFLOPS ULP-HPC Microsoft Technical Computing Initiative HPC-GPGPU: Large-Scale Commodity Accelerated Clusters and its Application to Advanced Structural Proteomics 1) General-Purpose Computation Using Graphics Hardware. 2) Mark, W.R., Glanville, R.S., Akeley, K. and Kilgard, M.J.: Cg: A System for Programming Graphics Hardware in a C-like Language, ACM Trans. Graphics (Proc. SIGGRAPH 2003 ), Vol.22, No.3, pp (2003). 3) Buck, I., Foley, T., Horn, D., Sugerman, J., Fatahalian, K., Houston, M. and Hanrahan, P.: Brook for GPUs: Stream Computing on Graphics Hardware, SIG- GRAPH 04: ACM Transactions on Graphics, Vol.23, No.3, pp (2004). 4) NVIDIA CUDA Compute Unified Device Architecture. 5) Stock, M.J. and Gharakhani, A.: Toward efficient GPU-accelerated N-body simulations, 46th AIAA Aerospace Sciences Meeting and Exhibit, AIAA (2008). 6) Larsen, E.S. and McAllister, D.: Fast Matrix Multiplies using Graphics Hardware, The 2001 ACM/IEEE conference on supercomputing (CDROM ), ACM Press (2001). 7) Cooley, J.W. and Tukey, J.W.: An Algorithm for the Machine Calculation of Complex Fourier Series, Math. Comput., Vol.19, pp (1965). 8) Moreland, K. and Angel, E.: The FFT on a GPU, Proc. SIGGRAPH/Eurographics Workshop on Graphics Hardware 2003, pp (2003). 9) Govindaraju, N.K., Larsen, S., Gray, J. and Manocha, D.: A Memory Model for Scientific Algorithms on Graphics Processors, The 2006 ACM/IEEE conference on supercomputing (CDROM ), IEEE (2006). 10) Swarztrauber, P.N.: FFT algorithms for vector computers, Parallel Computing, Vol.1, pp (1984). 11) Van Loan, C.: Computational Frameworks for the Fast Fourier Transform, SIAM Press, Philadelphia, PA (1992). 12) Frigo, M. and Johnson, S.G.: The Design and Implementation of FFTW3, Proc. IEEE, Vol.93, No.2, pp (2005). special issue on Program Generation, Optimization, and Platform Adaptation. ( ) ( ) FFT IEEE-CS GPU HPC

9 239 CUDA 3 FFT GCOE ACM IEEE-CS NAREGI 2006 TSUBAME JSPS Award ISOTAS 96 ECOOP 97 ISCOPE 99 ACM OOPSLA 02 IEEE CCGrid 03 HPCAsia 2005 Grid 2006 IEEE/ACM Supercomputing 04 Network Track Chair Reflection 01 IEEE CCGrid 06 Global Grid Forum Steering Group