GridFTP-APT: Automatic Parallelism Tuning Mechanism for Data Transfer Protocol GridFTP

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "GridFTP-APT: Automatic Parallelism Tuning Mechanism for Data Transfer Protocol GridFTP"

Ευθαλία Τρικούπης
5 χρόνια πριν
Προβολές:

1 THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS TECHNICAL REPORT OF IEICE. -APT: TCP TCP TCP TCP -APT ( with Automatic Parallelism Tuning) -APT TCP -APT: Automatic Parallelism Tuning Mechanism for Data Transfer Protocol Takeshi ITO, Hiroyuki OHSAKI, and Makoto IMASE Graduate School of Information Science and Technology, Osaka University 1-5 Yamadaoka, Suita, Osaka, Japan {t-itou,oosaki,imase}@ist.osaka-u.ac.jp Abstract has been used as a data transfer protocol to effectively transfer a large volume of data in Grid computing. supports a feature called parallel data transfer that improves throughput by establishing multiple TCP connections in parallel. However, in order to achieve high throughput, the number of TCP connections should be optimized based on the network status. In this paper, we propose an automatic parallelism tuning mechanism called -APT ( with Automatic Parallelism Tuning) that adjusts the number of parallel TCP connections only using information measurable in a Grid middleware. Through simulation experiments, we demonstrate that -APT significantly improves the performance of in various network environment. Key words Grid,, Parallel TCP Connections, Golden Section Search Method 1 [1, 2] TCP TCP TCP TCP TCP TCP [3] TCP 1

2 TCP [3] [3] TCP TCP [4] TCP 2 TCP [5] [5] ( TCP TCP ) [5] [5] [6] ( ) [5] TCP TCP 3 [6] TCP -APT ( with Automatic Parallelism Tuning) -APT -APT TCP -APT TCP -APT -APT TCP -APT -APT 4 -APT 5 2 FTP (File Transfer Protocol) [7 9] GGF (Global Grid Forum) [1] client TCP socket buffer size TCP connection parallel file transfer Grid network parallelism TCP socket buffer partial data transfer 1 Fig. 1 Parallel data transfer in server file server TCP TCP FTP TCP [1] [1] Globus Toolkit [11] 1 ( v1) TCP TCP GGF v1 2 ( v2) [2] v2 v1 TCP v2 OPTS RETR TCP ( 1) TCP TCP TCP [12] (1) TCP TCP TCP TCP AIMD TCP (2) TCP 1 TCP N TCP N TCP (3) TCP TCP 1 file 2

3 2 N TCP N TCP N (1) TCP (2) TCP TCP N TCP N 3 TCP 3. 1 TCP TCP Globus Toolkit TCP TCP TCP TCP TCP TCP TCP APT [5] ( NW G min R, N (1 ( )) p ) 6+21p 3+ (1) 2 R p p ( 2+ 2 BR N ( ) BR 2 ) 1 (2) N G N TCP W TCP TCP B R TCP p (1) G TCP N TCP N G TCP TCP -APT -APT TCP -APT TCP -APT (Golden Section Search Method) [13] f(x) [x l : x r] f(x) x -APT TCP [1] ( ) ERET ESTO T ERET ESTO [1] X T G = X/T 3. 3 TCP TCP ( ) TCP TCP TCP ( ) TCP N G(N) k TCP N k -APT 1 TCP N N N N TCP 2 G(N) 3 (N 2,N 1,N) (4) G(N) <G(N 1) 3

4 4 TCP N (2) N α N α(> 1) TCP 1 N (l, m, r) TCP -APT TCP N 1 TCP N { l +(m l)ν if m l>r m N (3) m +(r m)ν otherwise ν (= (3 5)/2) TCP 1 N 2 G(N) 3 (4) G(N) >G(m) (1) { (l, m, N) if m<n (l, m, r) (N,m,r) otherwise 4 (1) { (m, N, r) if m<n (l, m, r) (4) (l, N, m) otherwise 3. 4 ( ) X TCP TCP TCP [14] TCP TCP TCP TCP -APT goodput APT ( ) Fig. 2 Example of -APT operation (searching for bracket) -APT -APT X G(N 1) G(N 1)/G(N 2) X G(N 1) G(N 1) G(N 2) G(N 1) G(N 2) X X N W R W TCP R 2 G(N 2) X X αg(n 1) TCP (l, m, r) 2 { ((1 ξ) G(l)+ξG(m)) if N<m X ((1 ξ) G(m) +ξg(r)) otherwise ξ { N l m l N m r m if N<m otherwise APT -APT 2 - APT N =1 α =2 k k -APT TCP ( ) 4 4

5 goodput file client sending host TCP socket buffer W [Kbyte] control channel data channel RED router chunk router buffer size L [packet] # of TCP connections N bottleneck link bandwidth 1[Mbit/s] τ[ms] RED router router buffer size L [packet] server receiving host TCP socket buffer W [Kbyte] 3 -APT ( TCP ) Fig. 3 Example of -APT operation (searching for the optimal number of TCP connections) TCP N 1(=N ) TCP N TCP N 4 8 (2, 4, 8) 3 -APT TCP 3 k k (2, 4, 8) TCP (2, 4, 8) 5 TCP N (3) N =6 5 G(6) G(4) <G(6) (4) (4, 6, 8) TCP N (4, 6, 8) (4, 6, 7) (5, 6, 7) (5, 6, 7) N =6 TCP 4 -APT APT ns-2 [15] [ms] -APT TCP -APT TCP 15 [s] N =13 1 [s] 5 [s] TCP 4 Fig. 4 Network topology used in simulation 1 Table 1 Parameter configuration used in simulation 1 [Mbit/s] 1, 2 [ms] RED 1 [packet] RED max th 75 RED min th 25 RED max p.1 RED w q.2 TCP 64 [Kbyte] TCP 1 [byte] TCP N 4 TCP α 2 1. [s] APT 5 -APT TCP (τ = 1 [ms]) Fig. 5 in -APT for τ =1[ms] -APT 6 TCP TCP TCP APT 1 [s] 8 [Mbit/s] 5 [s] 85.3 [Mbit/s] RED 1 [packet] TCP 85 [Mbit/s] -APT 15 [s] 2 [ms] -APT TCP [s] TCP 1 [ms] ( 7) 5

6 goodput [Mbit/s] APT N=1 N=4 N=8 N=16 N=32 6 -APT (τ =1[ms]) Fig. 6 Evolution of goodput in -APT for τ =1[ms] APT 7 -APT TCP (τ = 2 [ms]) Fig. 7 in -APT for τ =2[ms] goodput [Mbit/s] APT N=1 N=4 N=8 N=16 N=32 8 -APT (τ =2[ms]) Fig. 8 Evolution of goodput in -APT for τ =2[ms] TCP TCP N =25 1 [ms] (N =13) 8 -APT TCP 8 -APT TCP TCP 67.5 [Mbit/s] -APT 5 TCP -APT -APT TCP TCP -APT -APT -APT -APT (NAREGI) ( ) [1] W. Allcock et al., : Protocol extensions to FTP for the Grid, GGF Document Series GFD.2, Apr. 23. Also available as [2] I. Mandrichenko, W. Allcock, and T. Perelmutov, v2 protocol description, GGF Document Series GFD.47, May 25. Also available as [3] S. Thulasidasan, W. Feng, and M. K. Gardner, Optimizing through dynamic right-sizing, in Proceedings of IEEE International Symposium on High Performance Distributed Computing, June 23. [4] T. J. Hacker and B. D. Athey, The end-to-end performance effects of parallel TCP sockets on a lossy wide-area network, in Proceedings of the 16th IEEE-CS/ACM International Parallel and Distributed Processing Symposium (IPDPS), Aug. 21. [5] T. Ito, H. Ohsaki, and M. Imase, On parameter tuning of data transfer protocol in wide-area Grid computing, in Proceedings of Second International Workshop on Networks for Grid Applications (GridNets 25), pp , Oct. 25. [6] T. Ito, H. Ohsaki, and M. Imase, Automatic parameter configuration mechanism for data transfer protocol, to be presented at the 26 International Symposium on Applications and the Internet (SAINT 26), Jan. 26. [7] J. Postel and J. Reynolds, File transfer protocol (FTP), Request for Comments (RFC) 959, Oct [8] R. Elz and P. Hethmon, FTP security extensions, Request for Comments (RFC) 2228, Oct [9] P. Hethmon and R. Elz, Feature negotiation mechanism for the file transfer protocol, Request for Comments (RFC) 2389, Aug [1] Global Grid Forum. [11] Globus Toolkit. available at [12] L. Qiu, Y. Zhang, and S. Keshav, On individual and aggregate TCP performance, in Proceedings of Internetl Conference on Network Protocols, pp , Oct [13] W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press, [14] N. Ehsan and M. Liu, Analysis of TCP transient behavior and its effect on file transfer latency, in Proceedings of IEEE International Conference on Communications (ICC23), vol. 26, pp , May 23. [15] The network simulator ns2. available at edu/nsnam/ns/. 6

Σχετικά έγγραφα

GPGPU. Grover. On Large Scale Simulation of Grover s Algorithm by Using GPGPU

GPGPU. Grover. On Large Scale Simulation of Grover s Algorithm by Using GPGPU GPGPU Grover 1, 2 1 3 4 Grover Grover OpenMP GPGPU Grover qubit OpenMP GPGPU, 1.47 qubit On Large Scale Simulation of Grover s Algorithm by Using GPGPU Hiroshi Shibata, 1, 2 Tomoya Suzuki, 1 Seiya Okubo