ISS 0002239/ C 2777/ TP Journal of Computer Research and Development 42 (6) : 039046, 2005 RAID,2 2,3 2 2 ( 00876) 2 ( 30007) 3 ( 300222) (cu bj @snacomcn) Research on Perf ormance Bounds of et worked RAID Storage Systems Cu Baojang,2, Lu J un 2,3, Wang Gang 2, and Lu Jng 2 ( Inf orm aton Securty Cent re, Bejng U nversty of Posts and Telecom m uncatons, Bejng 00876) 2 ( Depart ment of Com puter Scence and Technology, anka U nversty, Tanjn 30007) 3 ( Depart ment of Inf orm aton Scence and Technology, Tanjn U nversty of Fnance and Economcs, Tanjn 300222) Abstract Prevous work on performance evaluaton of networked storage systems has been mostly qualta2 tve, and t he quanttatve analytcal met hod and model are stll lmted A quanttatve analytcal model based on CQ2FC (closed queueng networks wt h fnte capacty) s presented accordng to t he data flow of dstrbuted networked software RAID (dns2raid) In order to cope wth the state space exploson problem of CQ2FC soluton, a novel approxmate performance bounds analyss (APBA) met hod s proposed, whch has lower computatonal complexty t han ot her approxmate analytcal met hods n t he lterat ure Exper2 mental testng results show that, the performance bounds of dns2raid based on CQ2FC calculated by APBA method can reflect the actual throughput and IΠO response tme bounds n lght load, heavy load and over load respectvely, and can offer the maxmal system load as well ey words networked storage ; performance bounds ; queueng networks ; performance analyss,, RAID 2APBA,,APBA, APBA, RAID, RAID, ; ; ; TP302 [ DAS ],, [ 2,3 ], [4 ] :2004-03 - 0 ; :2004-08 - 23 : (6027303) ; (2002005502) ; (0438003) 995-2005 Tsnghua Tongfang Optcal Dsc Co., Ltd. All rghts reserved.
040 2005, 42 (6) RAID SCSI, RAID, [ 5 ] IP,, Lnux, [ 6 ] EBD server, EBD SA, clent, EBD clent EBD server Socket, Socket, EBD RAID ( ),,, [7,8 ] throughput approxmaton, network decomposton, approxmate MVA, matchng state space,,, [7 ] Fg S2RAID storage system archtecture, BAS CQ2 FC APBA (approxmate performance bounds analy2 RAID IΠO, ss),, APBA, ll rw block RAID5, RAID5 rad5 make request IΠO IΠO EBD :2 RAID, RAID, ;3 IΠO EBD,EBD,4 APBA EBD RAID,5 EBD,clent, Socket, EBD 2 RAID server EBD clent server, 2 22 RAID Lnux RAID, PC Cluster RAID CQ2FC, RAID 2 LA, RAID, IP RAID EBD, CPU, RAID IΠO 3, 2 RAID CPU RAID5 CPU,, RAID, 995-2005 Tsnghua Tongfang Optcal Dsc Co., Ltd. All rghts reserved.
: RAID 04 DMA, IΠO MΠMΠ, Fg 2 Queueng network model for S2RAID storage system 2 RAID 23,, CPU, [9 ] D Csn CPU D Cm D Csn = - P m S T P mrw m, EBD Clent ( T snpro S mr5 + T S TU snp I P num ), (2) sn TCPΠIP T snpro D Cm = T mpro S m S TU m + ; mrw, S TU sn, T snp IP ( - P m ) mrw T mp I P num, (), S m ; T mpro ; S TU m ; P m D ; - P m m = ( - P m ) mrw I P num Frames e, T Rate e CPU T Rate e, Frames e RAID5 RAID5, CPU RAID5, mrw 2, D sn = - P m mrw T mp EBD Clent IP S T P mrw I P num Frames e (4) m T Rate e IΠO TCPΠIP I P num D d = - P m - P sn S T P m S T P snrw sn IP, I P num = S mr5 ΠM S S, S mr5 ( seek + latency + S TU sn ) S mr5, (5) RAID5, T Rate d S TU sn, S mr5 = S m S T P m Π P sn ; - P sn S T P m -, S T P m ; seek latency M S S TCP ; T Rate d 995-2005 Tsnghua Tongfang Optcal Dsc Co., Ltd. All rghts reserved. (3)
042 2005, 42 (6) ; snrw CQ2FC, IΠO RAID5, {,,, }, FCFS RAID5, RAID, p j j, 2 IΠO, v, p j RAID5 ; snrw 3 ; v snrw ; S T P sn v = k v j p j, {,, }, D j =, {,, }, D = v Π, D max = max 24 { D, (,, ) }, D sum = (vst rato) v D, (,, =, ), D avg = D sum Π B ( : ), (,, ), B mn = mn { B, (,, ) }, M v = v j p j, {,, } (6) j = B, B mn,, p j j = RAID BAS, CQ2FC,, C,, < B : Table Transton Probabltes for Wrte Operaton of S2RAID s odes [0 ] RAID, BAS, Transton Probablty Cm m sn Csn Dsn j Cm P m - P m 0 0 0 m 0 0 ΠS T P m 0 0 sn 0 0 0 0 Csn 0 0 0 P sn ( - P sn )ΠS T P sn Dsn j 0 0 0 0 = k [ M - P +, M ), P = mn{, B mn },, Cm, m CPU CQ2FC ; sn, Csn () [, B mn ] CPU B mn, Lttle,, (,, S T P m ) ; Dsn j X B ( ) j IΠO, j (,, S T P sn ) X ( ), (6), v Cm, RAID X B ( ) = X ( ) = (7) v T R ( ) T R ( ) 3 CQ2FC, [ ] MVA [ ] : 3 T R ( ) = [ + gn, CQ2 ( - ) ], (8) FC, gn ( - ) -, C 32 CQ2FC 3 CQ2 FC, B mn, B mn,, 995-2005 Tsnghua Tongfang Optcal Dsc Co., Ltd. All rghts reserved., [, M ) [, B mn ], ( B mn, M - P + ), =
: RAID 043 [0 ] v T R ( ) = = = D sum + = X B ( ) D [ + gn ( - ) ] = D gn ( - ) D sum + ( - ) D avg, (9) (0) (6) D sum + ( - ) D avg, v B T U ( ) : U ( ) = X B ( ) D,, e, D max X B ( ) D max = D max () X B ( ) (0) (), X B ( ) max{ D max, D sum + ( - ) D avg } (2) ( B mn, M - P + ) (2), e = - ( M - ) (8), j, j [2, Π ] j j,, [8 ] T R ( ) = [ + gn ( - ) ] + B T, (3) j { j p j 0, gn j ( ) = B j } (9), B T B T (9) S b ( e) S mn b ( e) : 0, S mn b ( e) = S mn b ( e - ) + B T mn be,, X B ( ) = = v v { [ + gn = ( - ) ] + B T } [ + gn ( - ) ] D sum + ( - ) D avg (4) () X,, B ( ) { max{ D max, D sum + ( - ) D avg } + X B ( ) S mn b ( - ( M - ) ) } (2) X B ( ) max{ D max, D sum + ( - ) D avg } 3, (2), (5) ( 2) (3) [ M - P +, M ) (5) e = - ( M - ), [ M - P +, M ),,,,, X B ( ) =,, e <, e e, e = [ + gn ( - ) ] + = v = v B T, v B T e S b ( e), v B T S b ( e), e = - ( M - ), (7) = S b ( e) S b ( e) = S b ( e - ) + B T be, B T be = mn b,, be- { v B T, =,, }, B T mn = B TM I, B T mn be = mn mn b,, mn be- { v B TM I, =,, }, e = - ( M - ) (20) (7) - ( M -, 995-2005 Tsnghua Tongfang Optcal Dsc Co., Ltd. All rghts reserved. (20), = v B T S mn b ( e), e = ) (6), CQ2FC 33 APBA CQ2FC
044 2005, 42 (6) Table 2 Tme Complexty Comparsons Between APBA 4, 3, B = Method and Others 2 APBA 4, B 2 = 5, B 3 = 3, B 4 = 2, CQ2FC =, 2 = 4, 3 = 2, 4 = 2, v =, v 2 =, [3 v 3 =, v 4 = ] Approxmate Analytcal Method APBA 3 : Tme Complexty etwork Decomposton O ( r 4 B max 3 ) Varable Queue Capacty Decomposton O ( 3 ) Matchng State Space O ( 3 + 2 ) Approxmate MVA O ( 3 + r ) APBA O ( lg ) Annotate : teratve number r, B max = max{ B, (,, ) } RAID 4 Fg 3 Throughputs of CQ2FC 3 CQ2FC Lnux73 PC, PIII800CPU,28MB 2,00M D2Lnk, 5, 4, B = AMD600CPU,64MB,00M D2Lnk, 2, B 2 = 4, B 3 = 3, B 4 = 2, B 5 = 4, 3 36 GB SCSI,4MB = 4, 2 = 3, 3 = 2, 4 =, 5 = 2,,49ms seek,299ms, v =, v 2 =, v 3 =, v 4 =, v 5 = 35MBΠs RAID [3 ], APBA IΠO, IΠO 4 : 2MB Fg 4 Throughputs of CQ2FC 4 CQ2FC, APBA CQ2FC Table 3 Model Conf guratons 3,APBA, v B TM I, O ( lg ) [7 CQ2FC ], 2, 4 RAID RAID5, RAID, 6 B, 8 B, 3 TCP 460B, 3 B Cm, B Csn, B m, B sn, B D CPU, CPU IΠO CPU Parameters Values Parameters Values T mpro 0078ms Frames e 58B T mp 0034ms B Cm 64MB T snpro 0047ms B Csn 30MB T snp 002ms B m 4MB P m 043 B sn 4MB P sn 0677 B D 4MB 995-2005 Tsnghua Tongfang Optcal Dsc Co., Ltd. All rghts reserved.
: RAID 045 5 RAID 998 264275 2 Yngpng Lu, Davd H C Du Performance study of SCSI2based APBA 5 storage subsystems IEEE Communcatons Magazne, 2003, 4, (8) : 7682, 3 Stephen Aken, Drk Grunwald, Andrew R Pleszkun, et al A, performance analyss of the SCSI protocol In : Proc MSS 03, Washngton : IEEE Computer Socety Press, 2003 2334 4 Xubn He,Praveen Beedanagar, Dan ZHou Performance evalua2, ton of dstrbuted SCSI rad The 2th Int l Conf PACT2003,, ew Orleans, 2003,, 5 Wee Teck g, Bruce Hllyer, Elzabeth Shrver, et al Obtanng M, hgh performance for storage outsourcng Conference on Fle and,, Storage Technologes( FSAT 02), Monterey, Calforna, 2002 6 Yao2Long Zhu, Shun2Yu Zhu, Hu Xong Performance analyss APBA and testng of the storage area network The 9th IEEE Sympo2 RAID, sum on Mass Storage Systems and Technologes, Maryland, USA, 2002 7 S Balsamo, A Ranero Closed queueng networks wth fnte ca2 Fg 5 Comparson of the theoretc and expermental throughput M Reser, S S Lavenberg Mean value analyss of closed mul2 tchan queueng networks J ACM, 980, 27 (2) : 33322 2 5 C H Sauer, M Chandy Computer Systems Performance Modelng Englewood Clffs, J : Prentce2Hall, 98 5 RAID Cu Baojang, born n 973 Lecturer, re2, RAID ceved the Ph D degree from the anka, Unversty n 2004, Hs man research area RAID, are nformaton securty, remote dsaster tol2 APBA,erance and network storage,973,,,, APBA, RAID Lu Jun, born n 963 Assocate professor,, Ph D canddate, Hs man research area RAID are network storage,parallel and dstrbuted, systems Rakesh Barve, Elzabeth Shrver, Phllp B Gbbons Modelng and optmzng IΠO throughput of multple dsks on a bus In : Proc Sgmetrcs 98ΠPerformance 98. ew York : ACM Press, 995-2005 Tsnghua Tongfang Optcal Dsc Co., Ltd. All rghts reserved. pacty queues : Approxmate analyss The 4th European Smula2 ton Multconference2Smulaton and Modelng : Enablers for a Bet2 ter Qualty of Lfe ( ESM2000), Ghent, Belgum, 2000 8 I F Akyldz Mean value analyss of blockng queueng net2 works IEEE Trans on Soft Eng, 988, 4 (4) : 48428 9 E D Lazowska, J Zahorjan, G S Graham, et al Quantta2 tve System Performance : Computer System Analyss Usng Queueng etwork Models ew Jersey : Prentce2Hall, 984 0 Raf O Onvural, Survey of closed queueng networks wth block2 ng ACM Comput Surv, 990, 22 (2) : 832 3 R O Onvural, H G Perrors Approxmate throughput analy2 ss n cyclc queueng networks wth fnte buffers IEEE Trans Software Engneerng, 989, 5 (6) : 800808,963,,
046 2005, 42 (6) Wang Gang, born n 974 Assocate profes2 sor, receved the Ph D degree from the anka Unversty n 2002, Hs man re2 search area are data layout,parallel and ds2 trbuted systems,974,, Lu Jng, born n 942 Professor and Ph. D. supervsor. Hs man research area are parallel and dstrbuted systems, mass stor2 age, desgn and analyss of algorthms,942,, Research Background Up to now most studes on the performance evaluaton of networked storage systems are qualtatve, and the quanttatve analyt2 cal method and model are stll lmted, especally for dstrbuted networked RAID (D2RAID) storage system In order to analyze the system performance, a CQ2FC (closed queueng networks wth fnte capacty) quanttatve analytcal model of D2RAID storage system s presented Meanwhle, we put forward an APBA (approxmate performance bounds analyss) method to work out the per2 formance bounds of CQ2FC model, whch has less tme complexty than other methods The performance bounds of D2RAID based on CQ2FC calculated by APBA are compared wth the exact solutons and expermental results, and found to be n agreement Ths paper s sponsored by the atonal atural Scence Foundaton of Chna ( Grant o6027303), Educaton Mnstry Doctoral Research Foundaton of Chna ( Grant o200200502) and Tanjn Muncpal Scence and Technology Development Foundaton ( Grant o 0438003) 995-2005 Tsnghua Tongfang Optcal Dsc Co., Ltd. All rghts reserved.