31 3 2014 3 DOI: 10.7641/CTA.2014.30690 Control Theory & Applcatons Vol. 31 No. 3 Mar. 2014 q,,,, (, 102249) : L 2 (square support vector machne algorthm, LS SVM),, q = 2. q LS SVM, 0 < q <, q. cq, c q,,,.,, L 2LS SVM. : q ; (LS SVM); : TP181 : A q norm regularzng least-square-support-vector-machne lnear classfer algorthm va teratve reweghted conjugate gradent LIU Jan-we, LI Ha-en, LIU Yuan, FU Je, LUO Xong-ln (Research Insttute of Automaton, Chna Unversty of Petroleum, Bejng 102249, Chna) Abstract: The L 2 norm penalty least-square-support-vector-machne algorthm (LS SVM) has been extensvely studed and s probably the most wdely used machne learnng algorthm. The regularzaton parameter n LS SVM s predetered wth default value q = 2. Based on the teratve reweghted conjugate gradent algorthm, the q norm regularzng LS-SVM s proposed wth 0 < q < 1, a ratonal number. We desgn a grd method to change the value of two adjustable parameters, the regularzaton parameter c and the order q of regularzaton, by performance ndcators of predcton error rate. On the selected values of c and q, usng the teratve reweghted conjugate gradent algorthm for solvng classfcaton object functon and fndng the mum predcton error, we can mprove the feature selecton and predct the error rate. The expermental results on real datasets n dfferent felds ndcate that the predcton performance s more accurate than L 2 norm LS-SVM, and can carry out feature selecton. Key words: q-norm regularzaton; LS SVM; Iteratve reweghted conjugate gradent method 1 (Introducton) Vapnk (support vector machne, SVM), L 2 (L 2 norm support vector machne, L 2 SVM [1] ). (x, y ),,, m, x R n, y { 1, +1}, SVM J(w) = pen(w) + C m L(y, wx ), w f(x)=sgn( w, x +b), J(w) pen(w), L(y, wx). L 2 SVM w J(w) = 1 2 wt w + C m max(1 y, w T x, 0). (1) L 2 (L 2 -norm least square support vector machne, L 2 LS SVM) SVM, SVM C m ξ 2 + 1 wξ 2 2 w 2 2, (2) s.t. y [ x, w + b] = 1 ξ, = 1,, m, [2 3] : 2013 07 08; : 2013 12 02.. E-mal: lujw@cup.edu.cn; Tel.: 86 10-89733306. : 973 (2012CB720500); (21006127); ( ) (JCXK 2011 07).
3 : q 335 [ ] [ ] [ ] 0 y T b 0 w 1 SVM, =, (3) y Ω + I/C a 1 v [16], Lu Yufeng L 0 L 1 pen(w) : a = [a 1 a n ], Ω = z T = λ w z, 0 + (1 λ) w 1 SVM [17], z T, L 0 SVM = [x 1 y 1 x m y m ], y = [y 1 y m ], 1 v = L 1 SVM, [1 1]. Lu Yufeng L 1, L 2 L 2 LS SVM pen(w) = w q (q {1, 2}) SVM [18]., L 2 SVM. L 1, Hu,,., Zou : pen(w) = λ w 1 + (1 λ) w 2,, [19]. ; 2 L 1, L 2.,. Su-In Lee L 1., Lopez 1 Norm SVM [4] [20], Zhenqu Lu, L q [21],,, LopezL 0, (recever-operatng characterstc, ROC) (2) wa [5] C m ξ 2 + 1 (area under the curve, AUC) n λ j a 2 w,ξ,a 2 2 j, q. [22 24], s.t. y [( m a k x k )x + b] = 1 ξ, = 1,, m. L q, 0 < q < 1. k=1 q SVM, [6]q SVM, (1) L 2 L q w q = ( m w j q ) 1/q, 0 < q < 1, (1) w v, v < w < v,, [7]. [8 12], [13] L q LS SVM, LS SVM s = m a y x x j + b, ξ 2 s s k k=1,,m µ = max s k s, k k=1,,m k=1,,m (2) C m ξ 2 2 C m µ ξ 2 2 (3), Ω (Cµ ) 1, β q, β [ ] [ ] 0 y T 0 s.t. β =, y Ω + I/C 1 v β = [b a] T, 1 q 2,, Enrque BrtoIRMS [14]. Kloft L q SVM [15], q > 1., L 1, L 2 SVM,. Zhu J L 1 pen(w) =,, LS SVM,,, L q q, 0 < q <. LS SVM, q,, : q,,, q q LS SVM, q. q LS SVM(L q LS SVM) : w,ξ 1 2 m ξ 2 + c q w q q, (4) s.t. y (x T w + b) = 1 ξ, = 1, 2,, m, 0 < q <. (4), q = 2, L 2 LS SVM, q =1, L 1 LS SVM. 0<q <1, q =1, q = 2,? [25 27], L q LS SVM., Tany Zhou[28] Nesterov (Nesterov s method)svm, SVM, SVM Perf [29], Pegasos [30], SVM Lght [31] LIBSVM [32],
336 31,.,, q c, q 2 0 2 13, 14 ; c 2 4 2 13, 18. q c (4),,,, q. k, w (k) + λ (k) d (k) q 2 w (k 1) + λ (k 1) d (k 1) q 2, k, w (k) +λ (k) d (k) q 2 k 1 w (k 1) + λ (k 1) d (k 1) q 2, n n w (k) w (k 1) + λ (k) d (k) q + λ (k 1) d (k 1) q 2 w (k) + λ (k) d (k) 2, q,,.,, :,,, ;,, w,, [33 34].,, [35 36],,.,,,.,,,,,,,. 5: 1) q LS SVM, 0 < q,, q,. 2) L q LS SVM, q c, q cl q LS SVM. 3),,, q,. 4) w, w, w,,. 5) L q LS SVM Tr11 Arcene Sylva PIE 32 32 Yaleb 32 32,cq, LS SVM. 2 L q LS SVM(L q LS SVM classfer) 2.1 (Objectve functon) X m n, m n. m, n, X m n x R m, y {±1}, w R m, L q LS SVM w,b f (w, b) = 1 N m L(y, x T w + b) + η w q q, (5) : w q = ( n w j q ) 1/q, 0 < q <, w j w j, L( ), η. L q LS SVM f 1 m 1(w, ξ) = ξ 2 + c w,ξ w,ξ 2 q w q q, (6) s.t. y (x T w + b) = 1 ξ, = 1, 2,, m, c (5) η [37]., w T :=[w 1 w n b] T, x T :=[x 1 x n 1], : f 1 m 2(w, ξ) = ξ 2 + c w,ξ w,ξ 2 q w q q, (7) s.t. y (x T w) = 1 ξ, = 1, 2,, m., ξ f 2(w) = 1 m (1 y (x T w)) 2 + c w 2 q w q q. (8) 2.2 λ (k) (Soluton for conjugate gradent step-sze) h(w) = w q = ( n w j q ) 1/q, k(w) = h q (w), sgn w j = w j w j,
3 : q 337 0 < q <, 1 m (y 2 (x T ) 2 (λ (k) d (k) ) 2 + k(w) 2 = q w q 2 j w j. (9) w j f(w) w j, f = m y x,j (y (x T ) 1) + w j c w j q 2 w j, j = 1, 2,, n. (10) g (k) := f g k, w w (k) k, λ (k) k. : w (k+1) = w (k) λ (k) g (k), g (k) = m y () x (,j) (y (x T w (k) 1) + c w (k) j q 2 w (k) j., w w 0, d (1), d (1) = g (1). k, (8) w (k+1) = w (k) + λ (k) d (k), (11) f 2 (w) = 1 m (1 y (x T (w (k) + λ (k) d (k) ))) 2 + 2 c q w(k) + λ (k) d (k) q q. (12) w (k) + λ (k) d (k) q 2 w (k 1) + λ (k 1) d (k 1) q 2, k w (k) + λ (k) d (k) q 2 k 1 w (k 1) + λ (k 1) d (k 1) q 2, f 2 (w) 1 m (1 y (x T (w (k) + λ (k) d (k) ))) 2 + 2 c n w (k 1) +λ (k 1) d (k 1) q 2 w (k) +λ (k) d (k) 2. q (13) k w (k) d (k), (13) λ (k) : f 2 (λ) λ (k) 1 m (y 2 (x T ) 2 (w (k) + λ (k) d (k) ) 2 2 2y (x T (w (k) + λ (k) d (k) ))) + c q w(k 1) + λ (k 1) d (k 1) q 2 w (k) + λ (k) d (k) 2 = 2y 2 (x T ) 2 w (k) λ (k) d (k) 2y (x T λ (k) d (k) ))+ c q w(k 1) + λ (k 1) d (k 1) q 2 (2w (k) λ (k) d (k) + (λ (k) d (k) ) 2 ). (14), λ (k),. c 1 = c q w(k 1) + λ (k 1) d (k 1) q 2, (14) f 2 (λ (k) ) λ (k) m (y (x T ) 2 λ (k) (d (k) ) 2 + 2y 2 (x T ) 2 w (k) d (k) 2y (x T d (k) )) + c 1 (2w (k) d (k) + 2λ (k) (d (k) ) 2 ). (15) f 2(λ (k) ) = 0, λ (k) λ (k) m y (x T d (k) ) y 2 (x T ) 2 w (k) d (k) (k) c 1 w (k) d y 2 (x T ) 2 (d. (k) ) 2 + c 1 (d (k) ) 2 (16) 2.3 (Soluton for teratve reweghted conjugate gradent model) (8) c 1 = c q w(k 1) + λ (k 1) d (k 1) q 2, w f 2(w)= 1 2 (1 Y Xw)T (1 Y Xw)+c 1 w T w, (17) : Y =dag{y 1, y 2,, y m }, X =(x T 1, x T 2,, x T m) T, w =(w (1), w (2),, w (m) ) T. (17): w f 2(w) = 1 2 wt Gw + b T w + a, (18) : G = (Y X) T (Y X) + c 1 I, b = 2Y X, a = 1. [38]β k : β k = gt k+1(gk+1 T gk T ) d T k (gt k+1 gt k ) = gt k+1g k+1 gk Tg. (19) k d (1) = g (1), d (k+1) = g (k+1) + β k+1 d (k), β k+1 = g(k+1), g (k+1). g (k), g (k) w (k+1) = w (k) + λd (k), (20) : d (k) k,,
338 31, λ., g (k), d (k) 0, g (k) d (k),. f (k) k, τ 1 : f (k) f (k 1) > τ 2 ; τ 2, (g (k), d (k) ) < τ 2, d (k),, (g (k), d (k) ) τ 2,, g (k)., τ 1 = 1e 5, τ 2 = 4 9 π. 2.4 (Iteratve algorthm of reweghted conjugate gradent) L q LS SVM: 1), (x 1, y 1 ),, (x n, y n ),, c q. 2) L q LS SVM. w = [1 1], t = 1, τ 1 = 1e 5, τ 2 = 4 9 π, τ 3 = 1e 5; : rw: w 0, rw()=0; w 0, rw() = c w q 2, tmp1 = y. X = [ m y x,1 m y x,2 tmp2 = tmp1 w 1. m y x,n ] T, 3) c, q,, 500, t = 1g (1), d (1) = g (1). t > 1 k, f (k) f (k 1) < τ 1 τ 1, g (k) = tmp1 T tmp2 + rw T w (k), d (k) d (k) = g (k) + β k d (k 1), β k = g(k), g (k) g (k 1), g (k 1)., τ 2, (g (k), d (k) ) < τ 2, d (k), (g (k), d (k) τ 2, d (k) = g (k) ; 4) w (k+1) = w (k) + λd (k), w (k). w w j w w, 0. 5),.. 1. 1. L q LS SVM Procedure Iter-Reweg-ConjG Input: pars of samples(x 1, y 1 ),, (x n, y n ) Output: w Intalzaton: setc = (ones(1, 18) 2). ( 4:13), q = 0.2 : 0.2 : 4 w = [1,, 1], t = 1, τ 1 = 1e 5, τ 2 = 4 9 π, τ 3 = 1e 5 for to length(c) do for to length(q) do for t=1 to 500 do f t=1 then g (k) = tmp1 T tmp2 + rw T w (k) d (1) = g (1) else f f (t) f (t 1) > τ 1 then g (t) (t) = tmp1t tmp2+rwt w d (t) = g (t) + β t d (t 1) β t = g(t), g (t) g (t 1), g (t 1) computaton: (g (t), d (t) ) f (g (t), d (t) ) τ 2 then d (t) = g (t) end f end f end f update: w t+1 := w t + λ (t) d (t) select features: f w (t) < τ 3 w (t) then w (t) = 0 end f end for end for end for return w end Iter-Reweg-ConjG 3 (Complexty analyss) O(m 2 ), m.,. c, q,, L q LS SVM O(m 2 s t), s t c, q,,,,
3 : q 339. 110 L p SVM, c, q, s t, L 2 LS SVM s. 1 10 L p SVM Table 1 The runnng tme results of L p SVM on 10 datasets c q L q LS SVM /s 57 2215 4096 3.6 0.19 96 4026 0.25 1.0 0.13 62 2000 0.25 1.2 1.15 78 3750 64 4.0 1.86 PIE 0102 340 1024 2048 1.4 2.58 Yaleb 0102 128 1024 2048 1.2 1.45 Yaleb 2627 128 1024 4096 1.2 2.05 Tr11 143 6424 32 3.2 1.87 Arcene 100 10000 128 2.4 5.32 Gsette 6000 5000 8 2.4 11.14 4 (Experments) 4.1 (Pretreatment), L q LS SVM 3. 3 (bladder) (colon) (lymphoma). http://www.hes.fr/ znovyev/prncmanf2006/, http://llmpp.nh.gov/lymphoma/ data/rawdata/, http://perso. telecom-parstech.fr/ gfort/glm/programs.html. (bladder) 2215; (colon) 2000; (lymphoma) 4026.,. (bladder) 57, 42, 15; (colon) 62, 46, 16 ; (lymphoma) 96, 72, 24. L q LS SVM L 2 LS SVM,, : Tr11 (http: // glaros. dtc. umn. edu/ gkhome/cluto/cluto)arcene Sylva(http: //www.modelselect.nf.ethz.ch/) PIE 32 32 Yaleb 32 32 (http://www.zjucadcg.cn/ dengca/ Data/data.html)., [ 1,1], x, {1, 2,, m}, : 1, x < 1, f(x) = x, 1 < x < 1, 1, x 1.. : g(x) = f(x) + 1. 2,. 4.2 L 2 LS SVM(Concluson and comparson to L 2 LS SVM) q 0<q < (bladder) (colon) (lymphoma), q. q > 4, 0 < q < 4, q q. cq,, q, qc. q, c., (c, q). 2, c, q. 2, L 2, L 1, L 0. 2 3 Table 2 Predcton error rates of the three cancel datasets c q /% (bladder) 4096 3.6 13.07 (colon) 0.25 1 12.95 (lymphoma) 0.25 1.2 4.72,. q = 1, q (0, 2], q c, (c, q), 10, q, c,,.,. (c, q), 10,. :
340 31 cq,. 3(a),,, 3(b),. 3(a) Table 3(a) Select features numbers on best classfcaton error /% (bladder) 13.07 558 (colon) 12.95 553.13 (lymphoma) 4.72 1401.6 3(b) Table 3(b) Classfcaton error on best feature selecton /% (bladder) 3.375 20.95 (colon) 7 42.19 (lymphoma) 10.875 7.29 L 2 LS SVM, LS SVMlab v1.7 (http: //www. esat. kuleuven.be/ssta/ lssvmlab/) L q LS SVM. LS SVM, 10, C, 3 10 sg C 4(a). 4(b) L q LS SVMqL 2 LS SVM sgc, 3. 4L 2 LS SVM, q, q L 2 LS SVM. 4(a) 3 C sg Table 4(a) The values of regularzaton parameter C and kernel wdth sg on the three real datasets C sg (bladder) 1.79 2080 (colon) 2.07 2369 (lymphoma) 3.84 7325 4(b) 3 Table 4(b) Comparson of test error rates on the three cancel datasets LS SVM/% q /% (bladder) 16.93 13.07 (colon) 16.95 12.95 (lymphoma) 13.79 4.72, L q LS SVM L 2 LS SVM. 10 : 3 Tr11 Arcene Sylva PIE 32 32 Yaleb 32 32. Yaleb 32 32,, 0 255. 38, 59 64, 1024. 1 2, 128, 1 64, 264. 1, 2 Yaleb 0102 ; 26 27, 128, Yaleb 2627 ; PIE 0102. 5 L q LS SVM, C, q. L 2 LS SVM 10,. 5 10 Table 5 Predcton error rates of the ten datasets and compare wth L 2 LS SVM C q L q LS SVM / % L 2 LS SVM / % 57 2215 4096 3.6 13.07 16.93 96 4026 0.25 1.0 12.95 16.95 62 2000 0.25 1.2 4.72 13.79 78 3750 64 4.0 10.69 14.71 PIE 0102 340 1024 2048 1.4 6.86 5.56 Yaleb 0102 128 1024 2048 1.2 2.60 3.30 Yaleb 2627 128 1024 4096 1.2 6.58 5.97 Tr11 143 6424 32 3.2 17.82 21.36 Arcene 100 10000 128 2.4 6.67 8.92 Gsette 6000 5000 8 2.4 9.43 11.28
3 : q 341 5 10 100, L q LS SVM LS SVM, PIE 0102 Yaleb 2627 L q LS SVM L 2 LS SVM. L q LS SVM. 5 (Concluson and prospect) L q LS SVM, L 2, L 1, L 0, q, q, 0 < q <, c, q,.,.,, Tr11 Arcene Sylva PIE 32 32 Yaleb 32 32,,, L 2 LS SVM.,, LS SVM,. (References): [1],,. p [J]., 2012, 38(1): 76 87. (LIU Janwe, LI Shuangcheng, LUO Xongln. Classfcaton algorthm of support vector machne va p-norm regularzaton [J]. Acta Automatca Snca, 2012, 38(1): 76 87.) [2] SUYKENS J A K, VAN GESTEL T, DE BRABANTER K, et al. Least Squares Support Vector Machnes [M]. Sngapore: World Scentfc, 2002. [3] SUYKENS J A K, VANDEWALLE J. Least squares support vector machne classfers [J]. Neural Processng Letters, 1999, 9(3): 293 300. [4] LOPEZ J, DORRONSORO J R. Least 1 norm SVMs: a new SVM varant between standard and LS SVMs [C] //Proceedngs of the 18th European Symposum on Artfcal Neural Networks. Bruges, Belgum, 2010, 135 140. [5] LOPEZ J, DE BRABANTER J, SUYKENS J A K, et al. Sparse L s- SVMs wth L 0-norm mzaton [C] //Proceedngs of the 19th European Symposum on Artfcal Neural Networks. Bruges, Belgum: Elsever, 2011, 189 194. [6] TAN J Y, ZHANG C H, DENG N Y. Cancer related gene dentfcaton va p-norm support vector machne [C] //Proceedngs of the Fourth Internatonal Conference on Computatonal Systems Bology. Suzhou, Chna: World Publshng Corporaton, 2010: 101 108. [7] BRADLEY P S, MANGASARIAN O L, STREET W N. Feature selecton va mathematcal programg [J]. INFORMS Journal on Computng, 1998, 10(2): 209 217. [8] SUYKENS J A K, DE BRABANTER J, LUKAS L, et al. Weghted least squares support vector machnes: robustness and sparse approxmaton [J]. Neurocomputng, 2002, 48(1): 85 105. [9] VALYON J, HORVATH G. A weghted generalzed LS SVM [J]. Electrcal Engneerng and Computer Scence, 2003, 47(3/4): 229 251. [10] ZHOU L, LAI K K, YEN J. Credt scorng models wth AUC maxmzaton based on weghted SVM [J]. Internatonal Journal of Informaton Technology & Decson Makng, 2009, 8(4): 677 696. [11] LESKI J M. Iteratvely reweghted least squares classfer and ts L 2-and L 1-regularzed Kernel versons [J]. Bulletn of The Polsh Academy of Scences: Techncal Scences, 2010, 58(1): 171 182. [12] SUN D, LI J, WEI L. Credt rsk evaluaton: Support vector machnes wth adaptve L q penalty [J]. Journal of Southeast Unversty (Englsh Edton), 2008, 24(S): 3 6. [13] LIU J, LI J, XU W, et al. Weghted L q adaptve least squares support vector machne classfers-robust and sparse approxmaton [J]. Expert Systems wth Applcatons, 2011, 38(3): 2253 2259. [14] BRITO A E. Iteratve adaptve extrapolaton appled to SAR mage formaton and snusodal recovery [D]. EI Paso: The Unversty of Texas, 2001. [15] KLOFT M, BREFELD U, SONNENBURG S, et al. Effcent and accurate L p-norm multple kernel learnng [J]. Advances n Neural Informaton Processng Systems, 2009, 22(22): 997 1005. [16] ZHU J, ROSSET S, HASTIE T, et al. 1-norm support vector machnes [J]. Advances n Neural Informaton Processng Systems, 2004, 16(1): 49 56. [17] LIU Y, WU Y. Varable selecton va a combnaton of the L 0 and L 1 penaltes [J]. Journal of Computatonal and Graphcal Statstcs, 2007, 16(4): 782 798. [18] LIU Y, HELEN ZHANG H, PARK C, et al. Support vector machnes wth adaptve L q penaltes [J]. Computatonal Statstcs & Data Analyss, 2007, 51(12): 6380 6394. [19] MARTIN S. Tranng support vector machnes usng Glbert s algorthm [C] //Proceedngs of the Ffth IEEE Internatonal Conference on Data Mnng. Houston, USA: IEEE, 2005: 306 313. [20] LEE S I, LEE H, ABBEEL P, et al. Effcent L 1 regularzed logstc regresson [C] //Proceedngs of the 21st Natonal Conference on Artfcal Intellgence. Menlo Park, CA: AAAI Press, 2006: 401 408. [21] LIU Z, JIANG F, TIAN G, et al. Sparse logstc regresson wth Lp penalty for bomarker dentfcaton [J]. Statstcal Applcatons n Genetcs and molecular Bology, 2007, 6(1): 1 22. [22] BRADLEY P S, MANGASARIAN O L. Feature selecton va concave mzaton and support vector machnes [C] //Proceedngs of the 15th Internatonal Conference on Machne Learnng. Madson, USA: Morgan Kaufmann, 1998, 82 90. [23] FOUCART S, LAI M J. Sparsest solutons of underdetered lnear system va L q-mzaton for 0 < q < 1 [J]. Appled and Computatonal Harmonc Analyss, 2009, 26(3): 395 407. [24] FAN J, LI R. Varable selecton va nonconcave penalzed lkelhood and ts oracle propertes [J]. Journal of the Amercan Statstcal Assocaton, 2001, 96(456): 1348 1360. [25] PYTLAK R. Conjugate Gradent Algorthms n Nonconvex Optmzaton [M]. New York: Sprnger, 2009.
342 31 [26] SCHIROTZEK W. Nonsmooth Analyss [M]. New York: Sprnger, 2007. [27] MISHRA S K, GIORGI G. Invexty and Optmzaton [M]. New York: Sprnger, 2008. [28] ZHOU T, TAO D, WU X. NESVM: a fast gradent method for support vector machnes [C] //Proceedngs of the Tenth IEEE Internatonal Conference on Data Mnng. Sydney, Australa: IEEE, 2010: 679 688. [29] JOACHIMS T. Tranng lnear svms n lnear tme [C] //Proceedngs of the 12th ACM SIGKDD Internatonal Conference on KnowledgeDscovery and Data Mnng. Phladelpha, USA: ACM, 2006: 217 226. [30] SHALEV-SHWARTZ S, SINGER Y, SREBRO N, et al. Pegasos: prmal estmated sub-gradent solver for svm [J]. Mathematcal Programg, 2011, 127(1): 3 30. [31] JOACHIMS T. Makng large-scale SVM learnng practcal [C] //Proceedngs of Advances n Kernel Methods-Support Vector Machnes. Cambrdge, USA: MIT Press, 1999: 169 184. [32] CHANG C C, LIN C J. LIBSVM: a Lbrary for support vector machnes [J]. ACM Transactons on Intellgent Systems and Technology (TIST), 2011, 2(3): 27. [33] SINDHWANI V, KEERTHI S S. Large scale sem supervsed lnear SVMs [C] //Proceedngs of the 29th annual nternatonal ACM SIGIR conference on Research and development n nformaton retreval. New York: ACM, 2006: 477 484. [34] SONNENBURG S, FRANC V. COFFIN: a computatonal framework for lnear SVMs [C] //Proceedngs of the 27nd Internatonal Conference on Machne Learnng. Hafa, Israel: Omnpress, 2010: 999 1006. [35] LIU J, LI S C, LUO X. Iteratve reweghted nonnteger norm regularzng SVM for gene expresson data classfcaton [J/DB]. Computatonal and Mathematcal Methods n Medcne, 2013. ID 768404. http://dx.do.org/10.1155/2013/768404. [36] CHEN P C, HUANG S Y, CHEN W J, et al. A new regularzed least squares support vector regresson for gene selecton [J]. BMC Bonformatcs, 2009, 10(1): 10 44. [37] BERTSEKAS D P. Nonlnear Programg (Second edton) [M]. Belmont, England: Athena Scentfc, 1999. [38],. [M]. :, 1999. (YUAN Yaxang, SUN Wenyu. Optmzaton Theory and Method [M]. Bejng: Scence Press, 1999. : (1966 ),,,,, E-mal: lujw@ cup.edu.cn; (1988 ),,,, E-mal: lhaen1988@163.com; (1989 ),,,, E-mal: ysht 25@sna.com; (1988 ),,,, E-mal: luylujw@sna.com; (1963 ),,,,, E-mal: luoxl@cup.edu.cn.