Kernel Methods and their Application for Image Understanding

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "Kernel Methods and their Application for Image Understanding"

Καλλίστρατος Πύθιος Καψής
7 χρόνια πριν
Προβολές:

1 Vol 1 No SIG 12(CVIM 1) Jan 1960 Kernel Methods and their Application for Image Understanding Kenji Nishida and Takio Kurita Support vector machine (SVM) has been extended to build up nonlinear classifier using the kernel trick It is recognized as one of the best models for two class classification among the many methods currently known because it is devised to obtain high performance for unlearned data This paper reviews kernel methods centering on the SVM and introduces some examples of applications for image understanding 1 (Support Vector Machine, SVM) 1) 3) 21 4) 6) 7) 2 8) 10) Neuroscience Research Institute, National Institute of Advanced Industrial Science and Technology 1 9) Vapnik Optimal Separating Hyperplane 1990

obtain high performance for unlearned data This paper reviews kernel methods centering on the SVM and introduces some examples of applications for image understanding 1 (Support Vector

2 2 Jan 1960 shrinkage 2 y = sign(w T x h) (1) 2 (±1) w h sign(u) u > 0 1 u C 1C N x 1,, x N t 1,, t N Lagrange α i ( 0)i = 1,, N t i(w T x i h) 1, i = 1,, N (2) H1: w T x h = 1 H2: w T x h = w w h H2 H1 1 ( 1-1 ) t i(w T x i h) 1, (i = 1,, N) (3) L(w) = 1 2 w 2 (4) L( w, h, ) = 1 2 w 2 α i{t i(w T x i h) 1} (5) w h w = α it ix i (6) 0 = α it i (7) α i t i = 0 (8) 0 α i, i = 1,, N (9)

h = 1 2 2 1 w w h H2 H1 1 ( 1-1 ) t i(w T x i h) 1, (i = 1,, N) (3) L(w) = 1 2 w 2 (4) L( w, h, ) = 1

3 Vol 1 No SIG 12(CVIM 1) 3 L D( ) = α i 1 2 α iα jt it jx T i x j (10) i,j=1 Lagrange α i ( 0)i = 1,, N α i 0 α i > 0 x i w T x h = 1 w T x h = 1 α i 0 x i α i (i 0) w w = αi t i x i (11) S h w T x h = 1 w T x h = 1 1 w 211 x s, s S h = w T x s t s (12) α i (i 0) y = sign(w T x h ) = sign( α i t ix T i x h ) (13) α i = 0 α i > 0 γ H2 H1 ( 1-1 ) H1 H2 ξ i ( 0) ξ i w ξ i w (14) ξ i 0, t i (w T x i h) 1 ξ i, L(w, ) = 1 2 w 2 + γ ( i = 1,, N) (15) ξ i (16) Lagrange

= w T x s t s (12) α i (i 0) y = sign(w T x h ) = sign( α i t ix T i x h ) (13) α i = 0 α i > 0 γ 2 211 2 H2 H1 ( 1-1

4 4 Jan 1960 α i ν i L(w, h,, ) = 1 2 w 2 + γ ξ i α i {t i(w T x i h) (1 ξ i)} ν i ξ i (17) whν i 0 w = α it ix i (18) 0 = α i t i (19) α i = γ ν i (20) α i t i = 0 (21) 0 α i γ, i = 1,, N (22) L D ( ) = α i 1 2 α i α j t i t j x T i x j (23) i,j=1 α i H1 H2 ( ) H1 H2 α i α i = 0 H1 H2 0 < αi < γ H1 H2 α i = γ ξ i 0 H1 H2 22 x φ(x) φ L D φ(x 1) φ(x 2) φ(x 1 ) T φ(x 2 ) = K(x 1, x 2 ) (24) x 1 x 2 φ(x 1 ) φ(x 2 ) K(x 1, x 2 ) K K K(x 1, x 2) = (1 + x T 1 x 2) p (25) Gauss K(x 1, x 2 ) = exp ( x1 x 2 2 2σ 2 ) (26) K(x 1, x 2) = tanh ( ax T 1 x 2 b ) (27)

0 H1 H2 0 < αi < γ H1 H2 α i = γ ξ i 0 H1 H2 22 x φ(x) φ L D φ(x 1) φ(x 2) φ(x 1 ) T φ(x 2 ) = K(x 1, x 2 ) (24) x 1 x 2 φ(x 1 ) φ(x 2 )

5 Vol 1 No SIG 12(CVIM 1) 5 Kernel 2 x w y (10) (23) L D L D ( ) = α i 1 2 = α i 1 2 α i α j t i t j φ(x i ) T φ(x j ) i,j=1 α i α j t i t j K(x i, x j ) (28) i,j=1 (13) y = sign(w T φ(x) h ) = sign( α i t i φ(x i ) T φ(x) h ) = sign( α i t i K(x i, x) h ) (29) y = A T x (30) Gauss Radial Basis Function (RBF) A = [a ij ] (31) A Σ T Σ B ( ) ) 10) x = (x 1,, x M ) T K J = tr(ˆσ 1 T ˆΣ B ) (31) ˆΣ T ˆΣ B y Σ BA = Σ T AΛ (A T Σ T A = I) (32) Λ x

sign( α i t i K(x i, x) h ) (29) y = A T x (30) Gauss Radial Basis Function (RBF) A = [a ij ] (31) A Σ T Σ B 15 4 2 2 15 1

6 6 Jan 1960 Σ T = Σ B = (x i x T )(x i x T ) T K ω k ( x k x T )( x k x T ) T (33) k=1 ω k = N k /N x k x T C k x C k (30) Φ(x) 1 y = a T Φ(x) (34) a a = α iφ(x i) (35) y = α iφ(x i) T Φ(x) = α i K(x i, x) = T k i (x) (36) k i (x) = (K(x 1, x),, K(x N, x)) T T Σ (K) B J = T Σ (K) (37) Σ (K) B = Σ(K) λ (38) Σ (K) B Σ(K) Σ (K) = Σ(K) + αi (39) 0 3 (25)(26) (27) (Empirical Kernel Map) 9),12) (x 1,, x l ) Φ l : x K(, x) (x1,,x l ) = (K(x 1, x),, K(x l, x)) (40) (x 1,, x l ) (40) x (x 1,, x l ) k i (x) l l K K K ij = K(x i, x j ) (41) (13) y = sign( αi t ix T i x h ) = sign( αi t i k T i (x i )k i (x) h ) (42) i s (42) k T i (x i )k i (x) = KK (43) i s i s (42)

Σ(K) + αi (39) 0 3 (25)(26) (27) (Empirical Kernel Map) 9),12) (x 1,, x l ) Φ l : x K(, x) (x1,,x l ) = (K(x 1, x),, K(x l, x)) (40) (x 1,, x l ) (40) x (x 1,, x l )

7 Vol 1 No SIG 12(CVIM 1) 7 y = sign( α i t ikk h ) (44) (29) Φ l : x K 1/2 k i (x) (45) K 1/2 K λ i λ i K (5(B)DT(distance y = sign( α i t i x T i x h ) = sign( α i t ik T i (x i)k 1 2 K 1 2 ki(x) h ) i s = sign( α i t ikk 1 K h ) i s = sign( α i t ik(x i, x) h ) (46) (29) K(x, y) l (x 1,, x l ) l (45) 4 41 Chamfer Distance Gavrila Chamfer Distance 11) Gavrila 924 Chamfer Distance Chamfer Distance Gauss Chamfer distance (empirical kernel map) 9),12) Chamfer distance I, J(5(A)) I e, J e transformation) I d, J d (5(C)) DT I J chamfer distance D chamfer (I, J) = 1 J d (i) (47) I e i I e I e I I e J d (i) I J chamfer distance I e DT J d D chamfer (I, J) D chamfer (J, I) I J [ K = [K ij] = exp( D ] chamfer(i, j) ) (48) 2σ 2 45 k emp (x) = K 1 2 k(x) (49) MIT CBCL Chamfer Distance 905 False Positive 108

Gauss 300 900 Chamfer distance (empirical kernel map) 9),12) Chamfer distance I, J(5(A)) I e, J e transformation) I d, J d (5(C)) DT I J chamfer distance D chamfer (I, J) = 1 J d (i) (47) I e i I e I

8 8 Jan 1960 I e Je (A) (B) (C) D chamfer D chamfer 5 (A) (B) (C)DT (J,I) (i,j) False Positive 13) (False Positive ) ) C = {k(x i) i = 1,, n f } C = {k(x k ) k = 1,, n f }, (50) n f n f k f = 1 n f Σ f = 1 n f n f n f k(x i), (k(x i) k f )(k(x i) k f ) T (51) k T = ω f k + n f N ω f n f k(x k ) (52) k=1 = n f N N = n f + n f Σ (f) Σ (f) B + 1 N = ω f Σ f = ω f ( k f k T )( k f k T ) T n f (k(x k ) k T )(k(x k ) k T ) T (53) k=1 J = tr(ˆσ (f) 1 ˆΣ (f) B ) (54) y = A T k(x) A Σ (f) B A = Σ(f) AΛ (AT Σ (f) A = I) (55) y min(n f, N) = n f 2 1 Σ (f) = Σ(f) + αi (56) MIT CMU eb

f k(x k ) (52) k=1 = n f N N = n f + n f Σ (f) Σ (f) B + 1 N = ω f Σ f = ω f ( k f k T )( k f k T ) T n f (k(x k ) k T )(k(x k ) k T ) T (53) k=1 J =

9 Vol1 NoSIG12(CVIM1) % 9815 % 9950 % SVM 9834 % 9969 % 9790 % MATLAB R GUI SVM light : Cornell University T Joachims C Solaris, indows, Cygwin, Linux ) 1) VNVapnik, Statistical Learning Theory, John iley & Sons (1998) 2),,,, No444, pp52-58 (2000) 3), -,, Vol42, No7, pp (2001) 4) EOsuna, RFreund and FGirosi, Training Support Vector Machines: an Application to Face Detection, Proc of IEEE Computer Society Conf on Computer Vision and Pattern Recognition, pp , ) CPPapageorgiou and MOren and TPoggio, A General Framework for Object Detection, Proc Fifth Int l Conf on Computer Vision, 6) AMohan, CPPapageorgiou and TPoggio, Example-Based Object Detection in Images by Components, IEEE Trans on Pattern Analysis and Machine Intelligence, Vo23, No4, pp , ) THastie, RTibshirani, JFriedman, The Elements of Statistical Learning - Data Mining, Inference, and Prediction, Springer-Verlag, ) KRMuller, SMika, GRatsch, KTsuda, BScholkopf, An introduction to kernel-based learning algorithms, IEEE Trans On Neural Networks, Vol12, No2, pp , ) BScholkpf and AJSmola, Learning with Kernels, The MIT Press, ),, ) DMGavlira, Pedestrian Detection from a Moving Vehicle, Proc of Europian Conference URL on Computer Vision, pp37-49, ) BSchölkopf, SMika, C J C Burges, LIBSVM: National Taiwan University Chih- P Knirsch, KR Müller, GRätsch, and Jen Lin AJSmola, Input Space Versus Feature Space in Kernel-Based Methods, Trans on Neural C

and FGirosi, Training Support Vector Machines: an Application to Face Detection, Proc of IEEE Computer Society Conf on Computer Vision and Pattern Recognition, pp130-136, 1997 5) CPPapageorgiou and

10 10 Jan 1960 Networks, val10, No5, pp , ),,, VolJ84-D-II, No8, pp , ) TKurita, TTaguchi, A modification of kernel-based Fisher discriminant analysis for face detection, Proc of the Fifth Inter Conf on Automatic face and Gesture Recognition, May 2002, ashington, DC, pp , ),, 2005-CVIM-149, pp , 2005 ( ) ( ) ICOT (NRC) IEEE CS

of the Fifth Inter Conf on Automatic face and Gesture Recognition, 20-21 May 2002, ashington, DC,

Σχετικά έγγραφα

Detection and Recognition of Traffic Signal Using Machine Learning

1 1 1 Detection and Recognition of Traffic Signal Using Machine Learning Akihiro Nakano, 1 Hiroshi Koyasu 1 and Hitoshi Maekawa 1 To improve road safety by assisting the driver, traffic signal recognition