37 5 Vol. 37, No. 5 2011 5 ACTA AUTOMATICA SINICA May, 2011 1, 2 1 2 2 1 2.,,,.,,,,,. :. DOI,,, 10.3724/SP.J.1004.2011.00550 Fusing Multiple Features for Object Tracking Based on Uncertainty Measurement GU Xin 1, 2 WANG Hai-Tao 1 WANG Ling-Feng 2 WANG Ying 2 CHEN Ru-Bing 1 PAN Chun-Hong 2 Abstract This paper presents a novel tracking algorithm that fuses multiple features based on feature uncertainty measurement. It is based on the fact that tracking failure of particle filter often happens in the cases of low discriminative abilities of the observed features and disperse distributions of the sampled particles. To handle this failure, we first define a new feature uncertainty measurement method to adaptively adjust the relative contributions of different features. Then we introduce a self-adaptive feature fusion strategy to overcome the shortcomings of product and sum fusion ones. This strategy effectively sharpens the distribution of the fused posterior, and makes the tracking less sensitive to noises. Thereby, the tracking robustness is improved. An extensive number of comparative experiments show that the proposed algorithm is more stable and robust than the single feature, multiplicative fusion, and additive fusion tracking algorithms. Key words Object tracking, uncertainty measurement, particle filter, multiple features fusion [1].,, : [2] [3] [4] [5] [6] SIFT [7].,,, [8].,.,,.,,. [9] 2010-09-09 2010-12-27 Manuscript received September 9, 2010; accepted December 27, 2010 (60873161, 61005036, 61005013), (BK2010503), (SYG201024) Supported by National Natural Science Foundation of China (60873161, 61005036, 61005013), Natural Science Foundation of Jiangsu Province of China (BK2010503), and Project of Suzhou Science Technology Bureau (SYG201024) 1. 210016 2. 100190 1. College of Automation, Nanjing University of Aeronautics and Astronautics, Nanjing 210016 2. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190,. Birchfield [10],., Li [11],,,,.,,. Wang [12],,.,. [13],, Frobenius,.,.,, ;,,,
5 : 551 ;,,.,.,.,,,. 1, : x t = f t (x t 1, ξ t ) (1), x t t, ξ t., : z t = h t (x t, e t ) (2), z t x t, e t. {ξ 1:t }, {e 1:t }. {ξ 1:t }, {e 1:t },., {z 1:t } {x t }, p(x t z 1:t ).,. : 1) p(x t z 1:t 1 ) = p(x t x t 1 )p(x t 1 z 1:t 1 )dx t 1 (3) 2) p(x t z 1:t ) = p(z t x t )p(x t z 1:t 1 ) p(zt x t )p(x t z 1:t 1 )dx t (4), (3) (4) [14]. [15]., p(x t z 1:t ) p(x t z 1:t ) = N w t,j δ(x t x t,j ) (5) j=1 N ; w t,j t j,, N j=1 w j=1; δ( ). q(x t,j x t 1,j, z 1:t ), : w t,j w t 1,j p(z t x t,j )p(x t,j x t 1,j ) q(x t,j x t 1,j, z 1:t ) (6) p(x t x t 1 ), (6) w t,j w t 1,j p(z t x t,j ) (7), x, x = {c x, c y, l x, l y, θ}., c x, c y, l x, l y θ., (1)., : x t = Ax t 1 + W (8), A,. W 0. 2,.,,. 2.1,,. ( ).,.,., ( ),,.,,,..,,,.,.,,,.,.
552 37,.,,,., 1 : 1),, ;, ;,,. 2), ;, ; 1). 3), ;, ; 1). 4),, ;,, ;., : β i t+1 = σ t H(p i t) (9), βt+1 i t + 1 i. σ t t,. σ t = tr(σ), Σ, σ. H(p i t) t i,., : H(p i ) = N (p(z i x j ) log 2 p(z i x j )) (10) j=1, p(z i x j ) i j. H(p i ),. 2.2,. [11].,., n : p(z 1 z n x) = n p(z i x) (11) i=1 [12],., n : p(z 1 z n x) = n φ i p(z i x) (12) i=1, φ i i, : n i=1 φi = 1. Fig. 1 1 ( : ; : ) The diagrammatic sketch of four cases in process of particle filter: the spatial distributions of particles (top row), and the distributions of probability values in particles (bottom row)
5 : 553. 2 : (Feature 1, Feature 2), (Product rule).,.,.,,,,. (Sum rule),,,.,.,,,. ( p(z 1 x)p(z 2 x) + β 1 U(x)p(z 2 x) + β 2 U(x)p(z 1 x) + β 1 β 2 (U(x)) 2 ) (14) (14),, :,,.,. β 1 0, 1, (13) 1 p(z 1 x), 1. β 1, 1, (13) 1 1, 2. 2 :, (Our rule),,.,,,,. 3 2 ( 1; 2; ; ; ) Fig. 2 The comparison of multiple feature fusion ( Feature 1; Feature 2; Sum fusion result; Product fusion result; Our result.),. n,,,. : p(z 1 z n x) = n ( ) p(z i x) + β i U(x) i=1 1 + β i (13), β i i, U(x). n = 2, (13) : p(z 1, z 2 x) = 1 (1 + β 1 )(1 + β 2 ),.,,,. 3.1,. [1],,. B c h c (u) = δ(i(x, y) u) (15) u=1, I(x, y), B c.. [16].,, I(x, y), G α:
554 37 G(x, y) = I 2 x + I 2 y, ( ) α(x, y) = tan 1 Iy (16) 4,, B e,.,,.,., Bhattacharyya [17]. h tar h mod. B ρ(h tar, h mod ) = 1 htar (u)h mod (u) (17) u=1, [16] p(z i x) exp( λ i ρ 2 i (h mod, h tar )) (18) I x 4,,.,,,., ( 1 2), ( 3 4).,,. : ; N 100; U(x) = 1/N; B e = 18; B c = 216; B c = 64; λ 1. Table 1 1 The coefficients of two features, i {1, 2}, 1, 2. 3.2 : 1. x 0, h 1 0, h 2 0, {w 0,j = 1/N} N j=1, β0 1 = β0 2 = 0.5; 2. : x t = Ax t 1 + W, x t 1 x t ; 3. p(z 1 x t,j ), p(z 2 x t,j ),, (10) H(p 1 t ), H(p 2 t ); 4., N,, ; 5. (13) p(z 1, z 2 x), (9) β 1, β 2, ; 6. : w t,j = w t 1,j p(z t x t,j ), : x t = N w j=1 t,jx t,j ; 7., 1/( N t=1 w2 t,j) < N/2, {x t,j } N j=1 N {x t,j l} N l=1, w t,l = 1/N, ; 8. 2. (λ c) (λ e) 1 90 40 2 90 30 3 90 40 4 80 40 1 1.,,., :. 3, (37 ),,.,.,,.,,,,. 2..,,.,. 1 Otcbvs Benchmark Dataset Collection [Online], available: http://www.cse.ohio-state.edu/otcbvs-bench/, December 28, 2010
5 : 555. 4,,, 121,. (346 ),.,,,, 246,.,,.,,.,,.. 2 ( 5). 1, 2.,, :,,,, ;,, ;,, ;,, ;.,,. 3 ( Fig. 3 3 1 ( 1 : ; 2 : ; 3 : ) ( : 1, 28, 37, 43, 61) Some results on experiment 1 by using single feature: color (the first row), edge (the second row), and proposed multi-feature fusion method (the third row) (Frames: 1, 28, 37, 43, 61) 4 2 ( 1 : ; 2 : ; 3 : ) ( : 6, 206, 246, 346, 376) Fig. 4 Some results on experiment 2 by using single feature: color (the first row), edge (the second row), and proposed multi-feature fusion method (the third row) (Frames: 6, 206, 246, 346, 376)
556 37 Fig. 5 5 ( ), ( ) The representative video frames in the five stages (top row), and the uncertainty of two features (bottom row) 6 3 ( 1 : ; 2 : ; 3 : ) ( : 251, 456, 486, 546, 386) Fig. 6 Some results on experiment 3 by using product rule (the first row), sum rule (the second row), and proposed multi-feature fusion method (the third row) (Frames: 251, 456, 486, 546, 586) 1).. 6, (486 ),., ( ),,, ;.,.,,,,.,,. 4.. 7, (116 ),.,,,,,. (291 ),,.,.,. :
5 : 557 7 4 ( 1 : ; 2 : ; 3 : ) ( : 61, 116, 271, 291, 396) Fig. 7 Some results on experiment 4 by using: product rule (the first row), sum rule (the second row), and proposed multi-feature fusion method (the third row) (Frames: 61, 116, 271, 291, 396) Table 2 2 ( ) Comparison results ( interprets that we did not perform in this sequence) 1 150 31 ( ) 2 378 346 ( ) 246 ( ) 3 647 456 ( ) 486 ( ) 4 393 291 ( ) 116 ( ),,,. 2 (, ). :,, ( ),,,,.,,,,. P4 3.4 GHz 1024 M Matlab 7.6.,,,,. 3 fps ( / ).. Table 3 3 5 The comparison of computation costs of the five methods (fps) 1 25.8 24.7 11.5 11.4 10.9 2 21.3 21.5 10.1 10.0 9.7 3 24.9 24.1 10.9 11.0 10.3 4 25.6 24.5 12.3 12.1 11.8
558 37 5,,.,,,. ( ),.,, ;,,. References 1 Hou Zhi-Qiang, Han Chong-Zhao. A survey of visual tracking. Acta Automatica Sinica, 2006, 32(4): 603 617 (,.., 2006, 32(4): 603 617) (,,. SIFT., 2010, 36(8): 1204 1208) 8 Yilmaz A, Javed O, Shah M. Object tracking: a survey. ACM Computing Surveys, 2006, 38(4): 13 58 9 Wang Yong-Zhong, Liang Yan, Zhao Chun-Hui, Pan Quan. Kernel-based tracking based on adaptive fusion of multiple cues. Acta Automatica Sinica, 2008, 34(4): 393 399 (,,,.., 2008, 34(4): 393 399) 10 Birchfield S. Elliptical head tracking using intensity gradients and color histograms. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Santa Barbara, USA: IEEE, 1998. 232 237 11 Li P H, Chaumette F. Image cues fusion for object tracking based on particle filter. In: Proceedings of the 3rd International Workshop on Articulated Motion and Deformable Objects. Palma de Mallorca, Spain: Springer, 2004. 99 107 2 Perez P, Hue C, Vermaak J, Gangnet M. Color-based probabilistic tracking. In: Proceedings of the 7th European Conference on Computer Vision. London, UK: Springer, 2002. 661 675 3 Kim B G, Park D J. Unsupervised video object segmentation and tracking based on new edge features. Pattern Recognition Letters, 2004, 25(15): 1731 1742 4 Baker S, Matthews I. Lucas-Kanade 20 years on: a unifying framework. International Journal of Computer Vision, 2004, 56(3): 221 255 5 Bastos R, Dias J M S. Fully automated texture tracking based on natural features extraction and template matching. In: Proceedings of the ACM SIGCHI International Conference on Advances in Computer Entertainment Technology. New York, USA: ACM, 2005. 180 183 12 Wang X, Tang Z M. Modified particle filter-based infrared pedestrian tracking. Infrared Physics and Technology, 2010, 53(4): 280 287 13 Zhong Xiao-Pin, Xue Jian-Ru, Zheng Nan-Ning, Ping Lin- Jiang. An adaptive fusion strategy based multiple-cue tracking. Journal of Electronics and Information Technology, 2007, 29(5): 1017 1021 (,,,.., 2007, 29(5): 1017 1021) 14 Arulampalam M S, Maskell S, Gordon N, Clapp T. A tutorial on particle filters for online nonlinear/non-gaussian Bayesian tracking. IEEE Transactions on Signal Processing, 2002, 50(2): 174 188 15 Isard M, Blake A. Condensation conditional density propagation for visual tracking. International Journal of Computer Vision, 1998, 29(1): 5 28 6 Du W, Piater J. A probabilistic approach to integrating multiple cues in visual tracking. In: Proceedings of the 10th Europe on Conference on Computer Vision. Berlin, Germany: Springer, 2008. 225 238 16 Dalal N, Triggs B. Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA: IEEE, 2005. 886 893 7 Lin Hai-Feng, Ma Yu-Feng, Song Tao. Research on object tracking algorithm based on SIFT. Acta Automatica Sinica, 2010, 36(8): 1204 1208 17 Perez P, Vermaak J, Blake A. Data fusion for visual tracking with particles. Proceedings of the IEEE, 2004, 92(3): 495 513
5 : 559... E-mail: xingu396@gmail.com (GU Xin Master student at the College of Automation, Nanjing University of Aeronautics and Astronautics. His research interest covers visual object tracking and intelligent video surveillance. Corresponding author of this paper.).. E-mail: htwang2002@126.com (WANG Hai-Tao Associate professor at the College of Automation, Nanjing University of Aeronautics and Astronautics. His research interest covers photoelectric detecting technology and computer control technique.) ers video surveillance and medical image processing.).,. E-mail: ywang@nlpr.ia.ac.cn (WANG Ying Ph. D. candidate at the Institute of Automation, Chinese Academy of Sciences. His research interest covers image processing, probabilistic graphical models, and object tracking.).. E-mail: chenrb@szjl.com.cn (CHEN Ru-Bing Engineer at Suzhou Institute of Measurement and Testing Technology. His research interest covers measuring and testing technologies.).. E-mail: lfwang@nlpr.ia.ac.cn (WANG Ling-Feng Ph. D. candidate at the Institute of Automation, Chinese Academy of Sciences. His research interest cov-.. E-mail: chpan@nlpr.ia.ac.cn (PAN Chun-Hong Professor at the Institute of Automation, Chinese Academy of Sciences. His research interest covers image processing and computer vision.)