Vol. 48 No. SIG 1(CVIM 17) Feb. 2007 15 24 Gait Identification Using a View Transformation Model in the Frequency Domain Yasushi Makihara, Ryusuke Sagawa, Yasuhiro Mukaigawa, Tomio Echigo and Yasushi Yagi Gait identification have recently gained attention as methods of individual identification at a distance from a camera. However, appearance changes due to view direction changes cause difficulties for gait identification. We propose a method of gait identification using frequencydomain features and a view transformation model. We first extract frequency-domain features from a spatio-temporal gait silhouette volume. Next, our view transformation model is obtained with a training set of multiple persons from multiple view directions. In an identification phase, the model transforms gallery features into the same view direction as that of a probe feature. Experiments involving gait identification from 24 view directions demonstrate the effectiveness of our method. 1. 2),7),18),19),21),22) The Institute of Scientific and Industrial Research, Osaka University Department of Engineering Informatics, Osaka Electro- Communication University 3),6),8) 10),12),15),16) Urtasun 19) Spencer 18) Sarkar 16) Sagawa 15) x-y-t volume 1 78
Vol. 48 No. SIG 1(CVIM 17) 79 BenAbdelkader 1) Yu 23) 24) 17) 1 5) 4) Mukaigawa 11) Utsumi 20) View Transformation Model VTM 1 VTM VTM Utsumi 20) Gait Silhouette Volume GSV 2 GSV 3 GSV 4 VTM 5 6 2. GSV 2.1 2 (b) NEC TH1702MX 320 pixel 240 pixel 30 fps 2(c) (a) Input image with color camera (b) Background subtraction for (a) 1 Fig. 1 Problem setting of view transformation. (c) Input image with infrared-ray camera 2 (d) Background subtraction for (c) (c) Fig. 2 Comparison of background subtraction by color camera and that by infrared-ray camera (In (c), brighter value indicates higher temperature).
80 Feb. 2007 (a) time slice images (x-y plane, every 3 frames) (b) horizontal slice image (t-x plane at y = 27) (c) vertical slice image (t-y plane at x =6) 3 GSV Fig. 3 An example of GSV. 2(d) 1 2.2 30 30 pixel 20 pixel 30 pixel GSV GSV x-y t-x t-y 3 3(b) (c) 3. GSV 3.1 1 N gait GSV N gait =arg C(N) = x,y x,y max N [N min,n max ] T (N) n=0 T(N) n=0 g(x,y,n)2 x,y C(N) (1) g(x, y, n)g(x, y, n + N) T(N) n=0 g(x,y,n+n)2 (2) T (N) =N total N 1 (3) C(N) N GSV g(x, y, n) n (x, y) GSV [0,1] N total GSV N min =20 N max =40 3.2 1 S N gait {S i } i =1, 2,...,N sub i S i [in gait, (i +1)N gait 1] DFT N gait G i (x, y, k) = (i+1)n gait 1 n=in gait g(x, y, n)e jω0kn (4) A i (x, y, k) = 1 N gait G i (x, y, k). (5) ω 0 N gait G i (x, y, k) DFT k A i (x, y, k) N gait G i (x, y, k) k>2 DFT DFT A i (x, y, k) k =0, 1, 2 N A 20 30 3 = 1800 4 θ 5 VTM
Vol. 48 No. SIG 1(CVIM 17) 81 4 15 Fig. 4 Gait features for some subjects from each view (every 15 deg). Fig. 5 5 θ Definition of view direction θ at top view. 3.3 2 S i S j S i A i (x, y, k) N A a(s i ) d(s i, S j ) d(s i, S j )= a(s i ) a(s j ). (6) 2 S P S G {S P i } i =1, 2,... {S G j } j =1, 2,... D(S P, S G ) S P i {S G j } j =1, 2,... D(S P, S G )=Median i [min{d(s P i, S G j )}]. (7) j 4. 4.1 Utsumi 20) VTM Utsumi VTM GSV VTM θ K m k N A a m θ k M K SVD a 1 θ 1 a M θ 1 P θ1 ]....... =USV T =. [v 1 v M. a 1 θ K a M θ K P θk (8) U KN A M V M M S M M P θk US N A M v m M v m m P θk v θ k m θ i a m θ i a m θ i = P θi v m. (9) θ ref θ i â m θ i = P θi P + θ ref a m θ ref. (10) P + θ ref P θref 1 4 a m 0 a m 90 θ ref (1),...,θ ref (k) θ i + P θref (1) a m θ â m θ i = P θi ref (1)... (11) P θref (k) a m θ ref (k)
82 Feb. 2007 2.2 y i y j y i y i 4.2 5 θ ref I opp (θ ref ) θ ref I(θ ref ) I opp (θ ref ) I(θ ref +180) I(θ ref +180) I(θ ref ) pers. I(360 θ ref ) I(θ ref ) sym. 2 I(θ ref ) I(180 θ ref ) pers.+sym. θ ref (180 θ ref ) (θ ref + 180) (360 θ ref ) 0 180 θ ref 5. 5.1 20 15 24 736 10 20 VTM 10 24 240 20 k 20k VTM 10 NT PP 5) VTM VTM+ VTM NT PP 5.2 1 0 45 90 2 0 90 VTM+ 6 6(a) a â RMSE 7 VTM VTM (tr.) NT PP 0 180 0 180 0 7(a) θ ref =0 180 7(c) θ ref =90 270
Vol. 48 No. SIG 1(CVIM 17) 83 (a) θ ref =0 (b) θ ref =45 (a): original feature, (b) (e): transformed features from 0, (c) θ ref =90 45, 90, and 0 and 90 deg. reference Fig. 6 6 Transformed features. VTM(tr.) VTM VTM(tr.) 2 NT PP VTM+ NT PP VTM (d) θ ref (1) = 0,θ ref (2) = 90 7 RMSE Fig. 7 Root mean squared errors (RMSE) of transformed features. 0 6(b) 90 270 7(a) 90 6(d) 0 180
84 Feb. 2007 7(c) 45 135 315 6(c) 7(b) 0 90 7(a) (c) 0 90 2 6(e) 7(d) 5.3 20 ROC 14) 10% 0 45 90 1 0 90 2 8 PP VTM 0 8(a) NT PP NT PP PP VTM+ PP 45 8(b) 3 135 225 315 0 90 8(a) (c) 84% 0 90 2 8(d) 80% 94% 8(a) θ ref =0 180 8(c) θ ref =90 270 Fig. 8 (a) θ ref =0 (b) θ ref =45 (c) θ ref =90 (d) θ ref (1) = 0,θ ref (2) = 90 8 10% Verification rate at 10% false alarm. 9 Fig. 9 Averaged verification rate for the number of references. 9 best worst average best
Vol. 48 No. SIG 1(CVIM 17) 85 2 94% worst 3 87% 6. VTM VTM 20 24 739 10% VTM 45 1 84% 0 90 2 94% Sarkar 16) 30 10% 96% FRVT2002 13) 10% 96% 2 94% 1 HumanID Gait Challenge Problem Datasets 16) 1) BenAbdelkader, C.: Gait as a Biometric For Person Identification in Video, Ph.D. Thesis, Maryland Univ. (2002). 2) Cunado, D., Nixon, M. and Carter, J.: Automatic Extraction and Description of Human Gait Models for Recognition Purposes, Computer Vision and Image Understanding, Vol.90, No.1, pp.1 41 (2003). 3) Cuntoor, N., Kale, A. and Chellappa, R.: Combining Multiple Evidences for Gait Recognition, Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, Vol.3, pp.33 36 (2003). 4) Han, J., Bhanu, B. and Roy-Chowdhury, A.: A STUDY ON VIEW-INSENSITIVE GAIT RECOGNITION, Proc. IEEE Int. Conf. on Image Processing, Vol.3, pp.297 300 (2005). 5) Kale, A., Roy-Chowdhury, A. and Chellappa, R.: Towards a View Invariant Gait Recognition Algorithm, Proc. IEEE Conf. on Advanced Video and Signal Based Surveillance, pp.143 150 (2003). 6) Kobayashi, T. and Otsu, N.: Action and Simultaneous Multiple-Person Identification Using Cubic Higher-Order Local Auto-Correlation, Proc. 17th Int. Conf. on Pattern Recognition, Vol.3, pp.741 744 (2004). 7) Lee, L.: Gait Analysis for Classification, Ph.D. Thesis, Massachusetts Institute of Technology (2002). 8) Liu, Y., Collins, R. and Tsin, Y.: Gait sequence analysis using frieze patterns, Proc. 7th European Conf. on Computer Vision, Vol.2, pp.657 671 (2002). 9) Liu, Z. and Sarkar, S.: Simplest Representation Yet for Gait Recognition: Averaged Silhouette, Proc. 17th Int. Conf. on Pattern Recognition, Vol.1, pp.211 214 (2004). 10) Mowbray, S. and Nixon, M.: Automatic Gait Recognition via Fourier Descriptors of Deformable Objects, Proc. IEEE Conf. on Ad-
86 Feb. 2007 vanced Video and Signal Based Surveillance, pp.566 573 (2003). 11) Mukaigawa, Y., Nakamura, Y. and Ohta, Y.: Face Synthesis with Arbitrary Pose and Expression from Several Images An integration of Image-based and Model-based Approach, Proc. 3rd Asian Conf. on Computer Vision, Vol.1, pp.680 687 (1998). 12) Murase, H. and Sakai, R.: Moving Object Recognition in Eigenspace Representation: Gait Analysis and Lip Reading, Pattern Recognition Letters, Vol.17, pp.155 162 (1996). 13) Phillips, P., Blackburn, D., Bone, M., Grother, P., Micheals, R. and Tabassi, E.: Face Recogntion Vendor Test (2002). http://www.frvt.org 14) Phillips, P., Moon, H., Rizvi, S. and Rauss, P.: The FERET Evaluation Methodology for Face-Recognition Algorithms, Trans. of Pattern Analysis and Machine Intelligence, Vol.22, No.10, pp.1090 1104 (2000). 15) Sagawa, R., Makihara, Y., Echigo, T. and Yagi, Y.: Matching Gait Image Sequences in the Frequency Domain for Tracking People at a Distance, Proc. 7th Asian Conference on Computer Vision, Vol.2, pp.141 150 (2006). 16) Sarkar, S., Phillips, J., Liu, Z., Vega, I., Grother, P. and Bowyer, K.: The HumanID Gait Challenge Problem: Data Sets, Performance, and Analysis, Trans. Pattern Analysis and Machine Intelligence, Vol.27, No.2, pp.162 177 (2005). 17) Shakhnarovich, G., Lee, L. and Darrell, T.: Integrated Face and Gait Recognition from Multiple Views, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, Vol.1, pp.439 446 (2001). 18) Spencer, N. and Carter, J.: TOWARDS POSE INVARIANT GAIT RECONSTRUC- TION, Proc. IEEE Int. Conf. on Image Processing, Vol.3, pp.261 264 (2005). 19) Urtasun, R. and Fua, P.: 3D Tracking for Gait Characterization and Recognition, Proc. 6th IEEE Int. Conf. on Automatic Face and Gesture Recognition, pp.17 22 (2004). 20) Utsumi, A. and Tetsutani, N.: Adaptation of appearance model for human tracking using geometrical pixel value distributions, Proc. 6th Asian Conf. on Computer Vision, Vol.2, pp.794 799 (2004). 21) Wagg, D. and Nixon, M.: On Automated Model-Based Extraction and Analysis of Gait, Proc. 6th IEEE Int. Conf. on Automatic Face and Gesture Recognition, pp.11 16 (2004). 22) Yam, C., Nixon, M. and Carter, J.: Automated Person Recognition by Walking and Running via Model-based Approaches, Pattern Recognition, Vol.37, No.5, pp.1057 1072 (2004). 23) Yu, S., Tan, D. and Tan, T.: Modelling the Effect of View Angle Variation on Appearance- Based Gait Recognition, Proc. 7th Asian Conf. on Computer Vision, Vol.1, pp.807 816 (2006). 24) Nayar, S. 2 3 Vol.J77-D-II, No.11, pp.2179 2187 (1994). ( 18 5 5 ) ( 18 11 10 ) 2001 2002 2005 2006 79 18 18 1998 2000 2003 IEEE
Vol. 48 No. SIG 1(CVIM 17) 87 1997 2002 2003 2004 10 2004 11 IEEE 1980 1982 2003 1982 2003 2006 1983 1985 1990 2003 1995 1996 2002 1996 2003 ACM VRST2003 Honorable Mention Award IEEE