MachineDancing: 1 1 3 MachineDancing GP 1. 33DCG 3D CG MikuMikuDance (MMD) *1 MMD MMD ( ) () 3D 1 National Institute of Advanced Industrial Science and Technology (AIST) *1 http://www.geocities.jp/higuchuu4 1 MachineDancing: Kinect ( ) MachineDancing c 2014 Information Processing Society of Japan 1
[1][13] [5][14] [11] [12] 2 [3] 1 [8] HMM [11] HMM [16] (DBN) [7] (HDP-HMM) [17] Kernel Canonical Correlation Analysis (KCCA) [4] (GP) [9] Topological Gesture Analysis (TGA) [10][15] Gaussian Process Dynamical Models (GPDM) [18] Multi-layer Joint Gait-Pose Manifolds (multi-layer JGPM) [2] GP MachineDancing 1. HMM 2. GP 1 MachineDancing HMM 2. 2 3D koron [19] 2.1 2.1.1 MFCC 16 MFCC MFCC 16 32 2.1.2 3 (x 1,x 2,x 3 ) c 2014 Information Processing Society of Japan 2
2 3 3 3 3 3 3 (θ x,θ y,θ z ) Q (q 1,q 2,q 3 ) q 0 Q = q 0 + q 1 i + q 2 j + q 3 k i 2 = j 2 = k 2 = ijk = 1 ij = ji = k jk = kj = i ki = ik = j 4 20 2 20 3 3 19 4 1 79 2.2 2.2.1 A 2 A 3 : 0.5 1 1 1.0 1 4 4 1 4 1 3 4 1.5 3 0.5 1 1.0 2.2.2 n D n v n D n d [d 1,,d s,,d S ] T d s 79 S c 2014 Information Processing Society of Japan 3
S D n v n P (D n v n ) P (v n v n 1 ) N{D n } N {v n } N P ( {D n } N,{v n } N ) N = P (D n v n )P (v n v n 1 )(1) P (v 1 v 0 ) = P (v 1 ) 0 HMM ( ) P (D n v n )P (v n v n 1 ) P (v 1 )P (D n v n ) 2.2.3 2 D 1 D 2 2 D [d 1,,d S ] 11.0 d s = f (t s ) f t s d s ( 0.5 < t s < 1.0) f 2 d s f (t s ) d s f (t s ) 2 N ( 0,σ 2 I ) (2) N ( 0,σ 2 I ) σ 2 I f (t) φ j (t) f (t s ) = J a j φ j (t s ) (3) j=1 {a j } J j=1 0 Σ p a = (a 1,,a J ) N (0,Σ p ) (4) Representer φ j f3 J D RBF φ tj (t) = k(t,t j ) = exp ( λ2 ) t t j 2 (5) 2 P (D,a) = P (D a)p (a) a = P (D t;σ,λ) ( 1 exp (2π) 1 t α K α 2 tr( D T K 1 D )) (6) t t = (t 1,,t s,,t S ) T α d K RBF (K) ij = k(t i,t j )+σ 2 δ tit j δ 2.3 2.3.1 {v k } K k=1 L = N logp (D n t n,v ;σ vn,λ vn ) EM E-step: v σ v λ v M-step: 2 E-step M-step 2.3.1.1 E-step M {D 1,,D M } c 2014 Information Processing Society of Japan 4
v v 6 D D = [ D 1,,D M] T t t = [ t 1,,t M] T σ λ σ λ LL(σ,λ,v ) = logp (D t,v ;σ,λ) σ v λ v Scaled Conjugate Gradient [7] v v 2.3.1.2 M-step E-step {v k } K k=1 ˆD ) ˆD ˆv ˆv = argmax v logp (ˆD ˆt,v;σ v,λ v 2.3.2 D v σ v λ v = P (D v,d,t,t) 1 exp ( (2π) 12 ( tr Z Z) ) T ˆK 1 (7) t α K α Z = D AK 1 D, ˆK = B A T K 1 A (8) (A) ij = exp ( λ v 2 ) 2 t i t j 2 +σ 2 v δ t i t j (9) (B) ij = exp ( λ v 2 ) 2 t i t j 2 +σ 2 v δ tit j (10) 7 v v 2.4 2.4.1 v MFCC+ MFCC 3 4 {M n } N P (M v) {M n } N K{z k } K k=1 P (M n v) z P (M n v n ) = z = z P (M n,z v n ) P (M n z)p (z v n ) (11) P (M n z) z P (z v n ) 11P (M n z) P (z v n ) z P (z v n) = 1 2.4.2 v MFCC+ MFCC HMM (1) {v n } N P ( {M n } N,{D n } N ) = P ( {M n } N,{D n } N,{v n } N ) {v n} N = {v n} N N P (M n v n )P (D n v n )P (v n v n 1 ) (12) 2.5 P ( {M n } N,{D n } N ) c 2014 Information Processing Society of Japan 5
3. https://staff.aist.go.jp/s.fukayama/ MachineDancing/index-j.html 60 MMD vmd [19] 0.5 1.5 3D CG 4 : 1.53 4 {v n } N 2 {M n } N {v n} N {v n} N = argmax {v n} N N P (M n v n )P (v n v n 1 ) (13) HMM Viterbi {v n} N D new n v n D = [ D 1,,D M] T 3 4 4 2 koron3d 4. MachineDancing MachineDancing 3D 3D koron JST CREST OngaCREST [1] K. M. Chen, S. T. Shen and S. D. Prior: Using music and motion analysis to construct 3D animations and visualisations, Digital Creativity Vol. 19, No. 2 (2008). [2] M. Ding and G. Fan: Multi-layer joint gait-pose manc 2014 Information Processing Society of Japan 6
ifold for human motion modeling, In Proc. FG 2013 pp. 1 8, (2013). [3] R. Fan, S. Xu and W. Geng: Example-based automatic music-driven conventional dance motion synthesis, IEEE Transactions on Visualization and Computer Graphics Vol. 18, No. 3 (2012). [4] T. Hirose and T. Taniguchi: Abstraction multimodal low-dimensional representation from high-dimensional posture information and visual images, Journal of Robotics and Mechatronics Vol. 25, No. 1 (2013). [5] J. W. Kim, H. Fouad, J. L. Sibert and J. K. Hahn: Perceptually motivated automatic dance motion generation for music, Computer Animation and Virtual Worlds 2009 Vol. 20 (2009), pp. 375-384. [6] M. Lee, L. Lee and J. Park: Music similarity-based approach to generating dance motion sequence, Multimedia Tools and Applications Vol. 62, No. 3 (2013), pp. 895-912. [7] M. F. Møller: A scaled conjugate gradient algorithm for fast supervised learning, Neural Networks 6, 4 (1993), pp. 525-533. [8] K. Moon and V. Pavlović: Graphical models for human motion modelling. Human Motion, Computational Imaging and Vision Vol. 36 (2008), pp. 159-183. [9] T. Mukai and S. Kuriyama: Geostatistical Motion Interpolation, In Proc. ACM SIGGRAPH, Vol. 24, No. 3 (2005), pp. 1062-1070. [10] L. Naveda and M. Leman: The spatiotemporal representation of dance and music gestures using topological gesture analysis (TGA), Music Perception Vol. 28, No. 1 (2010), pp. 93-111. [11] F. Ofli, E. Erzin, Y. Yemez and A. M. Tekalp: Learn2dance: Learning statistical music-to-dance mappings for choreography synthesis, IEEE Transactions on Multimedia Vol. 14, No. 3 (2012). [12] S. Oore and Y. Akiyama: Learning to synthesize arm motion to music by example, In Proc. WSCG 2006 (2006), pp. 201-208. [13] C. Panagiotakis, A. Holzapfel, D. Michel and A. Argyros: A. Beat synchronous dance animation based on visual analysis of human motion and audio analysis of music tempo, In Proc. ISVC 2013 (2013), pp. 118-127. [14] T. Shiratori, A. Nakazawa and K. Ikeuchi: Synthesizing dance performance using musical and motion features, In Proc. ICRA 2006 (2006), pp. 3654-3659. [15] P. Sousa, J. L. Oliveira, L. P. Reis and F. Gouyon: Humanized robot dancing: Humanoid motion retargeting based in a metrical representation of human dance styles, In Proc. EPIA 2011 (2011), pp. 392-406. [16] T. Takeda, Y. Hirata and K. Kosuge: Dance step estimation method based on HMM for dance partner robot, IEEE Transactions on Industrial Electronics Vol. 54, No. 2 (2007). [17] T. Taniguchi, K. Hamahata and N. Iwahashi: Unsupervised segmentation of human motion data using sticky HDP-HMM and MDL-based chunking method for imitation learning, Advanced Robotics Vol. 25, No. 17(2011). [18] J. M. Wang, D. J. Fleet and A. Hertzmann: Gaussian process dynamical models for human motion, IEEE Transactions on Pattern Recognition and Machine Intelligence Vol. 30, No. 2 (2008). [19],,, M. Mauch, : Songle,, Vol. 54, No. 4 (2013), pp. 1363 1372. c 2014 Information Processing Society of Japan 7