: Volterra. Volterra 3. (i.i.d.) Volterra cross-correlation. 2.1 Volterra f[ ] y(t) = f[t, u( )] y(t) = h 0 +

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download ": Volterra. Volterra 3. (i.i.d.) Volterra cross-correlation. 2.1 Volterra f[ ] y(t) = f[t, u( )] y(t) = h 0 +"

Ἰουλία Μήτζου
6 χρόνια πριν
Προβολές:

1 IBM (LAB-S7B) IBM Research Tokyo, LAB-S7B, Shimo-Tsuruma, Yamato, Kanagawa , Japan RCAST, University of Tokyo, Komaba, Meguro-ku, Tokyo , Japan Volterra,,, JL 002/02/ C 2010 SICE 1. (i.i.d.) Bishop 2) 21) 10 kernelize 2000 L 1 1 Volterra : Volterra Volterra cross-correlation 31) 8), 9) 2.1 Volterra f[ ] y(t) = f[t, u( )] y(t) = h 0 + n=1 1 ds 1... ds n n! h n (s 1,..., s n )u(t s 1 )... u(t s n ) (1) u(t) Volterra 1 y(t) = h 0 + ds h 1 (s)u(t s) Volterra m u(t) m u R m 1 29)

2 Volterra y = h 0 + n=1 i 1 =1 h n i 1,...,i n u i1 u in (2) i n =1 Volterra h n i 1,...,i n t Volterra { } D (u (t), y (t) ) u (t) R m, y (t) R 1, t = 1, 2,..., N Volterra h 0, {h 1 i 1 },..., {h n i 1,...,i n }, Volterra Volterra m n φ 0 (u) φ 1 (u) [u 1,..., u m ] R m φ 2 (u) [u 1 u 1, u 1 u 2,..., u m u m ] R m2 (3) u φ n (u) m m n Volterra y = ηn φ n (u) n=0 η 0 = h 0 φ 1 φ 2 η 1 [h 1 1, h 1 2,..., h 1 m] R m (4) η 2 [h 1 1,1, h 1 2,1,..., h 1 m,m] R m2 (5) Volterra {η n } η [ η 0, η 1, η 2,..., η ] (6) φ(u) [ 1, φ 1, φ 2,..., φ ] (7) Volterra y = η φ(u) D η 2.3 Volterra φ n φ n (u) φ n (z) = u i1 u in z i1 z in = i 1 =1 i 1 =1 i n =1 (u i1 z i1 ) (u in z in ) = ( u z ) n i n =1 φ {β 1, β 2,... R} φ(u) [β 0 φ 0 (u), β 1 φ 1 (u),...β φ (u) ] φ k(u, z) φ(u) 2 φ(z) = β ( n u z ) n n=0 {β n 2 } p 2 0 k(u, z) = ( 1 + u z ) p β n 2 = 1 n! (8) (9) k(u, z) = exp ( u z ) (10) u k(u, z) D N N N K (k(u (i), u (j) )) 2.4 D Volterra u y 2 Volterra η Ψ(η λ) 1 N N y (n) η φ(u (n) ) 2 + λ η 2 (11) n=

3 λ 2 L 2 λ = 0 2 η η Φα (12) N α Φ [ ] Φ φ(u (1) ), φ(u (2) ),..., φ(u (N) ) R N Ψ α Ψ(α λ) 1 N y N Kα 2 + λα Kα (13) y N [ y (1), y (2),..., y (N)] kernel ridge regression (13) α α = [K + λni N ] 1 y N I N N (12) u y = φ(u) Φα = k(u) [K + λni N ] 1 y N (14) [ ] k(u) k(u, u (1) ), k(u, u (2) ),..., k(u, u (N) ) R N k N N N Volterra K (12) η N α Representer 33) 3. Volterra N 1 DBN : (a) ( ), (b) (Dynamic Bayesian Networks: DBN) 23) (Bayesian Networks: BN) 90 BN DBN x t+1 = f(x t, u t ) + v t (15) y t = g(x t ) + w t (16) DBN 1 (a) x t, x t v t, w t DBN f g 1 10), 11), 23) DBN 30), 34) DBN ( u t,y t) ( x t) 3 x t u t

4 24) (Switching Linear Dynamical System: SLDS) (SKF) 1 (b) DBN SLDS EM (Expectation-Maximization) E 2 s t ( : ) x t ( ) M E s t x t M E s t x t 25) s t Viterbi 12) SLDS 4), 7) 3.2 (15) (16) f g 1 Ghahramani Roweis 13) Radial Basis Function Networks (RBFN) EM E (Extended Kalman Smoothing) M RBFN 2 Kernel Kalman Filter (KKF) 27) EM 14) Hammerstein (Kernel Canonical Correlation Analysis: KCCA) Kawahara 17) 3 (Gaussian Process) 28) (Gaussian Process Dynamical Models: GPDM) 35) f, g f, g GP-Bayes 18) GP-ADF 5) {x t } 4 (Manifold Learning) ( 32) ) (Laplacian Eigenmap:LE) 1) 22) N N N

5 16) De Moor Nyström 3), 6) Nyström K 37) Nyström 36) 4.2 Pelckmans 26) ARX u M 15) Pelckmans (11) 2 L 1 2 L 1 Lasso (least absolute shrinkage and selection operator) Kukreja 19) Lasso SN Lasso λ η i 0 y Lasso Representer 33) 4.3 (SVM) SVM SVR support vector regression SVM(SVR) Langford 20) SVM ( ) ) L 1 1 M. Belkin and P. Niyogi. Laplacian eigenmaps and spectral techniques for embedding and clustering. In Advances in Neural Information Processing Systems 14, pages MIT Press, C. M. Bishop. Pattern Recognition and Machine Learning. Springer, K. D. Brabanter, P. Dreesen, P. Karsmakers, K. Pelckmans, J. D. Brabanter, J. Suykens, and B. D. Moor. Fixed-size LS-SVM applied to the Wiener-Hammerstein benchmark. In Proceedings of the 15th IFAC Symposium on System Identification, S. Chiappa, J. Kober, and J. Peters. Using Bayesian dynamical systems for motion template libraries. In Advances in Neural Information Processing Systems 21. MIT Press, M. P. Deisenroth, M. F. Huber, and U. D. Hanebeck. Analytic moment-based Gaussian process filtering. In Proceedings of the 26th Annual International Conference on Machine Learning, pages , T. Falck, K. Pelckmans, J. Suykens, and B. D. Moor. Identification of Wiener-Hammerstein systems using LS-SVMs. In Proceedings of the 15th IFAC Symposium on System Identification, pages ,

6 7 E. Fox, E. Sudderth, M. Jordan, and A. Willsky. Nonparametric Bayesian learning of switching linear dynamical systems. In Advances in Neural Information Processing Systems 21. MIT Press, M. O. Franz and B. Schölkopf. Implicit Wiener series for higherorder image analysis. In L. K. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems 17, pages , Cambridge, MA, MIT Press. 9 M. O. Franz and B. Schölkopf. A unifying view of Wiener and Volterra theory and polynomial kernel regression. Neural Computation, 18(12): , N. Friedman, K. Murphy, and S. Russell. Learning the structure of dynamic probabilistic networks. In Proc. Fourteenth Conference on Uncertainty in Artificial Intelligence (UAI-98), pages , Z. Ghahramani. Learning dynamic Bayesian networks. In Adaptive Processing of Sequences and Data Structures (Lecture notes in artificial intelligence), pages Springer-Verlag, Z. Ghahramani and G. E. Hinton. Variational learning for switching state-space models. Neural Computation, 12: , Z. Ghahramani and S. Roweis. Learning nonlinear dynamical systems using an EM algorithm. In Advances in Neural Information Processing Systems 11, pages MIT Press, I. Goethals, K. Pelckmans, J. A. K. Suykens, and B. D. Moor. Subspace identification of Hammerstein systems using least squares support vector machines. IEEE Trans. on Automatic Control, 50: , T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning Data Mining, Inference, and Prediction. Springer, H. Kashima, T. Idé, T. Kato, and M. Sugiyama. Recent advances and trends in large-scale kernel methods. Transactions on Information and Systems, E92-D(7): , Y. Kawahara, T. Yairi, and K. Machida. A kernel subspace method by stochastic realization for learning nonlinear dynamical systems. In B. Schölkopf, J. C. Platt, and T. Hofmann, editors, Advances in Neural Information Processing Systems 19, pages , Cambridge, MA, MIT Press. 18 J. Ko and D. Fox. Gp-bayesfilters: Bayesian filtering using Gaussian process prediction and observation models. Autonomous Robots, 27(1):75 90, S. L. Kukreja, J. Löfberg, and M. J. Brenner. A least absolute shrinkage and selection operator (LASSO) for nonlinear system identification. In Proceedings of the14th IFAC Symposium on System Identification, volume J. Langford, R. Salakhutdinov, and T. Zhang. Learning nonlinear dynamic models. In Proceedings of the 26th Annual International Conference on Machine Learning, pages , L. Ljung. Perspectives on system identification. In the IFAC Congress, Z. Lu, M. Carreira-Perpinan, and C. Sminchisescu. People tracking with the Laplacian eigenmaps latent variable model. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, pages MIT Press, Cambridge, MA, K. P. Murphy. Dynamic Bayesian Networks: Representation, Inference and Learning. PhD thesis, UC Berkeley, R. Murray-Smith and T. A. Johansen. Multiple Model Approaches to Modelling and Control. CRC Press, V. Pavlovic, J. M. Rehg, and J. Maccormick. Learning switching linear models of human motion. In Advances in Neural Information Processing Systems 13, pages The MIT Press, K. Pelckmans, K. Goethals, J. Suykens, and B. D. Moor. On model complexity control in identification of Hammerstein systems. In Proceedings of IEEE Conference on Decision and Control, volume 2, pages , L. Ralaivola and F. d Alche Buc. Time series filtering, smoothing and learning using the kernel Kalman filter. In Proc. of IEEE Int. Joint Conference on Neural Networks, pages , C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, A. G. B. Richard S. Sutton. Reinforcement Learning: An Introduction. The MIT Press, J. W. Robinson and A. J. Hartemink. Non-stationary dynamic Bayesian networks. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21, pages MIT Press, W. J. Rugh. Nonlinear System Theory The Volterra/Wiener Approach. The Johns Hopkins University Press, (Web version prepared in 2002.). 32 L. K. Saul, K. Q. Weinberger, J. H. Ham, F. Sha, and D. Lee. Chapter 16: Spectral methods for dimensionality reduction. In Semisupervised learning, pages MIT Press, B. Schölkopf and A. J. Smola. Learning with Kernels. The MIT Press, L. Song, M. Kolar, and E. Xing. Time-varying dynamic Bayesian networks. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems 22, pages MIT Press, J. Wang, D. Fleet, and A. Hertzmann. Gaussian process dynamical models. In Advances in Neural Information Processing Systems 18, pages MIT Press, C. K. I. Williams, C. E. Rasmussen, A. Schwaighofer, and V. Tresp. Observations on the Nyström method for Gaussian process prediction, pubs.html C. K. I. Williams and M. Seeger. Using the Nyström method to speed up kernel machines. In Advances in Neural Information Processing Systems 13, pages , IBM ACM SIGKDD

Σχετικά έγγραφα

Anomaly Detection with Neighborhood Preservation Principle

27 27 Workshop on Information-Based Induction Sciences (IBIS27) Tokyo, Japan, November 5-7, 27. Anomaly Detection with Neighborhood Preservation Principle Tsuyoshi Idé Abstract: We consider a task of anomaly