2 ICA. (ICA, Independent Component Analysis) (PCA, Principal Compoenent Analysis) x(t) =(x 1 (t),...,x m (t)) T t =0, 1, 2,... PCA 2 ICA.

Transcript

1 s T (ICA, Independent Component Analysis) (PCA, Principal Compoenent Analysis) x =(x,...,x m ) T t =,,,... PCA ICA m m n s x PCA x =As, () ICA ICA ICA Blind Source Separation A m n BSS s A x n n m n m W y =W x, () y WA = I (I n n ) y s ICA y permutation amplitude ICA n>m ICA [] [] x =As+n, (3) ICA ICA ICA Blind Source Separation (BSS) Blind Source Deconvolution BSS Blind Source Deconvolution BSS s =(s,...,s n ) T t =,,,... n () m = n Blind Source Deconvolution x =A s x i = a ik s k, (4) k a ik s k = a ik (τ)s k (t τ), τ=

2 y W x x x y =W x (5) x BSS m n A 3 ICA ICA PCA (Principal Component Analysis, ) ICA (3.) BSS (3.) gradient (3.3) Blind Source Deconvolution (3.4) 3. PCA ICA : Observed signals R R R T R = R V V x x = V x x N x x T = V N V V T = I, (7) t= PCA x, x, x x s s : Source signals s s 3: Applying PCA on observed signals PCA 3 ICA ICA 4 PCA s () x ( ) V = N xx T, (6) N t= y y y 4: Result of solving ICA problem y ICA ICA PCA ICA Whitening Sphering

3 ICA 3. s i y p(y) =p(y,,y n ) W x y y i p(y i ) y i p(y) p(y) = n i= p(y i) p(y) n i= p(y i) W p(y) n i= p(y i) Kullback-Leibler divergence ( {Y i } (i =,...,n) ) W K-L divergence p(y) KL(W) = p(y) log n i= p(y i) dy = H(Y ; W )+ n H(Y i ; W ). (8) i= H(Y ; W ) H(Y i ; W ) p(y)dy = p(x)dx p(y) = p(x)/ W ( W W ) (8) H(Y ; W ) H(Y i ; W ) p(x) W H(Y ; W ) = p(y) log p(y)dy = p(x)(log p(x) log W )dx = H(X) + log W, (9) H(Y i ; W ) = p(y) log p(y i )dy = p(x) log p(y i )dx () KL(W) p(y i ) W KL(W) W W KL(W) W = ( (W T ) E x [ϕ(y)x T ] ) = ( I E x [ϕ(y)y T ] ) (W T ) () ( log p(y ) ϕ(y) =,..., log p(y ) T n) y y n () W () (W T ) W T W [5] W ( I E x [ϕ(y)y T ] ) W (3) [4] η (4) W W t+ = W t + η ( I ϕ(y)y T ) W t. (4) ϕ(y) W [3] Cardoso Souloumiac[9] Jutten Herault[8] Bell Sejnowski [7] sigmoid Common [] Edgeworth Gram-Charlier [5] [6] (sub-gaussian) (super-gaussian) Sigmoid 3.3 { Q = n } log E[yi ] log det E[yy T ], (5) i= 3

4 (E[ ] ) FIR BSS [, 9] K-L divergence Molgedey Schuster [3] (5) [9] [3] ICA [7, 6] (5) x, s, A Fourier ω xx(t + τ) T = A ss(t + τ) T A T ˆx(ω) R s (τ) =Â(ω)ŝ(ω), = A... AT, (6) msec R sn (τ) x R si (τ) s i W y yy(t + τ) T = (WAs) ˆx(ω, t s )=Â(ω)ŝ(ω, t s ), (9) (WAs(t + τ)) T λ R s (τ) =..., (7) λ n R s n (τ),,...,n,,...,n λ i W τ y ((7) ) x τ i W W xx(t + τ i ) T W T = Λ i, i =,...,r, (8) ˆx(ω), ŝ(ω), Â(ω) ˆx(ω, t s ) ŝ(ω, t s ) x, s windowed Fourier BSS BSS perumutation amplitude Blind Source Deconvolution [7] Λ i W 4 [3] τ i 4. [6, 3] [] ICA EEG (Electroencephalogram, ), MEG (Magnetoencephalography, ), MRI (Magnetic Resonance Imaging, ) [7,, 8, 6, 8] optical recording Photo-detector 3.4 Blind Source Deconvolution (Multi-unit Recording) ICA EEG, MEG MRI 3.,3.3 FIR BSS EEG, MEG EEG MEG EEG MEG 4

5 ( ) EEG, MEG BSS ch ch ch 3 ch 4 ch 5 [ft] Time(msec) (Source 4) MRI ICA [] MRI x x t (t =,..., ) x x i (i =,..., ) EEG, MEG ICA MRI ICA 5: 4. ICA Source Source ICA [] MRI Source 3 ICA sparce Source 4 coding[5] ICA [5] Source Time(msec) Cocktail Party 6: [4] 3 Blind Source Deconvolution BSS MEG [6] MEG [4] URL ICA ICA 99 MEG 9channel 5 3 ICA ICA 6 (Source ) (Source,3) shiro/ 3 5

6 5 ICA [9] ICA 99 MEG ICA Blind Source Deconvolution on-line ICA ICA ICA MATLAB 4 [] S. Amari. ICA of temporally correlated signals learning algorithm. In Proceedings of International Workshop on Independent Component Analysis and Blind Signal Separation (ICA 99), pp. 3 8, Janurary 999. [].. Computer Today ( ), [3] S. Amari and J.-F. Cardoso. Blind source separation semiparametric statistical approach. IEEE Trans. Signal Processing, 45():69 7, November 997. [4] S. Amari, T. Chen, and A. Cichocki. Stability analysis of learning algorithms for blind source separation. Neural Networks, (8):345 35, August 997. [5] S. Amari, A. Cichocki, and H. H. Yang. A new learning algorightm for blind signal separation. In D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo eds., Advances in Neural Information Processing Systems, Vol. 8, pp The MIT Press, Cambridge MA, 996. [6] H. Attias. Independent factor analysis. Neural Computation, (4):83 85, May 999. [7] A. J. Bell and T. J. Sejnowski. An information maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6):9 59, 995. [8] J. Cao, N. Murata, S. Amari, A. Cichocki, and T. Takeda. ICA approach with pre & post-processing paris/ techniques. In Proceedings of 998 International Symposium on Nonlinear Theory and its Applications (NOLTA 98), Vol., pp. 87 9, September 998. [9] J.-F. Cardoso and A. Souloumiac. Blind beamforming for non Gaussian signals. IEE-Proceedings-F, 4(6):36 37, December 993. [] A. Cichocki, W. Kasprzak, and S. Amari. Multi-layer neural networks with a local adaptive learning rule for blind separation of source signals. In Proceedings of 995 International Symposium on Nonlinear Theory and its Applications (NOLTA 95), pp. 6 65, 995. [] P. Common and O. Grellier. Non-linear inversion of underdetermined mixtures. In Proceedings of International Workshop on Independent Component Analysis and Blind Signal Separation (ICA 99), pp , January 999. [] P. Comon. Independent component analysis, a new concept? Signal Processing, 36(3):87 34, April 994. [3] S. C. Douglas and A. Cichocki. Neural networks for blind decorrelation of signals. IEEE Trans. Signal Processing, 45():89 84, November 997. [4] F. Ehlers and H. Schuster. Blind separation of convolutive mixtures and an application in automatic speech recognition in noisy environment. IEEE Transactions on Signal Processing, 45():68 69, 997. [5] A. Hyvärinen, P. Hoyer, and E. Oja. Denoising of nongaussian data by independent component analysis and sparce coding. In Proceedings of International Workshop on Independent Component Analysis and Blind Signal Separation (ICA 99), pp , Janurary 999. [6],. Independent component analysis MEG., NC98-8:9 36, June 998. [7] S. Ikeda and N. Murata. An approach to blind source separation of speech signals. In Proceedings of 998 International Conference on Artificial Neural Networks (ICANN 98), pp , September 998. [8] C. Jutten and J. Herault. Separation of sources, part i. Signal Processing, 4():, July 99. [9] M. Kawamoto, K. Matsuoka, and N. Ohnishi. A method of blind separation for convolved nonstationary signals. Neurocomputing, (-3):57 7, 998. [] S. Makeig, T.-P. Jung, and T. J. Sejnowski. Using feedforward neural networks to monitor alertness from changes in EEG correlation and coherence. In D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo eds., Advances in Neural Information Processing Systems, Vol. 8, pp The MIT Press, Cambridge, MA, 996. [] K. Matsuoka, M. Ohya, and M. Kawamoto. A neural net for blind separation of nonstationary signals. Neural Networks, 8(3):4 49,

7 [] M. J. Mckeown, T.-P. Jung, S. Makeig, G. Brown, S. S. Kindermann, T.-W. Lee, and T. J. Sejnowski. Spatially independent activity patterns in functional magnetic resonance imaging data during the Stroop colornaming task. In Proceedings of the National Academy of Sciences, Vol. 95, pp. 83 8, February 998. [3] L. Molgedey and H. G. Schuster. Separation of a mixture of independent signals using time delayed correlations. Phys. Rev. Lett., 7(3): , 994. [4] N. Murata, S. Ikeda, and A. Ziehe. An approach to blind source separation based on temporal structure of speech signals. to appear in Neurocomputing. [5] B. A. Olshausen and D. J. Field. Emergence of simplecell receptive field properties by learning a sparce code for natural images. Nature, 38:67 69, 996. [6] P. Smaragdis. Blind separation of convolved mixtures in the frequency domain. Neurocomputing, (-3): 34, 998. [7] R. N. Vigário. Extraction of ocular artifacts from EEG using independent component analysis. Electroenceph. clin. Neurophysiol., 3:395 44, 997. [8] R. Vigário, V. Jousmäki, M. Hämäläinen, R. Hari, and E. Oja. Independent component analysis for identification of artifacts in Magnetoencephalographic recordings. In M. I. Jordan, M. J. Kearns, and S. A. Solla eds., Advances in Neural Information Processing Systems, Vol., pp The MIT Press, Cambridge MA, 998. [9] L. Zhang, S. Amari, and A. Cichocki. Natural gradient approach to blind separation of over- and undercomplete mixtures. In Proceedings of International Workshop on Independent Component Analysis and Blind Signal Separation (ICA 99), pp , January 999. [3] A. Ziehe and K.-R. Müller. TDSEP an efficient algorithm for blind separation using time structure. In Proceedings of 998 International Conference on Artificial Neu ral Networks (ICANN 98), pp , September 998. ( )