[2] REVERB 8 [3], [4] [5] [20] [6], [7], [8], [9], [10] [11] REVERB 8 1 [9] LDA 2 MLLT (SAT) [8] (basis fmllr) [12] (DNN) [10] DNN [11] [13] [14] Ka

HTML
DOWNLOAD

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "[2] REVERB 8 [3], [4] [5] [20] [6], [7], [8], [9], [10] [11] REVERB 8 *1 [9] LDA *2 MLLT (SAT) [8] (basis fmllr) [12] (DNN) [10] DNN [11] [13] [14] Ka"

Πολυδεύκης Σπανού
6 χρόνια πριν
Προβολές:

1 : REVERB 1,a) 1 2 REVERB 8 REVERB REVERB 6.76% 18.60% 68.8% 61.5% REVERB Effectiveness of dereverberation techniques and system combination approach for various reverberant environments: REVERB challenge Tachioka Yuuki 1,a) Narita Tomohiro 1 Watanabe Shinji 2 Abstract: The recently released REVERB challenge includes a reverberant speech recognition task. This paper focuses on state-of-the-art ASR techniques such as discriminative training of acoustic models including Gaussian mixture model, sub-space Gaussian mixture model, and deep neural networks, and various feature transformations after the proposed single channel dereverberation method with reverberation time estimation and multi-channel beamforming that enhances direct sound compared with the reflected sound. In addition, because it is necessary to handle these various environments in the challenge and the best performing system is different from environment to environment, we perform a system combination approach using different feature and different types of systems. Moreover, we use our discriminative training technique for system combination that improves system combination by making systems complementary. Experiments show the effectiveness of these approaches, reaching 6.76% and 18.60% word error rate on the REVERB simulated and real test sets, which are 68.8% and 61.5% relative improvements over the baseline. Keywords: Reverberation, Dereverberation, Discriminative training, Feature transformation, System combination, REVERB challenge 1 Information Technology R&D Center, Mitsubishi Electric Corporation, Kamakura, Kanagawa, , Japan 2 Mitsubishi Electric Research Laboratories a) Tachioka.Yuki@eb.MitsubishiElectric.co.jp 1. REVERB [1] 1

2 [2] REVERB 8 [3], [4] [5] [20] [6], [7], [8], [9], [10] [11] REVERB 8 *1 [9] LDA *2 MLLT (SAT) [8] (basis fmllr) [12] (DNN) [10] DNN [11] [13] [14] Kaldi [15] ) 2) *1 (LDA) [6] (MLLT) [7] *2 9 3) 2 (NLMS) (LDA MLLT fmllr) (MFCC) (PLP) ( (boosted MMI), cf ) 3 (GMM SGMM DNN) 4.4 ROVER CSP [5] ỹ t m (STFT) x t (m) ỹ t = m x t (m) exp( ȷωτ 1,m ) (1) t ω 1 m τ 1,m 2 (CSP) [3] [ τ 1,m = arg max S 1 xt (1) x t (m) ] x t (1) x t (m) (2) S STFT * CSP [16] [4] 3 CSP [17] 3.2 [2] T r x 2 ŷ 2 n 2 x t 2 = t w µ ŷ t µ 2 + n 2 (3) µ=0 2

3 <PLP> <MFCC> GMM (f-bmmi, w/o SAT) τ CSP Speech DS derev. NLMS {1,8} Feature extraction Feature Transform LDA,MLLT basis fmllr f-bmmi GMM (f-bmmi, w/ SAT) SGMM (bmmi, w/ SAT) ROVER Results Speech enhancement part (3.) DNN (bmmi, w/ SAT) for 8ch data ASR part (4.) 1 Schematic diagram of the proposed system. (CSP: cross spectrum phase analysis, DS: delay-and-sum beamformer, derev.: proposed dereverberation method, NLMS: normalized least-mean-squares algorithm, gray blocks are complementary systems for each system type) µ w µ ŷ 2 x 2 ŷ t µ 2 = η(t r ) x t µ 2 n 2 (4) η T r T r w 0 1 Eq. (5) t ŷ t 2 = x t 2 [ w µ η(tr ) x t µ 2 n 2] n 2 (5) µ=1 2 D( ) Polack [18] w µ D < µ µ < D w µ 0 w µ = α s /η(t r )e 2 φµ (6) φ α s η Eq. (5) (SS)[19] SS ŷ 2 β x 2 β x 2 ( ) β 2 T r [2] MMI MMI MMI (bmmi) [20] b( 0) F bmmi (λ) = r p λ (x r H sr ) κ p L (s r ) log s p λ (x r H s ) κ p L (s)e ba(s,sr) (7) x r r λ H sr H s s r s HMM p λ κ p L A(s, s r ) s s r 4.2 MMI (f-bmmi) h t I J M [9] y t = x t + Mh t (8) x t t I y t I h t J( I) M F f-bmmi (M) Eq. (7) x r r y r F f-bmmi (M) = r p λ (y r H sr ) κ p L (s r ) log s p λ (y r H s ) κ p L (s)e ba(s,sr) (9) 3

4 4.3 DNN DNN-HMM (CE) bmmi (7) [21] DNN HMM j p θ p θ (x r j) = p θ (j x r ) p 0 (j) (10) p 0 (j) j F bmmi (θ) Eq. (7) λ θ 4.4 [14] Q F F c (φ) = (1 + α c )F(φ) α c Q Q F(φ) (11) q=1 s r q 1 ( )s q,1 φ ( λ M θ) α c F bmmi f-bmmi α c F Eq. (11) REVERB [1] (5k) (WSJCAM0) SimData (Room 1 3) 0.5 m (near) 2 m (far) 6 SNR 20dB RealData 1 (Room 1) m (tr) 92 7,861 (eva) SimData 28 2,176 RealData (dev) SimData 20 1,484 RealData tr dev (WER) tri-gram 1 * NLMS 0 12 MFCC PLP MFCC 117 LDA 40 LDA HMM MLLT [11] fmllr [12] SAT [8] bmmi f-bmmi 0.1 Eq. (11) α c 0.75 *4 tri-phone (ML) ML DNN Kaldi [15] Povey 2 2M GMM SGMM [22] DNN 3 GMM f-bmmi SGMM DNN[21] bmmi MFCC PLP (dev) WER 4 2 Kaldi baseline WER derev. 8 derev. *3 (D = 9, α = 5, β = 0.05, a = 0.005, b = 0.6) *4 mono-phone ( sil ) 44 triphone 2,500 15,000 4

5 1 WER [%] by room and microphone distance on the REVERB Challenge (dev). SimData RealData Room 1 Room 2 Room 3 Avg Room 1 Avg Feature Type near far near far near far near far 1ch Kaldi baseline MFCC ML derev GMM +LDA+MLLT ML basis fmllr bmmi f-bmmi f-bmmi c SAT ML bmmi f-bmmi f-bmmi c SGMM ML bmmi bmmi c DNN CE bmmi bmmi c ROVER ch CSP+BF+derev. MFCC ML NLMS GMM +LDA+MLLT ML basis fmllr bmmi f-bmmi f-bmmi c SAT ML bmmi f-bmmi f-bmmi c SGMM ML bmmi bmmi c DNN CE bmmi bmmi c ROVER NLMS RealData WER 2.04% SimData 0.63% NLMS LDA MLLT WER 1 f-bmmi bmmi SGMM SimData GMM RealData GMM DNN SimData RealData SAT GMM DNN 2 SimData Real- Data DNN DNN 16 *5 *5 PLP MFCC 5

6 2 WER [%] on the REVERB Challenge (eva). MFCC feature was used for single system and MFCC and PLP features were used for ROVER). SimData RealData Room 1 Room 2 Room 3 Avg Room 1 Avg near far near far near far near far 1ch Kaldi baseline derev f-bmmi SAT+f-bMMI SGMM+bMMI DNN+bMMI ROVER ch CSP+BF+derev NLMS f-bmmi SAT+f-bMMI SGMM+bMMI DNN+bMMI ROVER (eva) DNN DNN (ROVER 5) WER SimData RealData 1ch 1.26% 2.13% 8ch 1.03% 2.22% 7. [1] Kinoshita, K. et al.: The REVERB Challenge: A Common Evaluation Framework for Dereverberation and Recognition of Reverberant Speech, Proc. of WASPAA (2013). [2] Tachioka, Y. et al.: Dereverberation Method with Reverberation Time Estimation Using Floored Ratio of Spectral Subtraction, Acoust. Sci. & Tech., Vol. 34, No. 3, pp (2013). [3] Knapp, C. and Carter, G.: The Generalized Correlation Method for Estimation of Time Delay, IEEE Trans. on Acoustics, Speech, and Signal Processing, Vol. 24, pp (1976). [4] Tachioka, Y. et al.: Direction of Arrival Estimation by Cross-power Spectrum Phase Analysis Using Prior Distributions and Voice Activity Detection Information, Acous. Sci. & Tech., Vol. 33, pp (2012). [5] Johnson, D. and Dudgeon, D.: Array Signal Processing, Prentice-Hall (1993). [6] Haeb-Umbach, R. and Ney, H.: Linear Discriminant Analysis for Improved Large Vocabulary Continuous Speech Recognition, ICASSP, pp (1992). [7] Gopinath, R.: Maximum Likelihood Modeling with Gaussian Distributions for Classification, ICASSP, pp (1998). [8] Anastasakos, T. et al.: A Compact Model for Speakeradaptive Training, ICSLP, pp (1996). [9] Povey, D. et al.: fmpe: Discriminatively Trained Features for Speech Recognition, ICASSP, pp (2005). [10] Hinton, G. et al.: Deep Neural Networks for Acoustic Modeling in Speech Recognition, IEEE Signal Processing Mag., Vol. 28, pp (2012). [11] Tachioka, Y. et al.: Discriminative Methods for Noise Robust Speech Recognition: A CHiME Challenge Benchmark, the 2nd CHiME Workshop on Machine Listening in Multisource Environments, pp (2013). [12] Povey, D. and Yao, K.: A Basis Representation of Constrained MLLR Transforms for Robust Adaptation, Computer Speech and Language, Vol. 26, pp (2012). [13] Fiscus, J.: A Post-processing System to Yield Reduced Error Word Rates: Recognizer Output Voting Error Reduction (ROVER), Proc. of ASRU, pp (1997). [14] Tachioka, Y. et al.: A Generalized Framework of Discriminative Training for System Combination, Proc. of ASRU, pp (2013). [15] Povey, D. et al.: The Kaldi Speech Recognition Toolkit, ASRU, pp. 1 4 (2011). [16] Suzuki, T. and Kaneda, Y.: Sound Source Direction Estimation Based on Subband Peak-hold Processing, The Journal of the Acoust. Soc. of Jpn, Vol. 65, pp (2009). [17] Nishiura, T. et al.: Localization of Multiple Sound Sources Based on a CSP Analysis with a Microphone Array, ICASSP, pp (2000). [18] Habets, E.: Speech Dereverberation Using Statistical Reverberation Models, Speech Dereverberation (Naylor, P. and Gaubitch, N., eds.), Springer (2010). [19] Boll, S.: Suppression of Acoustic Noise in Speech Using Spectral Subtraction, IEEE Trans. on Acoustics, Speech and Signal Processing, Vol. 27, No. 2, pp (1979). [20] Povey, D. et al.: Boosted MMI for Model and Feature-space Discriminative Training, ICASSP, pp (2008). [21] Veselý, K. et al.: Sequence-discriminative Training of Deep Neural Networks, INTERSPEECH (2013). [22] Povey, D. et al.: The Subspace Gaussian Mixture Model A Structured Model for Speech Recognition, Computer Speech and Language, Vol. 25, No. 2, pp (2011). 6

Σχετικά έγγραφα

[5] F 16.1% MFCC NMF D-CASE 17 [5] NMF NMF 3. [5] 1 NMF Deep Neural Network(DNN) FUSION 3.1 NMF NMF [12] S W H 1 Fig. 1 Our aoustic event detect

[5] F 16.1% MFCC NMF D-CASE 17 [5] NMF NMF 3. [5] 1 NMF Deep Neural Network(DNN) FUSION 3.1 NMF NMF [12] S W H 1 Fig. 1 Our aoustic event detect NMF 1 1,a) 1 AED NMF DNN IEEE D-CASE 2012 20% DNN NMF 1. Computational Auditory Scene Analysis: CASA [1] [2] [3] [4] [5] Non-negative Matrxi Factorization (NMF) NMF 2. CASA IEEE 1 Dept. Computer Science