Ψηφιακή Επεξεργασία Φωνής

Transcript

1 ΕΛΛΗΝΙΚΗ ΔΗΜΟΚΡΑΤΙΑ ΠΑΝΕΠΙΣΤΗΜΙΟ ΚΡΗΤΗΣ Ψηφιακή Επεξεργασία Φωνής Ενότητα 8η: Αναγνώριση Ομιλητή Στυλιανού Ιωάννης Τμήμα Επιστήμης Υπολογιστών

2 CS578- Speech Signal Processing Lecture 9: Speaker Recognition Yannis Stylianou University of Crete, Computer Science Dept., Multimedia Informatics Lab Univ. of Crete

3 Outline 1 Introduction 2 Spectral Features for Speaker Recognition Mel-Cepstrum Sub-Cepstrum 3 Speaker Recognition Algorithms Minimum-Distance Classifier Vector Quantization Gaussian Mixture Model - GMM 4 Non-Spectral Features in Speaker Recognition Glottal Flow Derivative, GFD Prosodic and other features 5 Acknowledgments 6 References

4 Terms Speaker identification Speaker verification Claimant (Target speaker) Imposter (Background speaker) False acceptances/false rejections Features Training stage/testing stage Mismatch conditions

5 Terms Speaker identification Speaker verification Claimant (Target speaker) Imposter (Background speaker) False acceptances/false rejections Features Training stage/testing stage Mismatch conditions

6 Terms Speaker identification Speaker verification Claimant (Target speaker) Imposter (Background speaker) False acceptances/false rejections Features Training stage/testing stage Mismatch conditions

7 Terms Speaker identification Speaker verification Claimant (Target speaker) Imposter (Background speaker) False acceptances/false rejections Features Training stage/testing stage Mismatch conditions

8 Terms Speaker identification Speaker verification Claimant (Target speaker) Imposter (Background speaker) False acceptances/false rejections Features Training stage/testing stage Mismatch conditions

9 Terms Speaker identification Speaker verification Claimant (Target speaker) Imposter (Background speaker) False acceptances/false rejections Features Training stage/testing stage Mismatch conditions

10 Terms Speaker identification Speaker verification Claimant (Target speaker) Imposter (Background speaker) False acceptances/false rejections Features Training stage/testing stage Mismatch conditions

11 Terms Speaker identification Speaker verification Claimant (Target speaker) Imposter (Background speaker) False acceptances/false rejections Features Training stage/testing stage Mismatch conditions

12 Overview Of A Speaker Verification System

13 Outline 1 Introduction 2 Spectral Features for Speaker Recognition Mel-Cepstrum Sub-Cepstrum 3 Speaker Recognition Algorithms Minimum-Distance Classifier Vector Quantization Gaussian Mixture Model - GMM 4 Non-Spectral Features in Speaker Recognition Glottal Flow Derivative, GFD Prosodic and other features 5 Acknowledgments 6 References

14 Cues for recognition: High Level Clarity Roughness Animation Magnitude Pitch intonation Articulation rate Dialect

15 Cues for recognition: High Level Vocal tract spectrum Instantaneous pitch Glottal flow excitation Modulations in formant trajectories

16 Mel-Cepstrum Compute STFT: X (n, ω k ) = x[m]w[n m]e jω k m m= where ω k = 2π k with N the DFT length N Apply mel-scale filters V l (ω k ) on X (n, ω k ) : V l (ω k ) X (n, ω k ) Compute the energy in each mel-frequency band: E mel (n, l) = 1 U l V l (ω k ) X (n, ω k ) 2 A k k=l l where L l and U l denote the lower and upper limit of the lth filter and U l A l = V l (ω k ) 2 k=l l Compute mel-cepstrum: C mel [n, m] = 1 R 1 log (E mel (n, l)) cos ( 2π R R lm) l=0 where R is the number of filters.

17 Mel-Cepstrum Compute STFT: X (n, ω k ) = x[m]w[n m]e jω k m m= where ω k = 2π k with N the DFT length N Apply mel-scale filters V l (ω k ) on X (n, ω k ) : V l (ω k ) X (n, ω k ) Compute the energy in each mel-frequency band: E mel (n, l) = 1 U l V l (ω k ) X (n, ω k ) 2 A k k=l l where L l and U l denote the lower and upper limit of the lth filter and U l A l = V l (ω k ) 2 k=l l Compute mel-cepstrum: C mel [n, m] = 1 R 1 log (E mel (n, l)) cos ( 2π R R lm) l=0 where R is the number of filters.

18 Mel-Cepstrum Compute STFT: X (n, ω k ) = x[m]w[n m]e jω k m m= where ω k = 2π k with N the DFT length N Apply mel-scale filters V l (ω k ) on X (n, ω k ) : V l (ω k ) X (n, ω k ) Compute the energy in each mel-frequency band: E mel (n, l) = 1 U l V l (ω k ) X (n, ω k ) 2 A k k=l l where L l and U l denote the lower and upper limit of the lth filter and U l A l = V l (ω k ) 2 k=l l Compute mel-cepstrum: C mel [n, m] = 1 R 1 log (E mel (n, l)) cos ( 2π R R lm) l=0 where R is the number of filters.

19 Mel-Cepstrum Compute STFT: X (n, ω k ) = x[m]w[n m]e jω k m m= where ω k = 2π k with N the DFT length N Apply mel-scale filters V l (ω k ) on X (n, ω k ) : V l (ω k ) X (n, ω k ) Compute the energy in each mel-frequency band: E mel (n, l) = 1 U l V l (ω k ) X (n, ω k ) 2 A k k=l l where L l and U l denote the lower and upper limit of the lth filter and U l A l = V l (ω k ) 2 k=l l Compute mel-cepstrum: C mel [n, m] = 1 R 1 log (E mel (n, l)) cos ( 2π R R lm) l=0 where R is the number of filters.

20 Triangular mel-scale filter bank

21 Sub-Cepstrum Convolve mel-scale filter impulse response u l [n] (subband filter) with x[n]: Compute energy: E sub (n, l) = X (n, ω l ) = x[n] u l [n] N/2 m= N/2 where p[n] is a smoothing filter. Compute subband cepstrum: p[n m] X (n, ω l ) 2 C sub [n, m] = 1 R 1 log (E sub (n, l)) cos ( 2π R R lm) l=0

22 Sub-Cepstrum Convolve mel-scale filter impulse response u l [n] (subband filter) with x[n]: Compute energy: E sub (n, l) = X (n, ω l ) = x[n] u l [n] N/2 m= N/2 where p[n] is a smoothing filter. Compute subband cepstrum: p[n m] X (n, ω l ) 2 C sub [n, m] = 1 R 1 log (E sub (n, l)) cos ( 2π R R lm) l=0

23 Sub-Cepstrum Convolve mel-scale filter impulse response u l [n] (subband filter) with x[n]: Compute energy: E sub (n, l) = X (n, ω l ) = x[n] u l [n] N/2 m= N/2 where p[n] is a smoothing filter. Compute subband cepstrum: p[n m] X (n, ω l ) 2 C sub [n, m] = 1 R 1 log (E sub (n, l)) cos ( 2π R R lm) l=0

24 Comparing Mel-Cepstrum and Sub-Cepstrum

25 Energies from mel-scale and subband filter banks

26 Outline 1 Introduction 2 Spectral Features for Speaker Recognition Mel-Cepstrum Sub-Cepstrum 3 Speaker Recognition Algorithms Minimum-Distance Classifier Vector Quantization Gaussian Mixture Model - GMM 4 Non-Spectral Features in Speaker Recognition Glottal Flow Derivative, GFD Prosodic and other features 5 Acknowledgments 6 References

27 Minimum-Distance Classifier Compute the average (mel or subband) cepstral features for the training data: C tr [n] = 1 M C tr [ml, n] M m=1 where L denotes the frame length. Compute the average cepstral features for the testing data: Compute a distance: For speaker verification: C ts [n] = 1 M M m=1 C ts [ml, n] D = 1 R 1 ( R 1 C tr [n] C ts [n]) 2 n=1 if D > Threshold, then speaker is verified

28 Minimum-Distance Classifier Compute the average (mel or subband) cepstral features for the training data: C tr [n] = 1 M C tr [ml, n] M m=1 where L denotes the frame length. Compute the average cepstral features for the testing data: Compute a distance: For speaker verification: C ts [n] = 1 M M m=1 C ts [ml, n] D = 1 R 1 ( R 1 C tr [n] C ts [n]) 2 n=1 if D > Threshold, then speaker is verified

29 Minimum-Distance Classifier Compute the average (mel or subband) cepstral features for the training data: C tr [n] = 1 M C tr [ml, n] M m=1 where L denotes the frame length. Compute the average cepstral features for the testing data: Compute a distance: For speaker verification: C ts [n] = 1 M M m=1 C ts [ml, n] D = 1 R 1 ( R 1 C tr [n] C ts [n]) 2 n=1 if D > Threshold, then speaker is verified

30 Minimum-Distance Classifier Compute the average (mel or subband) cepstral features for the training data: C tr [n] = 1 M C tr [ml, n] M m=1 where L denotes the frame length. Compute the average cepstral features for the testing data: Compute a distance: For speaker verification: C ts [n] = 1 M M m=1 C ts [ml, n] D = 1 R 1 ( R 1 C tr [n] C ts [n]) 2 n=1 if D > Threshold, then speaker is verified

31 Using Acoustic Classes Let assume we know the acoustic class of each speech segment For each class i compute the mean: C i tr [n] = 1 M M m=1 C i tr [ml, n] C i ts [n] = 1 M M m=1 C i ts [ml, n] Compute the Euclidean distance in each class: D i = 1 R 1 R 1 Average over all classes: n=1 D = 1 I ( C tr i I i=1 [n] C ts i [n]) 2 Use D as previously for speaker verification (or identification) D i

32 Using Acoustic Classes Let assume we know the acoustic class of each speech segment For each class i compute the mean: C i tr [n] = 1 M M m=1 C i tr [ml, n] C i ts [n] = 1 M M m=1 C i ts [ml, n] Compute the Euclidean distance in each class: D i = 1 R 1 R 1 Average over all classes: n=1 D = 1 I ( C tr i I i=1 [n] C ts i [n]) 2 Use D as previously for speaker verification (or identification) D i

33 Using Acoustic Classes Let assume we know the acoustic class of each speech segment For each class i compute the mean: C i tr [n] = 1 M M m=1 C i tr [ml, n] C i ts [n] = 1 M M m=1 C i ts [ml, n] Compute the Euclidean distance in each class: D i = 1 R 1 R 1 Average over all classes: n=1 D = 1 I ( C tr i I i=1 [n] C ts i [n]) 2 Use D as previously for speaker verification (or identification) D i

34 Using Acoustic Classes Let assume we know the acoustic class of each speech segment For each class i compute the mean: C i tr [n] = 1 M M m=1 C i tr [ml, n] C i ts [n] = 1 M M m=1 C i ts [ml, n] Compute the Euclidean distance in each class: D i = 1 R 1 R 1 Average over all classes: n=1 D = 1 I ( C tr i I i=1 [n] C ts i [n]) 2 Use D as previously for speaker verification (or identification) D i

35 Using Acoustic Classes Let assume we know the acoustic class of each speech segment For each class i compute the mean: C i tr [n] = 1 M M m=1 C i tr [ml, n] C i ts [n] = 1 M M m=1 C i ts [ml, n] Compute the Euclidean distance in each class: D i = 1 R 1 R 1 Average over all classes: n=1 D = 1 I ( C tr i I i=1 [n] C ts i [n]) 2 Use D as previously for speaker verification (or identification) D i

36 Multivariate Gaussian pdf Let x be a d 1 vector Gaussian pdf: g µ,σ (x) = 1 2π d Σ e 1 2 (x µ)t Σ 1 (x µ) where µ is the mean vector and Σ the covariance matrix. Estimation of the mean: ˆµ = 1 N N i=1 x i Estimation of the (unbiased) covariance matrix: ˆΣ = 1 N 1 N (x i µ)(x i µ) T i=1

37 Multivariate Gaussian pdf Let x be a d 1 vector Gaussian pdf: g µ,σ (x) = 1 2π d Σ e 1 2 (x µ)t Σ 1 (x µ) where µ is the mean vector and Σ the covariance matrix. Estimation of the mean: ˆµ = 1 N N i=1 x i Estimation of the (unbiased) covariance matrix: ˆΣ = 1 N 1 N (x i µ)(x i µ) T i=1

38 Multivariate Gaussian pdf Let x be a d 1 vector Gaussian pdf: g µ,σ (x) = 1 2π d Σ e 1 2 (x µ)t Σ 1 (x µ) where µ is the mean vector and Σ the covariance matrix. Estimation of the mean: ˆµ = 1 N N i=1 x i Estimation of the (unbiased) covariance matrix: ˆΣ = 1 N 1 N (x i µ)(x i µ) T i=1

39 Gaussian Mixture Model - GMM Mixture of Gaussian PDFs K p(x θ) = p(q k x)g µk,σ k (x) k=1 where Speaker model, θ for k = 1, 2,, K K p(q k x) = 1 k=1 θ = {p k, µ k, Σ k }

40 Speaker Identification If we have estimated S target speaker models θ j with j = 1, 2,, S. Maximum a posteriori probability classification: Maximum Likelihood: max θ j P(θ j x i ) = p(x i θ j )P(θ j ) S j=1 p(x i θ j ) max θ j p(x i θ j ) if X = {x 0, x 1, x M 1 } and assuming frames are independent: Speaker identification: p(x θ j ) = M 1 i=0 p(x i θ j ) M 1 Ŝ = max log [p(xi θ j )] 1 j S

41 Speaker Identification If we have estimated S target speaker models θ j with j = 1, 2,, S. Maximum a posteriori probability classification: Maximum Likelihood: max θ j P(θ j x i ) = p(x i θ j )P(θ j ) S j=1 p(x i θ j ) max θ j p(x i θ j ) if X = {x 0, x 1, x M 1 } and assuming frames are independent: Speaker identification: p(x θ j ) = M 1 i=0 p(x i θ j ) M 1 Ŝ = max log [p(xi θ j )] 1 j S

42 Speaker Identification If we have estimated S target speaker models θ j with j = 1, 2,, S. Maximum a posteriori probability classification: Maximum Likelihood: max θ j P(θ j x i ) = p(x i θ j )P(θ j ) S j=1 p(x i θ j ) max θ j p(x i θ j ) if X = {x 0, x 1, x M 1 } and assuming frames are independent: Speaker identification: p(x θ j ) = M 1 i=0 p(x i θ j ) M 1 Ŝ = max log [p(xi θ j )] 1 j S

43 Speaker Identification If we have estimated S target speaker models θ j with j = 1, 2,, S. Maximum a posteriori probability classification: Maximum Likelihood: max θ j P(θ j x i ) = p(x i θ j )P(θ j ) S j=1 p(x i θ j ) max θ j p(x i θ j ) if X = {x 0, x 1, x M 1 } and assuming frames are independent: Speaker identification: p(x θ j ) = M 1 i=0 p(x i θ j ) M 1 Ŝ = max log [p(xi θ j )] 1 j S

44 Speaker Identification If we have estimated S target speaker models θ j with j = 1, 2,, S. Maximum a posteriori probability classification: Maximum Likelihood: max θ j P(θ j x i ) = p(x i θ j )P(θ j ) S j=1 p(x i θ j ) max θ j p(x i θ j ) if X = {x 0, x 1, x M 1 } and assuming frames are independent: Speaker identification: p(x θ j ) = M 1 i=0 p(x i θ j ) M 1 Ŝ = max log [p(xi θ j )] 1 j S

45 Speaker Verification If we have estimated a GMM for the target speaker θ t and a GMM for a collection of imposters (background model), θ BUM Compute the ratio: P(θ t X) P(θ BUM X) = p(x θ t )P(θ t ) p(x θ BUM )P(θ BUM ) Compute the log-likelihood ratio: Λ(X) = log [p(x θ t )] log [p(x θ BUM )] Compare with a threshold Λ(X) λ, accept Λ(X) < λ, reject

46 Speaker Verification If we have estimated a GMM for the target speaker θ t and a GMM for a collection of imposters (background model), θ BUM Compute the ratio: P(θ t X) P(θ BUM X) = p(x θ t )P(θ t ) p(x θ BUM )P(θ BUM ) Compute the log-likelihood ratio: Λ(X) = log [p(x θ t )] log [p(x θ BUM )] Compare with a threshold Λ(X) λ, accept Λ(X) < λ, reject

47 Speaker Verification If we have estimated a GMM for the target speaker θ t and a GMM for a collection of imposters (background model), θ BUM Compute the ratio: P(θ t X) P(θ BUM X) = p(x θ t )P(θ t ) p(x θ BUM )P(θ BUM ) Compute the log-likelihood ratio: Λ(X) = log [p(x θ t )] log [p(x θ BUM )] Compare with a threshold Λ(X) λ, accept Λ(X) < λ, reject

48 Speaker Verification If we have estimated a GMM for the target speaker θ t and a GMM for a collection of imposters (background model), θ BUM Compute the ratio: P(θ t X) P(θ BUM X) = p(x θ t )P(θ t ) p(x θ BUM )P(θ BUM ) Compute the log-likelihood ratio: Λ(X) = log [p(x θ t )] log [p(x θ BUM )] Compare with a threshold Λ(X) λ, accept Λ(X) < λ, reject

49 GMM-based recognition systems

50 Performance of GMM-based recognition systems 19 mel-scale coeff ( ), 8-component GMM with diagonal covariance matrix.

51 Outline 1 Introduction 2 Spectral Features for Speaker Recognition Mel-Cepstrum Sub-Cepstrum 3 Speaker Recognition Algorithms Minimum-Distance Classifier Vector Quantization Gaussian Mixture Model - GMM 4 Non-Spectral Features in Speaker Recognition Glottal Flow Derivative, GFD Prosodic and other features 5 Acknowledgments 6 References

52 Liljencrants-Fant (LF) model for GFD 7-parameters LF model: u LF (t) = 0, 0 t < T o = E o e α(t To) sin [Ω 0 (t T o )], T o t < T e = E 1 [e β(t Te) e β(tc Te) ], T e t < T c

53 Example of a glottal flow derivative estimate [1]

54 Comparing histograms for two speakers based on GFD estimates [1]

55 Speaker identification performance using GFD parameters [1] Table: Using GFD estimates Features Male Female Coarse: 7 LF 58.3% 68.2% Fine: 5 energy 39.5% 41.8% Source: 12 LF & energy 69.1% 73.6% Table: Using mel-cepstrum on GFD estimates Features Male Female Modeled GFD: 41.1% 51.8% GFD: 95.1% 95.5%

56 Prosodic and other features Left: females, Right: males

57 Explaining the performance of prosody for SID

58 Outline 1 Introduction 2 Spectral Features for Speaker Recognition Mel-Cepstrum Sub-Cepstrum 3 Speaker Recognition Algorithms Minimum-Distance Classifier Vector Quantization Gaussian Mixture Model - GMM 4 Non-Spectral Features in Speaker Recognition Glottal Flow Derivative, GFD Prosodic and other features 5 Acknowledgments 6 References

59 Acknowledgments Most, if not all, figures in this lecture are coming from the book: T. F. Quatieri: Discrete-Time Speech Signal Processing, principles and practice 2002, Prentice Hall and have been used after permission from Prentice Hall

60 Outline 1 Introduction 2 Spectral Features for Speaker Recognition Mel-Cepstrum Sub-Cepstrum 3 Speaker Recognition Algorithms Minimum-Distance Classifier Vector Quantization Gaussian Mixture Model - GMM 4 Non-Spectral Features in Speaker Recognition Glottal Flow Derivative, GFD Prosodic and other features 5 Acknowledgments 6 References

61 M. Plumpe, T. F. Quatieri, and D. Reynolds, Modeling of the Glottal Flow Derivative Waveform with Application to Speaker Identification, IEEE Trans. Audio, Speech, and Language Processing, vol. 1, pp , Sept

62

63 Τέλος Ενότητας

64 Χρηματοδότηση Το παρόν εκπαιδευτικό υλικό έχει αναπτυχθεί στα πλαίσια του εκπαιδευτικού έργου του διδάσκοντα. Το έργο «Ανοικτά Ακαδημαϊκά Μαθήματα στο Πανεπιστήμιο Κρήτης» έχει χρηματοδοτήσει μόνο τη αναδιαμόρφωση του εκπαιδευτικού υλικού. Το έργο υλοποιείται στο πλαίσιο του Επιχειρησιακού Προγράμματος «Εκπαίδευση και Δια Βίου Μάθηση» και συγχρηματοδοτείται από την Ευρωπαϊκή Ένωση (Ευρωπαϊκό Κοινωνικό Ταμείο) και από εθνικούς πόρους. 3

65 Σημειώματα

66 Σημείωμα αδειοδότησης Το παρόν υλικό διατίθεται με τους όρους της άδειας χρήσης Creative Commons Αναφορά, Μη Εμπορική Χρήση, Όχι Παράγωγο Έργο 4.0 [1] ή μεταγενέστερη, Διεθνής Έκδοση. Εξαιρούνται τα αυτοτελή έργα τρίτων π.χ. φωτογραφίες, διαγράμματα κ.λ.π., τα οποία εμπεριέχονται σε αυτό και τα οποία αναφέρονται μαζί με τους όρους χρήσης τους στο «Σημείωμα Χρήσης Έργων Τρίτων». [1] Ως Μη Εμπορική ορίζεται η χρήση: που δεν περιλαμβάνει άμεσο ή έμμεσο οικονομικό όφελος από την χρήση του έργου, για το διανομέα του έργου και αδειοδόχο που δεν περιλαμβάνει οικονομική συναλλαγή ως προϋπόθεση για τη χρήση ή πρόσβαση στο έργο που δεν προσπορίζει στο διανομέα του έργου και αδειοδόχο έμμεσο οικονομικό όφελος (π.χ. διαφημίσεις) από την προβολή του έργου σε διαδικτυακό τόπο Ο δικαιούχος μπορεί να παρέχει στον αδειοδόχο ξεχωριστή άδεια να χρησιμοποιεί το έργο για εμπορική χρήση, εφόσον αυτό του ζητηθεί.. 5

67 Σημείωμα Αναφοράς Copyright Πανεπιστήμιο Κρήτης, Στυλιανού Ιωάννης. «Ψηφιακή Επεξεργασία Φωνής. Αναγνώριση Ομιλητή». Έκδοση: 1.0. Ηράκλειο/Ρέθυμνο Διαθέσιμο από τη δικτυακή διεύθυνση:

68 Διατήρηση Σημειωμάτων Οποιαδήποτε αναπαραγωγή ή διασκευή του υλικού θα πρέπει να συμπεριλαμβάνει: το Σημείωμα Αναφοράς το Σημείωμα Αδειοδότησης τη δήλωση Διατήρησης Σημειωμάτων το Σημείωμα Χρήσης Έργων Τρίτων (εφόσον υπάρχει) μαζί με τους συνοδευόμενους υπερσυνδέσμους.

69 Σημείωμα Χρήσης Έργων Τρίτων Το Έργο αυτό κάνει χρήση των ακόλουθων έργων: Εικόνες/Σχήματα/Διαγράμματα/Φωτογραφίες Εικόνες/σχήματα/διαγράμματα/φωτογραφίες που περιέχονται σε αυτό το αρχείο προέρχονται από το βιβλίο: Τίτλος: Discrete-time Speech Signal Processing: Principles and Practice Prentice-Hall signal processing series, ISSN Συγγραφέας: Thomas F. Quatieri Εκδότης: Prentice Hall PTR, 2002 ISBN: X, Μέγεθος: 781 σελίδες και αναπαράγονται μετά από άδεια του εκδότη.