1 η Σειρά Ασκήσεων Θεόδωρος Αλεξόπουλος. Αναγνώριση Προτύπων και Νευρωνικά Δίκτυα

Transcript

1 Σχολή Εφαρμοσμένων Μαθηματικών και Φυσικών Επιστημών Εθνικό Μετσόβιο Πολυτεχνείο Αναγνώριση Προτύπων και Νευρωνικά Δίκτυα η Σειρά Ασκήσεων Θεόδωρος Αλεξόπουλος

2 Άδεια Χρήσης Το παρόν εκπαιδευτικό υλικό υπόκειται σε άδειες χρήσης Creaive Commons. Για εκπαιδευτικό υλικό όπως εικόνες που υπόκειται σε άδεια χρήσης άλλου τύπου αυτή πρέπει να αναγράφεται ρητώς.

3 Jeìdwroc Alexìpouloc Anaplhrw c Kajhgh c Theodoros Alexopoulos Associae Professor EJNIKO METSOBIO POLUTEQNEIO NATIONAL TECHNICAL UNIVERSITY SQOLH EFARMOSMENWN MAJHMATIKWN KAI DEPARTMENT OF PHYSICS FUSIKWN EPISTHMWN - TOMEAS FUSIKHS ZOGRAFOU CAMPUS HRWWN POLUTEQNEIOU ATHENS - GREECE AJHNA 57 8 Phone : Fax: Thl: Fax: Theodoros.Alexopoulos@cern.ch heoalex@cenral.nua.gr hml:// Anagn rish ProÔpwn & Neurwnikˆ DÐkua Probl maa (Kefˆlaio: Saisik JewrÐa ou Bayes (Episrof : 4 NoembrÐou 3. O akìloujoc pðnakac ou sq maoc ( mac dðnei ic upì sunj kh pijanìhec miac uqaðac meablh c X gia reic klˆseic ω ω kai ω 3. 'Esw ìi gnwrðzoume ic a priori pijanìhec P (ω = 3 kai P (ω = 3. UpologÐse hn pijanìha lˆjouc hc axinìmhshc qrhsimopoi nac on kanìna apìfashc Bayes. ParahreÐse ìi x p(x/ω i = kai h uqaða meablh X paðrnei imèc so diˆshma [ 6]. LÔsh: Gia ic a priori pijanìhec èqoume: P (ω = P (ω = 3 P (ω 3 = P (ω P (ω = 4. Oi imèc P (ω i p(x/ω i i = 3 pou ja qrhsimopoi soume gia hn axinìmhsh wn klˆsewn faðnonai son pðnaka. ParahroÔme ìi x P (ω ip(x/ω i = P (ω i. O axinomh c Bayes ja apofasðsei gia hn klˆsh ω i ìan: P (ω i p(x/ω i > P (ω j p(x/ω j j i.

4 i x p(x=x/ i X= X= X=3 X=4 X=5 X= Sq ma : ParahreÐse ìi x p(x/ω i = kai h uqaða meablh X paðrnei imèc so diˆshma [ 6]. P (ω i p(x/ω i x = x = x = 3 x = 4 x = 5 x = 6 ω ω ω PÐnakac : [ Sq ma :

5 Epomènwc apì on pðnaka h apìfash so q ro wn qarakhrisik n shmeðwn eðnai au pou faðneai so sq ma (. H pijanìha lˆjouc axinìmhshc eðnai: P e = P c = [ ] = 48 ìpou P c eðnai h pijanìha hc orj c axinìmhshc.. Na upologðsee hn pijanìha ou lˆjouc axinìmhshc kaˆ Bayes gia duo klˆseic ω ω ìpou jewroôme deðgmaa se -diasˆseic apì gkaousianèc kaanomèc pou perigrˆfonai apì ic akìloujec puknìhec pijanìhac: me kai me µ = µ = p( x/ω N( µ Σ ( ( 4 Σ = 9 p( x/ω N( µ Σ ( Jewr se ìi h a priori pijanìha P (ω = 5. LÔsh: ( 4 Σ = 9. ParahroÔme ìi gia ouc pðnakec diasporˆc Σ = Σ èqoume: Σ = Σ = Σ. O anðsrofoc pðnakac diasporˆc eðnai: Σ = 35 Oi sunar seic diˆkrishc g ( x g ( x eðnai: ( 9 4. me g i ( x = (Σ i µ i x µ i Σ i µ i + ln P (ω i P (ω = 5 P (ω = 75 kai h sunˆrhsh apìfashc eðnai: d( x = g ( x g ( x 3

6 d( ( x = Σ µ x µ Σ µ ( Σ µ x + µ Σ µ + ln ( P (ω P (ω d( x = 4x + 9x 3 ìpou x = ( x x. H epifˆneia apìfashc eðnai: d( x = 4x + 9x 3 = w x + w = ìpou w = ( 4/35 9/35 w = 3/35 = 885. OrÐzoume mia nèa meablh y = w x. Tìe oi upì sunj kh pijanìhec eðnai: p( ( x /ω i N µ i Σ i p( w x /ω i N ( w µ i w Σ i w p( w x /ω N ( 89 p( w x /ω N ( Dhlad o prìblhmˆ mac ègine prìblhma mðac diˆsashc ìpwc faðneai so sq ma (3. H pijanìha orj c axinìmhshc eðnai: P c = P (ω p( w x /ω d( w x + P (ω R p( w x /ω d( ω x R P c = N ( 89 dy N ( dy P c = 4 ( Φ + 3 ( Φ 83 4

7 5 5 \ Sq ma 3: P c = 4 4 Φ( Φ( P c = 4 4 ( ( P c = JewreÐse mia mèrhsh se reic diasˆseic x = (x x x 3. 'Esw ìi èqoume 4 deðgmaa apì hn klˆsh ω kai 4 deðgmaa apì hn klˆsh ω : ω : { ( ( ( ( } ω : { ( ( ( ( }. Na upojèsee ìi h meablh x eðnai gkaousian ìpou mporeðe na qrhsimopoi see ic akìloujec sqèseic gia on prosdiorismì hc mèshc im c kai ou pðnaka diasporˆc: µ = N N x k k= ìpou N eðnai o pl joc wn deigmˆwn. Σ = N N x k x k µ µ k= 5

8 Na qrhsimopoi see on kanìna apìfashc Bayes gia on prosdiorismì hc epifˆneiac apìfashc. ProsdiorÐse hn olik pijanìha gia hn orj axinìmhsh ou kanìna apìfashc Bayes. LÔsh: Oi mèsec imèc eðnai: kai oi pðnakec diasporˆc: µ = 4 3 Σ = Σ = 6 µ = Efìson Σ = Σ o kanìnac ou Bayes ja mac d sei èna axinomh elˆqishc apìsashc Mahalanobis me sunar seic diˆkrishc: ìpou Epomènwc: H epifˆneia apìfashc eðnai: Σ = g i ( ( x = Σ µ i x µ i Σ µ i kai x = x x x 3. g ( x = 4x 3 g ( x = 4x + 8x + 8x 3. d( x = g ( x g ( x = 8x 8x 8x = ( x x x 3 w x + w = + = 6

9 ìpou w = kai w = /. Efìson Σ = Σ kai p( x /ω i N( µ i Σ i ìe: p( w x /ω i N( w µ i w Σ i w. Epomènwc oi nèec monodiˆsaec puknìhec pijanìhac eðnai: p( w x /ω N( p( w x /ω N( \ Z W [ 5 5 H pijanìha ou lˆjouc axinìmhshc eðnai: [ Φ P e = Sq ma 4: ( Φ ( ] = 4..4 'Esw a deðgmaa: ( ( (3 (3 ( 3 7

10 an koun se mia klˆsh ω. Epiplèon èsw a deðgmaa: (7 9 (8 9 (9 8 (9 9 (8 an koun se mia klˆsh ω. 'Esw ìi a deðgmaa wn duo klˆsewn proèrqonai apì gkaousianèc puknìhec pijanìhac. Na breðe hn epifˆneia apìfashc efarmìzonac on kanìna Bayes. JewreÐse isopðjanec klˆseic. LÔsh: 'Oi mèsec imèc wn dôo kaanom n eðnai: µ = ( /5 µ = ( 4/5 9 ìpou qrhsimopoi same h sqèsh: µ i = 5 x k i = (gia ic dôo klˆseic 5 Gia on prosdiorismì wn mèswn im n kai gia ouc pðnakec diasporˆc: k= Epomènwc ja èqoume: Σ i = 5 5 x k x k µ i µ i k= Σ = {( ( ( ( ( } ( 4 6 (/5 (/ (/5 ( /5 /5 Σ = /5 /5 ( Σ 7 9 = DeÐxe ìi Σ = Σ. Oi sunar seic diˆkrishc g ( x g ( x kai h sunˆrhsh apìfashc eðnai: d( x = g ( x g ( x d( x = { ( Σ µ x } { ( µ Σ µ Σ } µ x µ Σ µ 8

11 d( x = ( µ µ Σ ( x µ Σ µ Σ µ ìpou x = ( x x. Epomènwc h epifˆneia apìfashc eðnai: d( x = x + 35x 6 = x + 35x 6 =..5 (a H epifˆneia apìfashc gia duo klˆseic ω kai ω pou eðnai kaanemhmènec kaˆ Gauss me pðnakec diasporˆc: Σ = Σ σ I kai mèsec imèc µ µ kai a priori pijanìhec: P (ω P (ω eðnai mia uperepifˆneia pou perigrˆfeai apì hn akìloujh grammik exðswsh: A ( x A =. Na ekfrˆsee ic paramèrouc A kai A wc sunˆrhsh wn µ µ Σ P (ω kai P (ω. (b O Ðdioc kanìnac apìfashc Bayes gia duo kahgorðec me pðnakec diasporˆc: Σ Σ P (ω P (ω ja d sei mia epifˆneia apìfashc pou perigrˆfeai apì hn exðswsh: x B x + A x + C =. Na kajorðsee ic paramèrouc B A kai C. LÔsh: (a H sunˆrhsh diˆkrishc g i ( x eðnai: g i ( x = x Σ x + ( µ i Σ x µ i Σ µ i + ln P (ω i 9

12 me epifˆneia apìfashc d( x = g ( x g ( x = g ( x = g( x 'Esw ìi: Tìe h sqèsh ( ja gðnei: ìpou C = ( µ µ Σ x ( µ Σ µ µ Σ µ + ln P (ω P (ω ( µ Σ µ µ Σ µ ln P (ω P (ω. ( µ µ Σ ( x C( µ + µ ( µ µ Σ ( µ + µ A ( x A = A = Σ ( µ µ =. ( = ParahroÔme ìi: A = / ( µ Σ µ µ Σ µ ln P (ω P (ω ( µ µ Σ ( µ + ( µ + µ. µ ( µ µ Σ ( µ + µ = µ Σ µ µ Σ µ. Epomènwc A = ( µ + ln (P (ω /P (ω µ ( µ µ Σ ( µ + ( µ + µ. µ (b Gia Σ Σ kai P (ω P (ω h sunˆrhsh diˆkrishc eðnai:

13 H epifˆneia apìfashc d ( x eðnai: g i ( x = x Σ ( i x + µ i Σ x i µ i Σ i ln Σ i + ln P (ω i. g ( x = g ( x x (Σ Σ x + ( µ Σ µ Σ x ( µ Σ µ µ Σ µ + + ln Σ Σ + ln P (ω P (ω = x B x + A x + C = ìpou profan c èqoume orðsei ic akìloujec paramèrouc: B = (Σ Σ kai C = A = Σ µ Σ µ ( µ Σ µ µ Σ µ + + ln Σ Σ + ln P (ω P (ω.

14 .6 (a JewreÐse èna prìblhma anagn rishc proôpwn me M-kahgorÐec miac diˆsashc ìpou kˆje kahgorða qarakhrðzeai apì mia kaanom puknìhac pijanìhac Rayleigh: p(x/ω i = { (x/σ i e x /σi gia x gia x <. BreÐe h sunˆrhsh apìfashc Bayes ou probl maoc upojèonac ìi oi a priori pijanìhec eðnai P (ω i = /M. (b Epanalˆbae o Ðdio prìblhma shn perðpwsh gia hn kaanom puknìhac pijanìhac: p(x/ω i = { (x/ti e x/t i gia x gia x <. LÔsh: (a H sunˆrhsh diˆkrishc eðnai: g i (x = p(x/ω i P (ω i mporoôme na qrhsimopoi soume o fusikì logˆrijmo g i (x = ln p(x/ω i + ln P (ω i g i (x = ln x ln σ i x σ i + ln M g i (x = ln x ln σ i x σ i ln M. Gia dôo klˆseic i j o axinomh c ja mac d sei mia epifˆneia apìfashc: ln x = d ij (x = g i (x g j (x = ( σi σ j x 4 ln ( σj σ i ( σ i / ( = σj σ i σ j. (b

15 H sunˆrhsh diˆkrishc eðnai: ìpou kai h epifˆneia apìfashc eðnai: g i (x = ln p(x/ω i ln P (ω i p(x/ω i = x e x/t i P (ω Ti i = d ij (x = g i (x g j (x ( ln T i xti ( ln T j xtj = x T = ln(t j/t i. T i T j S[ 3 S[ 3 [ 7 [ Sq ma 5: H pijanìha ou lˆjouc axinìmhshc eðnai: P e = [ xt x ] e x/t j x dx + e x/t i dx Tj x T Tj P e = ( [x T e x T /T j + e xt /Ti + ( e x T /T j + e x T /T i ] T j T i P e = [ ( ( ] e x T /T xt i + e x T /T xt j + T i T j 3

16 ìpou x T eðnai o shmeðo apìfashc: x T = ln(t j/t i. T i T j.7 'Esw ìi duo klˆseic perigrˆfonai apì ic sunar seic puknìhac pijanìhac ou sq - maoc (6. BreÐe o sônoro apìfashc Bayes. Na upojèsee ìi èqoume isopðjanec klˆseic ω kai ω. S[ S[ [ Sq ma 6: LÔsh: Apì o sq ma (6 èqoume: p(x/ω = x + p(x/ω = x. P (ω = P (ω =. H sunˆrhsh diˆkrishc eðnai: g i (x = p(x/ω i P (ω i kai h epifˆneia apìfashc eðnai: 4

17 d (x = g (x g (x = ( x + (x = H pijanìha ou lˆjouc hc axinìmhshc eðnai: P e = P (ω 4/3 x = 4 3. (x dx + P (ω P e = 4/3 ( = 3. ( x + dx.8 Upojèoume ìi oi akìloujec duo kahgorðec (ω ω perigrˆfonai apì gkaousianèc sunar - seic puknìhac pijanìhac: kai ω : { ( ( ( ( } ω : { (4 4 (6 4 (6 6 (4 6 }. 'Esw ìi oi a priori pijanìhec eðnai Ðsec P (ω = P (ω. Na breðe hn exðswsh pou perigrˆfei o sônoro apìfashc Bayes meaxô wn duo au n kahgori n. LÔsh: 'Eqoume P (ω = P (ω = ìpou x j eðnai a prìupa hc klˆshc ω µ = 4 µ = 4 4 j= 4 j= x j = ( x j = ( 5 5 5

18 [ Z Z [ Sq ma 7: ìpou x j eðnai a prìupa hc klˆshc ω. Oi pðnakec diasporˆc eðnai: Σ = 4 [( Σ = 4 ( + ( 4 x j x j µ µ j= ( ( = kai omoðwc breðe oi o pðnakac diasporˆc Σ eðnai: ( ( + ( = I ( ( + ] Σ = 4 4 x j x j = I. j= Efìson Σ = Σ = Σ = I oi sunar seic diˆkrishc eðnai: g ( x = x Σ µ µ Σ µ g ( x = x µ ( µ µ = ( x x ( ( g ( x = x + x. 'Omoia gia h g ( x èqoume: 6

19 g ( x = x Σ µ µ Σ µ Epomènwc h epifˆneia apìfashc eðnai: g ( x = 5x + 5x 5. d ( x = g ( x g ( x = ìpwc faðneai so sq ma (7. x + x 6 =.9 Na epanalˆbee o prìblhma (.8 gia ic akìloujec klˆseic: ω : { ( ( ( ( } kai ω : { ( ( ( ( }. ParahreÐse ìi oi duo auèc klˆseic den eðnai grammikˆ diaqwrðsimec. LÔsh: [ Z Z [ Sq ma 8: Oi mèsec imèc wn dôo klˆsewn eðnai: 7

20 ìpou a deðgmaa x j kai x j eðnai: x j kai x j Oi pðnakec diasporˆc eðnai: µ = 4 µ = 4 {( {( 4 j= 4 j= ( ( x j = ( x j = ( ( ( ( ( } }. 4 kai Σ = 4 [( Σ = 4 j= ( + + x j x j µ µ ( ] ( ( Σ = ( = I ( Σ = [( 4 ( + ( + ( ( + ( ( Σ = ] ( ( ( + ( = I. ( + ( ( ( + ParahroÔme ìi Σ Σ epomènwc oi dôo auèc klˆseic ω ω den eðnai grammikˆ diaqwrðsimec. Oi sunar seic diˆkrishc eðnai: g ( x = ln Σ ( x µ Σ ( x µ 8

21 g ( x = ln(/4 ( x µ ( x µ ìpou g ( x = ln 4 ( ( x + x = ln x + x x = ( x x. 'Omoia: g ( x = ln Σ ( x µ Σ ( x µ g ( x = ln 4 4 x x Epomènwc h epifˆneia apìfashc eðnai: g ( x = ln 4 ( x 4 + x ( = ln x 4 + x. d ( x = g ( x g ( x = ln 3 4 (x + x = x + x = 8 ln. 3 9

22 Χρηματοδότηση Το παρόν εκπαιδευτικό υλικό έχει αναπτυχθεί στα πλαίσια του εκπαιδευτικού έργου του διδάσκοντα Το έργο «Ανοικτά Ακαδημαϊκά Μαθήματα ΕΜΠ» έχει χρηματοδοτήσει μόνο την αναδιαμόρφωση του εκπαιδευτικού υλικού. Το έργο υλοποιείται στο πλαίσιο του Επιχειρησιακού Προγράμματος «Εκπαίδευση και Δια Βίου Μάθηση» και συγχρηματοδοτείται από την Ευρωπαϊκή Ένωση (Ευρωπαϊκό Κοινωνικό Ταμείο και από εθνικούς πόρους.