CS 3750 Mache Learg Lectre 9 Latet varable moel Varatoal appromato. Mlo arecht mlo@c.ptt.e 539 Seott Sqare CS 750 Mache Learg Cooperatve vector qatzer Latet varable : meoalty bary var Oberve varable : real vale var meoalty CS 750 Mache Learg
Cooperatve vector qatzer Moel: Latet var : Beroll trbto parameter: Obervable varable : ormal trbto parameter: W Σ W Σ We ame Σ I W CS 750 Mache Learg : bary var : real vale var Jot for oe tace of a : / / ep W W........ Cooperatve vector qatzer Or obectve: Lear the parameter of the moel W Oe ca e the ata lelhoo or lelhoo a optmze.. Learg f a are obervable Log lelhoo: : bary var : real vale var Solto: ce a eay W W c CS 750 Mache Learg
CS 750 Mache Learg Cooperatve vector qatzer : real vale var : bary var } { Log lelhoo of ata: Learg f oly are obervable Solto: oe ot let beeft from the ecompoto M: e to or ch cae Or obectve: Lear the parameter of the moel Oe ca e the ata lelhoo or lelhoo a optmze.. W CS 750 Mache Learg M Let be a et of all varable th he or mg vale Average both e th for ' ' ' ' ' ' ' ' Log-lelhoo of ata Log-lelhoo of ata
M algorthm Algorthm geeral formlato Italze parameter Repeat Set '. pectato tep '. Mamzato tep arg ma ' ' tl o or mall mprovemet ' roblem: poteror probablte ' efe over CS 750 Mache Learg oteror ' ' M algorthm for or moel ' ach ata pot reqre to calclate probablte If larger the th a bottleec!!! CS 750 Mache Learg
CS 750 Mache Learg Varatoal appromato Let be a et of all varable th he or mg vale ervato Average both e th KL Log-lelhoo of ata Log-lelhoo of ata CS 750 Mache Learg Varatoal appromato KL Appromato: mamze arameter: } { } { [ ]. } { KL Why? Mamzato of phe p the loer bo o the -lelhoo
Varatoal appromato Comparo: M e tre poteror ' Varatoal M e a rrogate poteror M: ' ' Varatoal M: KL CS 750 Mache Learg Varatoal M Let be a et of all varable th he or mg vale tep: Optmze M tep Optmze th repect to hle eepg fe th repect to hle eepg ote: f the poteror the the varatoal M rece to the taar M CS 750 Mache Learg
Varatoal M So hat the eal? Why hol e e the varatoal M? ope: If e chooe ell the optmzato of both a ll become eay A ell behave choce for the mea fel appromato CS 750 Mache Learg Ampto: Mea el Appromato the mea fel appromato. Varable the trbto are epeet varable. completely factorze: or or CV moel e varable are bary orce.... CS 750 Mache Learg
CS 750 Mache Learg Mea el Appromato ctoal for the mea fel: } { } { W W 3 Ame t oe ata pot a correpog : CS 750 Mache Learg Mea el Appromato ctoal. art : δ
CS 750 Mache Learg Mea el Appromato ctoal. art : ctoal. art 3: CS 750 Mache Learg Mea el Appromato ctoal : arameter: W Mea fel parameter: [ ] δ
CS 750 Mache Learg Mea el Appromato ctoal for all ata pot: arameter: W Mea fel parameter: [ ] δ CS 750 Mache Learg Varatoal M: tep Optmzato of the fctoal th repect to : 0 et g e g efe a fe pot eqato Iterate a et fe pot eqato for all ee.. a for all
Varatoal M: M tep Optmzato of the fctoal th repect to. Start th : or ata pot et 0 Cloe form olto CS 750 Mache Learg Varatoal M: M tep Optmzato of the fctoal th repect to. arameter : 0 v v v v v v W........ W K or each varable v: he eqato efe a et of lear eqato that ca be olve CS 750 Mache Learg
Bayea CV Moel y K ε Bayea moel: trbto over parameter K y S Ι CS 750 Mache Learg Moel Specfcato X { } S { } {... } W{ } { } oberve ata latet orce probablty of K eght matr Varace of W reco of oe CS 750 Mache Learg
CS 750 Mache Learg ror β α c Gamma b a Gamma Beta K K K 0 W CS 750 Mache Learg Why the Bayea moel? Very efl for Bayea Moel Selecto Ame e o ot o the mber of orce Bayea core tell ho goo the trctre Beeft of the Bayea core: mboe Occam Razor revet overft M M M M M M Margal lelhoo
CS 750 Mache Learg Varatoal appromato X X X X X Appromato: lelhoo of ata Where a trbto th fferet parameterzato CS 750 Mache Learg Varatoal appromato KL X Appromato: lelhoo of obervable ata Optmzato of phg p the loer bo o the lelhoo of obervable ata o to chooe? he: X KL tace
Varatoal Baye appromato valato of tractable Meafel appromato K Allo aalytcal evalato of CS 750 Mache Learg VB learg Lear Moel th a M le algorthm VB Optmze tmate tate of latet varable * ep VBM Optmze tmate parameter * ep CS 750 Mache Learg
CS 750 Mache Learg VB W W W tr tr y CS 750 Mache Learg VBM β α c Gamma b a Gamma Beta K W K Σ m W
CS 750 Mache Learg VBM cot ag Σ m Σ β β α α CS 750 Mache Learg VBM cot { tr c c b b a a W W W y