Latent variable models Variational approximations.

Save this PDF as:
 WORD  PNG  TXT  JPG

Μέγεθος: px
Εμφάνιση ξεκινά από τη σελίδα:

Download "Latent variable models Variational approximations."

Transcript

1 CS 3750 Mache Learg Lectre 9 Latet varable moel Varatoal appromato. Mlo arecht 539 Seott Sqare CS 750 Mache Learg Cooperatve vector qatzer Latet varable : meoalty bary var Oberve varable : real vale var meoalty CS 750 Mache Learg

2 Cooperatve vector qatzer Moel: Latet var : ~ Beroll trbto parameter: Obervable varable : ~ ormal trbto parameter: W Σ W Σ We ame Σ I W CS 750 Mache Learg : bary var : real vale var Jot for oe tace of a : / / ep W W Cooperatve vector qatzer Or obectve: Lear the parameter of the moel W Oe ca e the ata lelhoo or lelhoo a optmze Learg f a are obervable Log lelhoo: : bary var : real vale var Solto: ce a eay W W CS 750 Mache Learg

3 CS 750 Mache Learg Cooperatve vector qatzer : real vale var : bary var Log lelhoo of ata: Learg f oly are obervable Solto: oe ot let beeft from the ecompoto M: e to or ch cae Or obectve: Lear the parameter of the moel Oe ca e the ata lelhoo or lelhoo a optmze W CS 750 Mache Learg M Let be a et of all varable th he or mg vale ervato Average both e th for Log-lelhoo of ata Log-lelhoo of ata

4 M algorthm Algorthm geeral formlato Italze parameter Repeat Set. pectato tep. Mamzato tep arg ma tl o or mall mprovemet roblem: poteror probablte efe over CS 750 Mache Learg oteror M algorthm for or moel ach ata pot reqre to calclate probablte If larger the th a bottleec!!! CS 750 Mache Learg

5 CS 750 Mache Learg Varatoal appromato Let be a et of all varable th he or mg vale ervato Average both e th Log-lelhoo of ata Log-lelhoo of ata CS 750 Mache Learg Varatoal appromato Let be a et of all varable th he or mg vale Appromato: mamze arameter: [ ].

6 Varatoal M Let be a et of all varable th he or mg vale tep: Optmze M tep Optmze th repect to hle eepg fe th repect to hle eepg ote: f the poteror the the varatoal M rece to the taar M CS 750 Mache Learg Varatoal M So hat the eal? Why hol e e the varatoal M? ope: If e chooe ell the optmzato of both a ll become eay A ell behave choce for Ue mea fel appromato CS 750 Mache Learg

7 CS 750 Mache Learg Mea el Appromato Ampto: the mea fel appromato. Varable the trbto are epeet varable. completely factorze: or or CV moel e varable are bary orce CS 750 Mache Learg Mea el Appromato ctoal for the mea fel: W W 3

8 CS 750 Mache Learg Mea el Appromato ctoal. art : δ CS 750 Mache Learg Mea el Appromato ctoal. art : ctoal. art 3:

9 CS 750 Mache Learg Mea el Appromato ctoal : arameter: W Mea fel parameter: [ ] δ CS 750 Mache Learg Mea el Appromato ctoal for all ata pot: arameter: W Mea fel parameter: [ ] δ

10 CS 750 Mache Learg Varatoal M: tep Optmzato of the fctoal th repect to : 0 et g e g efe a fe pot eqato Iterate a et fe pot eqato for all ee CS 750 Mache Learg Varatoal M: M tep Optmzato of the fctoal th repect to. Start th : 0 et or ata pot or ata pot Cloe form olto

11 CS 750 Mache Learg Varatoal M: M tep Optmzato of the fctoal th repect to. arameter : 0 v v v v v v W K W or each varable v: he eqato efe a et of lear eqato that ca be olve CS 750 Mache Learg Mea el appromato Let be a et of all varable th he or mg vale tep: Optmze Appromato: mamze arameter: [ ]. [ ].