Probabilistic & Unsupervised Learning. Factored Variational Approximations and Variational Bayes. Expectations in Statistical Modelling

Transcription

1 Expecaons n Sascal Modellng Probablsc & Unsupervsed Learnng Parameer esmaon ˆ = argmax dz P(Z )P(X Z, ) Facored Varaonal Approxmaons and Varaonal Bayes Maneesh Sahan maneesh@gasby.ucl.ac.uk Gasby Compuaonal Neuroscence Un, and MSc ML/CSML, Dep Compuer Scence Unversy College London erm, Auumn 08 (or, usng EM) new = argmax dz P(Z X, old ) log P(X, Z ) Predcon p(x D, m) = d p( D, m)p(x, D, m) Model selecon or weghng (by margnal lkelhood) p(d m) = d p( m)p(d, m) hese negrals are ofen nracable: Analyc nracably: negrals may no have closed form n non-lnear, non-gaussan models numercal negraon. Compuaonal nracably: Numercal negral (or sum f Z or are dscree) may be exponenal n daa or model sze. Inracables and approxmaons Dsrbued models Inference compuaonal nracably Facored varaonal approx Loopy BP/EP/Power EP LP relaxaons/ convexfed BP Gbbs samplng, oher MCMC Inference analyc nracably Laplace approxmaon (global) Paramerc varaonal approx Message approxmaons (lnearsed, sgma-pon, Laplace) Assumed-densy mehods and Expecaon-Propagaon (Sequenal) Mone-Carlo mehods s () s () s () s () s () s () s () s () x x x x Learnng nracable paron funcon Samplng parameers Consrasve dvergence Score-machng Model selecon Laplace approxmaon / BIC Varaonal Bayes (Annealed) mporance samplng Reversble jump MCMC No a complee ls! Consder an FHMM wh M sae varables akng on K values each. Moralsaon pus smulaneous saes (s (), s (),..., s (M) ) no a sngle clque rangulaon exends clques o sze M + Each sae akes K values sums over K M+ erms. Facoral pror Facoral poseror (explanng away). Varaonal mehods approxmae he poseror, ofen n a facored form. o see how hey work, we need o revew he free-energy nerpreaon of EM.

2 he Free Energy for a Laen Varable Model Observed daa X = {x }; Laen varables Z = {z }; Parameers. Goal: Maxmze he log lkelhood wr (.e. ML learnng): l() = log P(X ) = log P(Z, X )dz Any dsrbuon, q(z), over he hdden varables can be used o oban a lower bound on he log lkelhood usng Jensen s nequaly: P(Z, X ) P(Z, X ) l() = log q(z) dz q(z) log dz def = F(q, ) q(z) q(z) he E and M seps of EM he log lkelhood s bounded below by: F(q, ) = log P(Z, X ) q(z) + H[q = l() KL[q(Z) P(Z X, ) EM alernaes beween: E sep: opmse F(q, ) wr dsrbuon over hdden varables holdng parameers fxed: q(z) log P(Z, X ) q(z) dz = = q(z) log P(Z, X ) dz q(z) log P(Z, X ) dz + H[q, q(z) log q(z) dz q (k) (Z) := argmax q(z) F ( q(z), (k )) = P(Z X, (k ) ) M sep: maxmse F(q, ) wr parameers holdng hdden dsrbuon fxed: (k) := argmax F ( q (k) (Z), ) = argmax log P(Z, X ) q (k) (Z) where H[q s he enropy of q(z). So: F(q, ) = log P(Z, X ) q(z) + H[q EM as Coordnae Ascen n F EM Never Decreases he Lkelhood he E and M seps ogeher never decrease he log lkelhood: l ( (k )) = E sep F( q (k), (k )) M sep F ( q (k), (k)) Jensen l ( (k)), he E sep brngs he free energy o he lkelhood. he M-sep maxmses he free energy wr. F l by Jensen or, equvalenly, from he non-negavy of KL If he M-sep s execued so ha (k) (k ) ff F ncreases, hen he overall EM eraon wll sep o a new value of ff he lkelhood ncreases.

3 Free-energy-based varaonal approxmaon Wha f fndng expeced suffcen sas under P(Z X, ) s compuaonally nracable? For he generalsed EM algorhm, we argued ha nracable maxmsaons could be replaced by graden M-seps. Each sep ncreases he lkelhood. A fxed pon of he graden M-sep mus be a a mode of he expeced log-jon. For he E-sep we could: Parameerse q = q ρ(z) and ake a graden sep n ρ. Assume some smplfed form for q, usually facored: q = q (Z ) where Z paron Z, and maxmse whn hs form. In eher case, we choose q from whn a lmed se Q: VE sep: maxmse F(q, ) wr consraned laen dsrbuon gven parameers: q (k) (Z) := argmax F ( q(z), (k )). q(z) Q Consran Wha do we lose? Wha does resrcng q o Q cos us? Recall ha he free-energy s bounded above by Jensen: F(q, ) l( ML ) hus, as long as every sep ncreases F, convergence s sll guaraneed. Bu, snce P(Z X, (k) ) may no le n Q, we no longer saurae he bound afer he E-sep. hus, he lkelhood may no ncrease on each full EM sep. l ( (k )) / = E sep F( q (k), (k )) M sep F ( q (k), (k)) Jensen l ( (k)), M sep: unchanged (k) := argmax F ( q (k) (Z), ) = argmax q (k) (Z) log p(z, X )dz, hs means we may no (and usually won ) converge o a maxmum of l. he hope s ha by ncreasng a lower bound on l we wll fnd a decen soluon. [Noe ha f P(Z X, ML ) Q, hen ML s a fxed pon of he varaonal algorhm. Unlke n GEM, he fxed pon may no be a an unconsraned opmum of F. KL dvergence Recall ha hus, F(q, ) = log P(X, Z ) q(z) + H[q s equvalen o: = log P(X ) + log P(Z X, ) q(z) log q(z) q(z) = log P(X ) q(z) KL[q P(Z X, ). E sep maxmse F(q, ) wr he dsrbuon over laens, gven parameers: q (k) (Z) := argmax q(z) Q F ( q(z), (k )). E sep mnmse KL[q p(z X, ) wr dsrbuon over laens, gven parameers: q (k) q(z) (Z) := argmn q(z) log q(z) Q p(z X, (k ) ) dz So, n each E sep, he algorhm s ryng o fnd he bes approxmaon o P(Z X ) n Q n a KL sense. hs s relaed o deas n nformaon geomery. I also suggess generalsaons o oher dsance measures. Facored Varaonal E-sep he mos common form of varaonal approxmaon parons Z no dsjon ses Z wh Q = { q q(z) = q } (Z ). In hs case he E-sep s self erave: (Facored VE sep) : maxmse F(q, ) wr q (Z ) gven oher q j and parameers: q (k) (Z ) := argmax q (Z ) F ( q (Z ) j q j(z j), (k )). q updaes eraed o convergence o complee VE-sep. In fac, every (VE) -sep separaely ncreases F, so any schedule of (VE) - and M-seps wll converge. Choce can be dcaed by praccal ssues (rarely effcen o fully converge E-sep before updang parameers).

4 Facored Varaonal E-sep he Facored Varaonal E-sep has a general form. he free energy s: ( F q j(z j), (k )) = j = log P(X, Z (k ) ) q j j (Z j ) [ + H dz q (Z ) log P(X, Z (k ) ) j j q j(z j) + H[q + H[q j q j (Z j ) j Mean-feld approxmaons If Z = z (.e., q s facored over all varables) hen he varaonal echnque s ofen called a mean feld approxmaon. Suppose P(X, Z) has suffcen sascs ha are separable n he laen varables: e.g. he Bolzmann machne P(X, Z) = ( Z exp W js s j + ) b s j Now, akng he varaonal dervave of he Lagrangan (enforcng normalsaon of q ): ( ( )) δ F + λ q = log P(X, Z (k ) ) log q (Z ) q(z) δq q j (Z j ) q + λ (Z ) j wh some s Z and ohers observed. Expecaons wr a fully-facored q dsrbue over all s Z log P(X, Z) q = W js q s j qj + b s q j (= 0) q (Z ) exp log P(X, Z (k ) ) q j j (Z j ) In general, hs depends only on he expeced suffcen sascs under q j. hus, agan, we don acually need he enre dsrbuons, jus he relevan expecaons (now for approxmae nference as well as learnng). (where q for s X s a dela funcon on he observed value). hus, we can updae each q n urn gven he means (or, n general, mean suffcen sascs) of he ohers. Each varable sees he mean feld mposed by s neghbours, and we updae hese felds unl hey all agree. Mean-feld FHMM Mean-feld FHMM s () s () s () s () s () s () s () s () q(s :M : ) = q m (s m ) m, s () s () s () s () s () s () s () s () q(s :M : ) = q m (s m ) m, x x x x q m (s m ) exp log P(s :M :, x : ) (m,) (sm ) = exp log P(s τ µ s µ τ ) + µ τ τ [ log m exp P(s s ) m + log P(x q s :M m ) log P(x τ s :M τ ) }{{} α m log Φ () e m j j q m (j) e log A (x ) q Cf. forward-backward: α () j α (j)φ j A (x ) (m,) + log P(s m + s m ) q m + } {{ } β m () e j log Φ m j q m + (j) β () j Φ j A j (x + )β + (j) x x x x [ log q m (s m m ) exp P(s s ) m + log P(x q s :M m ) }{{} α m log Φ () e m j j q m (j) e log A (x ) q Cf. forward-backward: α () j α (j)φ j A (x ) Yelds a message-passng algorhm lke forward-backward Updaes depend only on mmedae neghbours n chan Chans couple only hrough jon oupu Mulple passes; messages depend on (approxmae) margnals + log P(s m + s m ) q m + } {{ } β m () e j log Φ m j q m + (j) β () j Φ j A j (x + )β + (j) Evdence does no appear explcly n backward message (cf Kalman smoohng)

5 Srucured varaonal approxmaon q(z) need no be compleely facorzed. For example, suppose Z can be paroned no ses Z and Z such ha compung he expeced suffcen sascs under P(Z Z, X ) and P(Z Z, X ) would be racable. hen he facored approxmaon q(z) = q(z )q(z ) s racable. In parcular, any facorsaon of q(z) no a produc of dsrbuons on rees, yelds a racable approxmaon. C A D B C + A + B + B +... D + C + A + D + Sucured FHMM s () s () s () s () s () s () s () s () x x x x q m (s m : ) exp log P(s :M :, x : ) (s m : ) = exp log P(s µ s µ ) + µ [ exp log P(s m s ) m + = P(s m s m ) e log P(x s :M For he FHMM we can facor he chans: log P(x s :M ) log P(x s :M ) ) (s m ) q(s :M : ) = m (s m ) q m (s m : ) hs looks lke a sandard HMM jon, wh a modfed lkelhood erm cycle hrough mulple forward-backward passes, updang lkelhood erms each me. Messages on an arbrary graph Non-facored varaonal mehods Consder a DAG: A C B P(X, Z) = k P(V k pa(v k)) he erm varaonal approxmaon s used whenever a bound on he lkelhood (or on anoher esmaon cos funcon) s opmsed, bu does no necessarly become gh. and le q(z) = q (Z ) for dsjon ses {Z }. E We have ha he VE updae for q s gven by q (Z ) exp log p(z, X ) q (Z) where q (Z) denoes averagng wh respec o qj(zj) for all j hen: log q (Z ) = log P(V k pa(v k)) k D q (Z) + cons = j Z log P(Z j pa(z j)) q (Z) + j ch(z ) log P(V j pa(v j)) q (Z) + cons hs defnes messages ha are passed beween nodes n he graph. Each node receves messages from s Markov boundary: parens, chldren and parens of chldren (all neghbours n he correspondng facor graph). Many furher varaonal approxmaons have been developed, ncludng: paramerc forms (e.g. Gaussan) for non-lnear models closed form updaes n specal cases numercal or samplng-based compuaon of expecaons recognon neworks or amorsaon o esmae varaonal parameers non-free-energy-based bounds (boh upper and lower) on he lkelhood. We can also see MAP- or zero-emperaure EM and recognon models as paramerc forms of varaonal nference. Varaonal mehods can also be used o fnd an approxmae poseror on he parameers.

6 Varaonal Bayes Varaonal Bayesan EM... So far, we have appled Jensen s bound and facorsaons o help wh negrals over laen varables. We can do he same for negrals over parameers n order o bound he log margnal lkelhood or evdence. log P(X M) = log dz d P(X, Z, M)P( M) P(X, Z, M) = max dz d Q(Z, ) log Q Q(Z, ) P(X, Z, M) max dz d Q Z(Z)Q () log Q Z,Q Q Z(Z)Q () he consran ha he dsrbuon Q mus facor no he produc Q y(z)q () leads o he varaonal Bayesan EM algorhm or jus Varaonal Bayes. Some call hs he Evdence Lower Bound (ELBO). I m no fond of ha erm. Coordnae maxmzaon of he VB free-energy lower bound p(x, Z, M) F(Q Z, Q ) = dz d Q Z(Z)Q () log Q Z(Z)Q () leads o EM-lke updaes: Q Z(Z) exp log P(Z,X ) Q () Q () P() exp log P(Z,X ) QZ (Z) E-lke sep M-lke sep Maxmzng F s equvalen o mnmzng KL-dvergence beween he approxmae poseror, Q()Q(Z) and he rue poseror, P(, Z X ). P(X, Z, ) log P(X ) F(Q Z, Q ) = log P(X ) dz d Q Z(Z)Q () log Q Z(Z)Q () = dz d Q Z(Z)Q () log QZ(Z)Q () = KL(Q P) P(Z, X ) Conjugae-Exponenal models Le s focus on conjugae-exponenal (CE) laen-varable models: Condon (). he jon probably over varables s n he exponenal famly: { } P(Z, X ) = f (Z, X ) g() exp φ() (Z, X ) Conjugae-Exponenal examples In he CE famly: Gaussan mxures facor analyss, probablsc PCA hdden Markov models and facoral HMMs where φ() s he vecor of naural parameers, are suffcen sascs lnear dynamcal sysems and swchng models Condon (). he pror over parameers s conjugae o hs jon probably: { } P( ν, τ) = h(ν, τ) g() ν exp φ() τ where ν and τ are hyperparameers of he pror. dscree-varable belef neworks Oher as ye undream-of models combnaons of Gaussan, Gamma, Posson, Drchle, Wshar, Mulnomal and ohers. No n he CE famly: Bolzmann machnes, MRFs (no smple conjugacy) Conjugae prors are compuaonally convenen and have an nuve nerpreaon: ν: number of pseudo-observaons τ : values of pseudo-observaons logsc regresson (no smple conjugacy) sgmod belef neworks (no exponenal) ndependen componens analyss (no exponenal) Noe: one can ofen approxmae such models wh a suable choce from he CE famly.

7 Conjugae-exponenal VB he Varaonal Bayesan EM algorhm Gven an d daa se D = (x,... x n), f he model s CE hen: Q () s also conjugae,.e. Q () P() exp = h(ν, τ )g() ν e φ() τ h( ν, τ )g() ν e φ() τ log P(z, x ) g() n e Q Z log f (Z,X ) Q Z e φ() wh ν = ν + n and τ = τ + (z, x ) QZ only need o rack ν, τ. (z,x ) Q Z EM for MAP esmaon Goal: maxmze P( X, m) wr E Sep: compue Q Z(Z) p(z X, ) M Sep: argmax Properes: dz Q Z(Z) log P(Z, X, ) Varaonal Bayesan EM Goal: maxmse bound on P(X m) wr Q VB-E Sep: compue Q Z(Z) p(z X, φ) VB-M Sep: Q () exp dz Q Z(Z) log P(Z, X, ) Q Z(Z) = n = Qz (z) akes he same form as n he E-sep of regular EM Q z (z ) exp log P(z, x ) Q f (z, x )e φ() Q (z,x ) = P(z x, φ()) wh naural parameers φ() = φ() Q nference unchanged from regular EM. Reduces o he EM algorhm f Q () = δ( ). F m ncreases monooncally, and ncorporaes he model complexy penaly. Analycal parameer dsrbuons (bu no consraned o be Gaussan). VB-E sep has same complexy as correspondng E sep. We can use he juncon ree, belef propagaon, Kalman fler, ec, algorhms n he VB-E sep of VB-EM, bu usng expeced naural parameers, φ. VB and model selecon Varaonal Bayesan EM yelds an approxmae poseror Q over model parameers. I also yelds an opmsed lower bound on he model evdence max F M(Q Z, Q ) P(D M) hese lower bounds can be compared amongs models o learn he rgh (srucure, connecvy... of he) model If a connuous doman of models s specfed by a hyperparameer η, hen he VB free energy depends on ha parameer: P(X, Z, η) F(Q Z, Q, η) = dz d Q Z(Z)Q () log P(X η) Q Z(Z)Q () A hyper-m sep maxmses he curren bound wr η: η argmax dz d Q Z(Z)Q () log P(X, Z, η) η ARD for unsupervsed learnng Recall ha ARD (auomac relevance deermnaon) was a hyperparameer mehod o selec relevan or useful npus n regresson. A smlar dea used wh varaonal Bayesan mehods can learn a laen dmensonaly. Consder facor analyss: x N (Λz, Ψ) z N (0, I) wh a column-wse pror Λ : N ( 0, α I ) he VB free energy s F(Q Z(Z), Q Λ (Λ), Ψ, α) = log P(X, Z Λ, Ψ) + log P(Λ α) + log P(Ψ) Q Z Q Λ +... and so hyperparameer opmsaon requres α argmax log P(Λ α) QΛ Now Q Λ s Gaussan, wh he same form as n lnear regresson, bu wh expeced momens of z appearng n place of he npus. Opmsaon wr he dsrbuons, Ψ and α n urn causes some α o dverge as n regresson ARD. In hs case, hese parameers selec relevan laen dmensons, effecvely learnng he dmensonaly of z.

8 Augmened Varaonal Mehods Sparse GP approxmaons In our examples so far, he approxmae varaonal dsrbuon has been over he naural laen varables (and parameers) of he generave model. GP predcons: y X, Y, x N ( K x X (K XX + σ I) Y, K x x K x X (K XX+σ I) K Xx + σ ) Evdence (for learnng kernel hyperparameers): log P(Y X) = log π(kxx + σ I) Y (KXX + σ I) Y Somemes may be useful o nroduce addonal laen varables, solely o acheve compuaonal racably. Compung eher form requres nverng he N N marx K XX, n O(N ) me. One proposal o make hs more effcen s o fnd (or selec) a smaller se of possbly fcous measuremens U a npus Z such ha wo examples are GP regresson and he GPLVM. P(y Z, U, x ) P(y X, Y, x ). Wha values should U and Z ake? Varaonal Sparse GP approxmaons Wre F for he (smooh) GP funcon values ha underle Y (so Y N ( F, σ I ) ). Inroduce laen measuremens U a npus Z (and negrae over U). he lkelhood can be wren P(Y X) = df du P(Y, F, U X, Z) = df du P(Y F)P(F U, X, Z)P(U Z) Varaonal Sparse GP approxmaons F(q(U),, Z) = log P(Y F) P(F U) + log P(U Z) log q(u) q(u) Now P(F U) s fxed by he generave model (raher han beng subjec o free opmsaon). So we can evaluae ha expecaon: Now, boh U and F are laen, so we nroduce a varaonal dsrbuon q(f, U) o form a free-energy. F(q(F, U), ) = log P(Y F)P(F U, X, Z)P(U Z) q(f, U) q(f,u) Now, choose he varaonal form q(f, U) = P(F U, X, Z)q(U). ha s, fx F U whou reference o Y so nformaon abou Y wll need o be compressed no q(u). hen F(q(F, U),, Z) = log = P(Y F) P(F U, X, Z) P(U Z) P(F U, X, Z) q(u) P(F U)q(U) log P(Y F) P(F U) + log P(U Z) log q(u) q(u) So, log P(Y F) P(F U) = log πσ I [(Y σ r F)(Y F) P(F U) = log πσ I [ σ r (Y F P(F U) )(Y F P(F U) ) σ r [ Σ F U = log N ( Y K XZ K ZZ U, σ I ) σ r [ K XX K XZ K ZZ K ZX F(q(U),, Z) = log N ( Y K XZ K ZZ U, σ I ) + log P(U Z) log q(u) q(u) σ r [ K XX K XZ K ZZ K ZX.

9 Varaonal Sparse GP approxmaons A few references F(q(U),, Z) = log N ( Y K XZ K U, ZZ σ I ) P(U Z) q(u) q(u) σ r [ K XX K XZ K ZZ K ZX. he expecaon s he free energy of a PPCA-lke model wh normal pror U N (0, K ZZ ) and loadng marx K XZ K ZZ. he maxmum of hs free energy s he log-lkelhood (acheved wh q equal o he poseror under he PPCA-lke model). hs gves F(q (U),, Z) = log N ( Y 0, K XZ K ZZ K ZZ K ZZ K ZX + σ I ) σ r [ K XX K XZ K ZZ K ZX. Noe ha we have elmnaed all erms n K XX. We can opmse he free energy numercally wh respec o Z and o adjus he GP pror and qualy of varaonal approxmaon. A smlar approach can be used o learn X f hey are unobserved (.e. n he GPLVM). Assume q(x, F, U) = q(x)p(f X, U)q(U). hen F = log P(Y, F, U X) log P(X) q(u)q(x) whch smplfes no racable componens n much he same way as above. Jordan, Ghahraman, Jaakkola, Saul, 999. An nroducon o varaonal mehods for graphcal models. Machne Learnng 7:8. Aas, 000. A varaonal Bayesan framework for graphcal models. NIPS. hp:// Beal, 00. Varaonal algorhms for approxmae Bayesan nference. PhD hess, Gasby Un, UCL. hp:// Wnn, 00. Varaonal message passng and s applcaons. PhD hess, Cambrdge. hp://johnwnn.org/publcaons/hess.hml; also VIBES sofware for conjugae-exponenal graphs. Some complexes: MacKay, 00. A problem wh varaonal free energy mnmzaon. hp:// urner, MS, 0. wo problems wh varaonal expecaon maxmsaon for me-seres models. In Barber, Cemgl, Chappa, eds., Bayesan me Seres Models. hp:// Berkes, urner, MS, 008. On sparsy and overcompleeness n mage models. NIPS 0. hp:// Gordano, R, Broderck,, and Jordan, MI, 05. Lnear response mehods for accurae covarance esmaes from mean feld varaonal Bayes. NIPS