Expectation- Maximization & Baum-Welch. Slides: Roded Sharan, Jan 15; revised by Ron Shamir, Nov 15

Transcription

1 Expecaion- Maximizaion & Baum-Welch Slides: Roded Sharan, Jan 15; revised by Ron Shamir, Nov 15 1

2 The goal Inpu: incomplee daa originaing from a probabiliy disribuion wih some unknown parameers Wan o find he parameer values ha maximize he likelihood EM approach ha helps when maximum likelihood soluion canno be direcly compued. Seeks a local maximum by ieraively solving wo easier subproblems 2

3 Coin flipping: complee daa Coins A, B wih unknown heads probs. A, B Goal: esimae A, B Experimen: Repea x5: choose A or B wih prob. 1/2, flip 10 imes, record resuls. x=(x 1, x 5 ) : no of H in se 1, 5 Y=(y 1, y 5 ) : coin used in se 1, 5 Do & Bazoglou, NBT 08 3

4 Coin flipping: complee daa Naural guess: i = fracion of H in flips of coin i This is acually he ML soluion: maximizes P(x,y ) (ex.) Wha if we do no know which coin was used in 4 each round?

5 Coin flipping: incomplee daa Now (y 1, y 5 ) are hidden / laen variables. Canno compue H prob for each coin If we guessed Y correcly we could. Idea: guess iniial 0 A, 0 B Use A, B o compue he mos likely coin for each se, ge new y Use he resuling y o recompue A, B using ML, ge +1 A, +1 B Repea ill convergence EM: use probabiliies raher han he single mos likely compleion y 5

6 Coin flipping: incomplee daa 6

7 The probabilisic seing Inpu: daa X coming from a probabilisic model wih hidden informaion y Goal: Learn he model s parameers so ha he likelihood is maximized.

8 Mixure of wo Gaussians Kalai e al. Disenangling Gaussians CACM 2012 Our inpu generaes he black disribuion. We wan o color each sample red/blue and find he parameers of he wo disribuions o maximize he daa probabiliy. (assume known) P( y 1) p ; P( y 2) p 1 p i 1 i ( xi ) j P( xi yi j) exp ( p,, ) 1 1 2

9 The likelihood funcion P( y 1) p ; P( y 2) p 1 p i 1 i ( xi ) j P( xi yi j) exp L( ) P( x ) P( x, y j ) i i i i i j 2 p j ( xi ) j log L( ) log exp 2 i j 2 2 To be coninued

10 ( P Q) KL divergence Def: The Kullback-Liebler divergence (aka relaive enropy) of discree probabiliy disribuions P and Q: D KL i log(x) x-1 for all x>0 wih equaliy iff x=1 Q( x ) i Q( x ) i DKL ( P Q) P( xi ) log P( xi ) 1 i P( xi) i P( xi) Px ( ) log Q( x ) P( x ) Q( x ) 1 0 i i i i i i i Px ( ) i Qx ( Lemma: KL divergence is nonnegaive i ) i: sum over x s. P(x)>0 Q(x)=0 P(x)=0, 0log0=0 Wih equaliy iff PQ

11 The EM algorihm (i) Goal: max log P(x θ)=log (Σ P(x,y θ)) Sraegy: guess an iniial and ieraively adjus i, making sure ha he likelihood always improves. Assume we have a model θ ha we wish o improve o a new value. Bayes rule: P(x θ) = P(x,y θ) / P(y x,θ) Take log and muliply boh sides by P( y x, ) P( y x, ) log P( x ) P( y x, ) log P( x, y ) P( y x, ) log P( y x, ) P( y x, ) log P( x ) P( y x, ) log P( x, y ) P( y x, ) log P( y x, ) y y y log P( x ) P( y x, ) log P( x, y ) P( y x, ) log P( y x, ) y y

12 The EM algorihm (ii) log P( x ) P( y x, ) log P( x, y ) P( y x, ) log P( y x, ) y log P( x ) P( y x, ) log P( x, y ) P( y x, ) log P( y x, ) Define Define y P( y x, ) = Q( ) Q( ) P( y x, ) log P( y x, ) y Wan P( x ) P( x ) = logp(x ) - logp(x ) y Q( ) P( y x, ) log P( x, y ) y y KL Divergence 0 Q( ) Q( )

13 The EM algorihm (iii) Main componen: Q( ) P( y x, ) log P( x, y ) y log P(x,y θ) is called he complee log likelihood funcion Q is he expecaion of he complee log likelihood over he disribuion of y given he curren parameers θ The algorihm: repea E-sep: Calculae he Q funcion M-sep: Maximize Q(θ θ ) wih respec o θ Sopping crierion: improvemen in log likelihood ε Noe: local opimum guaraneed o be reached, no global. Saring poin maers! Try many..

14 Back o he Gaussian mixure model Q( ) P( y x, ) log P( x, y ) y P( x, y ) P( x, y ) P( x, y j ) y ij 1 0 y i y i i j j i i i i i j y ij log P( x, y ) y log P( x, y j ) i j ij i i Q( ) P( y x, ) y log P( x, y j ) i j y y i j ij i i P( y x, ) y log P( x, y j ) ij i i 14

15 Applicaion (con.) Q( ) P( y 1 x, )log P( x, y j ) w : P( y 1 x, ) i j ij i i i P( x, y j ) P( x, y j ) i i ij ij i i i j 1 Q( ) wij log log log p j i j 2 ( x ) i 2 2 j 2 Now wrie he derivaives and equae o zero o ge he opimal parameers +1 =( 1 +1, 2 +1, p 1 +1 ) 15

16 EM for HMM: The Baum- Welch algorihm 16

17 Reminder: HMM Markovian ransiion prob. a kl Hidden saes j Emission prob. e k (b) Observed oupu symbols x i pah = 1,, M Given sequence X = (x 1,,x M ): a kl = P( i =l i-1 =k), e k (b) = P(x i =b i =k) Model=(, Q, ) P(X,) = a 0, 1 i=1 L e i(x i ) a i,i+1 Goal: Finding pah * maximizing P(X,) 17

18 Max likelihood in HMM y=π, =( a kl, e k (b) ) he log likelihood is log P( x ) log P( x, ) And he Q funcion is: Q( ) P( x, ) log P( x, ) 18

19 Compuing Q M E ( b, ) M M A ( ) k kl k 1 b k 1 l1 (, ) [ ( )] k kl P x e b a Emission probabiliy, sae k characer b Number of imes we saw b from k in pah π Transiion probabiliy, sae k o sae l Number of ransiions from k o l in pah π

20 Compuing Q (ii) M M M Q( ) P( x, ) Ek ( b, ) log( ek ( b)) Akl ( ) log akl k 1 b k1 l1 M M M P( x, ) Ek ( b, ) log( ek ( b)) P( x, ) Akl ( ) log akl k 1 b k 1 l1 P( x, ) E ( b, ) E ( b) k k P( x, ) Akl ( ) Akl probabiliy value expecaion probabiliy value expecaion

21 Compuing Q (iii) So we wan o find a se of parameers θ +1 ha maximizes: M M M E ( b) log( e ( b)) A log a k k kl kl k1 b k 1 l1 f k (i) = P(x 0,,x i, i =k) E k (b), A kl can be compued using forward/backward: P( i =k, i+1 =l x, ) = [1/P(x)] f k (i) a kl e l (x i+1 ) b l (i+1) A kl = [1/P(x)] i f k (i) a kl e l (x i+1 ) b l (i+1) similarly, E k (b) = [1/P(x)] {i xi =b} f k (i) b k (i) b k (i) = P(x i+1, x L i =k) For maximizaion, selec: a ij k A ij A ik, e ( b) k b' Ek () b E ( b') k

22 Baum-Welch: EM for HMM Maximize: a ij k A ij A ik M M M E ( b) log( e ( b)) A log a k k kl kl k1 b k 1 l1 (denoe as a ), e ( b) chosen ij k b' Ek () b E ( b') k Muliply and divide by same facor Difference beween chosen se and some oher: a A a M M chosen M M chosen kl kl kl Akl log A ' log oher kk oher k 1 l1 akl k 1 k ' l1 Akk ' akl k ' M M chosen chosen a kl A kk ' akl log oher k 1 k ' l1 akl always posiive

23 Summary: Parameer Esimaion in HMM When Saes are Unknown Inpu: X 1,,X n indep raining sequences Baum-Welch alg. (1972): Expecaion: compue expeced no. of kl sae ransiions: P( i =k, i+1 =l X, ) = [1/P(x)] f k (i) a kl e l (x i+1 ) b l (i+1) A kl = j [1/P(X j )] i f kj (i) a kl e l (x j i+1) b lj (i+1) compue expeced no. of symbol b appearances in sae k E k (b) = j [1/P(X j )] {i x j i =b} f kj (i) b kj (i) (ex.) Maximizaion: re-compue new parameers from A, E using max. likelihood. repea (1)+(2) unil improvemen 23

24 Leonard Baum, many years afer he IDA Lloyd Welch, USC Elecrical Engineering 24

25 de novo moif discovery using EM Slides sources: Chaim Linhar, Dani Wider, Kaherina Kechris GE Ron Shamir 25

26 Transcripion Facors A ranscripion facor (TF) is a proein ha regulaes a gene by binding o a binding sie (BS) in is viciniy, specific o he TF. Binding sies vary in heir sequences. Their sequence paern is called a moif. 26

27 Moif profile Alignmen a G g a c T C c A a c g a c g T A g a c g C c A C c g a c g G A Profile C G T Line up he paerns by heir sar indexes s = (s 1, s 2,, s ) Consruc marix profile wih frequencies of each nucleoide in columns Consensus A C G T A C G T Moif finding: Given a se of co-regulaed genes, find a recurren moif in heir promoer regions. 27

28 An example: Implaning Moif AAAAAAAGGGGGGG agaccgggaacgaaaaaaaaagggggggggcgacacaagaaaacgagaagacgagaccggcgccgccg accccagagcagaaggaccggaaaaaaaagagacaaaacccgaaaaaaaaaaaggggggga gagacccgggagacaaaaaaaaggggggggcccccgagaaagaggacacgccagggccga gcgagaaggagaaaaaaaagggggggccacgcaacgcgaaccaacgcggacccaaaggcaagaccgaaaaggaga cccgcggaaggccgggaggcggacgagggaagcccaacggacaaaaaaaaaagggggggcaag gcaacagcggaaggaaaaaaaaaggggggggaccgcggcgcacccaaacagggggcgagcgcaa cggggcccgagaggcccccgaaaaaaaagggggggcaaagagagagcaacacgcggcggca aacgagaaaaaaaagggggggcggggcacaacaagaggagcccacagaagcgagacacaga ggcccaggcaaaagcccaacgacaaaggaagaagaaccgcaaaaaaaaagggggggaccgaaagggaag cgggagcaacgacagacacggcaagccgcccggggacaaagcacgaagcaaaaaaaaggggggga 28

29 Where is he Implaned Moif? (*) agaccgggaacgaaaaaaaaagggggggggcgacacaagaaaacgagaagacgagaccggcgccgccg accccagagcagaaggaccggaaaaaaaagagacaaaacccgaaaaaaaaaaaggggggga gagacccgggagacaaaaaaaaggggggggcccccgagaaagaggacacgccagggccga gcgagaaggagaaaaaaaagggggggccacgcaacgcgaaccaacgcggacccaaaggcaagaccgaaaaggaga cccgcggaaggccgggaggcggacgagggaagcccaacggacaaaaaaaaaagggggggcaag gcaacagcggaaggaaaaaaaaaggggggggaccgcggcgcacccaaacagggggcgagcgcaa cggggcccgagaggcccccgaaaaaaaagggggggcaaagagagagcaacacgcggcggca aacgagaaaaaaaagggggggcggggcacaacaagaggagcccacagaagcgagacacaga ggcccaggcaaaagcccaacgacaaaggaagaagaaccgcaaaaaaaaagggggggaccgaaagggaag cgggagcaacgacagacacggcaagccgcccggggacaaagcacgaagcaaaaaaaaggggggga 29

30 Implaning Moif AAAAAAGGGGGGG wih Four Muaions agaccgggaacgaagaagaaagggggggcgacacaagaaaacgagaagacgagaccggcgccgccg accccagagcagaaggaccggaaaaaaaagagacaaaacccgaaacaaaaaacggcggga gagacccgggagacaaaaaaggaggggcccccgagaaagaggacacgccagggccga gcgagaaggagcaaaaaaagggagccacgcaacgcgaaccaacgcggacccaaaggcaagaccgaaaaggaga cccgcggaaggccgggaggcggacgagggaagcccaacggacaaaaaaaaggaagggcaag gcaacagcggaaggaaacaaaagggcgggaccgcggcgcacccaaacagggggcgagcgcaa cggggcccgagaggcccccgaaaacaaggagggccaaagagagagcaacacgcggcggca aacgagaaaaaaagggagcccggggcacaacaagaggagcccacagaagcgagacacaga ggcccaggcaaaagcccaacgacaaaggaagaagaaccgcaacaaaaaggagcggaccgaaagggaag cgggagcaacgacagacacggcaagccgcccggggacaaagcacgaagcacaaaaaggagcgga 30

31 Where is he Moif??? agaccgggaacgaagaagaaagggggggcgacacaagaaaacgagaagacgagaccggcgccgccg accccagagcagaaggaccggaaaaaaaagagacaaaacccgaaacaaaaaacggcggga gagacccgggagacaaaaaaggaggggcccccgagaaagaggacacgccagggccga gcgagaaggagcaaaaaaagggagccacgcaacgcgaaccaacgcggacccaaaggcaagaccgaaaaggaga cccgcggaaggccgggaggcggacgagggaagcccaacggacaaaaaaaaggaagggcaag gcaacagcggaaggaaacaaaagggcgggaccgcggcgcacccaaacagggggcgagcgcaa cggggcccgagaggcccccgaaaacaaggagggccaaagagagagcaacacgcggcggca aacgagaaaaaaagggagcccggggcacaacaagaggagcccacagaagcgagacacaga ggcccaggcaaaagcccaacgacaaaggaagaagaaccgcaacaaaaaggagcggaccgaaagggaag cgggagcaacgacagacacggcaagccgcccggggacaaagcacgaagcacaaaaaggagcgga 31

32 MEME Muliple EM for Moif Eliciaion [Bailey, Elkan ISMB 94] Goal: Given a se of sequences, find a moif (PWM) ha maximizes he expeced likelihood of he daa Technique: EM (Expecaion Maximizaion) (based on [Lawrence, Reilly 90]) GE Ron Shamir 32

33 GE Ron Shamir The Mixure Model Daa: X = (X 1,,X n ) : all (overlapping) l-mers in he inpu sequences Assume X i s were generaed by a wo-componen mixure model - θ = (θ 1, θ 2 ) : Model #1: θ 1 = moif model: f i,b = prob. of base b a pos i in moif, 1 i l Model #2: θ 2 = background (BG) model: f 0,b = prob. of base b Mixing parameer: λ = (λ 1, λ 2 ) λ j = prob. ha model #j is used (λ 1 +λ 2 =1) Assume independence beween l-mers 33

34 Log Likelihood Missing daa: Z = (Z 1,,Z n ) : Z i = (Z i1, Z i2 ); Z ij = 1 if X i from model #j ; 0 o/w Complee Likelihood of model given daa: L (θ, λ X, Z) = p (X, Z θ, λ) = Π i=1 n p (X i, Z i θ, λ) p (X i, Z i θ, λ) = p (X i Z i,θ, λ) p (Z i ) = = λ 1 p(x i θ 1 ) if Z i1 =1; λ 2 p(x i θ 2 ) if Z i2 =1 log L = Σ i=1 n Σ j=1,2 Z ij log (λ j p(x i θ j )) GE Ron Shamir 34

35 MEME: Algorihm Goal: Maximize E[log L] Ouline of EM algorihm: Choose saring θ, λ Repea unil convergence of θ: E-sep: Re-esimae Z from θ, λ, X M-sep: Re-esimae θ, λ from X, Z Repea all of he above for various θ, λ GE Ron Shamir 35

36 E-sep Compue expecaion of log L over Z: E[log L] = Σ i=1 n Σ j=1,2 Z ij log (λ j p(x i θ j )) where: Z ij = p(z ij =1 θ,λ,x i ) = = p(z ij =1, X i θ,λ) / p(x i θ,λ) = = p(z ij =1, X i θ,λ) / Σ k=1,2 p(z ik =1, X i θ,λ) = = λ j p(x i θ j ) / Σ k=1,2 λ k p(x i θ k ) GE Ron Shamir 36

37 M-sep Find θ,λ ha maximize E[log L]=Q(,, ): E[log L] = Σ i=1 n Σ j=1,2 Z ij log (λ j p(x i θ j )) Finding λ: Suffices o maximize L 1 = Σ i=1 n Σ j=1,2 Z ij log λ j λ 1 +λ 2 =1 L 1 = Σ i=1 n (Z i1 log λ 1 + Z i2 log (1-λ 1 )) dl 1 /dλ 1 = Σ i=1 n (Z i1 / λ 1 Z i2 / (1-λ 1 )) GE Ron Shamir 37

38 MEME: Algorihm M-sep (con.): dl 1 /dλ 1 = Σ i=1 n (Z i1 / λ 1 Z i2 / (1-λ 1 )) = 0 λ 1 Σ i=1 n Z i2 = (1-λ 1 ) Σ i=1 n Z i1 λ 1 ( Σ i=1 n (Z i1 +Z i2 ) ) = Σ i=1 n Z i1 λ 1 = ( Σ i=1 n Z i1 ) / n λ 2 = 1- λ 1 = ( Σ i=1 n Z i2 ) / n Finding θ: GE Ron Shamir 38

39 GE Ron Shamir 39

40 Tim Bailey, Charles Elkan Senior Research Fellow Insiue for Molecular Bioscience, Universiy of Queensland, Brisbane, Ausralia Professor Deparmen of Compuer Science and Engineering Universiy of California, San Diego GE Ron Shamir 40

41 FIN GE Ron Shamir 41