Hidden Markov Models. Sequence to Sequence maps. Markov model. Hidden Markov model. Basic Definitions. Viterbi Decoding

Transcription

1 Outline Basic Definitions Martin Emms August 17, 2007 Empirical outcomes Baum-Welch and POS tagging Supervised learning and higher order models Outline Basic Definitions Sequence to Sequence maps examples of sequence to sequence maps in language processing speech recognition sequence of acoustic data sequence of words OCR sequence of optical data sequence of letters POS tagging sequence of words sequence of POS tags Named Entity Rec sequence of words sequence of entity classes Inf Ext sequence of words seq of template fillers Markov model set of states a probabilistic transition relation between states initial-state probability assignment Hidden Markov model additional probabilisitic output function: observation symbols are output from a state are a technique which has been applied to such maps

2 Basic Definitions Basic Definitions An example one wants tries a the pause In this example states are part-of-speech tags observation symbols are words P(tag now is tag prev was ) = 05 P(word now is wants tag now is ) = 0002 Some notation hidden states represented by integers 1 N observation symbols represented by integers 1 M S: sequence of hidden states O: sequence of observation symbols π: assigns to i the prob of starting in i A: assigns to i, j the prob P(s t = j s t 1 = i), abbreviated a ij B: assigns to i, k the prob P(o t = k s t = i), abbreviated b i (k) an HMM is specified by the triple (π, A, B) Basic Definitions Basic Definitions Joint state+observation probability Decoding: most probable state sequence An HMM fundamentally assigns a joint probability to a state+observation sequence: P(O 1,T, S 1,T ) = P(s 1 )P(o 1 s 1 ) re-expressed with π, A, B: P(O 1,T, S 1,T ) = π(s 1 )b s1 (o 1 ) TY P(o t s t )P(s t s t 1 ) t=2 TY b st (o t )a st 1 s t t=2 decode(o) = most probable hidden sequence S for O decode(o 1,T ) = arg max S 1,T P(S 1,T O 1,T ) (1) = arg max S 1,T P(S 1,T, O 1,T )/P(O 1,T ) (2) = arg max S 1,T P(S 1,T, O 1,T ) (3)

3 Basic Definitions Observation probability Decoding want to find most-probable hidden state sequence for a given sequence of visible observations Relevant to unsupervised training is the fact that the model assigns a probability to an observation sequence, by summing over all possible state sequences P(O 1,T ) = X all S 1,T P(O 1,T, S 1,T ) (4) decode(o 1,T ) = arg max S 1,T [(π(s 1 )b s1 (o 1 ) N possible states N T possible state sequences for O 1,T TY b st (o t )a st 1 s t )] not computationally feasible to simply enumerate the possible state sequences Viterbi algorithm is an efficient, dynamic programming method for avoiding this t=2 Decoding Viterbi in words best_path(t, i): best path through the HMM which accounts for the first t obs symbols and ends with state i abs_best_path(t): best path through the HMM which accounts for the first t obs symbols abs_best_path(t) is what you eventually want, but since clearly best_path(t, i) suffices abs_best_path(t) = max best_path(t, i) 1 i N best_path(t,) is easy to calculate from best_path(t 1, ) For a given state j at time t consider every possible immediate predecessor i for j and compare best_path(t 1, i) a ij b j (o t ) take the maximum

4 Viterbi Pseudo code Initialisation: for ( i = 1 ; i N ; i ++) { best_path(1, i) = π(i)b i (o 1 ) ; } Iteration: for ( t = 2 ; t T ; t ++) { for ( j = 1 ; j N ; j ++) { max = 0 ; for ( i = 1 ; i N ; i ++) { p = best_path(t 1, i)prob a ij b i (o t ) ; i f ( p > max ) { max = p ; prev_state = i ; } } best_path(t, j)prob = max ; best_path(t, j)prev_state = prev_state } } The cost of this algorithm will be of the order of N 2 T, compared to the brute force cost N T A trellis assign parts of the equation to each arc in this trellis A trellis A trellis (001)(001) (5)(002) (5) (45) (5) (002) (004)(001) (001) (001) (0001)(002) (5)(002) (45) (0002) (001) (0002) (5) (45) (01) (45) (5) (002) (01)(001) (01) (001) π() P(one ) x P( ) P(a ) x P( )P(wants ) x P( ) P(pause ) problem is now to find the best path through this trellis where total path prob is product of segments

5 Viterbi initialisation best_path to wants/ 4e 6 (004)(001) 4e 6 (0001)(002) (5)(002) (001)(001) 1e 6 best_path(1,) = 4e 6 best_path(1,) = 1e 6 INITIALISATION 1e 6 for best_path(2,) compare (4e 6)(2e 7) = 8e 13 (1e 6)(1e 3) = 1e 9 winner! best_path to wants/ best_path(2,) finished 4e 6 1e 9 1e 6 (45) (0002) (0001)(0002) for best_path(2,) compare (4e 6)(9e 5) = 36e 10 winner! (1e 6)(2e 8) = 2e 14 36e 10 best_path(2,) = 1e 9 best_path(2,) = 36e 10

6 best_path to a/ best_path(3,) finished 1e 9 36e 10 (5) (45) (01) (45) 225e 10 for best_path(3,) compare (1e 9)(225e 1) = 225e 10 (36e 10)(45e 3) = 16e 12 (winner!) best_path(3,) = 225e 10 best_path to pause/ best_path to pause/ 225e 10 (5) (002) 225e 10 (01)(001) best_path(4,) = (225e 10)(1e 3) = 225e 13 best_path(4,) = (225e 10)(1e 5) = 225e 15

7 best_path to pause/ best_path[4] finished 225e e e 15 (01) (001) 225e 13 best_path(4,) = (225e 10)(1e 5) = 225e 15 best_path(4,) = = 225e 13 best_path(4,) = = 225e 15 best_path(4,) = = 225e 15 (final max) Recall from Lec1: Supervised case for Seq-to-Seq Recall from Lec1: Unsupervised case for Seq-to-Seq supervised case: learn from f: sequence over vocab Σ sequence over (Σ labels C) 8 >< lots and lots of this: >: last/jj week/nn IBM/NNP bought/vbd Lotus/NNP ie training data is (equivalent to) large corpus of input/output pairs unsupervised case: learn from f: sequence over vocab Σ sequence over (Σ labels C) 8 >< lots and lots of this: >: last week IBM bought Lotus ie training data is (equivalent to) large corpus of possible inputs

8 Baum-Welch Unsupervised HMM training Recall Lec 3: K-means clustering Baum-Welch re-estimation: given an obs sequence O, an iterative method which refines parameters (π, A, B) to maximise P(O) an instance of Expectation-Maximisation learning Assignment using current means f 1 f k, generate clusters c 1 c k by assigning each point to its nearest mean Update assuming this categorisation, update means by taking centroids of the clusters this make a better fit of the cluster centres to the cluster members go back to Assignment The Baum-Welch HMM-training algorithm has a similar structure Both it and K-means are instances of Expectation-Maximisation algorithms in K-means the assignment phase corresponds to the Expectation phase, and update to the Maximisation phase Baum-Welch Properties Baum-Welch (brute-force version) Expectations using current parameters (π, A, B), consider P(path O) path 1 P(path 1 O) path n P(path n O) use the path probs to generate expected counts E(X) for an event X E(X) = X path count(x in path) P(path O) X might be an ij transition thus get a distribution for different types of event Maximisation generate new (ˆpi, Â, ˆB) from expectations eg from E(ij), E(i), re-estimate a ij : â ij = E(ij) E(i) this maximises the likelihood of the distribution just obtained now go back to the E phase Baum-Welch in outline is: u n t i l a fixed p o i n t { enumerate a l l paths use c u r r e n t (π i, A i, B i ) to generate expected counts from expected counts generate new (π i+1, A i+1, B i+1 ) } observations O get likelier: if iteration changes parameters from λ to λ : P λ (O) P λ (O) Caveat One: if O is short, Baum-Welch will give poor performance on unseen data the classic over-fitting problem Caveat Two: there may be many local maxima, so there is no guarantee that the re-estimation will converge to the best model Caveat Three: the re-estimation increases P(O), but you really want to increase accuracy(decode(o)) Caveat Four: prohibitively expensive to compute expectations via an emumeration of possible paths this is what Baum-Welch cleverly avoids

9 worked example of the (brute-force) re-estimation Suppose tagged corpus the/at wins/n stopped/v /STOP the/at cup/n wins/n /STOP he/pro wins/v the/at cup/n /STOP (= tags0) (= tags2) (= tags3) initialising π 0, A 0 and B 0 with relative frequences in this corpus: π 0 P(s 1 = AT) = 066 P(s 1 = PRO) = 033 A 0 P(N AT) = 1 P(V N) = 025 P(N N) = 025 P(STOP N) = 05 P(V PRO) = 1 P(STOP V) = 05 P(AT V) = 05 B 0 P(the AT) = 1 P(wins N) = 05 P(cup N) = 05 P(he PRO) = 1 P( STOP) = 1 P(stopped V) = 05 P(wins V) = 05 Tagging the words with these probs observations possible taggings O 0 the wins stopped the/at wins/n stopped/v /STOP (= tags0) O 1 the cup wins the/at cup/n wins/v /STOP (= tags1) the/at cup/n wins/n /STOP (= tags2) O 2 he wins the cup he/pro wins/v the/at cup/n /STOP (= tags3) O 0 and O 2 get 1 tagging O 1 gets two which wins between tags1 and tags2? the ratio P(tags1) and P(tags2) P(tags1) P(V N)P(wins V)P(STOP V) = P(tags2) P(N N)P(wins N)P(STOP N) relevant parts of A 0 and B 0 P(V N) = 025 P(wins V) = 05 P(STOP V) = 05 P(N N) = 025 P(wins N) = 05 P(STOP N) = 05 hence 2 taggings of the cup wins are equi-probable A training iteration Now do training on a corpus which has 1 the wins stopped + 1 the cup wins + 10 he wins the cup compute expected frequencies of tags with P(tags obs) freq(obs) the complete conditional probs of each tagging, given each observation are: P(tags0) = P(obs0) = P(tags0 obs0) = 1 P(tags1) = P(obs1) = P(tags1 obs1) = 05 P(tags2) = P(obs1) = P(tags2 obs1) = 05 P(tags3) = P(obs2) = P(tags0 obs2) = 1 observations expected taggings O 0 the wins stopped 1 the/at wins/n stopped/v /STOP O 1 the cup wins 05 the/at cup/n wins/v /STOP 05 the/at cup/n wins/n /STOP O 2 he wins the cup 10 he/pro wins/v the/at cup/n /STOP derive expected event counts and new probabilities (π 1, A 1, B 1 ): A 1 E(N, V) = 15 E(N) = 125 P(V N) = 15/125 E(N, N) = 05 P(N N) = 05/125 E(N, STOP) = 105 P(STOP N) = 105/125 E(V, STOP) = 15 E(V) = 115 P(STOP V) = 15/115 E(V, AT) = 10 B 1 E(N, wins) = 15 E(N) = 125 P(wins N) = 15/125 E(N, cup) = 11 E(V, stopped) = 1 E(V) = 115 E(V, wins) = 105 P(wins V) = 105/115

10 after one iteration Avoiding the brute force cost With these numbers, A 1 P(V N) = 15/125 P(N N) = 05/125 P(STOP N) = 105/125 P(STOP V) = 15/115 B 1 P(wins N) = 15/125 P(wins V) = 105/115 P(the/AT cup/n wins/v /STOP) 15/ /115 15/115 = P(the/AT cup/n wins/n /STOP) 05/125 15/ /125 = so P(the/AT cup/n wins/v /STOP) > P(the/AT cup/n wins/n /STOP) and further iterations will magnify this trend recall that it would exponentially costly to enumerate all possible paths on each iteration strategy: define various clock-tick probabilities, and derive expectations by summing over all times α t (i) = probability of being in state i at time t and emitting the observation symbols O 1,t = P(o 1 o t, s t = i) β t (i) = the probability of emitting observation symbols O t+1,t, given being in state i at time t = P(o t+1 o T s t = i) α t (i)β t (i) = the probability emitting observations symbols O 1,T and being in state i at time t = P(o 1 o t, s t = i, o t+1 o T ) real Baum-Welch: summing the clock-tick probs α and β α(t, i) β(t, i) α t (i) and β t (i) can be recursively calculated in Viterbi-esque style expected counts can be calculated by summations over terms involving α and β i i occupation γ t (i) = the probability of state i at t given O = α t (i)β t (i)/p(o) o 1 α(t, i) o t β(t, i) t o t+1 o T transition ξ t (i, j) = the probability of transition ij at t given O = [α t 1 (i) a ij b j (o t ) β t (j)]/p(o) i net observation P(O) = P i α t(i)β t (i) (any t can be chosen) so re-estimations o 1 o t o t+1 o T α t (i) = probability of being in state i at time t and emitting â ij = E(ij) E(i) = t=2 ξ t(i, j) t=1 γ t(i) ˆb j (k) = E(jk) E(j) = t=1 o t =kγ t (j) t=1 γ t(j)

11 transition ξ t (i, j) summing the clock-tick transition probs sum over t = expectation of i to j given obs α(t 1, i) β(t, j) ξ(t i j) ξ(t i j) ξ (t i j) i A[i j] j i j i j i j o1 B[j o t ] o t 1 o t o t+1 o T transition ξ t (i, j) = the probability of transition ij at t given O = [α t 1 (i) a ij b j (o t ) β t (j)]/p(o) E(ij) = t=2 ξ t(i, j) so â ij = E(ij) E(i) = t t t t=2 ξ t(i, j) t=1 γ t(i) summing the clock-tick occupation probs multiple observations sum over t where obs is k = expectation of ob k with state j given obs for single observation sequence, in occupation and transition formulae: γ(t j k) γ(t j k) γ (t j k) γ t (i) = α t (i)β t (i)/p(o) ξ t (i, j) = α t 1 (i)a ij b j (o t )β t (j)/p(o) j j o at t = k o at t = k o at t = k E(ob is k at j) = t=1 o t =kγ t (j) so ˆb j (k) = E(jk) E(j) = t=1 o t =kγ t (j) t=1 γ t(j) j P(O) denominators not strictly necessary, as ratios taken in re-estimation but for multiple observations sequences, O 1 O Q, expectations summed, and each P(O q ) matters â ij = ˆb j (k) = P q E P Oq (ij) Pq E Oq (i) = q P P q E Oq (ik) Pq E Oq (i) = t=2 ξoq t (i, j) q t=1 γoq t (i) P q t=1 o t =kγt Oq (j) P t=1 γoq t (j) q

12 Empirical Outcomes Empirical outcomes Baum-Welch and POS tagging HMMs and Baum-Welch re-estimation have achieved most notable successes in Speech-Recogition We ll give some results for the application to POS tagging so-called full lexicon standard initialisation for unsupervised tagger training A: all a ij equiprobable B: b i (w) = 0, for any (w, i) not in dictionary/annotated corpus otherwise equiprobable training testing initial error final error Merialdo k 2k 23% 13% Wang and Shuurmans k 45k 34% 187% More on empirical outcomes for POS tagging if starting from a low performing initial state, Baum-Welch helps but if starting from a high performing initial state, obtained using counts from a large annotated corpus, Baum-Welch hurts (Elworthy 94) so Baum-Welch increases accuracy ablated lexicon: if b i (w) set to 0 if in corpus P(i w) < 01, error rate at end is 62% Supervised learning and higher order models Supervised learning and higher order models Supervised HMM training Brants TnT tagger supervised training of HMMs: some direct evidence for events involving the hidden states is involved evidence purely A, the hidden state sequences: in the application of HMMs to speech recognition, this might take the form simply of text, providing evidence for word-to-word transitions For POS tagging, evidence is typically a POS-tagged corpus, providing direct evidence thereby for A (tag-to-tag) and B (tag-to-word) Brants(2000) uses the Penn Tree Bank for training The equation underlying the model conditions tag on 2 preceding tags P(O 1,T, S 1,T ) = P(s 1 )P(o 1 s 1 ) P(s 2 s 1 )P(o 2 s 2 ) Q T t=3 P(o t s t )P(s t s t 2 s t 1 ) achieves 967% accuracy this performance is typical of that attained by a number of people with broadly similar approaches

13 Interpolation even with a large corpus, the real probabilities are misrepresented by the observed relative frequencies: data sparsity unseen word: w not seen in the training data: simple counting sets the output B probabilities P(w i) = 0 unseen POS trigram: xyz not seen in the training data: simple counting sets the transition A probabilities P(z xy) = 0 smoothing and interpolation are techniques to combat data sparsity smoothing: methods which raise the probability of the apparently impossible, while inevitably lowering the probability of the apparently frequent interpolation: methods which basically combine models which use successively less conditioning for transition probabilities, use a weighted combination of estimates based on less and less conditioning information (Brants TnT tagger does this) A(xyz) = λ 0ˆP0 (z xy) raw trigram estimate +λ 1ˆP1 (z y) raw bigram estimate +λ 2ˆP2 (z) raw unigram estimate λs sum to 1 the effect for 0-count xyz is A[xyz] = λ 1ˆP1 (z y) + λ 2ˆP2 (z) Setting weights for linear interpolation many variants: one method (Jelinek and Mercer 1980) for setting the λs, uses a variant of Baum-Welch to learn λs from annotated data blow-up each transition of a visible trigram Markov model into a 3 alternative transitions via hidden states: (w)x (w)x (w)x (x)y (x)y (x)y (y)z (y)z (y)z (x)y λ 2 λ 0 λ 0 λ 1 yλ 1 (x)yλ 2 fix ˆP 0 (z xy), ˆP 1 (z y), ˆP 2 (z) from one portion of data use Baum-Welch on other portion to restimate λ 0, λ 1, λ 2 P(z xy) P(z y) (y)z P(z) Jelinek-Mercer EM method when you work out the corresponding versions of ξ for this setting, and sum over all times it all simplifies out to expectation c 2 = E(λ 2 ) = X C 2 (xyz)λ 2 ˆP2 (z) λ allxyz 0ˆP0 (z xy) + λ 1ˆP1 (z y) + λ 2ˆP2 (z) c 1 = E(λ 1 ) = X C 2 (xyz)λ 1 ˆP1 (z y) λ allxyz 0ˆP0 (z xy) + λ 1ˆP1 (z y) + λ 2ˆP2 (z) c 0 = E(λ 0 ) = X C 2 (xyz)λ 0 ˆP0 (z xy) allxyz λ 0ˆP0 (z xy) + λ 1ˆP1 (z y) + λ 2ˆP2 (z) re-estimation ˆλ 2 = c 2 c 2 + c 1 + c 0 ˆλ1 = go back to expectation c 1 c 2 + c 1 + c 0 ˆλ0 = c 0 c 2 + c 1 + c 0

14 Leave-one-out method Further variants take each occurrence of xyz in turn based on the corpus with the occurrence xyz deleted if ˆP 0 (z xy) is max, c o = c o + 1 if ˆP 1 (z y) is max, c 1 = c if ˆP 2 (z)) is max, c 2 = c finally set ˆλ 2 = c 2 c 2 + c 1 + c 0 ˆλ 1 = c 1 c 2 + c 1 + c 0 ˆλ 0 = This is the method used in the TnT tagger c 0 c 2 + c 1 + c 0 versions where λs depend on the conditioning event Witten-Bell in next lecture Backoff Discounting don t take non-0 count C(X) at face-value also use less and less specific events, but in different way if C(X) 0, just use that if C(X) = 0, back-off to less specific X interwoven with the idea of discounting discount to C to leave aside an expected count for all the unseen events, treated as a single class 1 distribute expected count for all unseen events over particular unseen X according to probs of subevents X of X for example P back (y x) = j C (xy)/c(x) if C(xy) > 0 α(x)c(y)/n otherwise where α(x) is set to make sure that the probabilities sum to 1 1 add-one can be seen as simple version of this

15 Good-Turing Discounting Performance of Good-Turing if r = C(X), and n r is the number of events also with frequency r, the Good-Turing value for C (X) is C (X) = (r + 1) n r+1 n r sum the probs based on discounted C do not get 1 prob that the next observation will be novel is estimated to be P(novel) = n 1 /N r n r f emp f GT 1 2, 018, , , , , million word corpus split into C 1 and C 2 for bigrams which had C 1 -frequency of r, f emp (r) gives average C 2 -frequency f GT : Good-Turing estimate of the same thing Wrapping up See also HMMs and the Machine Learning of them have been applied to wide variety of Seq-to-Seq maps speech recognition sequence of acoustic signal data sequence of words OCR sequence of optical signal data sequence of letters POS tagging sequence of words sequence of POS tags Named Entity Rec sequence of words sequence of entity classe NP chunking sequence of words sequence identifying NPs Inf Ext sequence of words seq of template fillers Expectation/Maximisation algorithms have been widely applied to unsupervised learning Smoothing techniques to combine specific and unspecific are important HMMs are a joint probability models of input+annotation combined there is also a lot of work on discriminative models of P(annotation input) Maximum-Entropy methods Conditional Random Fields