Hidden Markov Model Induction by Bayesian Model Merging

Transcription

1 To pper in: C. L. Giles, S. J. Hnson, & J. D. Cown, eds., Advnces in Neurl Informtion Processing Systems 5, Sn Mteo, CA, Morgn Kufmn, 1993 Hidden Mrkov Model Induction y Byesin Model Merging Andres Stolcke, Computer Science Division University of Cliforni Berkeley, CA stolcke@icsi.erkeley.edu Stephen Omohundro Interntionl Computer Science Institute 1947 Center Street, Suite 600 Berkeley, CA om@icsi.erkeley.edu Astrct This pper descries technique for lerning oth the numer of sttes nd the topology of Hidden Mrkov Models from exmples. The induction process strts with the most specific model consistent with the trining dt nd generlizes y successively merging sttes. Both the choice of sttes to merge nd the stopping criterion re guided y the Byesin posterior proility. We compre our lgorithm with the Bum-Welch method of estimting fixed-size models, nd find tht it cn induce miniml HMMs from dt in cses where fixed estimtion does not converge or requires redundnt prmeters to converge. 1 INTRODUCTION AND OVERVIEW Hidden Mrkov Models (HMMs) re well-studied pproch to the modelling of sequence dt. HMMs cn e viewed s stochstic generliztion of finite-stte utomt, where oth the trnsitions etween sttes nd the genertion of output symols re governed y proility distriutions. HMMs hve een importnt in speech recognition (Riner & Jung, 1986), cryptogrphy, nd more recently in other res such s protein clssifiction nd lignment (Hussler, Krogh, Min & Sjölnder, 1992; Bldi, Chuvin, Hunkpiller & McClure, 1993). Prctitioners hve typiclly chosen the HMM topology y hnd, so tht lerning the HMM from smple dt mens estimting only fixed numer of model prmeters. The stndrd pproch is to find mximum likelihood (ML) or mximum posteriori proility(map) estimte of the HMM prmeters. The Bum-Welch lgorithm uses dynmic progrmming

2 to pproximte these estimtes (Bum, Petrie, Soules & Weiss, 1970). A more generl prolem is to dditionlly find the est HMM topology. This includes oth the numer of sttes nd the connectivity (the non-zero trnsitions nd emissions). One could exhustively serch the model spce using the Bum-Welch lgorithm on fully connected models of vrying sizes, picking the model size nd topology with the highest posterior proility. (Mximum likelihood estimtion is not useful for this comprison since lrger models usully fit the dt etter.) This pproch is very costly nd Bum- Welch my get stuck t su-optiml locl mxim. Our comprtive results lter in the pper show tht this often occurs in prctice. The prolem cn e somewht llevited y smpling from severl initil conditions, ut t further increse in computtionl cost. The HMM induction method proposed in this pper tckles the structure lerning prolem in n incrementl wy. Rther thn estimting fixed-size model from scrtch for vrious sizes, the model size is djusted s new evidence rrives. There re two opposing tendencies in djusting the model size nd structure. Initilly new dt dds to the model size, ecuse the HMM hs to e ugmented to ccommodte the new smples. If enough dt of similr structure is ville, however, the lgorithm collpses the shred structure, decresing the model size. The merging of structure is lso wht drives generliztion, i.e., cretes HMMs tht generte dt not seen during trining. Beyond eing incrementl, our lgorithm is dt-driven, in tht the smples themselves completely determine the initil model shpe. Bum-Welch estimtion, y comprison, uses n initilly rndom set of prmeters for given-sized HMM nd itertively updtes them until point is found t which the smple likelihood is loclly mximl. Wht seems intuitively troulesome with this pproch is tht the initil model is completely uninformed y the dt. The smple dt directs the model formtion process only in n indirect mnner s the model pproches meningful shpe. 2 HIDDEN MARKOV MODELS For lck of spce we cnnot give full introduction to HMMs here; see Riner & Jung (1986) for detils. Briefly, n HMM consists of sttes nd trnsitions like Mrkov chin. In the discrete version considered here, it genertes strings y performing rndom wlks etween n initil nd finl stte, outputting symols t every stte in etween. The proility P(x M) tht model M genertes string x is determined y the conditionl proilities of mking trnsition from one stte to nother nd the proility of emitting ech symol from ech stte. Once these re given, the proility of prticulr pth through the model generting the string cn e computed s the product of ll trnsition nd emission proilities long the pth. The proility of string x is the sum of the proilities of ll pths generting x. For exmple, the model M 3 in Figure 1 genertes the strings,,, with proilities 2 3, 2 3 2, 2 3 3,, respectively. 3 HMM INDUCTION BY STATE MERGING 3.1 MODEL MERGING Omohundro (1992) hs proposed n pproch to sttisticl model inference in which initil

3 models simply replicte the dt nd generlize y similrity. As more dt is received, component models re fit from more complex model spces. This llows the formtion of ritrrily complex models without overfitting long the wy. The elementry step used in modifying the overll model is merging of su-models, collpsing the smple sets for the corresponding smple regions. The serch for su-models to merge is guided y n ttempt to scrifice s little of the smple likelihood s possile s result of the merging process. This serch cn e done very efficiently if () greedy serch strtegy cn e used, nd () likelihood computtions cn e done loclly for ech su-model nd don t require glol recomputtion on ech model updte. 3.2 STATE MERGING IN HMMS We hve pplied this generl pproch to the HMM lerning tsk. We descrie the lgorithm here mostly y presenting n exmple. The detils re ville in Stolcke & Omohundro (1993). To otin n initil model from the dt, we first construct n HMM which produces exctly the input strings. The strt stte hs s mny outgoing trnsitions s there re strings nd ech string is represented y unique pth with one stte per smple symol. The proility of entering these pths from the strt stte is uniformly distriuted. Within ech pth there is unique trnsition rc whose proility is 1. The emission proilities re 1 for ech stte to produce the corresponding symol. As n exmple, consider the regulr lnguge () + nd two smples drwn from it, the strings nd. The lgorithm constructs the initil model M 0 depicted in Figure 1. This is the most specific model ccounting for the oserved dt. It ssigns ech smple proility equl to its reltive frequency, nd is therefore mximum likelihood model for the dt. Lerning from the smple dt mens generlizing from it. This implies trding off model likelihood ginst some sort of is towrds simpler models, expressed y prior proility distriution over HMMs. Byesin nlysis provides forml sis for this trdeoff. Byes rule tells us tht the posterior model proility P(M x) is proportionl to the product of the model prior P(M) nd the likelihood of the dt P(x M). Smller or simpler models will hve higher prior nd this cn outweigh the drop in likelihood s long s the generliztion is conservtive nd keeps the model close to the dt. The choice of model priors is discussed in the next section. The fundmentl ide exploited here is tht the initil model M 0 cn e grdully trnsformed into the generting model y repetedly merging sttes. The intuition for this heuristic comes from the fct tht if we tke the pths tht generte the smples in n ctul generting HMM M nd unroll them to mke them completely disjoint, we otin M 0. The itertive merging process, then, is n ttempt to undo the unrolling, trcing serch through the model spce ck to the generting model. Merging two sttes q 1 nd q 2 in this context mens replcing q 1 nd q 2 y new stte r with trnsition distriution tht is weighted mixture of the trnsition proilities of q 1, q 2, nd with similr mixture distriution for the emissions. Trnsition proilities into q 1 or q 2 re dded up nd redirected to r. The weights used in forming the mixture distriutions re the reltive frequencies with which q 1 nd q 2 re visited in the current model. Repetedly performing such merging opertions yields sequence of models M 0, M 1,

4 M 0: F log L(x M 0) = M 1: log L(x M 1) = log L(x M 0) M 2: F log L(x M 2) = F M 3: F log L(x M 3) = log L(x M 2) Figure 1: Sequence of models otined y merging smples f, g. All trnsitions without specil nnottions hve proility 1; Output symols pper ove their respective sttes nd lso crry n implicit proility of 1. For ech model the log likelihood is given. M 2,, long which we cn serch for the MAP model. To mke the serch for M efficient, we use greedy strtegy: given M i, choose pir of sttes for merging tht mximizes P(M i+1 X). Continuing with the previous exmple, we find tht sttes 1 nd 3 in M 0 cn e merged without penlizing the likelihood. This is ecuse they hve identicl outputs nd the loss due to merging the outgoing trnsitions is compensted y the merging of the incoming trnsitions. The.5/.5 split is simply trnsferred to outgoing trnsitions of the merged stte. The sme sitution otins for sttes 2 nd 4 once 1 nd 3 re merged. From these two first merges we get model M 1 in Figure 1. By convention we reuse the smller of two stte indices to denote the merged stte. At this point the est merge turns out to e etween sttes 2 nd 6, giving model M 2. However, there is penlty in likelihood, which decreses to out.59 of its previous vlue. Under ll the resonle priors we considered (see elow), the posterior model proility still increses due to n increse in the prior. Note tht the trnsition proility rtio t stte 2 is now 2/1, since two smples mke use of the first trnsition, wheres only one tkes the second trnsition. Finlly, sttes 1 nd 5 cn e merged without penlty to give M 3, the miniml model tht genertes () +. Further merging t this point would reduce the likelihood y three orders of mgnitude. The resulting decrese in the posterior proility tells the lgorithm to stop

5 t this point. 3.3 MODEL PRIORS As noted previously, the likelihoods P(X M i ) long the sequence of models considered y the lgorithm is monotoniclly decresing. The prior P(M) must ccount for n overll increse in posterior proility, nd is therefore the driving force ehind generliztion. As in the work on Byesin lerning of clssifiction trees y Buntine (1992), we cn split the prior P(M) into term ccounting for the model structure, P(M s ), nd term for the djustle prmeters in fixed structure P(M p M s ). We initilly relied on the structurl prior only, incorporting n explicit is towrds smller models. Size here is some function of the numer of sttes nd/or trnsitions, M. Such prior cn e otined y mking P(M s )/e M, nd cn e viewed s description length prior tht penlizes models ccording to their coding length (Rissnen, 1983; Wllce & Freemn, 1987). The constnts in this MDL term hd to e djusted y hnd from exmples of desirle generliztion. For the prmeter prior P(M p M s ), it is stndrd prctice to pply some sort of smoothing or regulrizing prior to void overfitting the model prmeters. Since oth the trnsition nd the emission proilities re given y multinomil distriutions it is nturl to use Dirichlet conjugte prior in this cse (Berger, 1985). The effect of this prior is equivlent to hving numer of virtul smples for ech of the possile trnsitions nd emissions which re dded to the ctul smples when it comes to estimting the most likely prmeter settings. In our cse, the virtul smples mde equl use of ll potentil trnsitions nd emissions, dding is towrds uniform trnsition nd emission proilities. We found tht the Dirichlet priors y themselves produce n implicit is towrds smller models, phenomenon tht cn e explined s follows. The prior lone results in model with uniform, flt distriutions. Adding ctul smples hs the effect of putting umps into the posterior distriutions, so s to fit the dt. The more smples re ville, the more peked the posteriors will get round the mximum likelihood estimtes of the prmeters, incresing the MAP vlue. In estimting HMM prmeters, wht counts is not the totl numer of smples, ut the numer of smples per stte, since trnsition nd emission distriutions re locl to ech stte. As we merge sttes, the ville evidence gets shred y fewer sttes, thus llowing the remining sttes to produce etter fit to the dt. This phenomenon is similr, ut not identicl, to the Byesin Occm fctors tht prefer models with fewer prmeter (McKy, 1992). Occm fctors re result of integrting the posterior over the prmeter spce, something which we do not do ecuse of the computtionl complictions it introduces in HMMs (see elow). 3.4 APPROXIMATIONS At ech itertion step, our lgorithm evlutes the posterior resulting from every possile merge in the current HMM. To keep this procedure fesile, numer of pproximtions re incorported in the implementtion tht don t seem to ffect its qulittive properties. For the purpose of likelihood computtion, we consider only the most likely pth through the model for given smple string (the Viteri pth). This llows us to

6 express the likelihood in product form, computle from sufficient sttistics for ech trnsition nd emission. We ssume the Viteri pths re preserved y the merging opertion, tht is, the pths previously pssing through the merged sttes now go through the resulting new stte. This llows us to updte the sufficient sttistics incrementlly, nd mens only O(numer of sttes) likelihood terms need to e recomputed. The posterior proility of the model structure is pproximted y the posterior of the MAP estimtes for the model prmeters. Rigorously integrting over ll prmeter vlues is not fesile since vrying even single prmeter could chnge the pths of ll smples through the HMM. Finlly, it hs to e kept in mind tht our serch procedure long the sequence of merged models finds only locl optim, since we stop s soon s the posterior strts to decrese. A full serch of the spce would e much more costly. However, we found est-first look-hed strtegy to e sufficient in rre cses where locl mximum cused prolem. In those cses we continue merging long the est-first pth for fixed numer of steps (typiclly one) to check whether the posterior hs undergone just temporry decrese. 4 EXPERIMENTS We hve used vrious rtificil finite-stte lnguges to test our lgorithm nd compre its performnce to the stndrd Bum-Welch lgorithm. Tle 1 summrizes the results on the two smple lnguges c c nd The first of these contins contingency etween initil nd finl symols tht cn e hrd for lerning lgorithms to uncover. We used no explicit model size prior in our experiments fter we found tht the Dirichlet prior ws very roust in giving just the the right mount of is towrd smller models. 1 Summrizing the results, we found tht merging very relily found the generting model structure from very smll numer of smples. The prmeter vlues re determined y the smple set sttistics. The Bum-Welch lgorithm, much like ckpropgtion network, my e sensitive to its rndom initil prmeter settings. We therefore smpled from numer of initil conditions. Interestingly, we found tht Bum-Welch hs good chnce of settling into suoptiml HMM structure, especilly if the numer of sttes is the miniml numer required for the trget lnguge. It proved much esier to estimte correct lnguge models when extr sttes were provided. Also, incresing the smple size helped it converge to the trget model. 5 RELATED WORK Our pproch is relted to severl other pproches in the literture. The concept of stte merging is implicit in the notion of stte equivlence clsses, which is fundmentl to much of utomt theory (Hopcroft & Ullmn, 1979) nd hs een pplied 1 The numer of virtul smples per trnsition/emission ws held constnt t 0.1 throughout.

7 () Method Smple Entropy Cross-entropy Lnguge n Merging 8 m.p ±.020 c c 6 Merging 20 rndom ±.033 c c 6 Bum-Welch 8 m.p ±.023 (est) ( )c ( ) 6 (10 trils) ±.228 (worst) ( )c ( ) 6 Bum-Welch 20 rndom ±.031 (est) c c 6 (10 trils) ±.031 (worst) ( )c ( ) 6 Bum-Welch 8 m.p ±.271 c c 10 Bum-Welch 20 rndom ±.032 c c 10 () Method Smple Entropy Cross-entropy Lnguge n Merging 5 m.p ± Bum-Welch 5 m.p ±.161 (est) ( + + ) + 4 (3 trils) ±.007 (worst) ( + + ) + 4 Merging 10 rndom ± Bum-Welch 10 rndom ±.076 (est) (3 trils) ±.137 (worst) ( + + ) + 4 Tle 1: Results for merging nd Bum-Welch on two regulr lnguges: () c c nd () Smples were either the top most prole (m.p.) ones from the trget lnguge, or set of rndomly generted ones. Entropy is the verge negtive log proility on the trining set, wheres cross-entropy refers to the empiricl cross-entropy etween the induced model nd the generting model (the lower, the etter generliztion). n denotes the finl numer of model sttes for merging, or the fixed model size for Bum-Welch. For Bum-Welch, oth est nd worst performnce over severl initil conditions is listed. to utomt lerning s well (Angluin & Smith, 1983). Tomit (1982) is n exmple of finite-stte model spce serch guided y (nonproilistic) goodness mesure. Horning (1969) descries Byesin grmmr induction procedure tht serches the model spce exhustively for the MAP model. The procedure provly finds the glolly optiml grmmr in finite time, ut is infesile in prctice ecuse of its enumertive chrcter. The incrementl ugmenttion of the HMM y merging in new smples hs some of the flvor of the lgorithm used y Port & Feldmn (1991) to induce finite-stte model from positive-only, ordered exmples. Hussler et l. (1992) use limited HMM surgery (insertions nd deletions in liner HMM) to djust the model size to the dt, while keeping the topology unchnged. 6 FURTHER RESEARCH We re investigting severl rel-world pplictions for our method. One tsk is the construction of unified multiple-pronuncition word models for speech recognition. This is currently eing crried out in collortion with Chuck Wooters t ICSI, nd it ppers tht our merging lgorithm is le to produce linguisticlly dequte phonetic models. Another direction involves n extension of the model spce to stochstic context-free grmmrs, for which stndrd estimtion method nlogous to Bum-Welch exists (Lri

8 & Young, 1990). The notions of smple incorportion nd merging crry over to this domin (with merging now involving the non-terminls of the CFG), ut need to e complemented with mechnism tht dds new non-terminls to crete hierrchicl structure (which we cll chunking). Acknowledgements We would like to thnk Peter Cheesemn, Wry Buntine, Dvid Stoutmire, nd Jerry Feldmn for helpful discussions of the issues in this pper. References Angluin, D. & Smith, C. H. (1983), Inductive inference: Theory nd methods, ACM Computing Surveys 15(3), Bldi, P., Chuvin, Y., Hunkpiller, T. & McClure, M. A. (1993), Hidden Mrkov Models in moleculr iology: New lgorithms nd pplictions, this volume. Bum, L. E., Petrie, T., Soules, G. & Weiss, N. (1970), A mximiztion technique occuring in the sttisticl nlysis of proilistic functions in Mrkov chins, The Annls of Mthemticl Sttistics 41(1), Berger, J. O. (1985), Sttisticl Decision Theory nd Byesin Anlysis, Springer Verlg, New York. Buntine, W. (1992), Lerning clssifiction trees, in D. J. Hnd, ed., Artificil Intelligence Frontiers in Sttistics: AI nd Sttistics III, Chpmn & Hll. Hussler, D., Krogh, A., Min, I. S. & Sjölnder, K. (1992), Protein modeling using hidden Mrkov models: Anlysis of gloins, Technicl Report UCSC-CRL-92-23, Computer nd Informtion Sciences, University of Cliforni, Snt Cruz, C. Revised Sept Hopcroft, J. E. & Ullmn, J. D. (1979), Introduction to Automt Theory, Lnguges, nd Computtion, Addison-Wesley, Reding, Mss. Horning, J. J. (1969), A study of grmmticl inference, Technicl Report CS 139, Computer Science Deprtment, Stnford University, Stnford, C. Lri, K. & Young, S. J. (1990), The estimtion of stochstic context-free grmmrs using the Inside-Outside lgorithm, Computer Speech nd Lnguge 4, McKy, D. J. C. (1992), Byesin interpoltion, Neurl Computtion 4, Omohundro, S. M. (1992), Best-first model merging for dynmic lerning nd recognition, Technicl Report TR , Interntionl Computer Science Institute, Berkeley, C. Port, S. & Feldmn, J. A. (1991), Lerning utomt from ordered exmples, Mchine Lerning 7, Riner, L. R. & Jung, B. H. (1986), An introduction to Hidden Mrkov Models, IEEE ASSP Mgzine 3(1), Rissnen, J. (1983), A universl prior for integers nd estimtion y minimum description length, The Annls of Sttistics 11(2), Stolcke, A. & Omohundro, S. (1993), Best-first model merging for Hidden Mrkov Model induction, Technicl Report TR , Interntionl Computer Science Institute, Berkeley, C. Tomit, M. (1982), Dynmic construction of finite utomt from exmples using hill-climing, in Proceedings of the 4th Annul Conference of the Cognitive Science Society, Ann Aror, Mich., pp Wllce, C. S. & Freemn, P. R. (1987), Estimtion nd inference y compct coding, Journl of the Royl Sttisticl Society, Series B 49(3),