Nonlinear Parametric Hidden Markov Models

Transcription

1 M.I.T. Media Laboraory Percepual Compuing Secion Technical Repor No. Nonlinear Parameric Hidden Markov Models Andrew D. Wilson Aaron F. Bobick Vision and Modeling Group MIT Media Laboraory Ames S., Cambridge, MA 39 (drew, bobickmedia.mi.edu) Absrac In previous work [], we exended he hidden Markov model (HMM) framework o incorporae a global parameric variaion in he oupu probabiliies of he saes of he HMM. Developmen of he parameric HMM was moivaed by he ask of simulaneoiusly recognizing and inerpreing gesures ha exhibi meaningful variaion. Wih sandard HMMs, such global variaion confounds he recogniion process. In his paper we exend he parameric HMM approach o handle nonlinear (non-analyic) dependencies of he oupu disribuions on he parameer of ineres. We show a generalized expecaion-maximizaion (GEM) algorihm for raining he parameric HMM and a GEM algorihm o simulaneously recognize he gesure and esimae he value of he parameer. We presen resuls on a poining gesure, where he nonlinear approach permis he naural azimuh/elevaion parameerizaion of poining direcion. Inroducion In [] we inroduce parameric hidden Markov models (HMMs) as a echnique o simulaneously recognize and inerpre parameric gesure. By parameric gesure we mean gesures ha exhibi a meaningful variaion; an example is a poin gesure where he imporan parameer is direcion. A poin gesure is hen paramerized by wo values: he Caresian coordinaes ha indicae direcion. Alernaively, direcion can be specified by spherical coordinaes. We refer he reader o [] for a deailed moivaion of he parameeric HMM approach as i relaes o gesure recogniion and inerpreaion. We briefly menion here ha wihou resoring o manual inkering wih he feaure space, a sandard dynamic ime warping (DTW) or HMM approach o he recogniion of parameric gesures faces he difficuly ha he gesure can only be recognized once he value of he parameers ha deermine he form of he gesure are recovered, and likewise he process of recovering he parameers necessarily involves recognizing he gesure. Parameric HMMs exend he sandard HMM model o include a global parameric variaion in he oupu of he HMM saes. In [] a linear model was used o model he parameric variaion a each sae of he HMM. Using he linear model, we formulaed an expecaion-maximizaion (EM) mehod for raining he parameric HMM. During esing, he parameric HMM simulaneously recognizes he gesure and esimaes he quanifying parameers, also by an EM procedure. In his paper he parameric HMM approach is exended o handle siuaions in which he dependence of he sae oupu disibuions on he parameers is no linear. Nonlinear parameric HMMs accordingly model he dependence using a single 3-layer logisic neural nework a each sae. Before presening nonlinear parameric HMMs in full we reierae he mahemaical developemen of linear parameeric HMMs. Linear parameeric hidden Markov models. Model Parameric HMMs model he dependence on he parameer of ineres explicily. We begin wih he usual HMM formulaion [3] and change he form of he oupu probabiliy disribuion (usually a normal disribuion or a mixure model) o depend on he parameer, a vecor quaniy. In he sandard coninuous HMM model, a sequence is represened by movemen hrough a se of hidden saes. The Markovian propery is encoded in a se of ransiion probabiliies, wih a i = P(q = q ; = i) being he probabiliy of moving o sae a ime given he sysem was in sae i a ime ;. Associaed wih each sae is an oupu disribuion of he feaure vecor x given he sysem is really in sae a ime : P(x q = ). In a simple Gaussian HMM, he parameers o be esimaed are he a i,, and Σ. To inroduce he parameerizaion on we modify he oupu disribuions. The simples useful model is a linear dependenceof he mean of he Gaussian on. For each sae of he HMM we have: ˆ ()=W + () P(x q = )=N(x ˆ () Σ ) () In he work presened here all values of are considered equally likely and so he prior P( q = ) is ignored. Noe ha is consan for he enire observaion sequence, bu is free o vary from sequence o sequence. When necessary, we wrie he value of associaed wih a paricular sequence k as k.. Training Training consiss of seing he HMM parameers o maximize he probabiliy of he raining sequences. Each raining sequence is paired wih a value of hea. The Baum-Welch form of he expecaion-maximizaion (EM) algorihm is used o updae he Technically here are also he iniial sae parameers o be esimaed; in his work we use causal opologies wih a unique saring sae.

2 parameers of he oupu probabiliy disribuions. The expecaion sep of he Baum-Welch algorihm (also known as he forward/backward algorihm) compues he probabiliy ha he HMM was in sae a ime given he enire sequence x denoed as. I is convenien o consider he HMM s parse of he observaion sequence as being represened by. In raining, he parameers of he HMM are updaed in he maximizaion sep of he EM algorihm. In paricular, he parameers are updaed by choosing a o maximize he auxiliary funcion Q( ). may conain all he parameers in, or only a subse if several maximizaion seps are required o esimae all he parameers. As explained in he appendix, Q is he expeced value of he log probabiliy given he parse. In he appendix we derive he derivaive of Q for HMM s: Q = P(x q = ) P(x q = ) The parameers of he parameerized Gaussian HMM include W,, Σ and he Markov model ransiion probabiliies. Updaing W and separaely has he drawback ha when esimaing W only he old value of is available, and similarly if is esimaed firs. Insead, we define new variables: Z W Ω k k such ha ˆ = Z Ω k. We hen need o only updae Z in he maximizaion sep for he means. To derive an updae equaion for Z we maximize Q by seing equaion 3 o zero (selecing Z as he parameers in ) and solving for Z. Noe ha because each observaion sequence k in he raining se is associaed wih a paricular k, we can consider all observaion sequences in he raining se before updaing Z. Accordingly we denoe associaed wih sequence k as k. Subsiuing he Gaussian disribuion and he definiion of ˆ = Z Ω k ino equaion 3: Q = Z k = Σ ; = Σ ; k " kx kω T k ; (3) () k(x k ; ˆ ( k)) T Σ ; ˆ ( k) (5) Z k(x k ; ˆ ( k))ω T k (6) kz Ω kω T k Seing his derivaive o zero and solving for Z, we ge he updae equaion for Z : Z =" kx kωk#" T kω kω T k #; Once he means are esimaed, he covariance marices Σ are updaed in he usual way: Σ = P k # (7) (8) k (x k ; ˆ ( k))(x k ; ˆ ( k)) T (9) as is he marix of ransiion probabiliies [3]..3 Tesing In esing we are given an HMM and an inpu sequence. We wish o compue he value of and he probabiliy ha he HMM produced he sequence. As compared o he usual HMM formulaion, he parameerized HMM s esing procedure is complicaed by he dependence of he parse on he unknown. Here we presen only a echnique o exrac he value of, since for a given value of he probabiliy of he sequence x is easily compued by he Vierbi algorihm or by he forward/backward algorihm. We desire he value of which maximizes he probabiliy of he observaion sequence. Again an EM algorihm is appropriae: he expecaion sep is he same forward/backward algorihm used in raining. The forward/backward algorihm compues he opimal parse given a value of. In he corresponding maximizaion sep we updae o maximize Q, he log probabiliy of he sequence given he parse. To derive an updae equaion for, we sar wih he derivaive in equaion 3 from he previous secion and selec as. As wih Z, only he means depend upon yielding: = Q (x i ; ˆ ()) T Σ ; ˆ () Seing his derivaive o zero and solving for, we have: =" W T Σ ; W #;" W T Σ ; (x ; ) () () The values of and are ieraively updaed unil he change in is small. Wih he examples we have ried, less han en ieraions are sufficien. Noe ha for efficiency, many of he inner erms of he above expression may be pre-compued. 3 Non-linear parameeric hidden Markov models 3. Model Nonlinear parameric hidden Markov models omi he linear model of secion. in favor of a logisic neural nework wih one hidden layer. As wih linear parameric HMM s, he oupu of each nework is perurbed by Gaussian noise: P(x q = )=N(x ˆ () Σ ) () The oupu ˆ () of he nework associaed wih sae can be wrien as ˆ ()=W ( ) g(w ( ) + b ( ) )+b ( ) (3) where W ( ) denoes he marix of weighs from he inpu layer o he layer of hidden logisic unis, b ( ) he biases a each inpu uni, and g(x) he vecor-valued funcion ha compues he logisic funcion of each componen of is argumen. Similarly, W ( ) and b ( ) denoe he weighs and biases for he oupu layer. 3. Training As wih sandard HMMs and linear parameric HMMs, he parameers of he nonlinear parameeric HMM are updaed in he maximizaion sep of he raining EM algorihm by choosing a o maximize he auxiliary funcion Q( ). In he nonlinear parameric HMM, he parameers include he parameers of each neural nework as well as Σ and ransiion #

3 probabiliies a i. Unlike he linear parameric HMM i is no possible o maximize Q wih respec o analyically. Insead we rely on he generalized expecaion-maximizaion (GEM) algorihm in which Q is (approximaely) maximized in he maximizaion sep using opimizaion echniques. The expecaion sep is he same as in he linear parameric and sandard HMM formulaions (he forward/backward algorihm). Gradien ascen may be used o updae he nework parameers in each maximizaion sep of he GEM algorihm. When applied o muli-layer neural neworks, gradien ascen (or gradien descen when he goal is minimize error ) is ofen referred o as he backpropagaion algorihm []. Raher han reierae he gradien descen equaions for logisic neural neworks here, we noe ha he backpropagaion algorihm is appropriae for opimizing Q wih he modificaion ha he error o be propagaed backwards ino he nework has he form Σ ; (x k ; ˆ ()) () which is simply he usual error weighed by and Σ ;. Inuiively, his weighing seers eachnework o model he appropriae par of he inpu, much as he gaing funcion of a mixures of expers model [] selecs is expers. Also, his weighing may be derived from he form of Q (equaion 3). In each maximizaion sep of he GEM algorihm, i is no necessary o compleely maximize Q. As long as Q is increased for every maximizaion sep, he GEM algorihm is guaraneed o converge o a local maximimum in he same manner as EM. In fac, since he funcional Q changes wih every expecaion sep, a complee maximizaion of Q in he maximizaion sep is probably compuaionally waseful. Accordingly in our esing we run he gradien ascen algorihm (backpropagaion algorihm) a fixed number of ieraions for each GEM ieraion. 3.3 Tesing In esing we desire he value of which maximizes he probabiliy of he observaion sequence. Again an EM algorihm o compue is appropriae. As in he raining phase, we can no maximize Q analyically, and so a GEM algorihm is necessary. To opimize Q, we use a gradien-ascen algorihm: = Q (x i ; ˆ ()) T Σ ; ˆ () (5) ˆ () = W ( ) Λ(g (W ( ) + b ( ) ))W ( ) (6) where Λ(x) forms he diagonal marix from he componens of x, and g (x) denoes he derivaive of he vecor-valued funcion ha compues he logisic funcion of each componen of is argumen. In he resuls presened in his paper, we use a gradien ascen algorihm wih adapive sep size. In addiion i was found necessary o consrain he gradien ascen sep o preven he alogirhm from wandering ouside he bounds of he raining daa, where he oupu of he neural neworks is essenially undefined. This consrain is implemened by simply limiing any componen of he sep ha akes he value of ouside he bounds of he raining daa, esablished by he minimum and maximum raining values. As wih he EM raining algorihm of he linear parameric case, wih he examples we have ried less han en GEM ieraions are required. Discussion In [] we presen an example of a poining gesure parameerized by proecion of hand posiion ono he plane parallel and in fron of he user a he momen ha he arm is fully exended. The linear parameric HMM approach works well since he proecion is a linear operaion. The nonlinear varian of he parameric HMM inroduced in he previous secion is appropriae in siuaions in which he dependence of he sae oupu disribuions on he parameers is no linear, and can no be easily made linear wih a known coordinae ransformaion of he feaure space. In pracice, a useful consequence of nonlinear modeling for parameric HMMs is ha he parameer space may be chosen more freely in relaion o he observaion feaure space. For example, in a hand gesure recogniion sysem, he naural feaure space may be he spaial posiion of he hand, while a naural parameerizaion for a poining gesure is he spherical coordinaes of he poining direcion. Conversely, here is no guaranee ha any observaion feaure space will permi he parameric HMM o learn he parameerizaion. Coninuing wih he poining example, he nonlinear parameric HMM approach will learn he smooh mapping from spherical coordinaes of he poin o hand posiion a each sae unambiguously. Obviously, a feaure space ha does no include he x coordinae (across he body) will no be enough o capure he parameerizaion, while a feaure space ha neglecs he deph away from he body may work well enough. One difficuly in assessingwheher an observaionfeaure space and a parameer space are appropriae for one anoher is wheher he mapping from parameer o observaion feaures is smooh enough o be learned by neural neworks wih a reasonable number of hidden unis. While in heory a 3-layer logisic neural nework wih sufficienly many hidden unis is capable of learning any mapping, we would like o use as few hidden unis as possible and so choose our parameerizaion and observaion feaure space o give simple, learnable mappings. Cross-validaion is probably he only pracical auomaic procedure o evaluae parameer/observaion feaure space pairings, as well as he number of hidden unis in each neural nework. The compuaional complexiy of such approaches is a drawback of he nonlinear parameeric HMM approach. In summary, wih nonlinear parameric HMMs we are free o choose inuiive parameerizaions bu we mus be careful ha i is possible o learn he mapping from parameers o observaion feaures given a paricular observaion feaure space. 5 Resuls To es he performance of he nonlinear parameric HMM, we conduced an experimen similar o he poining experimen of [] bu wih a spherical coordinae parameerizaion raher han he proecion ono a plane in fron of he user. We used a Polhemus moion capure sysem o record he posiion of he user s wris a a frame rae of 3Hz. Fify such examples were colleced, each averaging 9 ime samples (abou second) in lengh. Thiry of he sequences were randomly seleced as he raining se; he remaining comprised he es se. Before raining, he value of he parameer mus be se for each raining example, as well as for each esing example o evaluae he abiliy of he parameric HMM o recover he parameerizaion. We direcly measured he value of by finding he poin a which he deph of he wris away from he user was greaes. This poin 3

4 an(x) 8 6 azimuh elevaion x Figure : The disribuion of for he poining daa ses over he angen funcion. was ransformed o spherical coordinaes (azimuh and elevaion) via he arcangen funcion. Noe ha for poining gesures ha are confined o a small area in fron of he user (as in he experimen presened in []) he linear parameeric HMM approach will work well enough, since for small values he angen funcion is approximaely linear. The poining gesures used in he presen experimen were more broad, ranging from -36 o +8 degrees elevaion and -77 o +8 degrees azimuh. Figure shows how he populaion of associaed wih he raining se is disribued over more han he nearly linear porion of he angen funcion. An eigh sae causal nonlinear parameric HMM was rained on he fory raining examples. To simplify raining we consrained he number of hidden unis of each sae o be equal; noe ha his consrain is no presen in he model bu makes choosing he number of hidden unis via cross validaion easier. We evaluaed performance on he esing se for various numbers of hidden unis and found ha hidden unis gave he bes esing performance. We did no evaluae he performance under varying amouns of raining daa or varying numbers of saes in he HMM. The oupu of he resuling eigh neural neworks is shown in Figure 3. The oupu of he neural neworks shows how he inpu s dependence on is mos dramaic in he middle of he sequence, or a he apex of he poining gesure. The average error over he esing se was compued o be abou 6. degrees elevaion and 7.5 degrees azimuh. For comparison, an eigh sae linear parameric HMM was rained on he same raining daa and yielded an average error over he same esing se of abou.9 degrees elevaion and 8.3 degrees azimuh. Lasly, we demonsrae recogniion performance of he nonlinear parameerized HMM on our poining daa. A one minue sequence was colleced ha conained a variey of movemens including six poins disribued hroughou. To simulaneously deec he gesure and recover, we used a 3 sample (one sec) window on he sequence. Figure shows he log probabiliy as a funcion of ime and he value of recovered for a number of recovered poining gesures. All of he poining gesures were recovered. 6 Conclusion The parameric hidden Markov model framework presened in [] has been generalized o handle nonlinear dependenciesof he sae oupu disribuions on he parameerizaion. We have shown ha where he linear parameric HMM employs he EM algorihm in raining and esing, he nonlinear varian similarly uses he GEM algorihm. The drawbacks of he of he generalized approach are wofold: he number of hidden unis for he neworks mus be chosen appropriaely during raining, and secondly, during esing he GEM algorihm is more compuaionally inensive han he EM algorihm of he linear approach. The nonlinear parameric HMM is able o model a much larger class of parameric gesures and movemens han he linear parameric HMM. A pracical benefi of he increased modeling abiliy is ha wih some care, he parameer space may be chosen independenly of he observaion feaure space. Theoreically, his should allow more inuiive parameerizaions, including perhaps hose derived from more subecive qualiies of he signal (e.g. he inensiy of a walk). Addiionally, i should be easier o ailor parameer spaces o specific gesures, wih all gesures employing he same observaion feaure space ha is indiginous o he sensors. We believe ha hese are signican advanages in modeling parameeric gesure and movemen. References [] C. M. Bishop. Neural neworks for paern recogniion. Clarendon Press, Oxford, 995. [] R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinon. Adapive mixures of local expers. Neural Compuaion, 3:79 87, 99. [3] L. R. Rabiner and B. H. Juang. An inroducion o hidden Markov models. IEEE ASSP Magazine, pages 6, January 986. [] A. D. Wilson and A. F. Bobick. Recogniion and inerpreaion of parameric gesure. Proc. In. Conf. Comp. Vis., 998. acceped for publicaion (see MIT Percepual Compuing Group Technical Repor, hp://wwwwhie.media.mi.edu/vismod).

5 3 log probabiliy 8., , , , , 6.7 ( 3.5, 79.).8, 3.9 ( 76.6, 7.3) ( 7., 5.) ( 7., 3.3) ( 7.6, 68.6) ( 53.7, 5.7) frame number Figure : Recogniion resuls are shown by he log probabiliy of he windowed sequence beginning a each frame number. The rue posiive sequences are labeled by he value of recovered by he EM esing algorihm and he value compued by direc measuremen (in paranheses). = x hea.5.5 hea y z Figure 3: The oupu of each nework is displayed (as a surface) for each of he observaion feaure coordinaes (x, y, and z), for each of he eigh saes of he rained parameric HMM. 5