This document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore.

Transcription

1 Ths documen s downloaded from DR-NTU, Nanyang Technologcal Unversy Lbrary, Sngapore. Tle A smplfed verb machng algorhm for word paron n vsual speech processng( Acceped verson ) Auhor(s) Foo, Say We; Yong, Lan; Lang, Dong Caon Foo, S. W., Yong, L., & Lang, D. (009). A smplfed verb machng algorhm for word paron n vsual speech processng. Proceedngs of 4h European Workshop on Image Analyss for Mulmeda Ineracve Servces, Dae 009 URL hp://hdl.handle.ne/00/6039 Rghs A smplfed verb machng algorhm for word paron n vsual speech processng copyrgh 009 World Scenfc Publshng Co. Ths conference's webse s locaed a hp://eproceedngs.worldscne.com/ / _0065.hml.

2 A SIMPLIFIED VITERBI MATCHING ALGORITHM FOR WORD PARTITION IN VISUAL SPEECH PROCESSING SAY WEI FOO School of EEE., Nanyang Technologcal Unversy, Sngapore, E-mal: YONG LIAN AND LIANG DONG ECE Dep., Naonal Unversy of Sngapore, Sngapore, 960 E-mal: {elelany, In hs paper, a novel smplfed Verb machng algorhm for sequence paron s presened. Ths mehod connecs he saes of dfferen HMMs o model he observed sequence. The mehod s appled o vsual speech processng o paron he word no vsual speech elemens. Expermenal resul shows ha good accuracy s acheved wh he proposed mehod.. Inroducon In auomac lp readng sysems usng Hdden Markov Models (HMMs), how o paron he word or senence no ndvdual recognon uns s an mporan opc of research. In radonal mehod, exhausve means such as level buldng s commonly appled o acheve hs goal. In hs paper, a novel aemp n solvng hs problem s repored. In he proposed mehod, he HMMs are reaed as deachable sae se. A word s paroned no vsual speech elemens by connecng he saes of dfferen HMMs. A smplfed Verb machng s performed whle rerevng he sae chan. The expermenal resuls ndcae ha he word can be paroned wh good accuracy f he basc vsual speech elemens are confgured properly.. Consrucon of Vseme Models and Sae Daabase The basc vsual speech elemens n Englsh are called vsemes. A vseme s a me seres ndcang he movemen of he lp ha s repeaed n dfferen word arculaon. In MPEG-4 mulmeda sandards, he vsemes are caegorzed no 4 groups wh one vseme correspondng wh ~5 vsually smlar phonemes []. Our nvesgaon on he dynamcal feaures of vsual speech shows ha he

3 a. Inal phase /o/ Arculaon phase Transon // End phase b. Fgure. a.) The hree phases of vseme producon b.) Segmenaon of he ranson sae from he producon of /o/ moon of he lp can be paroned no hree phases durng he producon of an ndependen vseme: The frs phase s called nal phase, sarng from a closed mouh o he begnnng of sound producon. The nex phase s called arculaon phase, whch s he course when sound s produced. The hrd phase s he end phase when he mouh resores o he relaxed sae. Among he above hree phases, he arculaon phase s he mos mporan for recognon because he dfference beween vsemes chefly les here and s relavely ndependen o he conex. The nal phase and he end phase, on he oher hand, are ransonal ones. They may change much under dfferen conexs. HMMs are used o model he vsemes n hs paper. Assume { S, S, L S } s N he sae se and { O, O, L O } s he symbol alphabe, an N-sae-M-symbol M HMM s deermned by: probables of he nal sae π = [ π ], sae ranson marx A = [ a j ] and symbol emsson marx B = [ b j ]. In he proposed sraegy, hree-sae lef-rgh HMMs are adoped o model he vsemes. If he parameers of he HMMs are properly confgured, he saes of he HMM can be physcally assocaed wh he hree phases of vseme producon []. Afer performed Baum-Welch esmaon, a number of vseme models (HMMs) are obaned. The arculaon saes of he HMMs are colleced as he recognon uns. On he oher hand, he ransons beween vsemes, whch are used o connec he recognon uns, are also segmened ou of wo vsemes jon ogeher, e.g. /o/. As llusraed n Fg. (b), he approxmae ranson phase s manually segmened from he mage sequence. The sascs of he ransons beween dfferen pars of vsemes are obaned by counng and averagng he symbols appeared n he process. These probably funcons are colleced o buld he ranson un se. 3. Smplfed Verb Machng Algorhm for Word Paron The purpose of sequence paron s o decode a sae chan ha he recognon uns and ranson uns appear alernaely. Gven he sae daabase Θ and T-

4 3 (K+L)*T nodes S + +L S + +L S + +L S + +L Recognon Uns Transon Uns Fgure. The probably rells ha s generaed from he forward process lengh observaon sequence X = { x, x L x }, he ask s o fnd a sae sequence T Q = { q, q Lq } o maxmze P ( Q X, Θ), whch s equvalen o maxmzng T P ( Q, X Θ). A Verb machng s mplemened as follows. Inalzaon: Assume he sequence sars from a ranson sae, e.g. from he mouh s closed, he frs column of he probably rells s obaned from (), 0 K = and ψ ( ) = 0 < K + L () π b ( x ) K < K + L where π s he nal probably of he -h sae. Because a word may sar from any ranson sae, a unform dsrbuon π = / L ( K < K + L) s appled o he ranson uns. Forward Process: A each momen, a sae wll eher repea self or rans o a un n anoher se. If he (-) h sae s a recognon un, he h sae can be he same recognon un as he (-) h sae or a ranson un. Two probables are hen compued. δ ) = a and δ ) max [ ( a )] K () ( ( = K < K + L and he h column of he probably rells s, = max[, ] b ( x ) and ψ ( ) = arg max[, ] b ( x ) (3) If he --h sae s a ranson sae, he followng probables are compued. δ ) = λ and δ ) max[ ( λ)] K < K + (4) ( ( = < K L where λ has he same meanng as sae reeraon probably a. I conrols he duraon of he ranson sae. Also, we have, = max[, ] b ( x ) and ψ ( ) = arg max[, ] b ( x ) (5) Termnaon: The las column of he probably rells s δ = max[ ( )] and q = arg max[ ] K + L (6) end δ T T K The opmal sae sequence s obaned by backrackng he nodes usng (7). q = ψ ( q ) = T, T, L (7) + + In (4), he enry λ s nroduced o buld a sae chan because he ranson un s merely a probably dsrbuon funcon. In applcaons, λ can be se o dfferen values o beer f he process nvesgaed. + L T

5 4 4. Expermens Some expermens are conduced o assess he performance of he proposed mehod for word paron. As lsed n Table, some words are decoded no vseme combnaons wh greaes probables are lsed n Table and he correc paron s marked wh a. I can be observed ha he correc parons always appear n he bes maches. Word Vseme combnaons Probably lp zoo wh black Table. Examples of word paron wh he proposed sraegy /l/ + rans. + /p/ 43% /l/ + rans. + /I/ + rans. + /p/ 3% /ch/ + rans. + /p/ % /U / 7% /z/ + rans. + /U/ 7% /sh/ + rans. 7% /w/ + rans. + /I/ + rans. + /z/ 35% /U/ + rans. + /z/ 30% /f/ + rans. + /s/ 9% /b/+rans.+/l/+rans.+/a/+rans.+/k/ 7% /b/+rans.+/l/+rans.+/a/+rans.+// 4% /b/ + rans.+ /A/ + rans. + /k/ 3% 5. Concluson In hs paper, he smplfed Verb algorhm s appled o lp readng as an example. Ths mehod can also be adoped n sequence paron of he followng condons:.) he saes of he HMM of he basc recognon uns can be classfed as varable saes and relavely consan saes..) I s dffcul o decode he arge sequence no an HMM chan of he basc recognon uns. References. M. Tekalp and J. Osermann, Face and -D mesh anmaon n MPEG-4, Image Communcaon J. (999). Say We Foo and Lang Dong, Recognon of Vsual Speech Elemens Usng Hdden Markov Models, Proc. PCM 00, LNCS53, (00) 3. L. R. Rabner and B. H. Juang, Fundamenals of speech recognon, Prence Hall Sgnal Processng Seres, (993) 4. Lang Dong and Say We Foo, Boosng of Hdden Markov Models and he Applcaon o Vsual Speech Analyss, Proc. IWSSIP, , (00)