Machine Translation Using Statistical Modeling

Transcription

1 Mchn Trnslton Usng Sttstcl Modlng Hrmn Ny nd F.. Och Achn Unvrsty o Tchnology Grmny Rrnc: Pttrn Rcognton n Sch nd ngug Procssng Ch Prsntd by Shh-Hung u

2 Outln ntroducton Sttstcl dcson thory nd lngustcs Algnmnt nd lxcon modls HMM BM modl -5 Srch Algnmnt tmlts From sngl words to word grou Sch trnslton Th ntgrtd roch Exrmntl rsults Summry

3 ntroducton Automtc trnslton o lngug or Mchn trnslton wrttn lngug or txt nut Sch trnslton sontnous sokn sch nut 3

4 Bys Dcson Rul or Wrttn ngug Trnslton Th Sttstcl Aroch Sch Rcognton Acoustc-ngustc Modlng + Sttstcl Dcson Thory Mchn Trnslton ngustc Modlng + Sttstcl Dcson Thory Advntgs n usng robblty dstrbutons Th robblts r drctly usd s scors. t s strghtorwrd to combn scors. Wk nd vgu dndncs cn b modld sly. 4

5 Bys Dcson Rul or Wrttn ngug Trnslton Mchn Trnslton Th strng trnslton modl { } { rg mx } ˆ rg mx ngug modl o th trgt lngug 5

6 Flow chrt 6

7 Algnmnt nd xcon Modls Modl th strng trnslton robblty W constrn ths modl by ssgnng ch sourc word to xctly on trgt word. φ φ φ 0 Two rochs to lgnmnt modlng r n mor dtl HMM BM -5 7

8 8 HMM ngth modl xcon modl Algnmnt modl s n lgnmnt :

9 9 HMM ngth modl xcon modl Algnmnt modl contxt dndnt HMM homognous HMM bsln HMM HMM Ty Thr css r consdrd. q q :

10 0 HMM mx mx mx mx mx mx mx mx mx mx Srch strtgy or bsln HMM DP: O V Rogr onon

11 Modls BM -5 Modl BM- nd BM-: zro-ordr dndnc bsln HMM Ty BM- nd BM- ngth modl Algnmnt modl xcon modl bsolut oston

12 Modls BM -5 Modl BM- nd BM-: zro-ordr dndnc [ ] [ ] [ ] [ ] [ ] [ ] [ ] { } [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] mxtur wght s lgnmnt robblts comonnt dstrbuton s lxcon robblts

13 Modls BM -5 Modl BM- nd BM-: zro-ordr dndnc Th modl BM- s scl cs wth unorm lgnmnt robblty modls BM Th mty word s ddd to th trgt sntnc to llow or sourc words whch hv no drct countrrt n th trgt sntnc. Formlly th conct o th mty word s ncorortd nto th lgnmnt modls by ddng th mty word t oston 0 to 0 th trgt sntnc nd lgnng ll sourc words wthout drct trnslton to ths mty word. 3

14 Modls BM -5 Modl BM-3: rtlty conct For ch trgt word thr s robblty dstrbuton ovr ts ossbl rtlts φ : φ Exrmntlly w obsrv tht th rtlts on vlus rom 0 to 4. rtlty : φ : δ Usng ths quton w cn strt wth n HMM or modl BM- nd thn comut ntl vlus or th rtlts. Th rtlty conct cn b usd to bttr modl trgt words hvng no countrrt n th sourc sntnc.. 0 φ 4

15 Modls BM -5 Modl BM-4 nd BM-5: nvrtd lgnmnts wth rst-ordr dndnc W ssum tht th robblty dstrbuton s th rsult o rocss consstng o thr sts. φ Slct rtlty or ch hyothszd trgt word For ch trgt word w gnrt th st o ssoctd sourc words ccordng to th rtlty φ whr th nl ostons r not scd yt. Th sourc words r rmut so tht th obsrvd squnc s roducd. nvrtd lgnmnt: b b : b b b s n nvtd lgnmnt 5

16 Rnmnts Modls BM -5 W must tk nto ccount tht th rtlty o word n oston my b drnt rom.g. or rtlty lrgr thn svrl ostons on th trgt xs hv to b roducd. Th dndnc on b dos not us th bsolut ostons but only rltv ostons. To rduc th numbr o r rmtrs th dndnc on th words b nd s rlcd by dndnc on th corrsondng rts-o-sch or word clsss G b nd G : b b G G b 6

17 Srch W us nvrtd lgnmnts s n th modl BM-4 whch dn mng rom trgt to sourc ostons rthr thn th othr wy round. W llow svrl ostons n th sourc lngug to b covrd.. w consdr mngs B o th orm: B : B { } For ths nvrtd lgnmnt mng wth sts B o sourc ostons w gn ssum sort o rst-ordr modl: B B whr w drod th dndnc on nd 7

18 Srch W rlc th sum ovr ll lgnmnts by th bst lgnmnt whch s rrrd to s mxmum roxmton n sch rcognton. Usng trgrm lngug modl w obtn th ollowng srch crtron: mx mx B B B B W cn s tht w cn buld u hyothss o rtl trgt sntncs n bottom-to-to strtgy ovr th oston o th trgt sntnc. Bm srch s usd to hndl th hug srch sc. Constrn: ll ostons o th sourc sntnc should b covrd xctly onc. 8

19 9 Srch B B B B B B B B B B mx mx mx mx mx mx mx mx mx mx

20 Algorthmc Drncs btwn Sch Rcognton nd ngug Trnslton Monotoncty n sch Rcognton thr s strct monotoncty btwn th squnc o coustc vctors nd th squnc o rcognzd words or honms. Ths s not th cs or mchn trnslton nd thror th srch roblm bcoms mor comlctd. Frtlty n mchn trnslton w hv to dcd whthr word s rsnt n th trgt strng or not. Thror t s mortnt to ssgn rtlty to ch word o th trgt vocbulry. n sch rcognton th countrrt o word s n HMM stt. Howvr w nvr tk dcsons bout stts but bout whol honm modls. Thr or th conct o rtlty s not rlly ndd n sch rcognton. 0

21 Algnmnt Tmlts: From Sngl Words to Word Grous W xtnd th roch to hndl word grous or hrss rthr thn sngl words. A whol grou o dcnt words n th sourc sntnc my b lgnd wth whol grou o dcnt words n th trgt lngug.

22 Algnmnt Tmlts: From Sngl Words to Word Grous W rst dcomos both th sourc sntnc nd th trgt sntnc nto squnc o word grous. k k k k k k k k + + k k k k k Thn th lgnmnt btwn word grous. Thn th lgnmnt btwn word grous. lgnmnt wthn word grou

23 Algnmnt Tmlts: From Sngl Words to Word Grous W ntroduc nw hddn vrbl whch wll b rrrd to s lgnmnt tmlt. z z z Th robblty z nd z r dtrmnd usng th lgnd trnng corus nd r st to zro th trl z dd not occur n th trnng corus. th trl dd occur n th trnng corus w us th ollowng modl or z z z whr z z z z 3

24 Srch To rorm th srch w us th ollowng modls As lngug modl w us clss-bsd n-grm.g. 3- or 5- grm lngug modl wth bckng-o. Tyclly ths s slghtly bttr thn th stndrd bgrm lngug modl. W ssum tht ll ossbl sgmnttons hv th sm robblty. Th lgnmnt modl t th tmlt lvl s n HMM-ty lgnmnt modl. Obvously s usul ll words n th sourc strng must b covrd. W hv to llow or ll ossbl sgmnttons o th sourc sntnc nto word grous or ll ossbl lgnmnts btwn th word grous nd or ossbl lgnmnts wthn th word grous. 4

25 Exrmntl Rsults VERBMOB Trnslton o sokn dlogus. n th domns o ontmnt schdulng nd trvl lnnng. A ntv Grmn skr nd ntv Englsh skr conduct dlogu whr thy cn only ntrct by skng nd lstnng to th VERBMOB systm. Corus Sokn dlogus wr rcordd. Ths dlogus wr mnully trnscrbd nd ltr mnully trnsltd by VERBMOB rtnrs. Ech o ths so-clld dlogu turns my consst o svrl sntncs sokn by th sm skr. Thr s no on-to-on corrsondnc btwn sourc nd trgt sntncs. 5

26 Exrmntl Rsults Th turns r slt nto shortr sgmnts usng unctuton mrks s otntl slt onts. A dynmc rogrmmng roch s usd to nd th otml sgmntton onts. th unctuton mrks n sourc nd trgt sntncs r not ncssrly dntcl 6

27 Exrmntl Rsults Oln Rsults W brly rort xrmntl oln rsults or th ollowng trnslton rochs: Sngl-word bsd roch Algnmnt tmlt roch Cscdd trnsducr roch 7

28 8 Sch Trnslton: Th ntgrtd Aroch Prncl T x tur vctors { } { } mx rg mx rg mx rg mx rg mx rg mx rg mx T T T T T T x x x x x x

29 Sch Trnslton: Th ntgrtd Aroch 9

30 30 Sch Trnslton: Th ntgrtd Aroch Prctcl mlmntton For th sk o smlcty bgrm dndnc wll b usd. y ssu Th quston o how th rqurmnt o hvng both wll-ormd sourc sntnc nd wll-ormd trgt sntnc t th sm tm s stsd. [ ] n lu o

31 Sch Trnslton: Th ntgrtd Aroch Summry No rochs ully mlmnts th ntgrtd coulng o rcognton nd trnslton rom sttstcl ont o vw. W consdr ths ntgrtd roch nd ts sutbl mlmntton to b n on quston or utur rsrch on sokn lngug trnslton. 3