INF5820/INF9820 LANGUAGE TECHNOLOGICAL APPLICATIONS. Jan Tore Lønning, Lecture 4, 14 Sep

Transcription

1 INF5820/INF9820 LANGUAGE TECHNOLOGICAL ALICATIONS Jn Tor Lønning Lctur 4 4 Sp. 206 tl@ii.uio.no

2 Tody 2 Sttisticl chin trnsltion: Th noisy chnnl odl Word-bsd Trining IBM odl

3 3

4 SMT xpl 4 En kokk lgd n rtt d bygg. 0.9 ch 0.6 d right 0.9 with 0.4 building 0.45 cook 0.3 crtd 0.25 stright 0.7 by 0.3 construction 0.33 prprd 0.5 court 0.2 o 0.2 brly 0. constructd 0.2 dish 0. cookd 0.05 cours 0.07 Siilrly or: pos 0-2 2x3 pos -3 pos 2-4 pos 3-5 4x5 pos 6-8 os4 pos 6 x3x3 ny os5 pos 7 5x3x3 ny right with 2.7x0-2 right with building.7x0-8 right o.5x0-0 right with construction 5.4x0-8 right by 9.7x0-2 right with brly 8.7x0-9 cours o.5x0-4 cours o brly.5x0-6

5 5

6 Alignnt 6 Lngth o English string: k 7 Lngth o orign string: 9 An lignnt is vctor o lngth ch ntry nubr btwn 0 nd k Th xpl: < 2 9 > < >

7 Alignnt 7 Artiicil rstrictions: Svrl orign words y b lignd with th s E word A orign word cnnot b lignd to or thn on E word

8 IBM Modl Considr ll possibl lignnts : For ch lignnt us th gnrtiv odl: Sipliy th odl k ssuptions 8

9 9 Figur 25.23

10 NULL Mry did not slp th grn witch Mri no dio un botd l bru vrd Choos th lngth o th orign string Which E word trnslts to th irst F word? Wht is th trnsltion o this word? Which E word trnslts to th -th F word givn th choics so r? Wht is th trnsltion o this word givn th choics so r? 0

11 Assuptions pproxitions is constnt indpndnt o nd E ll lignnts th s probbility dds to th word trnsltion probbility only dpnds on sourc word + k t

12 Sipliis to ε is norlistion ctor Forul 4.7 in th SMT book Th book gos not IBM odl + t k ε + t k ε 2

13 rtr stition 3 I th trining corpus ws lignd th odl could b lrnd by counting: t C C I w hd known th trnsltion probbilitis w could hv ound th ost probbl lignnt. W nithr know word probbilitis nor lignnt: Chickn nd gg probl EM-lgorith: w y lrn th two siultnously

14 Trining th id 4. Fro th trnsltion probbilitis w y stit probbilitis or th vrious lignnts W do not choos only th bst lignnt 2. Fro lignnt probbilitis w y rclcult trnsltion probbilitis By ltrnting btwn nd 2 th nubrs convrg towrds bttr rsults For IBM Modl it y b provd tht thy convrg towrds globl optiu

15 5

16 6

17 7

18 8

19 9

20 20

21 Two wys to dscrib th lgorith 2 Intuitiv Eicint rocd. Trnsltion prob. Alignnt prob 2. Trnsltion prob 2. Alignnt prob 3. Trnsltion prob Etc J&M sc xpl Intrctbl in prctic Sidstp lignnt probs:. Trnsltion prob 2. Trnsltion prob 3. Trnsltion prob Etc K:SMT sc xpl This is how it gts iplntd

22 Trining th intuitiv pproch 22. Initliz th prtr vlus t or pirs o words nd. With no ino initliz th uniorly: Ech word in th orign lngug is n qully likly trnsltion o th word. 2. For ch pir o sntncs in th corpus us t to clcult th probbilitis to ll possibl lignnts o th two sntncs. Clld th xpcttion stp pply odl to dt

23 Trining th intuitiv pproch Collct rctionl counts tc : «How ny tis is trnsltd s». First clcult this c ; or ch sntnc whr w count: how ny tis is lignd to by ch lignnt wighd by th probbility o th lignnt. 2. Thn dd ovr ll sntncs to gt tc c ;

24 Trining th intuitiv pproch Clcult th nw trnsltion probbilitis t tt tt Errors in orul 4.4 in K:SMT whr vris ovr ll orign words Clld th xiiztion stp stit odl ro counts 5. Rpt ro 2 s long s you lik

25 Assign probbilitis to lignnts Gol: coput Sinc w hv W know Hnc: + t k ε ' ' t t t k t k ε ε

26 Expl th intuitiv wy 26 Corpus : Dog brkd : Hund bt 2 : Dog bit dog 2 : Hund bt hund 3 English words: dog bit brkd 3 orign words: hund bt bt

27 Stp initiliztion 27 thunddog /3 tbtdog /3 tbtdog /3 thundbit /3 tbtbit /3 tbtbit /3 thundbrkd /3 tbtbrkd /3 tbtbrkd /3 thund0 /3 tbt0 /3 tbt0 /3 Unior Obsrv tht w includ th lst lin sinc n - word y b lignd to 0.

28 Stp 2: Alignnt probbilitis 28 : Dog brkd : Hund bt 2 : Dog bit dog 2 : Hund bt hund Sntnc pir : 9 possibl lignnts: <00> <0> <02> <0> <> <2><20><2> <22> Ech qully probbl: /9 cll this :.g. <0>/9 Sntnc pir 2: 64 possibl lignnts: <000> <00> <333> Ech qully probbl: /64 cll this 2. Or th hrd wy nxt slid

29 Stp 2: Th hrd wy Sntnc pir 2: 64 possibl lignnts: <000> <00> <333> Ech trnsltion probbility: /27 2 : Dog bit dog 2 : Hund bt hund > < ε ε ε ε ε ε hund t bit bt t dog hund t t t t t t k 64 64*27 64* 64* > < > < > < ε ε 29

30 Stp 3.: Collct rctionl counts 30 Clcult c ; or ch sntnc : Expl: hund dog : Thr r 3 lignnts tht connct th: <0> <> <2> : Dog brkd : Hund bt chunddog; <0>+ <>+ <2>3*/9 /3 chunddog; /3 cbtdog; /3 chundbrkd; /3 cbtbrkd; /3 chund0; /3 cbt0; /3

31 Stp 3.: Collct rc. counts ctd : bt bit 2 : Dog bit dog 2 : Hund bt hund 6 lignnts connct th: <x2z> or xz in {023} cbtbit; 2 2 6/64 /4 bt dog ll lignnts <xz> nd <x3z> or xz in {023} cbtdog; 2 2 2*6/64 /2 chunddog; 2 2 cbtdog; 2 2 /2 chundbit; 2 2 /2 cbtbit; 2 2 /4 chund0; 2 2 /2 cbt0; 2 2 /4

32 Stp 3.2: Totl counts 32 tc c ; tchunddog +/3 tcbtdog /2 tcbtdog /3 tc*dog4/3+/2+/3 3/6 tchundbit ½ tcbtbit ¼ tcbtbit 0 tc*bit3/4 tchundbrkd /3 tcbtbrkd 0 tcbtbrkd /3 tc*brkd 2/3 tchund0 ½+/3 tcbt0 /4 tcbt0 /3 tc*07/2

33 Stp 4: nw trns. probbilitis 33 t tt tt t xct dcil 0 hund 5/6/7/2 0/ bt /4/7/2 3/ bt /3/7/2 4/ dog hund 4/3/3/6 8/ dog bt /2/3/6 3/ dog bt /3/3/6 2/ bit hund /2/3/4 2/ bit bt /4/3/4 / brkd hund /3/2/3 /2 0.5 brkd bt /3/2/3 /2 0.5

34 Rpt: Stp 2 sntnc 34 9 dirnt lignnts c : Dog brkd : Hund bt / <00> thund0*tbt0 0/7*3/ <0> thund0*tbtdog 0/7*2/ <02> thund0*tbtbrkd 0/7*/ <0> thunddog*tbt0 8/3*3/ <> thunddog*tbtdog 8/3*2/ <2> thunddog*tbtbrkd 8/3*/ <20> thundbrkd*tbt0 /2*3/ <2> thundbrkd*tbtdog /2*2/ <22> thundbrkd*tbtbrkd /2*/ Su o s

35 Rpt: Stp 2 sntnc dirnt lignnts Ho work to nxt wk! How ny lignnts i th sntncs r 0 words long? Words k Align ill 25 billions Tht s why w nd srtr wy. To b continud