Locally Tree-shaped Sentence Automata and Resolution of Am biguity. Abstract

Transcription

1 50 Locally Tree-shaped Sentence Automata and Resolution of Am biguity Jussi Piitulainen Research Unit for Multilingual Language Technology Department o f General Linguistics University of Helsinki jpii t ula@ling. helsinki.fi Abstract T h e fra m ew ork o f fin ite state in tersection g ra m m a rs is d irected tow ards p ra ctica l representation and p arsing o f run ning tex t. A p r o b le m in parsin g has been th a t in tersection w ith successive rule a u to m a ta can p rod u ce p roh ib itiv e ly large in term ed iate sentence a u to m a ta. See (K osk en n iem i, T a p a n a in en and V ou tila in en 1992) and (K osk en n iem i 1990) and (V o u tilainen an d T a p a n a in en 1993). T h is p a p er sketches a d a ta stru ctu re and p arsing m e th o d th at m ay h elp to c o m p u te the in tersection o f a sentence a u to m a to n w ith all rule a u to m a ta w hile keepin g the size o f in term ed iate sentence a u to m a ta under co n tro l. 1 Representation of sentence readings Sen ten ce readin gs are finite sequences o f w ord fo rm s, m o rp h o sy n ta ctic la bels and b o u n d a ry la b els (V o u tila in e n 1994, V ou tila in en an d T a p a n a in en 1993). T h e stru c ture o f a sen ten ce readin g is such th a t each w ord fo rm readin g con tain s the w ord fo r m, a base fo rm, a few m o r p h o lo g ica l la bels and on e or tw o sy n ta ctic labels, and w ord fo r m readin gs are sep arated by b o u n d a ry labels. For e x a m p le, the correct readin g o f th e sen ten ce A n d tim e beg an seriously to p a ss. in figure 1 identifies tim e as a s u b je ct CSUBJ, and b e g a n as a m a in verb in a m a in clause 8NV MAIHC8, an d so o n. O th er readin gs m ig h t m isid en tify tim e as a verb or t o as a p rep osition, or co rre ctly id en tify tim e as a n oun b u t m isid en tify it sy n ta ctica lly as an o b je c t or an a p p o sitio n. A n in correct ch oice fo r a w ord b o u n d a ry la b el w ou ld in d ica te the presence o f m o re th an on e finite verb fo rm. T h e re are five altern a tiv e w ord b o u n d a ry labels; QQ b eg in s and ends a sentence, e m b e d d e d fin ite clauses are en closed betw een < an d #>, a b ou n d a ry w here a finite clause ends an d a new on e starts is la b eled w ith /, an d is used fo r others. A fin ite set o f readin gs is represented as an a cy clic fin ite state a u to m a to n called a sen ten ce a u to m a to n. W o rd fo rm s in th e sentence have all lex ica lly p ossib le m o r p h o sy n ta c tic readin gs as alternatives, and each w ord b ou n d a ry is m a d e fou r or som e tim e s five w ays a m b ig u ou s. T h e in itial n u m b er o f readings in a sentence a u to m a to n is the p r o d u c t o f the lex ical and w ord bou n d a ry a m b ig u ities in the sentence., pages 50-58

2 <cind> and CC OCC 00 0 <time> time N NOM SG «SUBJ 0 <began> begin V PAST VFIH OHV MAINCQ 0 <seriously> serious ADV OADVL 0 <to> to INFMARK Oaux 0 <pass> pass V INF (Dmv OBJ(B 0 <.> (Dfullstop 00 F igure 1 A sentence readin g T h e g ra m m a r is represented as a n u m b er o f fin ite sta te a u to m a ta called rule a u to m a ta. E ach rule a u to m a to n is con stru cted to a cce p t all readin gs th a t are to b e g ra m m a tica l and to reject so m e readin gs th a t are to b e u n g ra m m a tica l. In oth er w ords, the g ra m m a r is taken to b e the in tersection o f in d iv id u a l rules. Sentence a u to m a ta w ill b e draw n w ith th e th e start sta te on th e left, the d irection o f edges fro m left to righ t, an d the final state as a d o u b le circle on the right. 2 The problem T h e task is to com p u te the in tersection 5 n i i o n - n i 2 _ i o f a sen ten ce a u to m a to n S' w ith all rule a u to m a ta iio i, f i n - i - T h e m a in p ra ctica l p r o b le m is th a t som e tim e s in tersection w ith successive rule a u to m a ta p ro d u ce s p roh ib itiv e ly b ig in term edia te sentence a u tom a ta. A n y m e th o d th a t co m p u te s the sa m e final result can b e used instead o f the stra ig h tforw ard in tersection ; this p a p er p rop oses on e such m e th o d. T h e n um ber o f readings represented by the sentence a u to m a to n decreases m o n o to n ica lly du ring the parsing p rocess. T h e n u m ber o f states in th e sentence a u to m a to n ten ds to increcise at first; the in itial a u to m a to n can b e very c o m p a c t precisely becau se it con tain s all altern a tives, an d a sm aller set m a y b e represented by a bigger a u to m a to n. T h e rem ov al o f the dashed edges in figure 2 leaves tw o m in im a l a u to m a ta o f w hich the bigger represents th e sm aller set o f readin gs. T h is sh ow s h ow rem ov a l o f readin gs can require a d d itio n o f states an d edges. T h e tw o a u to m a ta in figure 2 represent a set o f fou r readin gs. a u to m a to n is m in im a l and the low er a u to m a to n is (a lm o s t) m a x im a l. T h e u p p er C on sid er now a rule th at rejects the sequen ce ( x, a, y, a, z ) and a ccep ts th e oth er three. N o edge can be rem ov ed fr o m the m in im a l a u to m a to n since every ed ge b e lo n g s to on e o f ( x, a, y, b, z ) and { x, b, y, a, z ) and ( x, b, y, b, z ). In the m a x im a l a u to m a to n, the dashed edges and th e state betw een th em b e lo n g o n ly to th e rejected seq u en ce and can be rem oved. T h e final sentence a u to m a to n 5 PI iio 0 PI i i n _ i represents th e set o f correct readings for the sentence. I f the g ra m m a r is accu rate, this a u to m a to n is again quite sm all becau se it represents a very sm all set o f readin gs. 51

3 o- - c r x y < r x y - F igure 2 A m in im a l a u to m a to n and a m a x im a l a u to m a to n 3 New representation of sentence automata T h is section in trod u ces a new rep resentation fo r sentence a u to m a ta. T h e new d a ta stru ctu re is d esign ed to fa cilita te sep aration o f tw o o p e ra tio n s th at intersection w ou ld com b in e, n am ely rem ov al o f readin gs and expan sion o f th e sentence a u to m a to n. T h is sep a ra tion sh o u ld help keep the size o f in term edia te a u to m a ta under co n tro l. E dges an d states can b e rem ov ed if th ey are n o t p arts o f any readin g that sh ou ld rem ain. In a tree-sh aped a u to m a to n, all rem o v a b le readin gs can b e rem ov ed by re m o v in g su ch edges and states. S ince it is strictly im p ra ctica l to m ake the w hole sen ten ce a u to m a to n tree-sh ap ed, th e a u to m a to n is m a d e tree-sh ap ed o n ly locally. In itially, th e lo c a l trees sp an readin gs o f w ord fo r m s, b u t th ey can b e e x p a n d ed to sp an lon ger intervals. T h e re m u st b e states w ith m ore than on e im m e d ia te predecessor. T h ese are im m e d ia te successors o f th e leaves o f lo c a l trees. S ince th e trees in itia lly span w ord fo rm s, the w ord b ou n d a ries corresp on d to co lle ctio n s o f the states th a t m a y have several im m e d ia te p red ecessors. T h ese c o lle ctio n s w ill b e called slices. E ach slice o f a sentence a u to m a to n con ta in s e x a ctly on e state o f every p a th betw een th e start state an d the final state. T h e first slice con ta in s the state th at im m e d ia te ly fo llo w s the start state, and the last slice con ta in s th e final state. Initially slices con ta in o n ly on e state. T h e tw o a u to m a ta in figure 3 represent a set o f a few readin gs o f the sentence T im e p a sse d. P u n ctu a tio n and so m e labels, n o ta b ly MAINC8, are o m itte d to save sp ace. T h e u p p er a u to m a to n is lo c a lly tree-sh ap ed ; the low er a u to m a to n is an ord in a ry m in im a l a u to m a to n. T h e rectan gles in the u p p er a u to m a to n represent th e three slices corresp on d in g to the three w ord b ou n d a ries in the sentence. E ach sta te in a slice is a r o o t o f a lo c a l tree. T h e leaves o f th e tree are im m ed ia te p red ecessors o f so m e r o o ts in th e n e x t slice. N ote th a t a state th a t is p rop erly betw een slices has o n ly on e im m e d ia te predecessor, so th a t th e trees d o n o t share stru ctu re. T h e d a ta stru ctu re representing a sentence a u to m a to n sh ou ld g ive easy access to the in fo rm a tio n needed in the a lg o rith m s to b e d escribed, in a d d itio n to the u n d erly in g a u to m a to n stru ctu re. F irst, the con ta in in g slice, if any, sh ou ld b e available g iven a state. S econ d, the p reviou s sta te or states sh ou ld b e available given a 52

4 80 <time>time 0-0 NON SG H ^ O-- O - O v 8SUBJ F igure 3 INF VFIN 8MV PCP2^ X )\ S C <passed>dass L o ca lly tree-sh ap ed vs. m in im a l V t > - ^ O M V 88 _ state. T h ird, the states in slices and im m e d ia te predecessors o f states in slices w ill b e assigned sets o f rule states th at sh ou ld b e available given such a state. It sh ou ld be easy to itera te over all edges th a t leave a state. 4 Operations on the new data structure T h e new d a ta stru ctu re is designed so th a t rem ov al and a d d itio n o f states an d edges are sep arate o p era tion s. T h is is in con tra st to in tersection th at n ot o n ly rem ov es readings bu t also expan d s the sentence a u to m a to n in a w ay th a t is d ifficu lt to con trol. In con tra st to sentence a u to m a ta, n ext to n o th in g is assu m ed a b o u t th e rep resentation o f rule a u to m a ta. A ll th a t is required is th a t it is p o ssib le to c o m p u te the state w here a given state takes a given s y m b o l, an d to d e cid e w hether there is any w ay to reach a final sta te fr o m a given state. A state is called a sink if there is n o w ay to reach a final state fr o m it. A rule can fa il b y g o in g in to a sink lo n g b efore reach in g th e end o f an in p u t sequence. 4.1 Initialization and propagation In the new representation, sentence a u to m a ta con ta in tw o sp ecia l kin ds o f states, n am ely the ro o ts o f the lo ca l trees and the leaves o f the lo c a l trees. T h e r o o ts are in slices, and each le a f is an im m e d ia te predecessor o f a r o o t. (T h e start sta te is also th ou gh t o f as a lea f.) R ea d in gs are rem ov ed in tw o step s ca lled p ro p a g a tio n and rem oval. T h e p ro p a g a tio n step assigns sets o f states o f a rule a u to m a to n to the ro o ts and leaves o f the lo ca l trees. In itially, a set con ta in in g th e start sta te o f th e rule a u to m a to n is assigned to the start state o f the sentence a u to m a to n. T h e re is on ly on e p a th fro m the start state o f the sentence a u to m a to n to the first r o o t, th e on e labeled 88; the set co n ta in in g the state to w hich the start state o f the rule a u to m a to n takes the la b el 88 is assigned to th e first r o o t. O n ce in itialized, the p rop a g a tio n step con tin u es by assign m en t o f sets o f rule states to the ro o ts and leaves o f each lo ca l tree in tu rn. F or ex a m p le, a certain rule 53

5 a u to m a to n goes to a sink state w hen it en cou nters the la bel MV after it has already en cou n tered fimv b u t n o t yet en cou n tered a clause b ou n d a ry. L et state 1 b e the start state, sta te 2 b e the state after the first MV an d state 5 b e the sink state. In the u p p er a u to m a to n in figure 4, the r o o t has been reached th rou g h p ath s th a t contain on e MV an d p a th s th at d o n o t con tain MV. T h erefore, the r o o t is assigned the set o f tw o rule states {1,2}. W h e n this set is p rop a g a te d fu rther, th e rule a u to m a to n stays in these tw o states until it en cou n ters the MV la b el; then the state set w ould b e {2,5} b u t th e sink sta te 5 is sim p ly ignored. In the low er a u to m a to n in figure 4, the r o o t has been reached o n ly th rough p a th s th at con ta in on e MV. T h e state set after th e secon d MV con tain s on ly the sink sta te and b e co m e s effectively em p ty. F igure 4 P ro p a g a tio n o f a rule sta te set T h e first r o o t has been assigned in the in itia lization step. T h ere is a unique p a th fro m the r o o t o f a loca l tree to a le a f o f th at tree. T h is p a th and th e rule states in the r o o t d eterm in e the set o f rule states th a t are assigned to the leaf. R u le states in the lea f are th ose to w hich so m e rule state in the r o o t takes the p a th from the r o o t to the leaf, e x clu d in g sinks. W h e n all the rule states w ou ld b e sinks, this set is em p ty. U nlike leaves, the ro o ts in a slice can have several predecessors. A r o o t is assigned the set o f rule states th at are reach a ble fr o m so m e rule state in som e o f its p red ecessors by so m e p a th fr o m the le a f to the r o o t. 4.2 Removing paths that block propagation T h e sets o f rule states in the leaves help to rem ov e states and edges. First, if a lea f is assigned the e m p ty set o f rule states, there is n o w ay for the rule to reach this leaf. T h e le a f can b e rem ov ed, and edges and states p reced in g it can b e rem ov ed back to the first state w ith m ore than on e im m e d ia te su ccessor. A ll bou n d a ry edges forw a rd can be rem ov ed, and if the fo llo w in g r o o t had n o oth er im m ed ia te 54

6 p redecessor, rem oval can con tin u e forw a rd. T h is m ea ns th at the edges la b e le d V VFIH (OMV QO in th e low er a u to m a to n in figure 4 can b e rem ov ed. S econ d, if all th e rule states in so m e le a f take so m e b o u n d a ry ed g e to a sink, th e ed ge can b e rem ov ed. If all th e edges fr o m th e le a f to a r o o t are rem o v a b le this way, rem oval can con tin u e back w ard and forw a rd. T h ird, a n on -fin al state in th e last slice can b e treated as if it w ere a sink. T h ere is n o w ay to reach an oth er fin al state fr o m th e last sta te in th e sen ten ce a u to m a to n. O n ly such readin gs are rem ov ed as w ou ld b e rem ov ed b y in tersection. H ow ever, n o t all such readin gs can b e rem ov ed this way. T o allow rem ov al o f fu rth er readin gs, the sentence a u to m a to n m ay need to be ex p an d ed. 4.3 Expanding the sentence automaton locally In the new m e th o d, the sentence a u to m a to n ca n n o t grow w h ile readin gs are rem ov ed. A sep arate o p e ra tio n o f lo c a l ex p a n sion is p r o v id e d to o p e n up rem ov al op p ortu n ities. O bserve th at th e in itial sentence a u to m a to n is tree-sh aped betw een any pair o f successive slices. T h e ex p a n sion m akes it tree-sh aped (rath er forest-sh a p e d ) betw een som e given pair o f slices. E ach sta te th a t is p ro p e rly betw een th e tw o slices w ill have e x a ctly on e im m ed ia te p redecessor. T h e ex a m p le sentence T im e p a ssed is so sh ort th a t there is o n ly on e w ay to ch oose a pair o f slices fo r expan sion. T h e in itial a u to m a to n is sh ow n in fig u re 3; the e x p a n d ed a u to m a to n is sh ow n in figure 5. F or lon ger sentences, su ch m a x im a l ex p an sion is n o t feasible. T h e first w ord fo r m tim e is three w ays a m b ig u o u s and th e w ord b o u n d a ry after it is fou r w ays am b ig u ou s; tog eth er tim e an d th e b o u n d a r y are tw elve w ays am big u ou s. E x p an sion m akes tw elve copies o f th e state in th e slice betw een the readings o f tim e and p a ssed, and th e tree o f p a ssed is tu rned in to a forest o f tw elve id en tica l trees. M uch o f the new stru ctu re is easy to rem ov e. P a rticu la rly in teresting are the three trees o f p a ssed th a t fo llo w the b o u n d a ry sy m b o l f l /. T h is b o u n d a r y s y m b o l separates finite clauses: there sh ou ld b e th e la b el VFIN b o th b e fo re and after it. T h is con strain t is n ot en ou g h to rem ov e an y th in g in th e a u to m a to n o f figure 3 since VFIH occu rs in on e o f the readings o f tim e, b u t in the a u to m a to n o f figure 5, tw o o f the three trees can b e rem ov ed. For ex a m p le, after QQ < tim e > t im e N NOM SG OSUBJ 0 / the rule sh o u ld b e in a sink state. A n o th e r im p o ssib ility is to have 0< and (0(D o r 0< and (0< as su ccessive b o u n d aries. T h e form er con strain t w ou ld allow rem ov a l o f m u ch o f th e stru ctu re in the ex p a n d ed a u to m a to n : the la bel OA w ou ld take a rule to a sink. P arts o f th e low er fou r trees o f p a ssed are rem ov a b le by th e con stra in t th at the b o u n d a ry s y m b o l betw een tw o fin ite verb fo rm s VFIH can n o t b e 8, so th a t the secon d VFIH w ou ld take a rule to a sink. 55

7 I9D :time> time o o ---- o F igure 5 A n e x p a n d ed sentence a u to m a to n 5 Com putation strategies T h e high level stra teg y to use is an open qu estion. T h e o p e ra tio n s o f p ro p a g a tio n, rem ov a l and ex p a n sion can be used in different orders. In this m e th o d, rules need to be used m o re than o n ce, and the ord er o f ex p an sion s and rem ovals m a y affect efficien cy sign ifican tly. T h e sim p le m in d ed b u t p roh ib itiv e ly expen siv e stra teg y w ou ld b e to first m a x im ize the sen ten ce a u to m a to n and then ju s t use every rule in turn. A n o th e r stra teg y w ou ld b e to exercise all rule a u to m a ta o n a sh ort interval in th e b e g in n in g o f th e sentece a u to m a to n first, an d then len gthen the interval. A th ird stra teg y w ou ld b e to e x p a n d sh ort intervals an d then p ro p a g a te and rem ove. M ore in telligent stra tegies m igh t take in to a cco u n t sp ecia l p rop erties o f each sen ten ce a u to m a to n, or sp ecia l p rop erties o f so m e rules. O r such intervals m igh t b e e x p a n d ed th a t con tain relatively few readings. K im m o K osk en n iem i (p erson a l c o m m u n ica tio n ) su ggested first p rop a g a tin g a rule in to the sen ten ce a u to m a to n and then stu d y in g the sets o f rule states to see w hether this rule can allow rem ov als after ex p a n sion. In the p ositiv e case, expan d an d then p ro p a g a te again. 56

8 6 Some details T h is section gives a m ore d etailed d escrip tion o f th e o p e ra tio n s o n th e lo c a lly treesh aped a u to m a ta than w as given b efore. A ll a u to m a ta here are d eterm in istic fin ite sta te a u tom a ta. A n a u to m a to n is a fin ite set o f u n iq u ely la b eled states, p a rtitio n e d in to final and n on -fin al states, w ith on e state d esign ated as th e start state. A ll states w ill b e seen as to ta l fu n ctio n s ta k in g sequences to states, as follow s. W h e n s and t are states and a p a th fro m s to t is la b eled b y a seq u en ce o f a lp h a b e t sy m b o ls w, the n o ta tio n s w design ates th e sta te t. T h e set o f all prefixes o f such sequences w th at s takes to a final sta te is ca lled the d o m a in o f s. A sta te w ith e m p ty d o m a in is called a sink; if w is n o t in th e d o m a in o f s, then s w is a sink. E ach state s satisfies th e equ a tion s s Q s an d s (a, 6,...) = (s (a )) ( 6,...) for all a, 6,... in the a lp h a b et. In a m in im a l a u to m a to n, a sink s is a n on -fin a l state th at satisfies s w = s fo r all sequen ces w. If s {a ) = t fo r so m e s y m b o l a, then s is sa id to b e an im m e d ia te p red ecessor o f t, and t is said to b e an im m e d ia te successor o f s. In the p rocess to b e sp ecified, a sen ten ce a u to m a to n S is represen ted as a sequ ence o f slices (5o,..., <S _i). It m u st b e rep laced by su ccessive sen ten ce a u to m a ta so th at the final result represents the in tersection o f th e o rig in a l sen ten ce a u to m a to n w ith a n um ber o f rule a u to m a ta R q,..., R t ^ i. 6.1 Propagation R u le states can be p ro p a g a te d fr o m S i to. A n in term ed ia te step is to p rop a g a te th em to the leaves o f each lo ca l tree w ith r o o t in S i. W h e n a p a th fr o m a r o o t to a le a f is la beled w and the set o f rule states assigned to th e r o o t is R, the set o f rule states in the le a f w ill b e { r t u r G i i } e x clu d in g sinks. A fter p ro p a g a tio n to the leaves, an im m e d ia te p red ecessor p o f a sta te t in contain s the correct set R ' o f rule states. If D is th e d o m a in o f p, th e set o f rule states in t w ill b e a su perset o f { r (d ) r G R ' and {d ) E D } e x clu d in g sinks. O th er im m e d ia te predecessors o f t p rod u ce oth er states. R u le states can be p ro p a g a te d fr o m the start sta te to each slice in turn. If the start state o f a rule is r o, the state in th e first slice w ill co n ta in ju s t th e state To { #). 6.2 Removal W h en rule states have been p ro p a g a te d fr o m S o to S i, it m a y b e p o ssib le to rem ov e edges an d states th at precede states in <S,-. If p is an im m e d ia te p redecessor o f so m e state s in Si and th e set o f rule states in p is em p ty, then the rule takes to a sink all sequen ces th a t la b el p a th s fr o m the start state to p. T h e state p can be rem ov ed tog eth er w ith p reced in g edges and states back to the first state th a t has m ore than on e ed ge in its d om a in. 57

9 I f th e set o f rule states in p is n o t em p ty b u t there are several edges fro m p to s, all the rule states can take so m e o f the edges to a sink. Such edges can be rem ov ed. 6.3 Expansion W h e n the sen ten ce a u to m a to n is e x p a n d ed betw een S i and 5 i+ 2i an d so m e state s in <Sj+i has k im m e d ia te predecessors, th e state s and its lo ca l tree are replaced w ith k copies, each w ith its ow n im m e d ia te predecessor o u t o f th e im m e d ia te predecessors o f th e orig in a l s. 7 Summary In tersection o f fin ite sta te a u to m a ta reduces readings bu t m a y ad d stru ctu re in the a u to m a ta. S ep a ra tion o f the tw o effects m a y help to keep the size o f a sentence a u to m a to n u nder co n tro l. K eep in g th e sentence a u to m a to n lo c a lly tree-sh aped m akes this sep a ra tion p ossible. F uture w ork w ill in volve im p lem en ta tion o f the parser and em p irica l stu d y o f variou s pra sin g stra tegies. Acknowledgements S p ecia l th an k s to A n ssi Y li-j y r ä, K ari G ra n ö, A tro V ou tila in en an d K im m o K oskenn iem i, an d thanks to T im o L ah tinen, T im o Jä rvin en, M ik k o L ou n ela and the tw o referees, fo r readin g an d com m en ts. T h e tw o e x a m p le sentences are from the D ou g la s A d a m s n ovel, D irk G e n tly s H olistic D e te ctiv e A g en cy, first pu blish ed in 1987 by W illia m H ein em ann L td. References K im m o K osk en n iem i F in ite-sta te P arsing and D isa m b ig u a tion. In H ans K arlgren (e d ). C O L I N G : P a p e r s P r e s e n t e d to th e 1 3 th I n te r n a tio n a l C o n f e r e n c e o n C o m p u t a tio n a l L in g u is tic s, Vol. 2. H elsinki, F in land. K im m o K osk en n iem i, Pasi T ap an ain en and A tro V ou tilain en C o m p ilin g and U sing F in ite-s ta te S y n ta ctic R ules. In P r o c e e d in g s o f C O L I N C -9 2, Vol. I. N antes, France. A tro V ou tila in en an d Pasi T a p a n a in en A m b ig u ity resolu tion in a red u ction - istic parser. In P r o c e e d in g s o f E A C L -9 S. U trecht, T h e N etherlands. A tro V ou tila in en D esig n in g a P arsing G ra m m a r. In T h r ee S tu d ies o f C r a m m a r -B a s e d S u r fa c e P a r s in g o f U n r e s tr ic te d E n g lish T ex t. P h.d. disserta tio n. H elsinki, F in la n d. 58