XML and Databases. Lecture 5 XML Validation using Automata. Sebastian Maneth NICTA and UNSW

Transcription

1 XML nd Dtbses Lecture 5 XML Vlidtion using Automt Sebstin Mneth NICTA nd UNSW CSE@UNSW -- Semester 1, 2009

2 Outline 2 1. Recp: deterministic Reg Expr s / Glushkov Automton 2. Complexity of DTD vlidtion 3. Beyond DTDs: XML Schem nd RELAX NG 4. Sttic Methods, bsed on Tree Automt

3 Previous Lecture 3 XML type definition lnguges wnt to specify certin subset of XML doc s = type of XML documents Remember The specifiction/type definition should be simple, so tht vlidtor cn be built utomticlly (nd efficiently) the vlidtor runs efficient on ny XML input (similr demnds s for prser) Type def. lnguge must be SIMPLE! (similrly: prser genertors use EBNF or smller subclsses: LL / LR) O(n^3) prsing

4 XML Type Definition Lnguges 4 DTD (Document Type Definition, W3C) Originted from SGML. Now prt of XML DTD my pper t the beginning of n XML document XML Schem (W3C) Now t version 1.1 HUGE lnguge, mny built-in simple types Reg Exprs must be deterministic (=1-unmbiguous) sme!! Unique Prticle Attribution Schems themselves: written in XML See the Schem Primer t RELAX NG (Osis) For tree structure definition, more powerful thn Schems&DTDs

5 XML Type Definition Lnguges 5 DTD (Document Type Definition) <!DOCTYPE root-element [ doctype declrtion ]> <!ELEMENT element-nme content-model> content-models EMTPY ANY (#PCDATA elem-nme_1 elem-nme_n)* deterministic Reg Expr over element nmes <!ATTLIST element-nme ttr-nme ttr-type ttr-defult..> Types: CDATA, (v1..), ID, IDREFs Defults: #REQUIRED, #IMPLIED, vlue, #FIXED

6 XML Type Definition Lnguges 6 DTD (Document Type Definition) <!DOCTYPE root-element [ doctype declrtion ]> <!ELEMENT element-nme content-model> content-models EMTPY ANY (#PCDATA elem-nme_1 elem-nme_n)* deterministic Reg Expr Most interesting / chllenging spect of DTDs <!ATTLIST element-nme ttr-nme ttr-type ttr-defult..> Types: CDATA, (v1..), ID, IDREFs Defults: #REQUIRED, #IMPLIED, vlue, #FIXED

7 7 Summry In order to check whether (lrge) document is vlid wrt to given DTD ( it vlidtes ) you need to check if children lists mtch the given Reg Expr s This cn be done efficiently, using finite-utomt (FAs)! To check if Reg Expr e is llowed in DTD we hve to construct prticulr finite utomton: the Glushkov utomton. Glu(e) must be deterministic. Glu(e) Note If Glu(e) is deterministic, then its size (# trnsitions) is liner in size(e)!

8 8 Summry In order to check whether (lrge) document is vlid wrt to given DTD ( it vlidtes ) you need to check if children lists mtch the given Reg Expr s This cn be done efficiently, using finite-utomt (FAs)! To check if Reg Expr e is llowed in DTD we hve to construct prticulr finite utomton: the Glushkov utomton. Glu(e) must be deterministic. Glu(e) Note If Glu(e) is deterministic, then its size (# trnsitions) is liner in size(e)! Question Cn you explin why this is the cse?

9 9 Summry In order to check whether (lrge) document is vlid wrt to given DTD ( it vlidtes ) you need to check if children lists mtch the given Reg Expr s This cn be done efficiently, using finite-utomt (FAs)! To check if Reg Expr e is llowed in DTD we hve to construct prticulr finite utomton: the Glushkov utomton. Glu(e) must be deterministic. Glu(e) Note If Glu(e) is deterministic, then its size (# trnsitions) is liner in size(e)! Question Cn you explin why this is the cse? not correct: liner in size(e) * #letters(e)

10 10 More Notes (1) From deterministic FA you cnnot necessrily obtin deterministic (= 1-unmbiguous) regulr expression!! Exmple: e = ( b)* ( b) NO 1-unmbigous reg exp exists for e To check if Reg Expr e is llowed in DTD we hve to construct prticulr finite utomton: the Glushkov utomton. Glu(e) must be deterministic. Glu(e) Note If Glu(e) is deterministic, then its size (# trnsitions) is liner in size(e)! Question Cn you explin why this is the cse? not correct: liner in size(e) * #letters(e)

11 More Notes For more detils: See pper by Brüggemnn-Klein, Linked from the course web-pge. 11 (1) From deterministic FA you cnnot necessrily obtin deterministic (= 1-unmbiguous) regulr expression!! Exmple: e = ( b)* ( b) NO 1-unmbigous reg exp exists for e (2) Glu(e) is closely relted to Thomson(e) [remove ε-trnsitions] nd to Berry/Sethi(e) [sme] nd Brzozowski(e) To check if Reg Expr e is llowed in DTD we hve to construct prticulr finite utomton: the Glushkov utomton. Glu(e) must be deterministic. Glu(e) Note If Glu(e) is deterministic, then its size (# trnsitions) is liner in size(e)! Question Cn you explin why this is the cse? not correct: liner in size(e) * #letters(e)

12 12 Glushkov utomton Glu(e) Ech letter-position in the Reg Expr e becomes one stte of Glu; plus, Glu hs one extr begin stte. FIRST( e ) = ll possible begin positions of words mtching e e.g. FIRST( R (E G) (EX)* ) = { R 1 }

13 Following slides from: 13

14 position 1 position 2 position

15 15

16 16

17 17

18 FIRST( e ) 18

19 19 Glushkov utomton G(e) Ech position in the Reg Expr e becomes one stte of G; plus, G hs one extr begin stte. FIRST( e ) = ll possible begin positions of words mtching e e.g. FIRST( R (E G) (EX)* ) = { R 1 } FOLLOW( e, x ) = ll possible positions following position x in e e.g. FOLLOW( R (E G) (EX)*, R 1 ) = { E 2, G 3 } From stte R 1 : dd E-trnsition to E 2 G-trnsition to G 3

20 FOLLOW( e, R 1 ) = 20

21 21

22 FOLLOW( e, E 2 ) = 22

23 FOLLOW( e, G 3 ) = 23

24 FOLLOW( e, E 4 ) = 24

25 FOLLOW( e, X 5 ) = 25

26 26 Glushkov utomton G(e) Ech position in the Reg Expr e becomes one stte of G; plus, G hs one extr begin stte. FIRST( e ) = ll possible begin positions of words mtching e e.g. FIRST( R (E G) (EX)* ) = { R 1 } FOLLOW( e, x ) = ll possible positions following position x in e e.g. FOLLOW( R (E G) (EX)*, R 1 ) = { E 2, G 3 } From stte R 1 : dd E-trnsition to E 2 G-trnsition to G 3 LAST( e ) = ll possible end positions of words mtching e e.g. LAST( R (E G) (EX)* ) = { E 2, G 3, X 5 }

27 27

28 Is this utomton deterministic?? 28

29 29 Glushkov utomton G(e) Another exmple (* b)* This FA is deterministic. 1 b 2 3 b b Which of these is deterministic? (b) (c) (b c) ( b)*c

30 30 Glushkov utomton G(e) Ech position in the Reg Expr e becomes one stte of G; plus, G hs one extr begin stte. FIRST( e ) = ll possible begin positions of words mtching e e.g. FIRST( R (E G) (EX)* ) = { R 1 } FOLLOW( e, x ) = ll possible positions following position x in e LAST( e ) = ll possible end positions of words mtching e Nïve implementtion: O(n^3) time, where n = size( e ) (for ech position: computing FOLLOW goes through every position t ech step, needs to compute union O( n*n*n )

31 31 Glushkov utomton G(e) Ech position in the Reg Expr e becomes one stte of G; plus, G hs one extr begin stte. FIRST( e ) = ll possible begin positions of words mtching e e.g. FIRST( R (E G) (EX)* ) = { R 1 } FOLLOW( e, x ) = ll possible positions following position x in e LAST( e ) = ll possible end positions of words mtching e Nïve implementtion: O(n^3) time, where n = size( e ) (for ech position: computing FOLLOW goes through every position t ech step, needs to compute union O( n*n*n ) Not relly needed. Cn be improved to O(n^2)

32 32 Glushkov utomton G(e) Ech position in the Reg Expr e becomes one stte of G; plus, G hs one extr begin stte. FIRST( e ) = ll possible begin positions of words mtching e e.g. FIRST( R (E G) (EX)* ) = { R 1 } FOLLOW( e, x ) = ll possible positions following position x in e LAST( e ) = ll possible end positions of words mtching e Nïve implementtion: O(n^3) time, where n = size( e ) (for ech position: computing FOLLOW goes through every position t ech step, needs to compute union O( n*n*n ) Cn be improved to O( size(e) + size(g(e)) ) Not relly needed. Cn be improved to O(n^2)

33 33 Glushkov utomton G(e) Note If G(e) is deterministic, then its size (# trnsitions) is qudrtic in size(e)! Liner in size(e) * #letters(e), if G(e) is deterministic! O( size(e) * #letters(e) ) Nïve implementtion: O(n^3) time, where n = size( e ) (for ech position: computing FOLLOW goes through every position t ech step, needs to compute union O( n*n*n ) Cn be improved to O( size(e) + size(g(e)) ) Not relly needed. Cn be improved to O(n^2)

34 To void these expensive running times 34 DTD requires tht FA=G(e) must be deterministic! n = length(w) m = size(e) Totl Running time O(n + m) If s = #letters(e) is ssumed fixed (not prt of the input) Otherwise: O(n + ms) How cn you implement regulr expression? Input: Reg Expr e, string w Question: Does w mtch e? deterministic FA: run on w tkes time liner in length(w) Algorithm FA = BuildFA(e); DFA = BuildDFA(FA); Size of FA is liner in size(e)=m Size of DFA is exponentil in m Unrestricted Reg Expr e Totl Running time O(n + 2^m) Other lterntive: O(nm)

35 35 Summry Deterministic (1-unmbiguous) content models give rise to efficient mtching lgorithms. (they void O(nm) or O(n+2^m) complexities) Disdvntges Hrd to know whether given reg expr is OK (deterministic) Det. reg exprs re NOT closed under union. (not so nice..) Question Cn you see why? Hint: find det. reg. exprs. e1 nd e2 such tht their union is equl to ( b)* ( b)

36 36 Now tht we know how the check ll the different content-models (in prticulr det. Reg Expr s) how to build full vlidtor for DTD? elem-nme_1 elem-nme_2 elem-nme_k RegExpr_1 RegExpr_2 RegExpr_k Automt A_1, A_2,, A_k The Vlidtion Problem Given DTD T nd document D, is D vlid wrt T? Top-Down Implementtion t element node w. lbel elem-nme_i, run utomton A_i check ttribute constrints check ID/IDREF constrints Totl Running time (Given A_1, A_2,, A_k) liner in the sum of sizes of the DTD nd the document. O( size(t) + size(d) )

37 DTDs hve the 37 lbel-gurded subtree exchnge property: t1, t2 trees in DTD lnguge T v1 node in t1, lbeled lb v2 node in t2, lbeled lb trees obtined by exchnging the subtrees rooted t v1 nd v2 re lso in T k locl content model only depends on lbel of prent t1 t2 v1 lb v2 lb

38 Beyond DTDs 38 Often, the expressive power of DTDs is not sufficient. Problem ech element nme hs precisely one content-model in DTD. Would like to distuingish, depending on the context (prent). deler used new cr cr model yer model cr hs different structure, in different contexts.

39 Beyond DTDs 39 Often, the expressive power of DTDs is not sufficient. Problem ech element nme hs precisely one content-model in DTD. Would like to distuingish, depending on the context (prent). deler used new speciliztion cr used cr new model yer model cr hs different structure, in different contexts.

40 Specilized DTDs 40 deler used, new used (cr used )* new (cr new )* cr used model, yer cr new model deler deler used new used new cr used cr new cr cr model yer model model yer model

41 Specilized DTDs deler used, new used (cr used )* new (cr new )* cr used model, yer cr new model New nottion. Use cpitlized TYPE Nmes Deler deler [Used, New] Used used [(Cr used )*] New new [(Cr new )*] Cr used cr [Model, Yer] Cr new cr [Model] 41 deler deler used new used new cr used cr new cr cr model yer model model yer model

42 New nottion. Use cpitlized TYPE Nmes 42 Deler deler [Used, New] Used used [(Cr used )*] New new [(Cr new )*] Cr used cr [Model, Yer] Cr new cr [Model] Not locl Let us cll this new concept grmmr. the locl restriction A grmmr G is locl, if for ny lbel[regexpr_1], lbel[regexpr_2] present in G it holds tht RegExpr_1 = RegExpr_2. By definition: Every DTD is locl grmmr, nd vice vers.

43 New nottion. Use cpitlized TYPE Nmes 43 Deler deler [Used, New] Used used [(Cr used )*] New new [(Cr new )*] Cr used cr [Model, Yer] Cr new cr [Model] Not locl Let us cll this new concept grmmr. the locl restriction A grmmr G is locl, if for ny lbel[regexpr_1], lbel[regexpr_2] present in G it holds tht RegExpr_1 = RegExpr_2. By definition: Every DTD is locl grmmr, nd vice vers. A grmmr G is single-type, if for ny lbel[regexpr_1], lbel[regexpr_2] occurring in the sme rule of G it holds tht RegExpr_1 = RegExpr_2. WRONG the single-type restriction

44 New nottion. Use cpitlized TYPE Nmes 44 competing Deler deler [Used, New] Used used [(Cr used )*] New new [(Cr new )*] Cr used cr [Model, Yer] Cr new cr [Model] Alterntively: Cll two TYPE Nmes T1 nd T2 competing if they hve the sme element nme (but not identicl rules) Clsses of Grmmrs locl no competing TYPE nmes! (DTDs) single-type TYPE nmes in the sme content model do not compete! (XML Schem s) regulr no restriction (RELAX NG)

45 New nottion. Use cpitlized TYPE Nmes 45 competing Deler deler [Used, New] Used used [(Cr used )*] New new [(Cr new )*] Cr used cr [Model, Yer] Cr new cr [Model] Question Are there single-type grmmrs (XML Schems) which cnnot be expressed by locl grmmrs (DTDs). Clsses of Grmmrs locl no competing TYPE nmes! (DTDs) single-type TYPE nmes in the sme content model do not compete! (XML Schem s) regulr no restriction (RELAX NG)

46 New nottion. Use cpitlized TYPE Nmes 46 competing but re not in sme content model! Person person [PersonNme, Gender, Spouse?, Pet*] PersonNme nme [First, Lst] Pet pet [Kind, PetNme] PetNme nme [#PCDATA] Question Are there single-type grmmrs (XML Schems) which cnnot be expressed by locl grmmrs (DTDs). YES! Clsses of Grmmrs locl no competing TYPE nmes! (DTDs) single-type TYPE nmes in the sme content model do not compete! (XML Schem s) regulr no restriction (RELAX NG)

47 New nottion. Use cpitlized TYPE Nmes 47 competing Deler deler [Used, New] Used used [(Cr used )*] New new [(Cr new )*] Cr used cr [Model, Yer] Cr new cr [Model] Through the use of TYPE Nmes (nonterminls / sttes) you cn distinguish deep context! deler used cr Cr new cr new cr new speciliztion through prent model yer model

48 New nottion. Use cpitlized TYPE Nmes 48 competing Deler deler [Used, New] Used used [(Cr used )*] New new [(Cr new )*] Cr used cr [Model, Yer] Cr new cr [Model] Through the use of TYPE Nmes (nonterminls / sttes) you cn distinguish deep context! deler Cn we model context tht is fr wy from the specilized node? used cr Cr new cr new cr new speciliztion through prent model yer model

49 New nottion. Use cpitlized TYPE Nmes 49 competing Deler deler [Used, New] Used used [(Cr used )*] New new [(Cr new )*] Cr used cr [Model, Yer] Cr new cr [Model] db root Through the use of TYPE Nmes (nonterminls / sttes) you cn distinguish deep context! deler specil user Cn we model context tht is fr wy from the specilized node? Sure! used cr cr My Cr new new cr speciliztion through following node model yer model #owners

50 DB db [Deler, User] Deler deler [Used, New] Used used [(Cr used )*] New new [(Cr new )*] Cr used cr [Model, Yer] Cr new cr [Model] Through the use of TYPE Nmes (nonterminls / sttes) you cn distinguish deep context! deler specil Cn we model context tht is fr wy from the specilized node? Sure! scr used sused used cr cr My Cr new new cr model yer model #owners Doc root [DB sdb] sdb db [sdeler, suser] sdeler deler [sused, New] sused used [(scr used )*] scr used cr [Model, Own, Yer] sdeler db root sdb suser user 50

51 DB db [Deler, User] Deler deler [Used, New] Used used [(Cr used )*] New new [(Cr new )*] Cr used cr [Model, Yer] Cr new cr [Model] Through the use of TYPE Nmes (nonterminls / sttes) you cn user distinguish deep context! deler specil Cn we model context tht is fr wy from the specilized node? Sure! scr used sused used cr cr My Cr new new cr model yer model #owners Doc root [DB sdb] sdb db [sdeler, suser] sdeler deler [sused, New] sused used [(scr used )*] scr used cr [Model, Own, Yer] sdeler db root sdb suser Question Sure this grmmr is not locl (DTD). But, is it single-type? 51

52 52 root root db db specil deler deler used new used new Question Is this grmmr single-type?

53 prev. exmple: probbly, not expressble in single-type (XML Schem). 53 Other exmple: Person MPerson FPerson MPerson person[nme, gender[mle], FSpouse?, Children?] FPerson person[nme, gender[femle], MSpouse?, Children?] Mle mle[] Femle femle[] FSpouse spouse[nme, gender[femle]] MSpouse spouse[nme, gender[mle]] Children children[person+] person s spouse must hve opposite gender. Note This exmple nd the Pet-exmple re tken from Hosoy s book (see course web pge).

54 prev. exmple: probbly, not expressble in single-type (XML Schem). 54 Other exmple: competing Reg Expr but not in content Person MPerson FPerson MPerson person[nme, gender[mle], FSpouse?, Children?] FPerson person[nme, gender[femle], MSpouse?, Children?] Mle mle[] Femle femle[] FSpouse spouse[nme, gender[femle]] MSpouse spouse[nme, gender[mle]] Children children[person+] person s spouse must hve opposite gender. BUT, is this even grmmr in our sense?

55 prev. exmple: probbly, not expressble in single-type (XML Schem). 55 Other exmple: competing Reg Expr in content Person root[mperson FPerson] MPerson person[nme, gender[mle], FSpouse?, Children?] FPerson person[nme, gender[femle], MSpouse?, Children?] Mle mle[] Femle femle[] FSpouse spouse[nme, gender[femle]] MSpouse spouse[nme, gender[mle]] Children children[person+] person s spouse must hve opposite gender. BUT, is this even grmmr in our sense? NO! Reg Expr only llowed inside content ( under n element nme ).

56 Clsses XML Type Formlisms 56 locl no competing TYPE nmes! (DTDs) single-type TYPE nmes in the sme content model do not compete! (XML Schem s) regulr no restriction (RELAX NG) Incresing Expressivness of defining sets of trees ( tree lnguges ) Questions Given two DTDs D1 nd D2 cn we check if ll documents vlid for D1 re lso vlid for D2? (DTD inclusion problem) D1 nd D2 describe the sme set of documents? (DTD equlity problem) Given Relx NG grmmr G, cn we check if there exists ny document tht is vlid for G? (emptiness problem) there is document vlid for G nd vlid for G2? (intersection & emptiness)

57 Clsses XML Type Formlisms 57 locl no competing TYPE nmes! (DTDs) single-type TYPE nmes in the sme content model do not compete! (XML Schem s) regulr no restriction (RELAX NG) Incresing Expressivness of defining sets of trees ( tree lnguges ) Questions Given two DTDs D1 nd D2 cn we check if ll documents vlid for D1 re lso vlid for D2? (DTD inclusion problem) D1 nd D2 describe the sme set of documents? (DTD equlity problem) Given Relx NG grmmr G, cn we check if there exists ny document tht is vlid for G? (emptiness problem) there is document vlid for G nd vlid for G2? (intersection & emptiness) If we cn do it for regulr tree grmmrs, then lso works for single-type/locl!!

58 58 All of the checks cn be done utomticlly, for regulr tree grmmrs! Tree Automt: very powerful frmework, Hve ll the good properties of string utomt! Yet, they re more expressive! equivlent to tree utomt Questions Given two DTDs D1 nd D2 cn we check if ll documents vlid for D1 re lso vlid for D2? (DTD inclusion problem) D1 nd D2 describe the sme set of documents? (DTD equlity problem) Given Relx NG grmmr G, cn we check if there exists ny document tht is vlid for G? (emptiness problem) there is document vlid for G nd vlid for G2? (intersection & emptiness) If we cn do it for regulr tree grmmrs, then lso works for single-type/locl!!

59 59 All of the checks cn be done utomticlly, for regulr tree grmmrs! Tree Automt: very powerful frmework, Hve ll the good properties of string utomt! Yet, they re more expressive! equivlent to tree utomt Note String utomt re not sufficient to check DTDs / Schems! Even if we only consider well-brcketed strings! Exmple 1 c c[, c, b ] empty b empty c empty Exmple 2 [ c, ] [, b ] / b / c empty

60 60 All of the checks cn be done utomticlly, for regulr tree grmmrs! constnt memory computtion equivlent to tree utomt Finite-stte utomt re importnt: Think you re in mze, with only fixed memory nd you cn only red the mze (cnnot mrk nything). wll Model by finite utomton. In stte q1, (to [N S E W], ) ( q2, [N S E W] ) q2, (to [N S E W], ) ( q3, [N S E W] ) q1 empty Cn n utomton serch the mze?

61 61 All of the checks cn be done utomticlly, for regulr tree grmmrs! constnt memory computtion equivlent to tree utomt Finite-stte utomt re importnt: Think you re in mze, with only fixed memory nd you cn only red the mze (cnnot mrk nything). wll Model by finite utomton. In stte q1, (to [N S E W], ) ( q2, [N S E W] ) q2, (to [N S E W], ) ( q3, [N S E W] ) q1 empty Cn n utomton serch the mze? No!! need mrkers ( pebbles ). How mny? 5? 2?

62 62 All of the checks cn be done utomticlly, for regulr tree grmmrs! constnt memory computtion equivlent to tree utomt Finite-stte utomt re importnt: In our context, e.g., for KMP (efficient string mtching) [Knuth/Morris/Prtt] generliztion using utomt. Used, e.g., in grep Compression Sttic nlysis of schems & queries (= everything you cn do *before* before running over the ctul dt )

63 4. Sttic Methods, bsed on Tree Automt 63 Person MPerson FPerson MPerson person [Nme, gender[mle], FSpouse?] FPerson person [Nme, gender[femle], MSpouse?] Regulr Tree Grmmr Rules of the form TypeNme Tree person Leves my be lbeled by TypeNmes person Nme gender FSpouse Nme gender Mle Mle Alterntively, regulr tree lnguges re defined by Tree Automt. stte, element-nme stte1, stte2 conventionlly, defined for binry/rnked trees.

64 4. Sttic Methods, bsed on Tree Automt 64 Given grmmrs D1 nd D2 cn we check if ll documents vlid for D1 re lso vlid for D2? (inclusion problem) D1 nd D2 describe the sme set of documents? (equlity problem) does there exists ny document tht is vlid for D1? (emptiness problem) there is document vlid for D1 *nd* vlid for D2? (intersection & emptiness) ALL these checks re possible for regulr tree grmmrs!! hence, they re lso solvble for DTDs / XML Schems / RELAX NG s (1) use binry tree encodings (2) trnslte XML Type Definition to Tree Grmmr (esy) Alterntively, regulr tree lnguges re defined by Tree Automt. stte, element-nme stte1, stte2 conventionlly, defined for binry/rnked trees.

65 4. Sttic Methods, bsed on Tree Automt 65 Given grmmrs D1 nd D2 cn we check if ll documents vlid for D1 re lso vlid for D2? (inclusion problem) D1 nd D2 describe the sme set of documents? (equlity problem) does there exists ny document tht is vlid for D1? (emptiness problem) there is document vlid for D1 *nd* vlid for D2? (intersection & emptiness) ALL these checks re possible for regulr tree grmmrs!! The checks bove give rise to very powerful optimiztion procedures for XML Dtbses! For exmple: documents d_1, d_2,, d_n re vlid for your schem Smll_xhtml. Are they lso vlid for schem XHTML? Check inclusion problem for Smll_html nd XHTML!

66 Automt 3 on words 66 1 b b 2 1 b b b

67 Automt 3 on words 67 1 b b 2 b b b 1

68 Automt 3 on words 68 1 b b 2 b b b 1

69 Automt 3 on words 69 1 b b 2 b b b 2

70 Automt 3 on words 70 1 b b 2 exp. blow-up exp. more succinct Expressiveness deteterministic = nondeterministic left-to-right = right-to-left b b b word is ccepted by the utomton

71 Automt 3 on words 71 1 b b 2 exp. blow-up exp. more succinct Expressiveness deteterministic = nondeterministic left-to-right = right-to-left b b b word is ccepted by the utomton Automt on trees 1. bottom-up LABEL( stte1, stte2 ) stte F OR AND T OR 0 1 0() F 1() T OR( F, F ) F OR( F, T ) T 0 1

72 Automt 3 on words 72 1 b b Expressiveness deteterministic = nondeterministic left-to-right = right-to-left 2 exp. blow-up exp. more succinct b b b word is ccepted by the utomton Automt on trees 1. bottom-up LABEL( stte1, stte2 ) stte F T OR AND T OR 0 F 1 0() F 1() T OR( F, F ) F OR( F, T ) T 0 1

73 Automt 3 on words 73 1 b b Expressiveness deteterministic = nondeterministic left-to-right = right-to-left 2 exp. blow-up exp. more succinct b b b word is ccepted by the utomton Automt on trees 1. bottom-up LABEL( stte1, stte2 ) stte F T OR F AND T OR 0 F 1 T 0() F 1() T OR( F, F ) F OR( F, T ) T AND( T, F ) F 0 1

74 Automt 3 on words 74 1 b b Expressiveness deteterministic = nondeterministic left-to-right = right-to-left 2 exp. blow-up exp. more succinct b b b word is ccepted by the utomton F Automt on trees T OR F T AND 1 T F OR 0 T 1. bottom-up LABEL( stte1, stte2 ) stte 0() F 1() T OR( F, F ) F OR( F, T ) T AND( T, F ) F Accepting Sttes = { T } 0 1 tree is ccepted by the utomton

75 Automt 3 on words 75 1 b b Expressiveness deteterministic = nondeterministic left-to-right = right-to-left 2 exp. blow-up exp. more succinct b b b word is ccepted by the utomton F Automt on trees T OR F T T OR 0 1 AND T 0 F 1 1. bottom-up LABEL( stte1, stte2 ) stte 0() F 1() T OR( F, F ) F OR( F, T ) T AND( T, F ) F tree is ccepted by the utomton Accepting Sttes = { T } This utomton is deterministic. nondeterminism LABEL( st1, st2 ) { st3, }

76 76 Question How much memory do you need exctly, to run such bottom-up tree utomton? F Automt on trees T OR F T T OR 0 1 AND T 0 F 1 1. bottom-up LABEL( stte1, stte2 ) stte 0() F 1() T OR( F, F ) F OR( F, T ) T AND( T, F ) F tree is ccepted by the utomton Accepting Sttes = { T } This utomton is deterministic. nondeterminism LABEL( st1, st2 ) { st3, }

77 Similrly s for word utomt: 77 For every nondeterministic bottom-up tree utomton there is n equivlent deterministic bottom-up tree utomton. Agin, the construction cn cuse exponentil size blow-up. 2. top-down stte, LABEL (stte1, stte2) b must contin $-lef,b = binry node lbels e,$ = lef node lbels top-most -node on the left-most pth must hve right-subtree which contins $-node.

78 Similrly s for word utomt: 78 For every nondeterministic bottom-up tree utomton there is n equivlent deterministic bottom-up tree utomton. Agin, the construction cn cuse exponentil size blow-up. 2. top-down stte, LABEL (stte1, stte2) b must contin $-lef,b = binry node lbels e,$ = lef node lbels top-most -node on the left-most pth must hve right-subtree which contins $-node. begin, (ny, find$) begin, b (begin, ny) find$, /b { (find$, ny), (ny, find$) } find$, $ ACC find$, e REJ ny, /b (ny,ny) ny, $/e ACC nondeterministic ccept tree if there exists n ccepting run (= ll leves go to ACC)

79 For every nondeterministic bottom-up tree utomton there is n equivlent deterministic bottom-up tree utomton. 79 Question Cn you find n equivlent bottom-up utomton for this exmple? 2. top-down stte, LABEL (stte1, stte2) b must contin $-lef,b = binry node lbels e,$ = lef node lbels top-most -node on the left-most pth must hve right-subtree which contins $-node. begin, (ny, find$) begin, b (begin, ny) find$, /b { (find$, ny), (ny, find$) } find$, $ ACC find$, e REJ ny, /b (ny,ny) ny, $/e ACC nondeterministic ccept tree if there exists n ccepting run

80 For every nondeterministic bottom-up tree utomton there is n equivlent deterministic bottom-up tree utomton, nd there is n equivlent nondeterministic top-down tree utomton. 80 Yes! you cn 2. top-down stte, LABEL (stte1, stte2) b must contin $-lef,b = binry node lbels e,$ = lef node lbels top-most -node on the left-most pth must hve right-subtree which contins $-node. begin, (ny, find$) begin, b (begin, ny) find$, /b { (find$, ny), (ny, find$) } find$, $ ACC find$, e REJ ny, /b (ny,ny) ny, $/e ACC nondeterministic ccept tree if there exists n ccepting run

81 For every nondeterministic bottom-up tree utomton there is n equivlent deterministic bottom-up tree utomton, nd there is n equivlent nondeterministic top-down tree utomton. 81 Question Is there n equivlent deterministic top-down utomton?? 2. top-down stte, LABEL (stte1, stte2) b must contin $-lef,b = binry node lbels e,$ = lef node lbels top-most -node on the left-most pth must hve right-subtree which contins $-node. begin, (ny, find$) begin, b (begin, ny) find$, /b { (find$, ny), (ny, find$) } find$, $ ACC find$, e REJ ny, /b (ny,ny) ny, $/e ACC nondeterministic ccept tree if there exists n ccepting run

82 For every nondeterministic bottom-up tree utomton there is n equivlent deterministic bottom-up tree utomton, nd there is n equivlent nondeterministic top-down tree utomton. 82 Question Is there n equivlent deterministic top-down utomton?? NO! nme nme first lst lst first This set of two trees cnnot be recognized by ny determinstic top-down tree utomton!! Why?

83 For every nondeterministic bottom-up tree utomton there is n equivlent deterministic bottom-up tree utomton, nd there is n equivlent nondeterministic top-down tree utomton. 83 Question Is there n equivlent deterministic top-down utomton?? NO! Questions Wht bout locl tree lnguges (defined by DTDs). Cn they be ccepted by deterministic top-down utomt? Wht bout single-type tree lnguges (defined by XML Schem s) Cn they be ccepted by deterministic top-down utomt?

84 For every nondeterministic bottom-up tree utomton there is n equivlent deterministic bottom-up tree utomton, nd there is n equivlent nondeterministic top-down tree utomton. 84 Question Is there n equivlent deterministic top-down utomton?? NO! Questions Wht bout locl tree lnguges (defined by DTDs). Cn they be ccepted by deterministic top-down utomt? Wht bout single-type tree lnguges (defined by XML Schem s) Cn they be ccepted by deterministic top-down utomt? Yes! Hence, there is no DTD / Schem for { nme[first,lst], nme[lst,first] }

85 For every deterministic bottom-up tree utomton there exists miniml unique equivlent one! 85 Equivlence is decidble In fct, YOU hve lredy produced miniml bottom-up tree utomt! The miniml DAG of tree t cn be seen s the miniml unique tree utomton tht only ccepts the tree t.

86 For every deterministic bottom-up tree utomton there exists miniml unique equivlent one! 86 Equivlence is decidble In fct, YOU hve lredy produced miniml bottom-up tree utomt! The miniml DAG of tree t cn be seen s the miniml unique tree utomton tht only ccepts the tree t. Question How expensive (complexity) to find mininml one? Sme s for word utomt?

87 Tree Automt re very useful concept in CS! 87 Hevily used in verifiction Derive property of complex object from the properties of its constituents h 1 h 2 g = glue 1 ( g 1, glue 2 ( h 1, h 2 ) ) glue 1 g 1 glue 2 g 1 h 1 h 2 Do ll grphs / chip-lyouts produced in this wy, hve property P? Use the hierrchicl construction history of n object, in order to work on prse tree insted of complex grph. From there, use tree utomt. Mny NP-complete grph problems become trctble on bounded-treewidth grphs!

88 XML Tree Automt ply crucil rule for 88 Efficient vlidtors ginst XML Types Optimiztions If doc1 is of TYPE1, then no need to vlidte ginst TYPE2, if we know TYPE2 included in TYPE1 - if only slightly different then only need to vlidte there - incrementl vlidtion ginst updtes -etc, etc. Efficient query evlutors, use richer utomt which cn select nodes nd produce query nswers Optimiztions If nswer of QUERY1 is in cche, then no need to evlute QUERY2, if included in QUERY1. - if every possible nswer set to QUERY1 (of TYPE X) is EMPTY, then no need to evlute on the rel dt! XML Type Checking for Progrmming Lnguges

89 The Future 89 In 5-10 yers from now: You cn write function in Progrmming Lnguge X Function foo(xml document D: TYPE1): TYPE2 { trverse D & compute output;... return output } Compiler (XML Type Checker) will complin, if your function does not compute documents of TYPE2. If no complint, then gurnteed: ALL outputs re ALWAYS of correct type!!)

90 The Future In 5-10 yers from now: You cn write function in Progrmming Lnguge X Function foo(xml document D: TYPE1): TYPE2 { trverse D & compute output;... return output } Experimentl PL s In this direction: CDuce XDuce 90 Compiler (XML Type Checker) will complin, if your function does not compute documents of TYPE2. If no complint, then correct type gurnteed. Compilers will hve to be ble to give sttic gurntees bout input/output behviour of progrm!

91 91 END Lecture 5