Lecture 3. XML Into RDBMS. XML and Databases. Memory Representations. Memory Representations. Traversals and Pre/Post-Encoding. Memory Representations

Transcription

1 Leture XML into RDBMS XML n Dtses Sestin Mneth NICTA n UNSW Leture XML Into RDBMS CSE@UNSW -- Semester, 00 Memory Representtions Memory Representtions Fts DOM is esy to use, ut memory hevy. in-memory size usully -0 times lrger thn size of originl file. SAX is very flexile. Using rrys or inry trees wo kwr pointers, in-memory size is pprox. sme s size of originl file. Cn e further improve using DAGs/shring-Grphs & oing/ompression for t vlues. Fts DOM is esy to use, ut memory hevy. in-memory size usully -0 times lrger thn size of originl file. SAX is very flexile. Using rrys or inry trees wo kwr pointers, in-memory size is pprox. sme s size of originl file. Cn e further improve using DAGs/shring-Grphs & oing/ompression for t vlues. TODAY How n we mp XML into reltionl DB? Memory Representtions Trversls n Pre/Post-Enoing Fts DOM is esy to use, ut memory hevy. in-memory size usully -0 times lrger thn size of originl file.. Pre-Orer Trversl (reursively). Post-Orer SAX is very flexile. Using rrys or inry trees wo kwr pointers, in-memory size is pprox. sme s size of originl file.. Pre-Orer (itertively). Into RDBMS with Pre/Post-enoing Cn e further improve using DAGs/shring-Grphs & oing/ompression for t vlues. TODAY TODAY How n we mp XML into reltionl DB? How n we mp XML into reltionl DB? ut first: Memory effiient tree trversls using e.g. DOM? ut first: Memory effiient tree trversls using e.g. DOM?

2 Tree Trversls Tree Trversls Strt t root noe; wnt to visit every noe. Strt t root noe; wnt to visit every noe. () reursively firstchil lstchil firstchil lstchil [ ] hilnoes [ ] hilnoes Tree Trversls Strt t root noe; wnt to visit every noe. Tree Trversls Strt t root noe; wnt to visit every noe. Tr() pr() Tr() pr() 0 () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) firstchil lstchil [ ] hilnoes firstchil lstchil [ ] hilnoes Tree Trversls Strt t root noe; wnt to visit every noe. () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) Tr() pr() Tr() pr() Tr() pr() Tr() pr() Tree Trversls Strt t root noe; wnt to visit every noe. () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) Tr() pr() Tr() pr() Tr() pr() Tr() pr() Tr() pr() Tr() pr() firstchil lstchil [ ] hilnoes firstchil lstchil [ ] hilnoes

3 firstchil Tree Trversls Strt t root noe; wnt to visit every noe. () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) lstchil [ ] hilnoes reursion epth = Tr() pr() Tr() pr() Tr() pr() Tr() pr() Tr() pr() Tr() pr() Tree Trversls Strt t root noe; wnt to visit every noe. () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) reursion epth = firstchil lstchil 0 [ ] hilnoes Tr() pr() Tr() pr() Tr() pr() Tr() pr() Tr() pr() Tr() pr() Tr() pr() Tr() pr() Tr() pr() Tr(0) pr(0) Tr() pr() Tr() pr() END Tree Trversls Tree Trversls Strt t root noe; wnt to visit every noe. () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) Memory nee proportionl Strt t root noe; wnt to visit every noe. () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) Memory nee proportionl Shoul e fine. Usully height (XML o) is smll. ( ) Shoul e fine. Usully height (XML o) is smll. ( ) Prolemti n reursion on hilren TR(n:Noe){ if(m=firstchil(n))!=nil then Tr(m); if(m=nextsiling(n))!=nil then Tr(m) Tree Trversls Tree Trversls Strt t root noe; wnt to visit every noe. () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) Memory nee proportionl Strt t root noe; wnt to visit every noe. () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) Memory nee proportionl Shoul e fine. Usully height (XML o) is smll. ( ) Shoul e fine. Usully height (XML o) is smll. ( ) Prolemti n reursion on hilren TR(n:Noe){ if(m=firstchil(n))!=nil then Tr(m); if(m=nextsiling(n))!=nil then Tr(m) Prolemti n reursion on hilren TR(n:Noe){ if(m=firstchil(n))!=nil then TR(m); if(m=nextsiling(n))!=nil then TR(m) A B C D E F TR();pr() TR();pr() TR();pr() TR(A);pr(A) TR(B);pr(B)

4 Tree Trversls Tree Trversls 0 Strt t root noe; wnt to visit every noe. () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) Memory nee proportionl Strt t root noe; wnt to visit every noe. () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) Memory nee proportionl Shoul e fine. Usully height (XML o) is smll. ( ) Shoul e fine. Usully height (XML o) is smll. ( ) Prolemti n reursion on hilren A B C D E F TR(n:Noe){ if(m=firstchil(n))!=nil then Tr(m); if(m=nextsiling(n))!=nil then Tr(m) TR();pr() TR();pr() TR();pr() TR(A);pr(A) TR(B);pr(B) Memory nee proportionl to mx. length of (firstchil nextsiling)*-pth Prolemti n reursion on hilren A B C D E F Tr(n:Noe){ if(m=firstchil(n))!=nil then Tr(m); if(m=nextsiling(n))!=nil then Tr(m) TR();pr() TR();pr() TR();pr() TR(A);pr(A) TR(B);pr(B) Cn e HUGE!!! Memory nee =size(tree) proportionl to mx. length of (firstchil nextsiling)*-pth Tree Trversls firstchil lstchil [ ] hilnoes Strt t root noe; wnt to visit every noe. () reursively Trverse(n:Noe){ For m in hilnoes(n) Trverse(m) Memory nee proportionl Shoul e fine. Usully height (XML o) is smll. ( ) Tr(n:Noe){ if(m=firstchil(n))!=nil then Tr(m); if(m=nextsiling(n))!=nil then Tr(m) Wht is the mx reursion epth on this tree? Cn e HUGE!!! Memory nee =size(tree) proportionl to mx. length of (firstchil nextsiling)*-pth Prolemti n reursion on hilren A B C D E F Rell inry tree (firstchil/nextsiling) enoing. In the inry tree, wht orrespons to this numer? TR();pr() TR();pr() TR();pr() TR(A);pr(A) TR(B);pr(B) mx. length of (firstchil nextsiling)*-pth Binry Tree Enoing Rell firstchil/nextsiling enoing. The firstchil eomes the left pointer The nextsiling eomes the right pointer per Noe pointers/ids + lel info oth, Trverse n TR n e exeute on the f/ns-inry tree enoing. Tree Trversls firstchil 0 ID f:ns:l (:-:) (-::) (::) (-::) (::) (-::) (-:-:) (-:-:) (-:0:) 0 (-,-:) firstchil 0

5 Tree Trversls Tree Trversls oth, Trverse n Tr n e exeute on the f/ns-inry tree enoing. For m in hilnoes(n) Trverse(n) oth, Trverse n Tr n e exeute on the f/ns-inry tree enoing. For m in hilnoes(n) Trverse(n) if(m=n->left)!=nil { while(m=n->right)!=nil Trverse(m) if(m=n->left)!=nil { while(m=n->right)!=nil Trverse(m) firstchil 0 if(m=firstchil(n))!=nil then TR(m); if(m=nextsiling(n))!=nil then TR(m) if(m=n->left)!=nil then TR(m) if(m=n->right)!=nil then TR(m) firstchil 0 if(m=firstchil(n))!=nil then Tr(m); if(m=nextsiling(n))!=nil then Tr(m) if(m=n->left)!=nil then TR(m) if(m=n->right)!=nil then TR(m) Reursion tkes re of the ft tht we o not hve prent pointers. (true for oth, unrnke & inry tree) Other Trversls Other Trversls We isusse the Pre-orer of the tree. (or, in XML-jrgon: oument-orer). Relize y Trverse & TR We isusse the Pre-orer of the tree. (or, in XML-jrgon: oument-orer). Relize y Trverse & TR Post-orer of tree =. Trverse left sutree in post-orer. Trverse right sutree in post-orer. Visit the root Reverse Polish Nottion Post-orer of tree =. Trverse left sutree in post-orer. Trverse right sutree in post-orer. Visit the root Reverse Polish Nottion In-orer Breth-First (left-to-right) level-orer Breth-First (left-to-right) level-orer Binry Serh Tree (inresing) q.enq(root); while(not q.empty){ Visit(q.eq); If(q->Left!=NIL) q.enq(q->left) If(q->Right!=NIL) q.enq(q->right) Memory nee proportionl to mx. #noes on one level q.enq(root); while(not q.empty){ Visit(q.eq); If(q->Left!=NIL) q.enq(q->left) If(q->Right!=NIL) q.enq(q->right) Other Trversls Pre-Orer 0 We isusse the Pre-orer of the tree. (or, in XML-jrgon: oument-orer). Relize y Trverse & TR Post-orer of tree =. Trverse left sutree in post-orer. Trverse right sutree in post-orer. Visit the root Reverse Polish Nottion We sw how to ompute Pre-orer (n Post n In) () reursively memory nee: O(mx_height) Cn e HUGE!!! Memory nee proportionl to mx. #noes on one level Breth-First (left-to-right) level-orer q.enq(root); while(not q.empty){ Visit(q.eq); If(q->Left!=NIL) q.enq(q->left) If(q->Right!=NIL) q.enq(q->right); 0

6 Pre-Orer Pre-Orer We sw how to ompute Pre-orer (n Post n In) () reursively memory nee: O(mx_height) How to ompute Pre-orer () itertively Memory nee? while(firstchil(n)!=nil) { n=firstchil(n); We sw how to ompute Pre-orer (n Post n In) () reursively memory nee: O(mx_height) pre() How to ompute Pre-orer () itertively Memory nee? while(firstchil(n)!=nil) { n=firstchil(n); Pre-Orer Pre-Orer We sw how to ompute Pre-orer (n Post n In) () reursively memory nee: O(mx_height) pre() How to ompute Pre-orer () itertively Memory nee? while(firstchil(n)!=nil) { n=firstchil(n); We sw how to ompute Pre-orer (n Post n In) () reursively memory nee: O(mx_height) pre() How to ompute Pre-orer () itertively Memory nee? repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n); Pre-Orer Pre-Orer We sw how to ompute Pre-orer (n Post n In) () reursively memory nee: O(mx_height) pre() How to ompute Pre-orer () itertively Memory nee? repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n); We sw how to ompute Pre-orer (n Post n In) () reursively memory nee: O(mx_height) pre() How to ompute Pre-orer () itertively Memory nee? repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n);

7 Pre-Orer Pre-Orer We sw how to ompute Pre-orer (n Post n In) () reursively memory nee: O(mx_height) pre() How to ompute Pre-orer () itertively Memory nee? repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n); We sw how to ompute Pre-orer (n Post n In) () reursively memory nee: O(mx_height) pre() How to ompute Pre-orer () itertively Memory nee? repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n); Pre-Orer Pre-Orer 0 We sw how to ompute Pre-orer (n Post n In) () reursively memory nee: O(mx_height) pre() How to ompute Pre-orer () itertively Memory nee? repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n); We sw how to ompute Pre-orer (n Post n In) () reursively memory nee: O(mx_height) pre() pre() How to ompute Pre-orer () itertively Memory nee? repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n); pre() pre() Pre-Orer Pre-Orer We sw how to ompute Pre-orer (n Post n In) () reursively memory nee: O(mx_height) pre() pre() pre() pre(0) How to ompute Pre-orer () itertively Memory nee? repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n); How to ompute Pre-orer () itertively Memory nee? pre() pre(0) pre() pre() No reursion! Nees onstnt memory! (only one pointer) repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n); pre() if(n=nil)then rek; pre() if(n=nil)then rek;

8 Pre-Orer Pre-Orer Given inry tree, (top-own, no prent) how muh memory o you nee to ompute pre? No reursion! Nees onstnt memory! (only one pointer) Given inry tree, (top-own, no prent) how muh memory o you nee to ompute pre? No reursion! Nees onstnt memory! (only one pointer) pre() pre() pre() pre(0) repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n); Cn you o it w. onstnt memory? pre() pre() pre() pre(0) repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n); pre() if(n=nil)then rek; pre() if(n=nil)then rek; Pre-Orer Pre-Orer Fun (MS ji) How muh memory you nee to hek for yles, in single-linke (pointer) list? No reursion! Nees onstnt memory! (only one pointer) Do you see how to o Post- n In-oers itertively? No reursion! Nees onstnt memory! (only one pointer) pre() pre() pre() pre(0) repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n); pre() pre() pre() pre(0) repet { while(firstchil(n)!=nil) { n=firstchil(n); while(nextsiling(n)=nil) { n=prent(n); n=nextsiling(n); pre() if(n=nil)then rek; pre() if(n=nil)then rek; From pre( ) Pre-Orer From pre( ) Pre-Orer Wht is this? we n ompute PreFollowing( n ) = { noes m with pre(m) > n PrePreeing( n ) = { noes m with pre(m) < n we n ompute PreFollowing( n ) = { noes m with pre(m) > n PrePreeing( n ) = { noes m with pre(m) < n 0 pre 0 0 pre 0

9 From pre( ) Pre-Orer Wht is this? From pre( ) Pre-Orer useful?? 0 we n ompute PreFollowing( n ) = { noes m with pre(m) > n PrePreeing( n ) = { noes m with pre(m) < n we n ompute PreFollowing( n ) = { noes m with pre(m) > n PrePreeing( n ) = { noes m with pre(m) < n Not tree (nvigtion) omplete 0 pre foll pre hil? ns? 0 pre 0 XML to RDBMS Enoing

10 pre( ) is not enough Other possiilities: XML to RDBMS Enoing lrge (unprse) text lo Shem-se enoing Ajeny-se enoing De Ens Not goo Goo possiility: use Pre- n Post-orer of noe! 0 Pre/Post Enoing A POST orer 0 0 PRE POST l 0 0 CREATE VIEW esennt AS SELECT r.pre,r.pre FROM R r, R r WHERE r.pre<r.pre AND r.post>r.post 0

11 Pre/Post Enoing A POST orer 0 0 PRE POST l 0 0 CREATE VIEW esennt AS SELECT r.pre,r.pre FROM R r, R r WHERE r.pre<r.pre AND r.post>r.post struturl join Pre/Post Enoing Strightforwr how to ompute, for given noe, esennts nestors following preeing s How to o lstchil prent hilnoes Cn you fin orresponing SQL queries? END Leture