A Functional Language for Hyperstreaming XSLT

Size: px

Start display at page:

Download "A Functional Language for Hyperstreaming XSLT"

Dwayne Barrett
6 years ago
Views:

1 A Functional Language for Hyperstreaming XSLT Pavel Labath 1, Joachim Niehren 2 1 Commenius University, Bratislava, Slovakia 2 Inria Lille, France May 2013

2 Motivation some types of tree transformations cannot be streamed at all with bounded memory practical example: given a book, produce a copy of the book with table of contents added Current approach of the XSLT working group is to restrict streaming support to a smaller fragment, where the situation above does not occur. Our approach: generalize the single input stream single output stream model output will be a collection of streams (a shredded stream) that can later be plugged together to form a simple stream

3 Outline Shredded trees X-Fun Compiler of XSLT to X-Fun Evaluation of X-Fun programs Implementation and Experiments

4 Trees, Hedges, Nested words trees, hedges (forests) over alphabet Σ nested word linearization of a tree/hedge a e.g., linearization of b c d is <a><c/><d/></a>

5 Shredded hedges, Shredded nested words a shredded hedge h = (h 0, (y i, h i ) n i=1 over Σ consists of variables y i, an ordinary root hedge h 0 over Σ {y 1,... y n }, and ordinary hedges h i over Σ {y i+1,..., y n } shredded nested word w = (w 0, (y i, w i ) n i=1 ) to convert a shredded hedge into an ordinary one, gradually substitute variables y i by corresponding hedges. unshred(h) = h 0 [h 1 /y 1 ] [h n /y n ] unshred(w) = w 0 [w 1 /y 1 ] [w n /y n ]

6 Example: A shredded nested word h = (h 0, (y 1, h 1 ), (y 2, h 2 ), (y 3, h 3 )) h 0 = <A>y 1 y 2 </A> h 1 = <C>y 3 </C> h 2 = <D/> h 3 = <E/>

7 Outline Shredded trees X-Fun Compiler of XSLT to X-Fun Evaluation of X-Fun programs Implementation and Experiments

8 Example <addresses > <address > <name>jemal Antidze </name> < com</ > <phone> </ phone> <phone> <secret/> </ address >... </ addresses > translate <ol > < l i > Jemal Antidze Phone: </ l i >... </ ol > </ phone>

9 Example: transformation algorithm f u n c t i o n convert_address ( x ) name = t e x t of the name c h i l d of x output node l i : output node p : output name for each c h i l d node x labeled phone and w i t h o u t a s e c r e t c h i l d : output node p : output t e x t Phone : output the t e x t of x main f u n c t i o n : output node o l : for each descendant node x labeled address : c a l l convert_address ( x )

10 Example: transformation in X-Fun l e t convert_address ( x ) = l e t name = for x in x / c h i l d : : name do t e x t ( x ) in t r e e ( l i, t r e e ( p, name) for x in x / c h i l d : : phone [ not ( c h i l d : : s e c r e t ) ] do t r e e ( p, Phone : t e x t ( x ) ) end end in t r e e ( ol, for x in r o o t / descendant : : address do convert_address ( x ) end ) end

11 X-Fun Syntax A ::= y (hedge variable) subtree(x) tree(l, A) T A 1 A n let D in A q(x, A 1,..., A n ) for x in x/p do A (V node (A) {x }) case N of F 1 A 1... F n A n compose x : A in A shred(a, T )

12 X-Fun features ability to construct shredded output ability to reprocess constructed output X-Fun programs can have three types of variables node variables (x) range over nodes of the input tree hedge variables (y) can contain parts of constructed output function variables q for functions at most one active node variable in every block

13 Relationship to TL [Maneth et al., 2005] a TL program is a Walking Macro Tree Transducer T = (Q, Σ,, q 0, R) rules R are of the form q(x 1 F, y 1,... y m ) A, where A is defined recursively as A ::= ε a(a) AA y j q(x 2 x 1 /P, A,..., A)

14 TL: semantics x/p = sequence of nodes in the input tree selected by (XPath) expression P when starting from node x. evaluation of q(x 2 x 1 /P, A 1,..., A m ): let x 1 /P = v 1,... v n. for each node v i : let q(x 1 F, y 1,..., y m ) A i be the first rule, such that v i F. replace q(x 2 x 1 /P, A 1,..., A m ) by A 1 [A 1/y 1, A m /y m ] A n[a 1 /y 1, A m /y m ]

15 TL: translation to X-Fun q(x 1 F 1 ) a(q(x2 x1/p)) q(x 1 F 2 ) b() translate q ( x ) = case x of F1 t r e e ( a, for x in x / P do q ( x ) ) F2 t r e e ( b, ε ) end

16 Outline Shredded trees X-Fun Compiler of XSLT to X-Fun Evaluation of X-Fun programs Implementation and Experiments

17 XSLT XSLT is a popular language for transforming XML documents our compiler can handle a large fragment of XSLT and enables us to automatically stream many existing transformations this also fascilitates comparison of X-Fun with already existing streaming implementations

18 XSLT: example < x s l : template name="q " match ="b " mode="m" > <C>< x s l : c a l l template name="p "/ > </C> </ x s l : template > < x s l : template name="p " match ="a " mode="m" > < x s l : apply templates s e l e c t =" c h i l d : : * " mode="m"/ > </ x s l : template > translate q ( x ) = t r e e (C, p ( x ) ) p ( x ) = t r e e (B, for x in x / c h i l d : : * do m( x ) ) m( x ) = case x in [ s e l f : : a ] p ( x ) [ s e l f : : b ] q ( x )

19 Outline Shredded trees X-Fun Compiler of XSLT to X-Fun Evaluation of X-Fun programs XPath streaming machine Functional engine Implementation and Experiments

20 XPath streaming machine (XSM) When processing an input tree, we shall create one XSM for every XPath expression and filter in the X-Fun program the machines will read the entire tree and evaluate the expression/filter the machines will operate synchronously possibility to quickly discard input

21 XPath streaming machine (XSM) has two input streams and a single output stream input stream 1: input nested word w input stream 2: word from {0, 1} of the same length as w value 1 in a given position signals the machine to start evaluating the expression or filter from the current node

22 XPath streaming machine output stream contains tuples (x, z, S) for XPath expressions and (x, S) for filters x - starting node of the expression or filter z - target node of the XPath expression S {select, reject, alive} whether the node got selected, rejected, or the result is not yet known

23 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM Output: input $

24 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM Output: input $

25 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM Output: (v 1, alive) input $

26 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM Output: (v 1, alive) input $

27 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM input $ Output: (v 1, alive) (v 2, alive)

28 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM input $ Output: (v 1, alive) (v 2, alive) (v 2, select)

29 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM input $ Output: (v 1, alive) (v 2, alive) (v 2, select)

30 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM input $ Output: (v 1, alive) (v 2, alive) (v 2, select)

31 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM input $ Output: (v 1, alive) (v 2, alive) (v 2, select)

32 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM input $ Output: (v 1, alive) (v 2, alive) (v 2, select)

33 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM input $ Output: (v 1, alive) (v 2, alive) (v 2, select)

34 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM input $ Output: (v 1, alive) (v 2, alive) (v 2, select) (v 1, reject)

35 XSM for filter [self::a/child::b] input 1 <a> <a> </a> </a> $ XSM input $ Output: (v 1, alive) (v 2, alive) (v 2, select) (v 1, reject)

36 Functional engine coordinates XSM sub-machines and drives them almost synchronously computes the second input stream for the sub-machines buffers parts of the input, if needed (for evaluating subtree(x) expressions) produces output by gradually unfolding/rewriting the program tree completed prefix is immediately flushed to the output

37 Speculative execution if we are unsure if we will need an output fragment, we need to compute it even if it may turn out to be redundant if an XSM for a path expression returns alive, we must start evaluating the loop body immediately same holds for filter expressions in case statements we evaluate all potential branches until we are sure which one is selected if a path XSM returns (, v, alive), we may need to start evaluating loop body even before we reach the loop in the main program evaluation tree

38 Implementation functional prototype implemented in Java supported fragment already exceeds the streamable fragment of Saxon some features still under construction we used FXP library ( with slight interface modifications, as an implementation of XSM XSLT compiler uses QuiXPath library ( for translation from XPath to FXP

39 Experiments experiments show that our unoptimized implementation is comparable to Saxon operating in streaming mode time ratio is between 2 and 6 in favor of Saxon ratio is slightly worse when comparing to Saxon operating in memory however, here we perform much better from memory usage point of view

40 Conclusion we have introduced a functional language X-Fun language combines streaming with arbitrary document navigation immediate practical aplications through translation from XSLT Outlook general for loops translation from XQuery, XProc support a larger fragment of XPath expressions...

41 Thank you

A uniform programming language for implementing XML standards

A uniform programming language for implementing XML standards Pavel Labath 1, Joachim Niehren 2 1 Commenius University, Bratislava, Slovakia 2 Inria Lille, France Sofsem 2015 Motivation Different data