Werner Nutt. Yehoshua Sagiv, Sara Shurin. The Hebrew University

Transcription

1 Deciding Equivalences among Aggregate Queries Werner Nutt German Research Center for AI (DFKI) Saarbrucken, Germany Yehoshua Sagiv, Sara Shurin The Hebrew University Jerusalem, Israel Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 1

2 Acknowledgement This work originated within the ESPRIT Long Term Research Project "Foundations of Data Warehouse Quality" (DWQ) Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 2

3 Motivation In recent year, increased interest in optimization of aggregate queries data warehousing decision support Aggregate queries are costly they touch many data items ; need for specialized optimization techniques Idea: Use previous results to answer new queries exploit redundancy! create redundancy! To do so, we have to be able to answer the question: \What can be computed from what?" (= the view usability problem) Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 3

4 Decision Support Queries Product Region Sales This Month Growth in Sales vs. Last Month Sales as % of Category Change in Sales as % of Category vs. Last Month Framis Central % 31% 3% Framis Eastern 179-3% 28% -1% Framis Western 55 5% 12% 1% Total Framis 344 6% 33% 1% Widget Central 66 2% 18% 2% Widget Eastern 102 4% 12% 5% Widget Western 39-9% 9% -1% Total Widget 207 1% 13% 4% Grand Total 551 4% 20% 2% Example of a business report. Exceptionally high values are marked with an asterisk (). Exceptionally low values are shown as bold. (The example is taken from R. Kimball, The Data Warehouse Toolkit, Addison Wesley) An SQL-query for the rst column: Select From Where p.product-name as Product, m.region name as Region, sum(f.sales) as Sales This Month sales fact f, product p, market m, time t f.product key = p.product key, f.market key = m.market key, f.time key = t.time key, p.product name in ('Framis', 'Widget'), t.month = 'May', t.year = 1996 Groupby p.product name, m.region name Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 4

5 Aggregate Queries: Abstract Notation SQL notation: Select p.a, s.b, max(r.c), sum(s.d), count(*) From p, r, s Where p.z = r.z, p.a = s.a, r.w = 'Joe', s.b < 10 Groupby p.a, s.b Abstract notation: q(a; B; max(c); sum(d); count) s(a; B; D) &p(a; Z) &r(z; C; W) & W = 0 Joe 0 & B 10 In general: q(x 1 ;:::;xm; 1 (y 1 );:::;n(yn)) R & C Short: q(x; (y)) R & C, with R C conjunction of relational atoms conjunction of comparisons Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 5

6 The View Usability Problem Given views and a query v i (x i ; i (y i )) R i & C i is there a query q(x; (y)) R & C; such that ~q(x; (y)) ~ R & ~ C; R consists of instantiations of the v i q and ~q are equivalent (i.e., q and ~q produce the same results over all databases) ; We need a syntactic characterization of equivalence! Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 6

7 Dimensions of the Problem q(x; (y) ) R & C Which aggregate functions? { min, max { count { sum { count distinct {... Queries { without comparisons { with comparisons over the: rationals, integers Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 7

8 Previous Work Equivalence of conjunctive queries Chandra/Merlin 1977, Klug 1988 View usability for conjunctive queries Levy et al Containment and equivalence of conjunctive queries under bag-semantics Chaudhuri/Vardi 1993 Equivalence preserving transformations of aggregate queries Levy/Mumick 1994, Gupta et al View usability for aggregate queries (sucient criteria) Srivastava et al View usability for data cubes Harinarayan et al. 1996, Gupta et al ; (Almost) no complete characterizations for aggregate queries! Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 8

9 Decouple Aggregations Observation: In q(prod; max(sales); sum(prot)) R & C, the aggregates max(sales), sum(prot), are functionally dependent on Prod. Denition: q(x; j (y j )) R & C is the j-th kernel of q(x; 1 (y 1 );:::;n(yn)) R & C. Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 9

10 Divide and Conquer Theorem: q(x; 1 (y 1 );:::;n(yn)) q 0 (x; 1 (y 1 );:::;n(yn)) are equivalent if and only if their kernels q j (x; j (y j )) q 0 j(x; j (y j )) are pairwise equivalent for all j 2 1::n. ; it suces to solve the equivalence problem for queries with a single aggregate term q(x; (y)) R & C (simple aggregate queries). Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 10

11 Aggregate Queries and Conjunctive Queries The core of q(x; (y)) is the conjunctive query R & C q(x; y) R & C: Examples: The core of is q(x; sum(y)) R & C q(x; y) R & C: The core of is q(x; count) R & C q(x) R & C: Strategy: Reduce equivalence of simple aggregate queries to properties of their cores. Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 11

12 Reminder on Conjunctive Queries Conjunctive queries have the form q(x) relational atom with variables R conjunction of relational atoms & C conjunction of comparisons q D := the result of q over database D q and q 0 are equivalent (written q q 0 ) i q D = q 0D for all db's D q is contained in q 0 (written q q 0 ) i q D q 0D for all db's D ; How can we check containment? Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 12

13 Query Homomorphisms An homomorphism from q 0 (x) R 0 & C 0 to q(x) R & C is a substitution such that x =x R 0 R C j= C 0. Theorem (Chandra/Merlin 77): For relational conjunctive queries: q q 0, there is an homomorphism from q 0 to q Finding an homomorphism is NP-complete! Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 13

14 Containment and Comparisons Classical example: q 0 p(u; v) &uv q p(y; z) &p(z; y) We have q q 0, but no homomorphism from q 0 to q. Idea: replace q with its linear expansion (q L ) L! q fy<zg p(y; z) &p(z; y) &y<z q fy=zg p(y; z) &p(z; y) &y = z q fy>zg p(y; z) &p(z; y) &y>z (case analysis) Theorem (Klug 88): q q 0, for every q L in (q L ) L, there is an homomorphism from q 0 to q L Containment with comparisons is P 2 -complete. (van der Meyden) Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 14

15 Relational Max-Queries q(x; max(y)) R q 0 (x; max(y)) R 0? Theorem: For relational max-queries: q q 0, the cores q and q 0 are equivalent Relational queries deliver the same max only if they deliver the same values! Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 15

16 Max-Queries with Comparisons The theorem fails if there are comparisons. q(max(y)) p(y) & p(z) &z<y q 0 (max(y)) p(y) & p(z 1 )&p(z 2 )&z 1 <z 2 Observations: q returns all elements of p, but the least q 0 returns all elements q and q 0 return answers if p has at least two elements ) the max-queries are equivalent. Which property of cores entails equivalence of max-queries? Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 16

17 Dominance Denition: q is dominated by q 0 i for every db, if q returns ( d; d), then q 0 returns some ( d; d0 ) such that d d 0. Proposition: For arbitrary max-queries: q q 0, the cores q and q 0 dominate each other ; How can we check dominance? Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 17

18 Dominance Mappings A dominance mapping from q 0 (x; y) R 0 & C 0 to q(x; y) R & C is like a homomorphism, except that C j= y (y) instead of y = (y). Theorem: q is dominated by q 0, for every q L in the lin. expansion of q, there is a dominance mapping from q 0 to q L Corollary: Equivalence of max-queries is P 2 -complete. Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 18

19 Relational Count-Queries q(x; count) R q 0 (x; count) R 0? q(x) and q 0 (x) return the same results with the same multiplicities over every db, i.e., q(x) and q 0 (x) are bag-set-equivalent Theorem (Chaudhuri/Vardi 93): If q, q 0 are relational queries, then q and q 0 are bag-set-equivalent, q and q0 are isomorphic I.e., q and q 0 are the same, up to renaming of variables. Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 19

20 Count-Queries with Comparisons The theorem fails if there are comparisons. q(count) p(x) &p(y) &p(z) & x<y& x<z q 0 (count) p(x) &p(y) &p(z) & x<z& y<z Observations: q and q 0 are not isomorphic q and q 0 are equivalent How can we check bag-set-equivalence? Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 20

21 Isomorphic Linear Expansions Let (q L ) L, (q 0 M ) M be linear expansions of q, q 0. Denition: (q L ) L and (q 0 M ) M are isomorphic i there is a bijection :(q L ) L! (q 0 M ) M such that q L and q 0 (L) are isomorphic, for all L. t q t L1 qm 0 1 X > t XXXXXXXXXXXXXXXX X Xz XXXXXXXXXXXXXXXX t Xz Z ZZ Z ZZ t Z ZZ * Z. ZZ. Z ZZ Z Z t Z~ q L2 t q 0 M 2 q L3 t q 0 M 3 q L4 t q 0 M 4 q Lk t q 0 M k Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 21

22 Equivalence of General Count-Queries Theorem: If q, q 0 are arbitrary conjunctive queries, then q and q 0 are bag-set-equivalent, q and q 0 have isomorphic linear expansions Corollary: The following can be decided with PSPACE: Bag-set-equivalence of conjunctive queries Equivalence of count-queries Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 22

23 Sum-Queries When are q(x; sum(y)) q 0 (x; sum(y))? Bag-set-equivalence of the cores is a sucient condition. But is it necessary? Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 23

24 Sum-Queries: Diculty 1 The cores of q and q 0 are not bag-set-equivalent: q(sum(y)) q 0 (sum(y)) p(1) & p(2) & p(3) & p(y) &1 y 3 p(1) & p(2) & p(3) & p(y) &1 y 2& p(z) &1 z 2 Over the integers, q and q 0 are equivalent, since 1+2+3= : Over the rationals, they are not. Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 24

25 Sum-Queries: Diculty 2 q(sum(y)) q 0 (sum(y)) p(y) &0<y& p(z) &0<z& p(w) &0 w p(y) &0 y & p(z) &0<z& p(w) &0 w q and q 0 return non-zero numbers with the same multiplicity...but q 0 may return 0, while q does not ) q and q 0 are equivalent, but the cores are not Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 25

26 Equivalence of Sum-Queries Diculties arise with comparisons and constants. Theorem: If q(x; sum(y)), q 0 (x; sum(y)) are sum-queries without comparisons or without constants, then q and q 0 are equivalent, the cores q and q0 are bag-set-equivalent In the general case, the characterization is complex. Theorem: For sum-queries with comparisons over the integers, or over the rationals, equivalence can be decided with PSPACE. Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 26

27 Linear Queries A query is linear if there are no multiple occurrences of the same predicate. Known: Containment of linear conjunctive queries under set-semantics is PTIMEdecidable. We have generalized this result: Theorem: For linear queries, the following problems are in PTIME: bag-set-equivalence of conjunctive queries equivalence of max-queries equivalence of count-queries equivalence of sum-queries Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 27

28 Conclusion Complete characterizations for the equivalence of aggregate queries with min, max, count, and sum. Characterization for special cases of queries with count distinct Polynomiality in the case of linear queries Foundation for solving the view usability problem for aggregate queries. Deciding Equivalences among Aggregate Queries PODS June 1998 { Slide 28