Foundations of Databases

Transcription

1 Foundations of Databases (Slides adapted from Thomas Eiter, Leonid Libkin and Werner Nutt) Foundations of Databases 1 Quer optimization: finding a good wa to evaluate a quer Queries are declarative, and can be translated into procedural languages in more than one wa Hence one has to choose the best (or at least good) procedural quer This happens in the contet of quer processing A quer processor turns queries and updates into sequences of operations on the database

2 Foundations of Databases 2 Stages Queries are translated into an etended relational algebra (operator + eecution method): Which algebra epressions will allow for an efficient eecution? Algebraic operators can be implemented b different methods (Eamples?): Which algorithm should be used for each operator? How do operators pass data (write into main memor, write on disk, pipeline into other operators)? Issues: Translate the quer into etended relational algebra ( quer plans ) Esimate the eecution of the plans: We need to know how data is stored, how it accessed, how large are intermediate results, etc. Decisions are based on general guidelines and statistical information Foundations of Databases 3 Overview of Quer Processing Start with a declarative quer: SELECT R.A, S.B, T.E FROM R,S,T WHERE R.A>5 AND S.B<3 AND T.D=T.E Translate into an algebra epression: π R.A,S.B,T.E (σ R.A>5 S.B<3 T.D=T.E (R S T )) Optimization step: rewrite to an equivalent but more efficient epression: π R.A,S.B,T.E (σ A>5 (R) σ B<3 (S) σ D=E (T ))) Wh ma this be more efficient? Is there a still more efficient plan?

3 Foundations of Databases 4 Optimization b Algebraic Equivalences Given a relational algebra epression E, find another epression E equivalent to E that is easier (faster) to evaluate. Basic question: Given two relational algebra epressions E 1, E 2, are the equivalent? If there were a method to decide equivalence, we could turn it into one to decide whether an epression E is empt, i.e., whether E(I) = for ever instance I. How does one show this? Problem: Testing whether E is undecidable for epressions of full relational algebra i.e., including union and difference. Good news: We can still list some useful equalities. Equivalence (and emptiness testing) is decidable for important classes of Foundations of Databases 5 queries (SPJR, SPC).

4 Foundations of Databases 6 Optimization b Algebraic Equivalences (cntd) Sstematic wa of quer optimization: Appl equivalences and are commutative and associative, hence applicable in an order Cascaded projections can be simplified: If the attributes A 1,..., A n are among B 1,..., B m, then π A1,...,A n (π B1,...,B m (E)) = π A1,...,A n (E) Cascaded selections might be merged: σ C1 (σ C2 (E)) = σ C1 C 2 (E) Commuting selection with join. If c onl involves attributes from E 1, then σ C (E 1 E 2 ) = σ C (E 1 ) E 2 Foundations of Databases 7 Optimization b Algebraic Equivalences (contd) Rules combining σ, π with and \. etc Commuting selection and union: Commuting selection and difference: Commuting projection and union: σ C (E 1 E 2 ) = σ C (E 1 ) σ C (E 2 ) σ C (E 1 \ E 2 ) = σ C (E 1 ) \ σ C (E 2 ) π A1,...,A n (E 1 E 2 ) = π A1,...,A n (E 1 ) π A1,...,A n (E 2 ) Question: what about projection and difference? Is π A (E 1 \ E 2 ) equal to π A (E 1 ) \ π A (E 2 )?

5 Foundations of Databases 8 Optimization of Conjunctive Queries Reminder: Conjunctive queries = SPJR queries = simple SELECT-FROM-WHERE SQL queries (onl AND and (in)equalit in the WHERE clause) Etremel common, and thus special optimization techniques have been developed Reminder: for two relational algebra epressions E 1, E 2, E 1 = E 2 is undecidable. But for conjunctive queries, even E 1 E 2 is decidable. Main goal of optimizing conjunctive queries: reduce the number of joins. Foundations of Databases 9 Optimization of Conjunctive Queries: An Eample Given a relation R with two attributes A, B Two SQL queries: Q1 Q2 SELECT DISTINCT R1.B, R1.A SELECT DISTINCT R3.A, R1.A FROM R R1, R R2 FROM R R1, R R2, R R3 WHERE R2.A=R1.B WHERE R1.B=R2.B AND R2.B=R3.A Are the equivalent? If the are, we can save one join operation. In relational algebra: Q 1 = π 2,1 (σ 2=3 (R R)) Q 2 = π 5,1 (σ 2=4 4=5 (R R R))

6 Foundations of Databases 10 Optimization of Conjunctive Queries (contd) We will show that Q 1 and Q 2 are equivalent! We cannot do this b using our equivalences for relational algebra epression. (Wh?) Alternative idea: CQs are like patterns that have to be matched b a database. If a CQ returns an answer over a db, a part of the db must look like the quer. Use rule based notation to find a representation of part of the db: Q 1 (, ) : R(, ), R(, z) Q 2 (, ) : R(, ), R(w, ), R(, u) Foundations of Databases 11 Containment is a Ke Propert Definition. (Quer containment) A quer Q is contained in a quer Q (written Q Q ) if Q(I) Q (I) for ever database instance I. Translations: If we can decide containment for a class of queries, then we can also decide equivalence. If Q is a class of queries that is closed under intersection, then the containment problem for Q can be reduced to the equivalence problem for Q.

7 Foundations of Databases 12 Checking Containment of Conjunctive Queries: Ideas We want to check containment of two CQs Q, Q We consider CQs without equalities and inequalities We also assume that Q and Q have the same arit and identical vectors of head variables, that is, the are defined as Q( ) : B, Q ( ) : B Intuition: a conjunctive quer Q( ) : B returns an answer over instance I if I matches the pattern B Instead of checking containment over all instances, we consider onl finitel man test instances (or better, one!) Foundations of Databases 13 Tableau Notation of Conjunctive Queries Tableau notation blurs the distinction between quer and database instance We first consider queries over a single relation Q 1 (, ) : R(, ), R(, z) Q 2 (, ) : R(, ), R(w, ), R(, u) Tableau: A B A B z w answer line u answer line Variables in the answer line (or summar) are called distinguished

8 Foundations of Databases 14 Tableau Homomorphisms Tableau (as well as database instances or first order structures) are compared b homomorphisms Idea: T is more general than T if there is a homomorphism from T to T Reminder: Terms are variables or constants A homomorphism δ from Tableau T 1 to tableau T 2 is a mapping such that δ : {variables of T 1 } {terms of T 2 } δ() = for ever distinguished if (t 1,..., t k ) is a row in T 1, then (δ(t 1 ),..., δ(t k )) is a row in T 2, where δ is etended to constants such that δ(a) = a for ever constant a Foundations of Databases 15 Tableau for Queries with Multiple Relations So far we assumed that there is onl one relation R, but what if there are man? Construct a tableau for the quer: Q(, ): S(, ), R(, z), R(, w), R(w, ) We create rows for each relation: A B S: A B R: z w w Formall, a tableau is just a database where variables can appear in tuples, plus a set of distinguished variables.

9 Foundations of Databases 16 Tableau Homomorphisms: General Case Let T 1, T 2 be tableau with the same distinguished variables A homomorphism δ from T 1 to T 2 is a mapping δ : {variables of T 1 } {terms of T 2 } such that δ(a) = a for ever constant δ() = for ever distinguished variable if t is a row of R in T 1, then δ( t) is a row of R in T 2, for ever relation R Foundations of Databases 17 The Homomorphism Theorem for Tableau Homomorphism Theorem: Let Q, Q be two conjunctive queries without equalities and inequalities that have the same distinguished variables and let T, T be their tableau. Then Q Q there eists a homomorphism from T to T

10 Foundations of Databases 18 Proof: Necessit Assume that Q Q Note that T, ignoring the declaration of distinguished variables, can be seen as a database instance Note also that Q(T ) Since Q Q, it follows that Q (T ) There is a valuation ν such ν satisfies Q over T ν( ) = Thus, defining δ() = ν() for ever variable in Q, we obtain a homomorphism from T to T Foundations of Databases 19 Proof: Sufficienc Assume there is a homomorphism δ from T to T. We will show that Q Q Let I be a db instance. We show that Q(I) Q (I) Let a Q(I). We show that a Q (I) There is a valuation ν : { variables of Q } dom(i) such that ν satisfies B, the bod of Q (that is, ν(b) I in logic programming perspective) ν( ) = a

11 Foundations of Databases 20 Proof: Sufficienc (cntd) Let ν := ν δ. Then ν is a valuation for Q with the following properties. First, ν ( ) = ν(δ( )) = ν( ) = a. Moreover, if R( t) is an atom of B, then δ(r( t)) = R(δ( t)) is an atom of B, since δ is a homomorphism, and ν (R( t)) = ν(δ(r( t))) = ν(r(δ( t))) is an atom in I, since ν satisfies B Hence, a Q (I) and the other claims are shown as well. Foundations of Databases 21 Appling the Homomorphism Theorem: Q 1 Q 2 T1 T2 A B A B f()=, f()= z w f(u)=z, f(w)= u Hence Q1 Q2 T1 T2 A B z A w B f()=, f()= f(z)=u u Hence Q2 Q1

12 Foundations of Databases 22 Quer Containment: Eercise Find all containments and equivalences among the following conjunctive queries: q 1 (, ) : r(, ), r(, z), r(z, w) q 2 (, ) : r(, ), r(, z), r(z, u), r(u, w) q 3 (, ) : r(, ), r(z, u), r(v, w), r(, z), r(, u), r(u, w) q 4 (, ) : r(, ), r(, 3), r(3, z), r(z, w) Foundations of Databases 23 Quer Containment: Compleit Given two conjunctive queries, how hard is it to test whether Q 1 Q 2? It is eas to transform them into tableau, from either SPJ relational algebra queries, or SQL queries, or rule-based queries. However, a polnomial algorithm for deciding equivalence is unlikel to eist: Theorem. Given two tableau, deciding the eistence of a homomorphism between them is NP-complete. In practice, quer epressions are small, and thus conjunctive quer optimization is nonetheless feasible in polnomial time

13 Foundations of Databases 24 Minimizing Conjunctive Queries Goal: Given a conjunctive quer Q, find an equivalent conjunctive quer Q with the minimum number of joins. Questions: How man such queries can eist? How different are the? Assumption: We consider onl CQs without equalities and inequalities. We call these queries simple conjunctive queries (SCQs). If nothing else is said in this chapter, CQs are simple CQs Terminolog: If Q( ) : R 1 ( u 1 ),..., R k ( u k ) is a CQ, then Q is a subquer of Q if Q is of the form where 1 i 1 < i 2 <... < i l k. Q ( ) : R i1 ( u i1 ),..., R il ( u il ) Foundations of Databases 25 Minimization: Background Theor Proposition 1. Let q be a SCQ with n atoms and q be an equivalent SCQ with m atoms where m < n. Then there eists a subquer q 0 of q such that q 0 has at most m atoms in the bod and q 0 is equivalent to q. Proposition 2. Let q and q be two equivalent minimal SCQs. Then q and q are identical up to renaming of variables. Conclusions: There is essentiall one minimal version of each SCQ Q. We can obtain it b dropping atoms from Q s bod.

14 Foundations of Databases 26 An Algorithm for Minimizing SCQs Given a conjunctive quer Q, transform it into a tableau T. Minimization algorithm: T := T ; repeat until no change end choose a row a in T ; if there is a homomorphism δ : T T \ {a} then T := T \ {a} Output: The quer Q corresponding to the tableau T Foundations of Databases 27 Questions about the Algorithm Does it terminate? Is Q equivalent to Q? Is Q of minimal length among the queries equivalent to Q?

15 Foundations of Databases 28 Minimizing SPJ/Conjunctive Queries: Eample R with three attributes A, B, C SPJ quer Q = π AB (σ B=4 (R)) π BC (π AB (R) π AC (σ B=4 (R))) Translate into relational calculus (instead of normalizing): ( z1 R(,, z 1 ) = 4 ) 1 ( ( z2 R( 1,, z 2 ) ) ( 1 R( 1, 1, z) 1 = 4 )) Simplif, b substituting the constant, and putting quantifiers forward: 1, z 1, z 2 (R(, 4, z 1 ) R( 1, 4, z 2 ) R( 1, 4, z) = 4) Conjunctive quer: Q(,, z) : R(, 4, z 1 ), R( 1, 4, z 2 ), R( 1, 4, z), = 4 Foundations of Databases 29 Minimizing SPJ/Conjunctive Queries (contd) Tableau T : 4 z z z 4 z Minimization, step 1: Is there a homomorphism from T to 1 4 z z 4 z? Answer: No. For an homomorphism δ, we have δ() = (wh?), thus the image of the first row is not in the smaller tableau.

16 Foundations of Databases 30 Minimizing SPJ/Conjunctive Queries (contd) Step 2: Is T equivalent to 4 z z 4 z? Answer: Yes. Homomorphism δ: δ(z 2 ) = z, all other variables sta the same. The new tableau is not equivalent to 4 z 1 4 z or 1 4 z 4 z Because δ() =, δ(z) = z, and the image of one of the rows is not present. Foundations of Databases 31 Minimizing SPJ/Conjunctive Queries (contd) Minimal tableau: 4 z z 4 z Back to conjunctive quer: Q (,, z) : R(,, z 1 ), R( 1,, z), = 4 An SPJ quer: σ B=4 (π AB (R) π BC (R)) Pushing selections: π AB (σ B=4 (R)) π BC (σ B=4 (R))

17 Foundations of Databases 32 Review of the Journe We started with π AB (σ B=4 (R)) π BC (π AB (R) π AC (σ B=4 (R))) Translated into a conjunctive quer Built a tableau and minimized it Translated back into conjunctive quer and SPJ quer Applied algebraic equivalences and obtained π AB (σ B=4 (R)) π BC (σ B=4 (R)) Savings: one join. Foundations of Databases 33 Minimization of Conjunctive Queries: Multiple Relations We consider again the quer: Q(, ): B(, ), R(, z), R(, w), R(w, ) The tableau was: A B B: A B R: z w w

18 Foundations of Databases 34 Minimization with Multiple Relations The algorithm is the same as before, but one has to tr rows in different relations. Consider the homomorphism where δ(z) = w, and δ is the identit for all other variables. Appling this to the tableau for Q ields B: A B R: A w B w This can t be further reduced, as for an homomorphism δ, δ() =, δ() =. Thus Q is equivalent to Q (, ) : B(, ), R(, w), R(w, ) One join is eliminated. Foundations of Databases 35 Quer Optimization and Functional Dependencies Additional equivalences can be inferred if integrit constraints are known We consider here functional dependencies Eample: Let R have attributes A, B, C. Assume that R satisfies A B. Then it holds that R = π AB (R) π AC (R) Tableau can help with these optimizations! π AB (R) π AC (R) as a conjunctive quer: Q(,, z): R(,, z 1 ), R(, 1, z)

19 Foundations of Databases 36 Quer Optimization and Functional Dependencies (contd) Tableau: z 1 1 z z Using the FD A B infer = 1 Net, minimize the resulting tableau: z 1 z z z z And this sas that the quer is equivalent to Q (,, z): R(,, z), that is, R This is known as the chase technique Foundations of Databases 37 Quer Optimization and Functional Dependencies (contd) General idea: simplif the tableau using functional dependencies and then minimize. Given: a conjunctive quer Q, and a set of FDs F Algorithm: Step 1. Construct the tableau T for Q Step 2. Appl algorithm CHASE(T, F ) Step 3. Minimize output of CHASE(T, F ) Step 4. Construct a quer from the tableau produced in Step 3

20 Foundations of Databases 38 The CHASE We assume that all FDs are of the form X A, where A is a single attribute. For simplicit, we also assume that the tableau has onl a single relation. The generalisation is straightforward. for each X A in F do for each t 1, t 2 in T such that t 1 [X] = t 2 [X] and t 1 [A] t 2 [A] do case t 1 [A], t 2 [A] of one nondistinguished variable replace the nondistinguished variable b the other term one distinguished variable, one distinguished variable or constant replace the distinguished variable b the other term two constants output and STOP end end Foundations of Databases 39 Quer Optimization and Functional Dependencies: Eample 2 R is over A, B, C; F = { B A } Q = π BC (σ A=4 (R)) π AB (R) Q as a conjunctive quer: Q(,, z) : R(4,, z), R(,, z 1 ) Tableau: 4 z CHASE 4 z minimize 4 z z 1 z 4 z 1 4 z 4 z Final result: Q(,, z) : R(,, z), = 4, that is, σ A=4 (R).

21 Foundations of Databases 40 Quer Optimization and Functional Dependencies: Eample 3 Same R and F ; the quer is: Q = π BC (σ A=4 (R)) π AB (σ A=5 (R)) As a conjunctive quer: Q(,, z) : R(4,, z), R(,, z 1 ), = 5 Tableau: 4 z 5 z 1 CHASE 5 z Final result: (the empt quer) This equivalence does not hold without the FD B A Foundations of Databases 41 Quer Optimization and Functional Dependencies: Eample 4 Sometimes simplifications are quite dramatic Same R, FD is A B, the quer is Q = π AB (R) π A (σ B=4 (R)) π AB (π AC (R) π BC (R)) Convert into conjunctive quer: Q(, ) : R(,, z 1 ), R(, 1, z), R( 1,, z), R(, 4, z 2 )

22 Foundations of Databases 42 Quer Optimization and Functional Dependencies: Eample 4 (contd) Tableau: z 1 1 z 1 z 4 z 2 CHASE 4 z 1 4 z 1 4 z 4 z 2 minimize 4 z 4 4 Foundations of Databases 43 Quer Optimization and Functional Dependencies: Eample 4 (contd) 4 z is translated into Q(, ) : R(,, z), = 4 4 or, equivalentl π AB (σ B=4 (R)). Thus, π AB (R) π A (σ B=4 (R)) π AB (π AC (R) π BC (R)) = π AB (σ B=4 (R)) in the presence of FD A B. Savings: 3 joins! This cannot be derived b algebraic manipulations, nor conjunctive quer minimization without using CHASE.

23 Foundations of Databases 44 Questions about the CHASE Does the CHASE algorithm terminate? What is the run time? What is the relation between a tableau and its CHASE d version? Quer containment wrt a set of FD s: How can we define this problem? Can we decide this problem? Quer minimsation wrt to a set of FDs Consider SCQs: we know from previous eercises that all such queries are satisfiable. Is the same true if we assume onl database instances that satisf a given set of FDs F? Foundations of Databases 45 Conjunctive Queries with Equalities and Inequalities Equalit / Inequalit atoms =, = a, z, etc Let T, T be the tableau of the parts of conjunctive queries Q and Q with ordinar relations Sufficienc: Q Q if there eists a homomorphism δ : T T such that for each (in)equalit atom t 1 θt 2 in Q, we have that δ(t 1 )θδ(t 2 ) is logicall implied b the equalit and inequalit atoms in Q However, eistence of a homomorphims is no more a necessar condition for containment. It holds under certain conditions, though. Note: Deciding whether a set of equalit / inequalit atoms A logicall implies an equalit / inequalit atom is eas.

24 Foundations of Databases 46 Quer Homomorphism: Definition Instead of tableau homomorphims, one often defines quer homomorphisms. An homomorphism from q ( ) : R, C to q( ) : R, C is a substitution δ such that δ( ) = δ(r ) R C = δ(c ). Here, δ( ), δ(r ) and δ(c ) are the etensions of δ to comple sntactic entities. Note that we view R, R, C, and C as sets of atoms. Foundations of Databases 47 Quer Homomorphism: Eample Q () : P (, ), R(, z), 3 Q() : P (, w), P (, ), R(, u), w 5, 2 An homomorphism from Q to Q is δ() =, δ() =, δ(z) = u.

25 Foundations of Databases 48 Homomorphisms: The General Case Eistence of a homomorphism is not a necessar, but a sufficient condition for containment. Theorem: Let Q, Q be two conjunctive queries, possibl with comparisons and inequalities, such that Q and Q have the same distinguished variables. Then Q Q if there eists a homomorphism from Q to Q. Foundations of Databases 49 Containment of Queries With Comparisons Classical eample: Q () : P (u, v), u v Q() : P (, z), P (z, ) Q is contained in Q, but there is no homomorphism from Q to Q.

26 Foundations of Databases 50 Checking Containment of Queries with Comparisons: Idea Q () : P (u, v), u v Q() : P (, z), P (z, ) Replace Q with its linear epansion (Q L ) L! Q { <z } : P (, z), P (z, ), < z Q { =z } : P (, z), P (z, ), = z Q { >z } : P (, z), P (z, ), > z (case analsis) Foundations of Databases 51 Klug s Theorem Theorem (Klug 88): If Q, Q are conjunctive queries with comparisons (Q L ) L is the linear epansion of Q, then: Q Q for ever Q L in (Q L ) L, there is an homomorphism from Q to Q L (analogous for disjunctive queries) Containment with comparisons is Π P 2 -complete. (van der Meden)

27 Foundations of Databases 52 Readings S. Abiteboul, R. Hull, and V. Vianu. Foundations of Databases. Addison-Wesle, Chapter 6 (Sections 6.1 and 6.2 for containment, equivalence and minimization of conjunctive queries) and Chapter 8 (Section 8.4 for the case with functional dependencies). If ou want to stud the same problems for the case of queries with inequalities, see the papers: Anthon C. Klug. On conjunctive queries containing inequalities. Journal of the ACM 35(1): (1988) Ron van der Meden. The Compleit of Quering Indefinite Data about Linearl Ordered Domains. Journal of Computer and Sstem Sciences. 54(1): (1997)