Relational operations

Transcription

1 Architecture

2 Relational operations We will consider how to implement and define physical operators for: Projection Selection Grouping Set operators Join Then we will discuss how the optimizer use them to generate physical query plans

3 Java Relational System (JRS):

4 Selectivity of selection The DBMS Catalog stores information about database tables and indexes. (1/10) (1/3) (1/3) (1/4)

5 Selectivity of selection (1/10)

6 Hystograms 105 records and A with 15 possible values in the range 17 to 31 Uniform distribution NonUniform distribution

7 Selectivity of selection with hystograms An Equi-Height histogram is used as an approximation of the actual distribution The active domain of A is divided into k intervals, containing a number of records of about N rec /k For each interval h i are known: min(h i ), max(h i ), N key (h i ), N rec (h i )

8 Equi-Height histograms

9 Equi-Height histograms s f (V = 24) = N rec (V=24)/N rec s f real = 6/105 = 0,057 s f uniform = 1/15 = 7/105= 0,066 s f histeqh = (26/3)/105 = 0,082

10 Physical operators for tables and sort Operators for R : TableScan (R) SortScan (R, {A i }) IndexScan (R, Idx) IndexSequentialScan (R, Idx) Operator to sort ( τ {Ai} ): Sort (O, {Ai})

11 Physical operators for π b {A i } Project (O, {A i }): to project the records of O without duplicates elimination C = C(O) IndexOnlyScan(R, Idx, {A i }): Idx an index on the attributes to project (or that contains them as prefix) C = N leaf (Idx) Is the result equivalent to a Project or to a Project+Distinct?

12 Physical ops for duplicate elimination Distinct (O): to eliminate duplicates from sorted records of O C = C(O) Actually, we only need them to be grouped: r i =r j and i<l<j => r i =r l =r j HashDistinct(O): to eliminate duplicates from records of O; C =?

13 Hashdistinct partitioning phase: duplicate elimination phase (h r h p ): C = C(O) + 2 N pag (O)

14 Sort Hash

15 Result size: Distinct(O) If A is the only O attribute Q = N key (A) If {A 1, A 2,..., A n } are the attributes Q = min( O /2, i N key (A i ) ) Why not just i N key (A i )?

16 Conclusion The sort method often preferred because produces a sorted result and because sort is heavily optimized DBMS: Informix uses Hash DB2, Oracle, Sybase ASE use Sort SQL Server e Sybase ASIQ use both methods

17 Simple selection With no index, and data unsorted: Relation scan with cost N pag (R). With an index on selection attribute: Use index to retrieve RIDs, then retrieve data records: C I + C D. Cost depends on qualifying tuples (size of R s f ), and clustering General selection conditions SELECT * FROM WHERE Students City = PI

18 Physical operators for selection Filter (O, ψ) C = C(O) E rec = s f (ψ) N rec (O) IndexFilter(R, Idx, ψ) = RidIndexFilter(Idx, ψ) + TableAccess(O, R) C = C I +C D E rec = s f (ψ) N rec (R)

19 Physical operators for selection IndexSequentialFilter(R, Idx, ψ) C = s f (ψ) x N leaf (Idx) IndexOnlyFilter(R, Idx, {A i }, ψ) C = s f (ψ) x N leaf (Idx) AndIndexFilter(R, {Idx i, ψ i }) C I = Σ i C I (Idx i ) C D = (E rec,n pag (R)) E rec = s f (ψ) N rec (R) OrIndexFilter(R, {Idx i, ψ i })

20 Conjunctive selection in DBMSs Informix, DB2 use intersection of RID sets Oracle, Sybase ASIQ use bitmaps Oracle use also HashJoin with index (later) on the attribute RID Sybase ASE uses one index only SQL Server use index join (later)

21 Exercises SELECT A, B Idx an index on A FROM R WHERE (A BETWEEN 50 AND 100) AND B > 20; SELECT A, B One index on A, B FROM R WHERE (A BETWEEN 50 AND 100) AND B > 20 ORDER BY A; SELECT DISTINCT A, B Two indexes on A and on B FROM R WHERE (A BETWEEN 50 AND 100) AND B > 20 ORDER BY B;

22 Physical operators for grouping As for duplicate elimination: Sorting Hashing Using an index on the grouping attributes (Duplicate elimination is grouping on all attributes with no aggregation)

23 Physical operators for ( {A i } γ {f j } ) GroupBy (O, {A i }, {f j }): to group the records of O sorted on the {A i }, computing functions in {f j }. {f j } contains the aggregation functions present in the SELECT and HAVING clauses. The operator returns records with attributes {A i } {f j } The records of O must be sorted on the {A i } Any permutation of {A i } is ok (Actually, they only need to be grouped on the {A i }) HashGroupBy (O, {A i }, {f j }) C? Cardinality?

24

25 Computing aggregations Each aggregation functions is an object o with methods start(), next(v), end() With the first record of a group: o.start() For each record of the group o.next(v) is invoked o.end() computes the final aggregate value.

26 Example p b A, T SELECT A, SUM(C) AS T FROM R WHERE A BETWEEN 50 AND 100 GROUP BY A HAVING COUNT(*) > 1; Project ({A, T}) Filter (N > 1) s N > 1 A g SUM(C) AS T, COUNT(*) AS N s... R Logical Tree GroupBy ({A}, {SUM(C) AS T, COUNT(*) AS N}) Sort ({A}) Filter (...) TableScan (R) Physical Tree... = A BETWEEN 50 AND 100

27 Physical ops for join: nested loops foreach r in O E do foreach s in O i do if r.r1 = s.s1 then add <r, s> to result C = C(O E ) + E rec (O E ) C(O I ) E rec = s f (C j ) E rec (O E ) E rec (O I ) Inequality join conditions?

28 Cost of nested loops foreach r in R do foreach s in S do if r.r1 = s.s1 then add <r, s> to result With R external: C = Npag(R) + Nrec(R) Npag(S) Npag(R) Nrec(R) Npag(R) Npag(S) With S external: C = Npag(S) + Nrec(S) Npag(R) Npag(S) Nrec(S) Npag(R) Npag(S) Hence

29 Page nested loops R Pk B S Fk C Pk B Fk C 10 a 20 b 30 c 40 d 10 f Join 20 g = 30 g 10 k 10 a 10 f 20 b 20 g 10 a 10 k 20 b 20 k 30 c 30 g 40 d 40 f 40 f 20 k C = Npag(R) + Npag(R) * Npag(S) The smallest relation is used as external

30 Block nested loops join C BNL = N pag (R) + Τ N pag (R) B N pag (S)

31 Index nested loop Hyp: There is an index on the join column of the internal relation (S) foreach r in R do foreach s in IndexFilter(S, I, s.s1=r.r1) do add <r, s> to result R(r1, r2) S(s1, s2) r1=s1

32 Index nested loop foreach r in R do foreach s in IndexFilter(S,I,s1 =r.r1) do add <r, s> to result Cost for R join S: C = N pag (R) + N rec (R) CaWithIdx(S) General Case: C = C(O E ) + E rec (O E ) (C I +C D )

33 Merge join Hyp: R and S are sorted on the join attribute, a key of the external relation C = C(O E ) + C(O I ) Inequality join conditions?... ri sj

34 Hash join Partitioning (R and S) Matching

35 Hash join: cost Assume N pag (R)/B < B and uniformity C = C(O E ) + C(O I ) + 2( N pag (O E ) + N pag (O I ) ) C = (log B (N pag (O E )) 2-2)*( N pag (O E ) + N pag (O I ) ) C(R join S) = 3 ( Npag(R) + Npag(S) ) What if N pag (R) < B?

36 Complex joins Join with a conjunctive condition Join with a disjunctive condition

37 Physical operators for join NestedLoop (O E,O I, ψ J ) PageNestedLoop (O E,O I, ψ J ) IndexNestedLoop (O E,O I, ψ J ) MergeJoin (O E,O I, ψ J ) HashJoin (O E,O I, ψ J )

38 Example SELECT ar, Sum(cR) FROM R, S WHERE R.PkR=S.FkR AND cs=1000 GROUP BY ar ORDER BY ar C GroupBy = C Sort = C IndexNestedLoop + 2 N pag (Project) N Pag (Project) = (E Rec (IndNestedLoop) (L ar + L cr ))/D pag C INL = C IndexFilterOnR + E rec (IndexFilterOnR) C(Filter) E rec (INL) = s f PkR = FkR E rec (IndexFilterOnR) E rec (Filter)

39 Example C(IndFilOnR) = C I + C D C I = N leaf (IcR)/N key (IcR) C D = (N rec (R)/ N key (IcR), N pag (R)) E rec (IndFilOnR) = N rec (R)/ N key (IcR) C(IndFilOnS) = C I + C D C I = N leaf (IFkR)/N key (IFkR) C D = (N rec (S)/ N key (IFkR), N pag (S)) E rec (Filter) = s f (cs=1000) N rec (S)

40 Set operators Union(O E,O I ), Except(O E,O I ), Intersect(O E,O I ) Operand sorted and without duplicates. HashUnion(O E,O I ), HashExcept(O E,O I ), HasIntersect(O E,O I ) Using hash. UnionAll(O E,O I ) trivial ExceptAll(O E,O I ), IntersectAll(O E,O I ) Not always supported

41 Example SELECT Name FROM Employees UNION SELECT Name FROM Professors ; Distinct Sort ({Name}) Union Distinct Sort ({Name}) Project ({Name}) Project ({Name}) TableScan (Employees) TableScan (Professors)

42 Summary Very few basic operators, hence the implementation of these operators can be carefully tuned. No universally superior technique for operators with many implementations We must consider available alternatives and select the best one: Query optimization... the next topic