' Characteristics of DSS Queries: read-mostly, complex, adhoc, Introduction Tremendous growth in Decision Support Systems (DSS). with large foundsets

Transcription

1 ' & Bitmap Index Design and Evaluation Chan Chee-Yong of Wisconsin-Madison University Ioannidis Yannis of Wisconsin-Madison University University of Athens 1

2 ' Characteristics of DSS Queries: read-mostly, complex, adhoc, Introduction Tremendous growth in Decision Support Systems (DSS). with large foundsets (i.e., high selectivity factors). \Resurrection" of interest in bitmap indexing. Not much known about space-time tradeos. & 2

3 A B 10 B 9 B 8 B 7 B 6 B 5 B 4 B 3 B 2 B 1 B 0 Example of a Bitmap Index

4 ' Compact representation of index (especially for attributes { low cardinality) ) space and I/O ecient. with Bitmap operations (AND, OR, XOR, NOT) are eciently { by hardware. supported Bitmap Index (cont.) Value-List Index [O'Neil & Quass, SIGMOD'97]. Advantages: & 4

5 Assumption: Attribute values are in f0; 1; 2;:::;C, 1g, where is the attribute cardinality. C ' Scope of Talk Bitmap Index Design for selection queries of the form: (A op c) where op 2 f; ;<;>;=; 6=g: { Range Query: op 2 f; ; <; >g. { Equality Query: op 2 f=; 6=g. 2-Dimensional Framework for Design Space. Space-Time Tradeo Study. & 5

6 A B 10 B 9 B 8 B 7 B 6 B 5 B 4 B 3 B 2 B 1 B 0 Example of a Value-List Index

7 Design space consists of 2 orthogonal dimensions by [Wong et al, VLDB'85]): (inspired Attribute Value Decomposition: determines number 1. size of index components. and Bitmap Encoding Scheme: determines encoding of 2. components. bitmap ' Space Of Bitmap Indexes Design Selection Queries for Index!! Component!! Bitmap & 7

8 Given a sequence of n numbers < b n ;b n,1;:::;b 1 >, attribute value A is decomposed into n digits each Each < b n ;b n,1;:::;b 1 > (base of index) denes an index. n-component 1 st Dimension: Attribute Value Decomposition ' A n A n,1 :::A 1, where A i is a base-b i digit. Example: C = 1000 and attribute value A = 256. < b n ;:::;b 1 > Decomposition of A < 1000 > 256 < 50; 20 > 12 (20) + 16 < 32; 32 > 8(32) + 0 < 5; 20; 10 > 1(20)(10) + 5(10) + 6 & 8

9 ,,,! Value Decomposition with Base < 3; 4 > Attribute A 2 A 1 A ,,,! ,,,! ,,,! ,,,! ,,,! ,,,! ,,,! ,,,! ,,,! ,,,! ,,,! 0 3 9

10 Equality Encoded Bitmap: B x i Range Encoded Bitmap: B x i 2 nd Dimension: Bitmap Encoding Schemes ' Consider the i th index component with base b i. Two basic ways to encode a value x (0 x < b i ): Encoding b i -bit Representation for value x Scheme b i, 1 x + 1 x x, 1 0 Equality Range = f records with A i = x g = f records with A i x g b i,1 i is not materialized since all its bits are set to 1. B & 10

11 A A 2 A 1 B 2 2 B 1 2 B 0 2 B 3 1 B 2 1 B 1 1 B 0 1 An Equality-Encoded Base-< 3; 4 > Index ,! 0 2,!

12 A A 2 A 1 B 1 2 B 0 2 B 2 1 B 1 1 B 0 1 A Range-Encoded Base-< 3; 4 > Index ,! 0 2,!

13 & Design Space of ' Bitmap Indexes ATTRIBUTE VALUE DECOMPOSITION < b, b,..., b> log C b < C > times < b 2,b 1 > BITMAP ENCODING SCHEME Equality Range Value-List Index Bit-Sliced Index..... < b, b b > 3 2,

14 & Space-Time Tradeo Issues Time ' Space-Optimal Time-Optimal under Space Constraint S Optimal Space-Time Tradeoff (knee) S Infeasible Region Time-Optimal Space 14

15 ' Uniform Query Distribution Assumption: space = fa op v : op 2 f; ;<;>;=; 6=g; 0 v < Cg, Query Analytical Cost Model Cost Metrics Space Number of bitmaps. Time Expected number of bitmap scans for a selection query evaluation. where C is the attribute cardinality. & 15

16 Time (Expected Number of Bitmap Scans) Comparison of Encoding Schemes Range-Encoded Index Equality-Encoded Index Space (Number of Bitmaps) Time (Expected Number of Bitmap Scans) Range-Encoded Index Equality-Encoded Index Space (Number of Bitmaps) (a) C = 100 (b) C =

17 Time-Optimal Index = < 2; 2;:::;2; { } {z C 2n,1 ' Space-Time Tradeo Results Class of n-component Indexes >. n,1 Space-Optimal = < b, 1;b, 1;:::;b, 1; { b;b;:::;b Index } {z } {z > n,r r where b = n p ; b r,1 (b, 1) n,r+1 < C b r (b, 1) n,r. C Time-Optimal Index = Single-component index. Space-Optimal Index = Maximal-component index. Knee Index 2-component space-optimal index. & 17

18 and Indexes, Time-Optimal C=100 Space-Optimal 7 7 n-comp. Time-Optimal Index 6 6 n-comp. Space-Optimal Index Time (Expected Number of Bitmap Scans) Space (Number of Bitmaps) 1 18

19 Time (Expected Number of Bitmap Scans) Knee Index, C = n-comp. Space-Optimal Index All Index Space (Number of Bitmaps) 1 19

20 Iteratively adjust the base of index to improve its 2. time-eciency. ' Space-Time Tradeo Results (cont.) Time-Optimal Index under Space Constraint Search space for the optimal solution is large! A 2-step Heuristic Approach: 1. Select an initial index that satises the space constraint. Heuristic Approach is near-optimal. & 20

21 Storage Schemes for Bitmap Compression Bitmap-Level Storage (BS) 6 files of N bits each Component-Level Storage (CS) 2 files of 3N bits each Index-Level Storage (IS) 1 file of 6N bits ( N = # tuples ) 21

22 ' Bitmap Compression Experimental Data (from TPC-D Benchmark): { Attribute: Lineitem.Qty with C = 50 and 6M tuples. { Indexes: 6 n-component space-optimal indexes. { Compression code: zlib library (a LZ77 variant). Notation: cbs, ccs, cis for compressed storage schemes. & 22

23 of Storage Schemes Compressibility to 1-comp. index under BS) (relative BS/CS/IS cbs ccs cis Compressibility Number of Components, n 23

24 30 25 BS cbs ccs Space-Time Tradeo of Compressed Indexes Time (secs) Space (MB) 24

25 General framework to explore design space of bitmap indexes selection queries. for Study of space-time tradeo issues oer guidelines for physical design using bitmap indexes. database More general class of selection queries; { A 2 fv 1 ;v 2 ;:::;v n g. e.g. ' Conclusions Future Work { Hybrid-encoded bitmap indexes. & 25