Implementation and Optimization Issues of the ROLAP Algebra

Transcription

1 General Research Report Implementation and Optimization Issues of the ROLAP Algebra F. Ramsak, M.S. (UIUC) Dr. V. Markl Prof. R. Bayer, Ph.D. Contents Motivation ROLAP Algebra Recap Optimization Issues Handling of Restrictions Aggregation Networks Future Work & Summary 1

2 Example DW Model TIME All Time Year Quarter Month Sales Cost Quantity Category Sector All Products PRODUCT Product Group Container Product Group Business Type Trade Type Trade Type Nation Region Region All Customer CUSTOMER User s View of a Report Sum - Sales Year Quarter Total Total Region Nation Asia China 1 4 Japan Asia Total Europe Europe Total Total France Germany Spain Grouping combinations used to fill pivot table: (1){Y,Q,R,N} (2){Y,Q,R} (3) {Y,Q} (4) {Y,R,N} (5){Y,R} (6){Y} () {R,N} (8) {R} (9){} = ALL 2

3 POT: Pivot Organized Tuples POT ( R, G, F) = PAD F g R ALL ( α ( γ ( )), ) h g G h G We may also write (R) G,F for POT(R,G,F). POT-Example POT(R,{{A},{A,B}},{sum(D)}) yields the table: A B Sum(D) a 1 ALL *** *** a n ALL *** a 1 b 1 *** *** a n *** *** a 1 b m *** *** a n b m *** A B sum(d) A sum(d) A B sum(d) α sum(d) α sum(d ) γ A A B sum(d) { A, B }( ALL) { A, B }( ALL) R γ A, B A B sum(d) 3

4 POT Extension: Group Filtering Filtering of generated groups (like with the HAVING clause in SQL) POT ( R, G, H, F) = PAD g G h G h ( σ H [ g ]( α F ( γ g ( R))), ALL) with H containing a predicate H[g] for each grouping g in G Group Filtering Example Report Years, Product-Group sales totals and sales/year for PGs with less than 10 Mio sales POT( Fact, {{ Y},{ PG},{ Y, PG}}, {{},{},{ sum( Sales) < 10Mio}}, { sum( Sales)}) 4

5 Straight Forward SQL Generation POT(R,{{A},{A,B}},{sum(D)}) maps directly to: SELECT A, ALL, sum(d) FROM R GROUP BY A UNION SELECT A, B, sum(d) FROM R GROUP BY A,B Disadvantages: Efficient execution depends on optimizer of underlying DBMS no UB-Tree support on SQL interface guaranteed Handling Restrictions Semantic of ALL value σ Year = F Pushing Restrictions Down Pushing Through POT: Restrictions on all groups σ ( POT( R, G, F)) = POT( σ ( R), G, F) ρ 1998 ( POT( R,{{ Year},{ PG}}, )) =? ρ Pushing down inside POT: Restrictions on individual groups may be pushed down (i.e., before grouping) if they do not contain constraints on the aggregation results 5

6 σ ( Year = 199 Year= 1998) { }( ), ALL Year Month { }( ), ALL Year Month γ Year γ Year, Month FACT { }( ), ALL Year Month { }( ), ALL Year Month γ Year γ Year, Month σ ( Year = 199 Year= 1998) FACT 6

7 Aggregation Networks Efficient generation of multiple groups Usage of previous generated (more finer) groups instead of fact table as starting point Only one access to the fact table for multiple groups Problems: Size of aggregation nets Hierarchy semantic reduces aggregation nets significantly UB-Tree & Tetris techniques have high potential to optimize aggregation nets Grouping requires sorting Sorted writing of large temporary results saves additional processing time Example of an Aggregation Network (Year) ( ) (Month) (Productgroup) (Year, Month) (Year, Productgroup) (Month, Productgroup) (Year, Month, Productgroup)

8 Aggregation Net with Hierarchies (Year) ( ) (Productgroup) Sort according to PG (or sorted writing+scan) (Year, Month) (Year, Productgroup) Tetris: sort according to Y (Year, Month, Productgroup) POT and Aggregation Nets { Year, Month} ( ALL) { Year, Month} ( ALL) γ Year γ Year,Month σ ( Year = 199 Year= 1998) Country= Germany Pr oductgroup= CTV FACT 8

9 Optimization Issues of AggregationNets Find minimal spanning tree for the specified groupings Vertices: groupings Edge weights: cost of computing new group# Cost factors: Group size Required sorting... Optimization Issues of Aggregation Nets (Year) ( ) C 6 C C 5 C 4 C 3 (Productgroup) (Year, Month) (Year, Productgroup) C 1 C 2 (Year, Month, Productgroup) 9

10 Summary and Future Work Aggregation networks have a very potential to speed up POT operations Standard grouping/aggregation algorithms may benefit significantly from UB-Tree/Tetris techniques Upon availability of resources: Implementation of basic ROLAP algebra processing as part of a master thesis 10