Local Cost Estimation for Global Query Optimization in a Multidatabase System. Outline

Transcription

1 Local os Esmaon for Global uery Opmzaon n a Muldaabase ysem Dr. ang Zhu The Unversy of Mchgan - Dearborn Inroducon Oulne hallenges for O n MDB uery amplng Mehod ualave Approach Fraconal Analyss and Probablsc Approach onclusons

2 . Inroducon Muldaabase ysem MDB Wha: a dsrbued sysem ha negraes daa from varous pre-exsng daabases managed by heerogeneous local DBMs Key feaure: local auonomy Why Global uery Opmzaon GO MDB Global query Global query opmzaon Good overall sysem performance MDB MDB erver erver DB2 DB2 Oracle Oracle 2

3 2. hallenges for Global uery Opmzaon n MDB GO for Tradonal DDB developed for a homogeneous envronmen Technques: opmal vs heursc searches jon vs semjon sraeges sac vs dynamc opmzaon sequenal vs parallel execuon many no suable for an MDB hallenges for GO n MDB aused by local auonomy: ome local opmzaon nformaon may no be avalable a global level Dfferen and changng local capables are assumed Heerogeneous daa formas and models may be used Implemenaon of local DBMs canno be changed More consrans need o be consdered durng global query opmzaon rucal challenge: ncomplee local nformaon 3

4 Proposed Technques albraon mehod Du e al. 92 Fuzzy approach Zhu e al. 94 Exended calbraon mehod Gardarn e al.96 os vecor daabase approach Adal e al.96 Generc cos model approach Naache e al.98 Garlc approach Roh e al. 99 uery samplng mehod Zhu e al. 94 &98 ualave approachzhu e al. 00 Fraconal analyss approach Zhu e al. 00 Probablsc approach Zhu e al uery amplng Mehod Key dea 4

5 lassfcaon of ueres Two exreme cases Informaon avalable haracerscs of queres: e.g. unary queres jon queres haracerscs of operand ables: e.g. number of uples ndexed columns haracerscs of local DBMs: e.g. suppored access mehods lassfcaon goal: each query class corresponds o one access mehod 5

6 lassfcaon rules: based on common rules for access mehods such as A unary query and a jon query use dfferen access mehods A clusered-ndex-based mehod s preferred o an non-clusered ndex-based mehod An ndex-based mehod s preferred o a sequenal scan mehod A clusered-ndex-based mehod s chosen for a query f has a conjuncve erm ha can use a clusered-ndex e.g. for query ec. σ. a= 2 R. b< 3 R. c 4 R R < clusered -ndexed lassfcaon mehods Boom-up mehod Top-down mehod Example of classfcaon G = { all queres } G = G unary queres G 2 jon queres G = G G 2 G 3 G 2 = G 2 G 22 G 23 6

7 G G G 2 3 = { π = { π s R.a = wherer.as a clusered - ndexed column } s R.a = wherer.as an ndexed column } - = G G b...b b...b σ σ F... Fm F... Fm G 2.. classfcaon can be furher refned R R a leas one F a leas one F m m G Relevan ssues omposonof rules Redundancyof rules lassfcaon algorhms Membershp esng amplng and cos formulas amplng mehod: mxed judgmen and probably samplng Use some nowledge o resrc a query class o a represenave subse Apply one or more ypes of probably samplng e.g. smple random samplng and cluser samplng o draw a sample Example For G F : : ample : { π R. a = σ b... b F... Fm - - clusered- ndexed P = { π α ey conjuncv e erm σ R } R. a= F R } 7

8 Dervaon of cos formulas Explanaory varables Basc se: ardnaly of operand ables ardnaly of resul able ze of nermedae resuls econdary se: Operand uple lenghs Resul uple lengh haracerscs of ndex ree ec elecon of varables A mxed forward and bacward procedure Bacward: remove nsgnfcan varables from he basc se Forward: add more sgnfcan varables from he secondary se Example of cos formula For a unary query class: = β + β coeffcen s 2 * N + β N operand able sze selecv y 3 * N * npus 8

9 Esmaon of coeffcens Mulple regresson analyss Valdaon of cos formulas andard error of esmaon oeffcen of oal deermnaon F-es Tes queres Expermen resuls on Oracle 9

10 Expermen resuls on DB2 For more deals:. Zhu and P.-A. Larson: olvng local cos esmaon problem for global query opmzaon n muldaabase sysems Dsrbued and Parallel Daabases Vol. 6 No lassfyng local queres for global query opmzaon n muldaabase sysems In l Journal of oop. Inf. ys. Vol. 9 No A query samplng mehod for esmang local cos parameers n an MDB Proc. of 0 h IEEE In l onf. On Daa Eng

11 4. ualave Approach Movaon: query cos may change dramacally n a dynamc envronmen Types of dynamc facors Frequenly-changng facors E.g. PU load I/Os per sec. amoun of memory beng used Occasonally-changng facors E.g. DBM confguraon parameers daa physcal dsrbuon physcal memory sze eady facors E.g. PU speed DBM release and ype

12 apure dynamc facors n cos models eady facors usually don cause problem Occasonally-changng facors perodcally rebuld he cos model va he query samplng mehod Frequenly-changng facors nfeasble o rebuld frequenly dffcul o nclude all dynamc varables: oo many; 2 unnown neracon forms use a new qualave approach Key dea onsder he combned effec of all dynamc facors on query cos Use he cos of a probng query o measure he sysem conenon level Dvde he conenon level no a number of dscree saes: e.g. hgh conenon medum conenon low conenon no conenon ec. Use a qualave varable n a cos model o ndcae conenon saes 2

13 os model wh qualave varable ualave varable a qualave varable wh m saes s represened by m- ndcaor varables : : 2 m os model : Z = Z2 = 0... Zm Z = 0 Z2 =... Zm Z = 0 Z = 0... Z 2 m = 0 = 0 = 0 Y = β m j n 0 β0 Z j + β + 0 m j 0 + β j= = j= Z j X ysem saes deermnaon How o deermne he sysem conenon saes? wo exremes: one sae nfne saes deermnaon va erave unform paron wh mergng adjusmen Phase I : unformly paron he range of probng query cos wh an ncremenal number of saes unl Rnew Rold / Rold and snew sold / sold are suffcenly small where R s 2 -- coeffcen of oal deermnaon -- sandard error of esmaon 3

14 Phase II : merge wo saes - and f no sgnfcan dfference n coeffcens for he cos model.e. f r = max { n} s oo small where { θ θ / θ } θ = β + β j 0 j -- adjused coeffcen of X for sae j Expermen resuls on Oracle 4

15 Expermen resuls on DB2 For more deals:. Zhu Y. un. Moheramgar: Developng os Models wh ualave Varables for Dynamc Muldaabase Envronmens Proc. of 6 h Inernaonal onf. on Daa Eng. IDE 2000 Feb

16 5. Fraconal Analyss and Probablsc Approach Movaon: a large cos query may experence mulple saes durng s execuon how o esmae s cos? mple soluons sngle sae analyss: consder only one prevalng conenon sae e.g. he nal sae average cos analyss: ae average of coss n all conenon saes Beer soluons? Yes Fraconal Analyss Approach Typcal load curve 6

17 Key dea alculae he fraconof cos n each sae and add hem up Le = { 2 = = N sarng and endng mes for query sarng a T T s M } all possble sequence s for > n sae cos esmae of n saes of saes occurred along max me nerval for n me nerval for he load curve 7

18 8 ase : ase 2: ub-case : hen f s = ] / [ : s wor for fracon of remanng / : s n for wor done esmaed fracon of hen f s s s > n done wor n done wor ]* / [ hen ]* / [ f = s s s General cos esmaon formula: Assumpons: Load curve s pror nown Load changes gradually ]* / [ he mnmum neger such ha s where ]* / [ + = + = + = + = m m m m m m T T m T T

19 Expermen Resuls on Oracle Probablsc Approach Movaon: how o deal wh a rapdly and randomly changng envronmen? Observaons Occurrence of a conenon sae s a random phenomenonand governed by laws of probably The sequence of occurrences of conenon saes can be consdered as a Marov chan Transon probably Pj for sae changng o sae j n he nex sep s nversely proporonal o he dsance beween and j Lm probably π j = lm P j n he lm probably where n Pjn s he probably for changng o j afer n seps 9

20 Properes Independen of nal sae Represen he long-run poronof me for he Marov chan beng n he sae asfy he sysem of lnear equaons: π j M = π Pj for j = 2... M subjec o π = os formula os ncurred n sae : Fracon of wor n sae : M Ideny: π * / = os formula: π M = /[ π = * M j= j = π * / = / ] Expermen resuls on Oracle 20

21 For more deals:. Zhu. Moheramgar Y. un: os Esmaon for ueres Experencng Mulple onenon aes n Dynamc Muldaabase Envronmens Knowledge and Informaon ysems prnger Verlag 200 o appear os esmaon for large queres va fraconal analyss and probablsc approach n dynamc muldaabase envronmens Lecure Noes n ompuer cence DEXA2000 Vol onclusons A crucal challenge for global query opmzaon n an MDB s ha some local cos nformaon may no be avalable a he global level A number of echnques have been proposed o esmae local cos parameers a he global level n an MDB uery samplng mehod s useful n esmang query coss n a sac MDB envronmen 2

22 ualave approach s useful n esmang coss of queres experencng one conenon sae n a dynamc envronmen Fraconal analyss approach s useful n esmang coss of queres experencng mulple conenon saes n a gradually changng dynamc envronmen Probablsc approach s useful n esmang coss of queres experencng mulple conenon saes n a rapdly changng dynamc envronmen Furher research needs o done n fuure For more nformaon: hp:// 22