Approximation Techniques for Spatial Data

Transcription

1 Appoximation Technique fo Spatial Data Abhinandan Da Conell Univeity Johanne Gehke Conell Univeity Miek Riedewald Conell Univeity ABSTRACT Spatial Databae Management Sytem (SDBMS), e.g., Geogaphical Infomation Sytem, that manage patial object uch a point, line, and hype-ectangle, often have vey high quey poceing cot. Accuate electivity etimation duing quey optimization theefoe i cucially impotant fo finding good quey plan, epecially when patial join ae involved. Selectivity etimation ha been tudied fo elational databae ytem, but to date ha only eceived little attention in SDBMS. In thi pape, we intoduce novel method that pemit high-quality electivity etimation fo patial join and ange queie. Ou technique can be contucted in a ingle can ove the input, handle inet and delete to the databae incementally, and hence they can alo be ued fo poceing of teaming patial data. In contat to peviou appoache, ou technique etun appoximate eult that come with povable pobabilitic quality guaantee. We peent a detailed analyi and expeimentally demontate the efficacy of the popoed technique. 1. INTRODUCTION In ecent yea, data management fo patial application uch a Geogaphic Infomation Sytem, the eath cience, and envionmental monitoing ha gained ignificant impotance. In patial data management, ecod in the databae have a patial extent, and ue can poe expeive queie uch a a patial join between two elation (join all object that ovelap o ae within cetain ditance of each othe) o a ange quey (epot all object in a elected ange, o etun an aggegate ove the elected object). Due to the patial extent of the object, executing patial queie can be vey expenive. When electing diffeent quey plan fo a patial quey, The autho ae uppoted by NSF gant IIS , CCF , and IIS , and by a gift fom Micooft. Any opinion, finding, concluion, o ecommendation expeed in thi pape ae thoe of the autho and do not neceaily eflect the view of the pono. Pemiion to make digital o had copie of all o pat of thi wok fo peonal o claoom ue i ganted without fee povided that copie ae not made o ditibuted fo pofit o commecial advantage, and that copie bea thi notice and the full citation on the fit page. To copy othewie, to epublih, to pot on eve o to editibute to lit, equie pio pecific pemiion and/o a fee. SIGMOD 2004 June 13-18, 2004, Pai, Fance. Copyight 2004 ACM /04/06... $5.00. we need accuate etimate of the cot of diffeent execution tategie. Thu a in elational ytem, we need accuate electivity (cadinality) etimate. Note that thee etimate can alo be ued fo fat appoximation of an aggegate quey, e.g., a claical example i the appoximate ange aggegate; anothe inteeting example i to ue appoximate join cadinality fo coelation analyi between data et. The cuent tate of the at in etimating the ize of patial join ue eithe ampling o hitogam, whee ecent wok ha hown that hitogam ae upeio fo a wide ange of poible quey clae [5, 26]. Howeve, exiting hitogam-baed technique till have ignificant dawback. Fit, they eithe give no o only vey conevative wotcae eo guaantee that ae uually ovely peimitic in pactice. Due to the lack of eo guaantee, it i not poible to quantify the tadeoff between toage and eult accuacy (except by pefoming extenive expeimental evaluation with a pioi known data ditibution). Second, exiting hitogam technique ae deigned fo tatic dataet, and equie uually eveal pae ove the data duing contuction. The only exception ae hitogam that ue a fixed patitioning of the pace (e.g., equi-width). Thee can be contucted in a ingle pa and can be maintained incementally, but they cannot adapt to kewed o changing data ditibution. Ou Contibution. In thi pape, we intoduce a novel famewok fo patial data that i baed on andomized pojection. Ou famewok can handle a lage et of challenging patial opeato, and it can handle object with patial extent. Ou appoach i gounded on andomizing technique fo computing mall, peudo-andom ketche of the patial dataet. The baic ketching technique wa oiginally intoduced fo on-line elf-join ize etimation by Alon, Matia, and Szegedy in thei eminal pape [4] and, a we demontate in ou wok, can be genealized to povide appoximate anwe to patial join queie with explicit and tunable pefomance guaantee on the appoximation eo. The main challenge in uing ketche fo patial data i to deign electivity etimation algoithm which ae baed on the ight way of counting event like point being contained in inteval. The algoithm hould not only detemine the coect etimate on expectation, but it eult hould alo have low vaiance. Moe concetely, the key contibution of ou wok can be ummaized a follow: We intoduce the fit ummay data tuctue fo patial data that allow etimation of the electivity of

2 patial queie with povable pefomance guaantee. It pemit a gaceful tadeoff between pace conumption and the quality of the eulting etimation. We apply ou patial ketch technique to patial join of inteval and ectangle (Section 4). All of ou technique ae accompanied by a thoough analytical evaluation in which we how pobabilitic bound on the quality of the eulting ummay data tuctue. We how the geneality of ou technique by dicuing eveal extenion: Join of hype-ectangle in a d- dimenional pace whee d > 2, patial join with othe join pedicate like contained, ǫ-join, and ange queie (Section 6). In a thoough expeimental evaluation, we compae ou technique with the bet peviouly known etimation technique fo patial join, the Geometic and the Eule hitogam (Section 7). Note that, even though we develop ou ketching algoithm in the claic databae context of toed elation, ou technique ae moe geneally applicable to cenaio whee only a ingle pa ove the data i poible fom team of patial data to huge Teabyte databae whee pefoming multiple pae ove the data fo the exact computation of quey eult may be pohibitively expenive. The et of the pape i oganized a follow. Section 2 define patial queie and dicue the ketching appoach of [4]. In Section 3 we intoduce the baic atomic ketche which will be ued to contuct electivity etimato. Selectivity etimation fo join of inteval et and ectangle et ae dicued in Section 4. Section 5 and 6 how how to genealize the technique to vitually any pactical etting and how to extend ou technique to the d-dimenional cae, othe join condition, and othe patial opeato. Expeimental eult ae dicued in Section 7, and Section 8 eview elated wok. Section 9 conclude thi aticle. 2. BACKGROUND 2.1 Queie We focu motly on electivity etimation fo patial join, moe peciely patial join of et of hype-ectangle and ε-join of point et. Thee ae the mot common type of join in patial DBMS. In the following, let N be a (dicete) metic pace and let = (1) (2) (d) be a hype-ectangle which i defined by ange (i) in dimenion i, 1 i d. A ange (i) i defined by it lowe and uppe endpoint l((i)) and u((i)) uch that (i) = {x N l((i)) x u((i))}. Hence = {x = (x 1, x 2,..., x d ) N d 1 i d : l((i)) x i u((i))}. Note that ou technique eaily genealize to multidimenional data pace with diffeent domain fo the dimenion. In Section 5.1 we dicu how to handle eal-valued domain. Definition 1. Let R and S be two et of hype-ectangle in d-dimenional pace N d. Functionovelap fo two hypeectangle (1) (2) (d) R and (1) (2) (d) S etun tue if 1 i d : l((i)) < l((i)) < u((i)) l((i)) < u((i)) < u((i)) l((i)) < l((i)) < u((i)) l((i)) < u((i)) < u((i)), and fale othewie. The patial join of R and S then i defined a R o S = {(, ) R S ovelap(,)}, and hence it electivity i R os R S. Accoding to thi definition object that only touch at thei boundaie ae not pat of the join eult. We will examine poible genealization in Section 6. Definition 2. Let A and B be two et of point in d- dimenional pace N d and dit be a function that etun the ditance between a point in A and a point in B. The ε-join of A and B then i defined a A ε B = {(a, b) a A b B dit(a, b) ε}. It electivity i the cadinality of the eult divided by the total numbe of point-pai, i.e., A εb A B. Typically the ditance dit(a, b) of two point a = (a 1,..., a d ) and b = (b 1,..., b d ) i computed by uing an L i-ditance dit Li, o dit i fo hot, whee dit i(a,b) = ( including the L ditance d a i b i i ) 1 i, i=1 dit (a, b) = max{ a 1 b 1,..., a d b d }. We alo examine electivity etimation fo ange queie. Definition 3. Let R be a et of d-dimenional hypeectangle and q = (q(1) q(2) q(d)) be the quey hype-ectangle uch that ange q(i) i elected in dimenion i. Thi ange quey elect the et Q(q, R) = { R ovelap(,q)}. The electivity i defined a Q(q,R) R Notice that fo all intoduced quey type the challenge i to compute the eult cadinality. Knowing the cadinality, we can compute the electivity by imply keeping tack of the input cadinalitie. Hence in the following we ae concened with etimating the cadinality of the eult of patial queie. 2.2 AMS Sketche Sketche wee popoed fo poceing data team. Conide a imple cenaio whee the goal i to etimate the ize of the elf-join SJ(A) of elation R ove one of it attibute R.A a the tuple of R ae teaming in; that i, we eek to appoximate the eult of quey Q = COUNT(R A R). W.l.o.g. let the domain of join attibute A be dom(a) = {0, 1,, dom(a) 1}, whee dom(a) denote the ize of the domain. Uing f(i) to denote the fequency of attibute value i in R.A, we can ewite quey Q a Q = SJ(A) = i dom(a) f(i)2 (i.e., the econd moment of A). In thei eminal pape, Alon, Matia, and Szegedy [4] pove that any deteminitic algoithm that poduce a tight appoximation to SJ(A) equie at leat Ω( dom(a) ) bit of toage, endeing uch olution impactical fo a data-team etting. Intead, they popoe a andomized technique that offe tong pobabilitic guaantee on the quality of the eulting SJ(A) appoximation while uing only logaithmic pace in dom(a). The baic idea of thei cheme i to define a andom vaiable Z that can be eaily computed ove the teaming value of R.A, uch that (1) Z i an unbiaed etimato fo SJ(A), i.e., E[Z] = SJ(A); and (2) Z ha ufficiently mall vaiance Va(Z) to povide tong pobabilitic guaantee

3 fo the quality of the etimate. Thi andom vaiable Z i contucted on-line fom the teaming value of R.A a follow: Select a family of fou-wie independent binay andom vaiable {ξ i : i = 1,..., dom(a) }, whee each ξ i { 1, +1} and P[ξ i = +1] = P[ξ i = 1] = 1/2 (i.e., E[ξ i] = 0). Infomally, the fou-wie independence condition mean that fo any 4-tuple of ξ i vaiable and fo any 4-tuple of { 1, +1} value, the pobability that the value of the vaiable coincide with thoe in the { 1, +1} 4-tuple i exactly 1/16 (the poduct of the equality pobabilitie fo each individual ξ i). Define Z = X 2, whee X = i dom(a) f(i)ξi. Note that X i imply a andomized linea pojection (inne poduct) of the fequency vecto of R i.a with the vecto of ξ i that can be efficiently geneated fom the teaming value of A a follow: Stat with X = 0 and imply add ξ i to X wheneve the i th value of A i obeved in the team. The cucial point hee i that, by employing known tool (e.g., othogonal aay) fo the explicit contuction of mall ample pace uppoting fou-wie independent andom vaiable, uch familie can be efficiently contucted on-line uing only O(log dom(a) ) pace [4]. Moe peciely, we do not explicitly toe the ξ i. Intead we toe a ingle eed (fo ξ i with i of length k bit, the eed ha length 2k+1 bit) fo the whole ξ-family. Wheneve a ξ i i needed fo the computation, it value i geneated on-the-fly fom the eed in time linea in the eed ize. 2.3 Booting Accuacy To impove the quality of the etimation guaantee one can ue a tandad booting technique (ee [4]) that maintain eveal independent identically-ditibuted (i.i.d.) intantiation of a andom vaiable and ue aveaging and median-election opeato to boot accuacy and pobabilitic confidence. The i.i.d. intance can be contucted by imply electing independent andom eed fo geneating the familie of fou-wie independent ξ i fo each intance. Let {Z i,j}, i {1, 2,..., k 1} and j {1, 2,..., k 2}, be a et of k 1 k 2 of uch i.i.d. intance of an etimato Z fo a quantity Q, uch that E[Z] = Q. Each Z i,j i a andomized linea pojection of the data team a dicued in Section 2.2. We ue the tem atomic ketch to decibe uch a ingle pojection, and the tem ketch fo the oveall ynopi coniting of k 1 k 2 i.i.d. intance of thee andom pojection. Figue 1 how a ketch and illutate how the booting wok. Fo each ow j we compute the aveage Z j = k1 i=1 Zi,j. Then we output the median of thee aveage { Z 1, Z 2,..., Z k2 }. The following lemma etablihe the quality guaantee. Lemma 1. [4] Uing 16 Va[Z] lg 1 independent copie of ε 2 E[Z] 2 φ Z we can guaantee that the computed etimate Z of E[Z] atifie P[ Z E[Z] > εe[z]] φ. Poof. By etting k 1 = 8 ε 2 Va[Z] E[Z] 2 and k 2 = 2 lg(1/φ) the lemma follow fom Chebychev inequality and Chenoff bound. Note that detemining the numbe of intance of Z actually equie knowledge of E[Z], the vey value we would like Sketch Z(1,1) Z(1,2) Z(1,k2) Z(2,1) Z(2,2) Z(2,k2) Z(k1,1) Z(k1,2) Z(k1,k2) Booted etimate of E[Z]: Aveage Z(1) Aveage Z(2) Aveage Z(k2) Median of the Z Figue 1: Booting the accuacy to etimate! Thi i a common poblem haed by peviou ketching technique (ee Section 8) and Online Aggegation [19, 18], in fact alo by any expeiment in the natual cience whee we want to etimate the eo in meauing an unknown quantity. We can geneally ue imple appoximation technique that povide a lowe bound on E[Z] ( anity bound a dicued in [3]), o ue hitoic data, e.g., peviouly computed exact anwe, to pedict futue value of E[Z]. The tadeoff hee i that the tighte thee bound ae, the tonge the quality guaantee etuned by the technique. 3. SKETCHES FOR SPATIAL DATA Technique deigned fo elational DBMS often do not eaily extend to patial data. Thi i mainly caued by the fact that SDBMS manage data with extent. A typical opeation duing the poceing of a patial quey i to check if two object ovelap, o if they ae within a cetain ditance ε of each othe. Ou technique adde thi poblem fo patial queie ove collection of hype-ectangle (including point, line, and ectangle a pecial cae). Mot SDBMS manage hype-ectangle like geogaphical map, envionmental meauement fo point on the eath uface, and o on. (Hype-)ectangle ae alo ued a bounding boxe of moe complex object (e.g., polygon, polyline) ince it i often moe efficient to fit compute a upe-et of the final eult baed on thee bounding boxe, and then to filte out the fale poitive in a final filteing tep. Fo poceing hype-ectangle, we identified that all of the tageted patial queie ely on the ame baic opeation: detemine if a point lie within an inteval. Hence by contucting a coeponding ketch, we have the baic ingedient fo electivity etimation fo patial queie. The actual challenge then i to detemine how to ue thee ketche to obtain a good quey eult cadinality etimate with high pobability, uing only mall pace. 3.1 Baic Sketche fo One Dimenion Fo implicity aume ou input data et R i a onedimenional et of inteval. We ue [a, b] to denote an inteval with lowe endpoint a and uppe endpoint b. To contuct ou ketch we ue a family of fou-wie independent andom vaiable {ξ i} (a intoduced in Section 2.2), uch that vaiable ξ i, i {0,..., n 1} = N, coepond to coodinate i N. An inteval [a, b] R intuitively i epeented by the coodinate of the point it contain, i.e., we ue ξ a +ξ a+1+ +ξ b to ketch it. Now aume we want to tet if a point c N lie within the inteval. Obeve that

4 ince the ξ-vaiable ae fou-wie (and hence alo paiwie) independent we have a c b E[(ξ a + ξ a ξ b )ξ c] = 1 c < a c > b E[(ξ a + ξ a ξ b )ξ c] = 0 Thi fom the bai of ou patial ketche. Fo data et R define the tandad atomic patial ketche V I = (ξ a + ξ a ξ b ) and V E = (ξ a + ξ b ) (1) Intuitively V I keep tack of the complete inteval, while V E ummaize only thei endpoint. Notice that the update cot of V I depend linealy on the inteval length ince we have to add the coeponding ξ i fo each point i N that i contained in the inteval. Fo lage coodinate domain, which ae not uncommon in patial application, thi quickly become cotly and alo eult in high vaiance of the etimate. Hence we intoduce dyadic patial ketche. Simila to [11, 17], we patition the domain N into inteval of ize 2 i. Fo implicity let n = N be a powe of 2, ay 2 h fo ome poitive intege h. 1 Fo each level 0 i h we patition N into 2 h i inteval of ize 2 i each. Hence fo level i = 0 we have point inteval, each coeponding to a ingle domain value, while fo level i = h we have a ingle inteval coveing the whole domain. Let D be the et of all dyadic inteval of all level ove N. Ou technique i baed on the following lemmata. Lemma 2. Let [a, b] be an inteval. It dyadic cove, D([a, b]), i defined to be the mallet et of dyadic inteval {δ 1, δ 2,..., δ m} uch that δ i = [d i 1, d i], 1 i m, and d 0 = a and d m = b. Then m 2log 2 n. Lemma 3. Fo each point a N, let D([a]) denote it dyadic point cove, which i the et of all dyadic inteval in D containing a. Thee ae exactly log 2 n+1 dyadic inteval in D that contain thi point. Each of thee dyadic inteval i at a diffeent level. Lemma 4. A point c N i contained in an inteval [a, b] iff thee i exactly one dyadic inteval δ D uch that δ D([a, b]) and δ D([c]). Poof. See [13]. Intead of having a ξ-vaiable fo each coodinate in N we will ue a ξ-vaiable fo each dyadic inteval ove N. Fo data et R, the coeponding atomic ketche fo inteval and endpoint then ae defined a: X I = ξ δ and X E = δ D([a,b]) δ D([a]) D([b]) ξ δ (2) Figue 2 how an example fo inteval and thei dyadic cove. To implify notation we will hencefoth ue ξ [a,b] = ξ δ and ξ [a] = ξ δ (3) δ D([a,b]) δ D([a]) 1 Othewie we jut pad the domain with additional value. Domain: Dyadic Inteval: Input inteval: δ2 δ1 δ3 δ4 δ5 δ6 δ7 Dyadic cove: D(): {δ2,δ6} D(l()): {δ4,δ2,δ1} D(u()): {δ6,δ3,δ1} D(): {δ5,δ3} D(l()): {δ5,δ2,δ1} D(u()): {δ7,δ3,δ1} Figue 2: Inteval and thei dyadic cove With thi notation we can ewite the dyadic atomic ketche a X I = ξ [a,b] ; X E = ( ξ [a] + ξ [b] ) (4) Oveall, uing dyadic ketche the update cot i educed to O(log 2 n) pe atomic ketch. Since the atomic ketche ae defined ove the et of dyadic inteval, which ha cadinality 2n 1, we need O(log 2 (2n 1)) = O(log 2 n) pace fo geneating each family of the ξ-vaiable. Fo vaiance analyi it will often be moe convenient to ewite the dyadic ketch definition into a diffeent fomat. Note that X I i computed by adding one o moe ξ δi fo each inteval [a, b] R, whee δ i D i a dyadic inteval ove N. Fo each dyadic inteval δ D let f I(δ) = {[a, b] R δ D([a, b])} be the numbe of inteval in R whoe cove contain δ. Similaly, let f E(δ) = {a a δ ( b N : [a, b] R [b, a] R)} be the numbe of inteval endpoint in R whoe cove contain dyadic inteval δ. Then we can wite X I = δ D f I(δ)ξ δ ; X E = δ D f E(δ)ξ δ (5) Fo intance fo inteval in Figue 2 we have f I(δ 2) = 1, f I(δ 6) = 1, and f I(δ i) = 0 fo i {1, 3,4, 5,7}. Fo an atomic ketch X we define it elf-join ize SJ(X) a E[X 2 ]. Fo example, fo ketch X I we have SJ(X I) = E[X 2 I ] = E[( δ D f I(δ)ξ δ ) 2 ] = δ D f 2 I (δ). The lat tep follow fom lineaity of expectation and the fou-wie (and hence two-wie) independence of the ξ- vaiable. 3.2 Sketche fo Multidimenional Data We dicu ketche fo d = 2, i.e., et of ectangle. The genealization to highe dimenionality then i taightfowad. A intoduced in Section 2.1, we wite a ectangle a the co-poduct of it ange in each dimenion, e.g., = [a, b] [c, d]. To keep tack of two-dimenional object, we need an independent ξ(i) family of fou-wie independent andom vaiable (ee Section 2.2) fo each dimenion i {1, 2}. Moe peciely, any ξ(1) j i independent of any ξ(2) k fo any j, k. The main obevation fo ketching a ectangle [a, b] [c, d] i that we (1) have to make ue that the ketch emembe that [a, b] and [c, d] belong togethe, and that (2) we have to keep tack not only of ectangle and point, but alo of hoizontal and vetical line, a will become clea below. The latte i impotant fo coect join ize etimation. The dyadic atomic ketche fo a two-dimenional data et

5 R ae defined a follow: X II = X IE = X EI = X EE = [a,b] [c,d] R [a,b] [c,d] R [a,b] [c,d] R [a,b] [c,d] R ξ(1) [a,b] ξ(2)[c,d] ξ(1) [a,b] ( ξ(2) [c] + ξ(2) [d] ) ( ξ(1) [a] + ξ(1) [b] ) ξ(2) [c,d] ( ξ(1) [a] + ξ(1) [b] )( ξ(2) [c] + ξ(2) [d] ) The ξ ae um ove the ξ-vaiable fo dyadic inteval that cove the coeponding inteval o endpoint a defined in the peviou ection. Intuitively, X II keep tack of the whole ectangle, X IE and X EI of thei hoizontal and vetical edge, epectively, and X EE of the ectangle cone point. The tandad atomic ketche V II, V IE, V EI, and V EE ae defined imilaly. Simila to Equation 5, we can ewite X II a X II = f II(δ(1), δ(2))ξ(1) δ(1) ξ(2) δ(2) (6) δ(1) δ(2) D 2 Hee vaiable f II(δ(1), δ(2)) denote the numbe of ectangle in R whoe dyadic cove contain ectangle δ(1) δ(2), defined by dyadic inteval δ(1) and δ(2) in dimenion 1 and 2, epectively. We can imilaly ewite the othe atomic ketche. A in Section 3.1 we define SJ(X) = E[X 2 ], hence we obtain in a imila manne SJ(X II) = fii(δ(1), 2 δ(2)) δ(1) δ(2) D 2 The elf-join ize fo the othe atomic ketche i imila. The genealization to highe dimenionality follow the ame patten. Let IE d = {I,E} d be the et of all ting of length d ove lette I and E. Fo w IE d the tem w[i] efe to the i-th lette in w. Fo a hype-ectangle let (i) be it ange in dimenion i. With N d we denote the data pace. The atomic ketche fo R ae then defined fo all w IE d a: X w = ξ(1) (1) ξ(2) (2) ξ(d) (d) (1) (2) (d) R uch that fo each dimenion i, ξ(i) (i) = ξ(i) [l((i)),u((i))] if w[i] = I, and ξ(i) (i) = ξ(i) [l((i))] + ξ(i) [u((i))] othewie (i.e., if w[i] = E). A befoe, each family ξ(i) fo dimenion i i a family of fou-wie independent { 1, 1} andom vaiable. Fo any i j the vaiable in ξ(i) ae independent fom the vaiable in ξ(j). 4. SPATIAL JOINS 4.1 Spatial Join of Inteval We decibe a counting pocedue that will be ued to compute the cadinality of the join of two et of inteval. Since inteval only ovelap if thei inteection i a non-empty one-dimenional object (ee Definition 1), we can afely aume that the data et do not contain any degeneate object, in thi cae point object, ince thee would not contibute to the join eult anyway. We alo aume that the data domain i finite. We will dicu in Section 5.1 how to handle eal valued coodinate. (1) dijunct (4) contain (2) meet (3) ovelap (6) identical (5) contain, meet Figue 3: Spatial elationhip between inteval Spatial Relationhip Figue 3 how all cae of patial elationhip between an inteval in R and a potential join patne in S (cae which can be obtained by wapping and ae omitted fo implicity). Accoding to ou definition, in cae (1) and (2) the inteval do not ovelap, while inteval in cae (3)-(6) ae ovelapping. If two inteval R and S fall into cae (i), we ay that they have the patial elationhip (i) Counting Inteection Finding an appopiate ketch that coectly count all ovelap, but doe not count cae (1) and (2) i a non-tivial poblem. None of the peviouly popoed appoache (cf. Section 8) can be diectly applied to compute a function like add 1 fo each pai of inteval (,) fo which one endpoint of i between the two endpoint of. In ode to ue ketche, we need a diffeent way of counting inteval inteection. A befoe, let l() and u() be the lowe and uppe endpoint of inteval, epectively (imila fo ). Intuitively, fo each pai of inteval R and S we want to add 0 to the join eult ize if they have patial elationhip (1) o (2), and 1 othewie. We obtain a fit appoximation by counting fo each pai (, ) how many of thei endpoint ae contained in the othe inteval. Fo intance, in cae (3).u i coveed by, and.l i coveed by, hence the count i 2. Oveall we obtain count 0, 2, 2, 2, 3, and 4 fo cae (1) to (6), epectively. If we divide thi eult by 2, we obtain count 0, 1, 1, 1, 1.5, 2. Howeve, fo coect join electivity etimation the count hould be 0, 0, 1, 1, 1, 1 (add 1 only fo cae (3) to (6)). Notice that the poblem cae with incoect count ae only thoe cae whee and have endpoint in common. Hence fo the emainde of thi ection we will aume the following. Aumption 1. None of the inteval in R ha an endpoint in common with any of the inteval in S. With the aumption it i eay to ee that cae (2), (5), and (6) ae eliminated, and ince ou technique count coectly fo cae (1), (3), and (4), we have a imple method fo calculating the join ize. We will how in Section 5.2 how to genealize ou algoithm if the aumption doe not hold An Atomic Sketch fo Inteval The imple counting pocedue can be diectly implemented with inteval and endpoint ketche a intoduced in Section 3.1. Hee we ue the dyadic ketche, i.e., we contuct atomic ketche X I and X E fo R, and the coeponding ketche Y I and Y E fo S.

6 We want to count how many time an endpoint of an inteval in S i contained in an inteval in R, and vice vea. To do thi we jut multiply the coeponding ketche, i.e., we define a andom vaiable Z = (X IY E + X EY I)/2. Thi andom vaiable i an unbiaed etimato fo the join cadinality. Lemma 5. Random vaiable Z ha the expected value E[Z] = R o S. Poof. See [13]. Fo ou example in Figue 2 we have X I = ξ 2 + ξ 6, X E = 2ξ 1 + ξ 2 + ξ 3 + ξ 4 + ξ 6, Y I = ξ 3 + ξ 5, and Y E = 2ξ 1 + ξ 2 + ξ 3 + ξ 5 + ξ 7. Fo Z we theefoe obtain Z = ((ξ 2 + ξ 6)(2ξ 1 + ξ 2 + ξ 3 + ξ 5 + ξ 7) +(2ξ 1 + ξ 2 + ξ 3 + ξ 4 + ξ 6)(ξ 3 + ξ 5))/2. Multiplying thee tem eult in a fomula that i the ummation of tem of the fom ξ i ξ j. Recall that the ξ-vaiable ae fou-wie (and hence alo paiwie) independent, theefoe it hold that E[ξ iξ j] = E[ξ i]e[ξ j] = 0 if i j. Futhemoe, becaue each ξ i eithe 1 o -1, it hold that E[ξ 2 i ] = 1. Thu fom the above fomula we obtain E[Z] = (E[ξ 2 2] + E[ξ 2 3])/2 = 1, which, a expected, i the coect value fo the numbe of inteecting inteval Vaiance Analyi Uing tandad tanfomation we obtain: Va[Z] = Va[(X IY E + X EY I)/2] = 1/4(Va[X IY E] + Va[X EY I] +2Cov[X IY E, X EY I]) Since in geneal, fo any andom vaiable X and Y, Cov[X, Y ] Va[X] Va[Y ]: Va[Z] 1/4(Va[X IY E] + Va[X EY I] +2 Va[X IY E] Va[X EY I]) = 1/4( Va[X IY E] + Va[X EY I]) 2 (7) To analyze the tem of Equation 7 we ue the definition of the atomic ketche in the fom of Equation 5. Fo thi type of ketche (tug-of-wa ketch) it can be hown that thei vaiance i bounded by twice the poduct of thei elf-join ize [3]: Va[X IY E] 2SJ(X I)SJ(Y E) and Va[X EY I] 2SJ(X E)SJ(Y I). Togethe with Equation 7 we have Va[Z] 1/2( SJ(X I)SJ(Y E) + SJ(X E)SJ(Y I)) 2. Since X I and X E togethe account fo all dyadic inteval that cove the inteval of R and thei endpoint (imilaly the Y ketche fo S), we define SJ(R) = SJ(X I) + SJ(X E) and SJ(S) = SJ(Y I) + SJ(Y E). Uing the Cauchy-Schwaz inequality it hold that ( SJ(X I) SJ(Y E) + SJ(X E) SJ(Y I)) 2 (SJ(X I) + SJ(X E))(SJ(Y E) + SJ(Y I)), hence: Va[Z] 1/2 SJ(R)SJ(S). (8) The Oveall Technique Having contucted the appopiate andom vaiable Z and computed an uppe bound on it vaiance, we can boot it accuacy a decibed in Section 2.3. The following theoem ummaize the eult. Theoem 1. Let R = { 1, 2,..., R } and S = { 1, 2,..., S } be two et of inteval that atify Aumption 1. Futhemoe let X (i) I = (i) ξ, [a,b] X(i) E = (i) (i) (i) ( ξ [a] + ξ [b] ), Y I = (i) (i) [c,d] S ξ [c,d], and Y E = (i) (i) [c,d] S ( ξ [c] + ξ [d]), 1 i 8SJ(R)SJ(S) lg 1, be atomic ε 2 E[Z] 2 φ ketche uch that Z i = (X (i) I Y (i) E + X(i) E Y (i) I )/2. The ξvaiable ae defined a in Equation 3 ove ξ (i) -familie of fou-wie independent { 1, 1} andom vaiable, which ae geneated fom eed (i) of ize O(log 2 n) bit and whee each (i) i independently choen. By computing the median of 2 lg(1/φ) aveage ove goup of 4 SJ(R)SJ(S) etimato Z i we obtain an etimate Z that with pobability ε 2 E[Z] 2 1 φ i within ε elative eo of the tue expected value E[Z] = R o S. Notice that the atomic ketch Z fo etimating the join ize only toe five value: a eed of ize O(log n) bit fo geneating the ξ-vaiable on-the-fly (duing update), and fou counte fo keeping tack of the cuent value of X I, X E, Y I, and Y E. A mentioned befoe, when an inteval i ineted into R (deleted fom R), we imply geneate the coeponding ξ-vaiable fo it inteval and endpoint cove fom the eed and add (ubtact) them fom X I and X E, epectively (imilaly fo S). The cot of geneating a ξ-vaiable fom the eed i linea in the eed ize, and each inteval o endpoint cove conit of O(log n) dyadic inteval. Hence the total update cot fo a ingle intance of Z i O(log 2 n). To compute Z etimate we imply combine the counte, eulting in a contant ovehead. Since we ue 8 SJ(R)SJ(S) lg 1 independent intance of Z, the total quey, update, and to- ε 2 E[Z] 2 φ age cot of ou inteval join ketch ae O( SJ(R)SJ(S) log 1 ), ε 2 E[Z] 2 φ log 1 log n), e- φ O( SJ(R)SJ(S) ε 2 E[Z] 2 pectively. log 1 φ log2 n), and O( SJ(R)SJ(S) ε 2 E[Z] Spatial Join of Rectangle The patial elationhip between inteval (cf. Section 4.1.1) genealize natually to highe dimenionality by examining the one-dimenional axi-paallel pojection of the hype-ectangle (thee pojection ae inteval). Figue 4 how elected example. 2 The patial elationhip of two hype-ectangle and i a d-tuple (i 1,..., i d ) whee i j i the patial elationhip of the pojection in dimenion j, a pe Figue 3. Hence and ovelap iff i j {3, 4,5, 6} fo all dimenion j Rectangle Sketche The imple counting pocedue fo inteval ovelap genealize to two-dimenional ectangle a follow. Let and be ectangle fom R and S, epectively. The counting pocedue add the following value: numbe of cone of 2 The patial elationhip ae lightly diffeent fom the one ued in [24, 25] ince we elected them to coepond to the ketche we ue.

7 (3, 3) ovelap (3, 4) ovelap (2, 3) no ovelap (4, 5) ovelap Figue 4: Spatial elationhip between ectangle which ae coveed by, numbe of hoizontal edge of which inteect vetical edge of, numbe of vetical edge of which inteect hoizontal edge of, and numbe of cone of which ae coveed by. The eade might eaily convince heelf that unde the aumption that and have no common cone coodinate in any dimenion, we will alway obtain a count of 4 fo inteecting ectangle (and 0 othewie). A it tun out, we can combine the two-dimenional ketche (ee Section 3.2) to obtain the coect count; uing atomic ketche X II, X IE, X EI, and X EE fo R, and the analogouly defined Y II, Y IE, Y EI, and Y EE fo S: Lemma 6. Let Z be a andom vaiable uch that Z = (X IIY EE + X IEY EI + X EIY IE + X EEY II)/4; and R and S be two et of ectangle that atify Aumption 1 in each dimenion. Then E[Z] = R o S and Va[Z] 1/16(8SJ(R)SJ(S)) = 1/2 SJ(R)SJ(S). Poof. It i eay to how that E[Z] = R o S, with the analyi being imila to the 1-dimenional cae. The vaiance analyi, on the othe hand, diffe fom the 1-dimenional cae mainly becaue the et of andom vaiable {ξ(1) δ(1) ξ(2) δ(2) : δ(1) δ(2) D 2 } doe not have the 4-wie independence popety. Thi might come a a upie ince the et of ξ(1) and the et of ξ(2)-vaiable ae both 4-wie independent, and the vaiable in one et ae independent of thoe in the othe. Howeve, notice that fo intance fo the 4-tuple {x 1 = ξ(1) δ(1)1 ξ(2) δ(2)1, x 2 = ξ(1) δ(1)1 ξ(2) δ(2)2, x 3 = ξ(1) δ(1)2 ξ(2) δ(2)1, x 4 = ξ(1) δ(1)2 ξ(2) δ(2)2 } it i eay to how that E[x 1x 2x 3x 4] = 1 0 = E[x 1]E[x 2]E[x 3]E[x 4]. Hence we cannot diectly ue vaiance bound fom ealie eult. We can obtain a bound on the vaiance of Z a follow. In the following, fo w IE 2 let w be the ting obtained fom w by eplacing I with E and vice vea. Then Va[Z] = 1 16 ( Va[X wy w] w IE 2 + Cov[X w1 Y w1, X w2 Y w2 ]) w 1 w ( Va[XwY w]) 2 w IE 2 The inequality follow ince, fo any andom vaiable X and Y, Cov[X, Y ] Va[X] Va[Y ]. With ome involved analyi we can how that fo any w IE 2, Va[X wy w] 8SJ(X w)sj(y w). With the above inequality we have: Va[Z] 1/16 8 ( SJ(Xw)SJ(Y w)) 2 w IE 2 Uing Cauchy-Schwaz and the fact that SJ(R) = w IE 2 SJ(X w) and SJ(S) = w IE 2 SJ(Y w) yield: Fo detail ee [13]. Va[Z] 1/16 (8SJ(R)SJ(S)). Hee SJ(R) = SJ(X II) + SJ(X IE) + SJ(X EI) + SJ(X EE), imilaly fo SJ(S) The Oveall Technique A fo inteval ketche, we boot the accuacy of the atomic ectangle ketch a decibed in Section 2.3 to obtain accuate etimate of R o S with quality guaantee. Theoem 2. Let R = { 1, 2,..., R } and S = { 1, 2,..., S } be two et of ectangle whoe endpoint have no coodinate in common in any dimenion. Futhemoe let X II = ξ(1) [a,b] [c,d] R [a,b] ξ(2)[c,d], X IE = ξ(1) [a,b] [c,d] R [a,b] ( ξ(2) [c] + ξ(2) [d] ), X EI = [a,b] [c,d] R ( ξ(1) [a] + ξ(1) [b] ) ξ(2) [c,d], X EE = [a,b] [c,d] R ( ξ(1) [a] + ξ(1) [b] )( ξ(2) [c] + ξ(2) [d] ), Y II = ξ(1) [a,b] [c,d] S [a,b] ξ(2)[c,d], Y IE = ξ(1) [a,b] [c,d] S [a,b] ( ξ(2) [c] + ξ(2) [d] ), Y EI = [a,b] [c,d] S ( ξ(1) [a] + ξ(1) [b] ) ξ(2) [c,d], and Y EE = [a,b] [c,d] S ( ξ(1) [a] + ξ(1) [b] )( ξ(2) [c] + ξ(2) [d] ), be atomic ketche uch that Z = (X IIY EE + X IEY EI + X EIY IE + X EEY II)/4. Uing 8 SJ(R)SJ(S) lg 1 independent ε 2 E[Z] 2 φ intance of Z we obtain an etimate Z that with pobability 1 φ i within ε elative eo of the tue expected value E[Z] = R o S. Note that the numbe of intance of Z fo the one- and two-dimenional cae incidentally i the ame (cf. Theoem 1). Howeve, the actual toage cot can be quite diffeent. Fit, the numbe of atomic ketche pe intance of Z ha doubled fo d = 2. Second, the elf-join ize fo R and S will be lage than fo the one-dimenional cae. Since update and quey cot depend appoximately linealy on the toage, thoe cot will be lage a well. In Section 6.1 we will how that ou technique uffe fom the cue of dimenionality, like any othe etimation o indexing technique. 5. GENERALIZATION In thi ection we dicu how to emove the eticting aumption we made fo ou technique. 5.1 Real-Valued Data Domain Note that fo ou technique to wok, domain N hould be finite. Thi i eential becaue a eed of ize 2k+1 limit u to a domain of ize 2 k. Theoetically thi pevent u fom uing ou technique fo eal-valued domain. Howeve, in pactice ou technique will wok jut fine. Thee i no patial application we know of that ue coodinate of unbounded peciion. Typically eal-valued coodinate ae toed a 32 o 64 bit ize floating point numbe clealy a finite domain. Ou ketch-baed etimation technique can handle uch domain vey well. Recall that the toage equiement of an atomic ketch i logaithmic in the domain ize. Hence ou technique cale vey well with inceaing domain ize. In contat, hitogam taditionally uffe fom the poblem that lage data domain eult in lage bucket and hence

8 poo appoximation. Hitogam-baed technique theefoe typically quantize the data pace and make aumption about the ditibution within a bucket in ode to obtain good etimate [23, 26]. 5.2 Spatial Join with Common Endpoint The counting algoithm ued o fa elied on Aumption 1 fo coectne. We can make thi aumption hold fo any data et a follow. Recall that the inteval endpoint ae dawn fom domain N = {0, 1,..., n 1}. Fit, we ceate a new domain M which contain all value fom N and in addition fo each pai of conecutive value i, i + 1 N two value i + and (i + 1) with i < i + < (i + 1) < (i + 1). Then we eplace each inteval S by a new inteval uch that fo l() = i, we et l( ) = i + and fo u() = j we et u( ) = j, i.e., we hink the inteval a little. Fo intance if all inteval endpoint ae intege coodinate, we could augment the domain by i 0.1 and i fo each intege in the domain and thu hink the S-inteval by 0.1 at each end. The patial join ize i not affected by thi tanfomation: Fo each cae we can eaily veify ovelap(,) ovelap(, ). All thi tanfomation doe i inceae the dimenion domain ize by at mot a facto of 3. The aymptotic cot of ou technique i not affected by thi tanfomation (cf. Section whee n now become 3n). We can apply the ame tanfomation to each dimenion of the data pace. Intead of uing the endpoint tanfomation, we can altenatively adapt ou oiginal imple counting pocedue to explicitly keep tack of common endpoint [13]. 6. EXTENSIONS 6.1 Join of Hype-Rectangle The one-and two-dimenional patial join etimato genealize natually to highe dimenionality d a the following theoem how. Theoem 3. Fo d-dimenional data et R and S let {X w} w IE d and {Y w} w IE d be two familie of atomic ketche ove the ame ξ(i) familie of fou-wie independent andom vaiable, uch that familie ξ(i) and ξ(j), 1 i, j d, fo i j ae independent of each othe. Set Z = 2 d w IE d X wy w. Then it hold that E[Z] = R o S and Va[Z] 3d 1 4 d SJ(R)SJ(S). Poof. The poof i faily involved and can be found in [13]. The atomic ketche ae a defined in Section 3.2. A befoe, fo w IE d we define w a the ting that i obtained fom w by eplacing I with E and vice vea (the complement of w). At a fit glance the vaiance might appea to be deceaing with inceaing dimenionality d due to the (3/4) d facto. Unfotunately thi i not the cae. The elf-join tem fo R and S each have 2 d contibuting um. Alo note that to compute Z we have to maintain 2 d atomic ketche fo each team. 6.2 Othe Join Condition We can extend ou algoithm to etimate the cadinality fo a lightly diffeent notion of ovelap, whee cae (2) (cf. Figue 3) i alo counted a ovelap. We can alo uppot othe join pedicate like containment join by maintaining the appopiate patial ketche. Due to pace containt we defe the detail to [13]. 6.3 ε-join Fo implicity we will fit dicu ε-join fo twodimenional data. Let A and B be two-dimenional et of data point ove pace N 2. In the following we aume that the L ditance i ued. We can etimate the ε-join cadinality baed on the following obevation. Let B be a data et obtained fom B by eplacing each point b B with the patial object b which i defined a b = {x x N 2 dit (x, b) ε}. Notice that fo the L ditance thi object b i a quae of idelength 2ε with b a it cente. Accoding to definition, it hold that dit (a, b) ε a i contained in b. Hence we can compute the cadinality of the ε-join by counting how many point of A ae contained in the quae of B. Thi tun out to be a pecial cae of the ectangle join whee endpoint might have equal coodinate. Since the object in A ae point, we do not need mot of the two-dimenional atomic ketche, except fo: X EE = Y II = (a 1,a 2 ) A ξ(1) [a1 ] ξ(2) [a2 ] [e,f] [g,h] B ξ(1)[e,f] ξ(2)[g,h], whee the ξ-vaiable ae a defined in Equation 3. Lemma 7. Fo andom vaiable Z = X EEY II it hold that E[Z] = A ε B and Va[Z] 8SJ(X EE)SJ(Y II). The genealization to any dimenionality follow the ame patten a fo the patial join. The following lemma ummaize the eult: Lemma 8. Fo d-dimenional point et A and B let B = {b b = {x x N d dit (x,b) ε}} be the et of hype-cube of idelength 2ε aound the point of B. Fo atomic ketche X E = (a 1,a 2,...,a d ) A ξ(1) [a1 ] ξ(2) [a2 ] ξ(d) [ad ] and Y I = l 1 l 2 l d B ξ(1) [l(l1 ),u(l 1 )] ξ(2) [l(l2 ),u(l 2 )] ξ(d) [l(ld ),u(l d )] let Z = X EY I. Then E[Z] = A ε B and Va[Z] (3 d 1)SJ(X E)SJ(Y I). Hee SJ(X E) = δ(1) δ(2) δ(d) D d f 2 (δ(1), δ(2),..., δ(d)) and SJ(Y I) = δ(1) δ(2) δ(d) D d g 2 (δ(1), δ(2),..., δ(d)) whee f(δ(1), δ(2),..., δ(d)) i the numbe of point in A which ae coveed by the hype-ectangle panned by the dyadic inteval δ(i). Similaly, g(δ(1), δ(2),..., δ(d)) i the numbe of hype-cube in B whoe dyadic cove contain δ(1) δ(2) δ(d). Notice that the contant facto of the vaiance bound eem upiingly high, compaed to the bound fo the moe geneal patial join (ee Theoem 3). Howeve, hee the elfjoin ize will be much lowe (1 tem veu 2 d tem added fo the patial join), and the numbe of atomic ketche i much lowe (2 pe intance of Z veu 2 2 d fo the patial join). 6.4 Range Queie

9 The ange quey i a pecial cae of the patial join, whee the econd data et conit only of a ingle hypeectangle the quey (ee Definition 3). Hence we could eadily apply the patial join etimation technique. The following optimization can futhe impove pefomance. We illutate it fo the cae of R being a one-dimenional et of inteval. Aume we want to etimate how many inteval ae elected by ange quey q = [u, v]. The eade might convince heelf that an inteval [a, b] R i elected iff eithe it uppe endpoint lie in [u, v] o if v lie in [a, b]. Notice that both condition ae mutually excluive and exhaut all poible cae of inteval being elected. To implement thi counting pocedue we only need two atomic ketche one that keep tack of the inteval a a whole (X I), and one fo thei uppe endpoint (X U): X I = ξ [a,b] ; X U = ξ [b]. Lemma 9. Fo inteval et R let X I and X U be atomic ketche a defined above. Fo ange quey q = [u, v] let Z = ξ [u,v] X U + ξ [v] X I. Then E[Z] = Q([u, v], R) and Va[Z] 2 (3 log 2 n+1) SJ(R) whee n i the ize of the domain N. The logaithmic facto i caued by the fact that the dyadic cove of [u, v] can conit of up to 2log 2 n inteval and the dyadic point cove of v contain log 2 n + 1 inteval. Compaed to the eult fo the patial join (ee Section 4.1.4) the elf-join ize hee doe not depend on the lowe inteval endpoint and hence will be malle. The genealization to d dimenion follow the ame patten a fo the patial join (jut eplace X E with X U). 6.5 Taking Data Popetie into Account In Section 3.1 we peented two diffeent atomic ketch appoache fo ummaizing inteval: tandad ketche and dyadic ketche. Fo each inteval [a, b] R, the tandad ketch add the ξ-vaiable fo all point in [a, b] to the inteval ketch V I, a cot of O(n). If the domain i lage and the data et contain many long inteval, then the dyadic atomic ketche ae clealy pefeable ince they guaantee O(log n) update cot. Howeve, if mot inteval ae vey hot, then the tandad ketch might delive bette etimate fo the ame toage cot. Recall that the dyadic endpoint ketch keep tack of all dyadic inteval coveing an endpoint, hence it will add the ξ-vaiable fo the dyadic inteval that cove the whole domain on each inteval inetion. Fo et with motly hot inteval thi might lead to a highe elf-join ize than if tandad atomic ketche wee ued. We theefoe popoe an adaptive appoach. Baed on tatitic about the inteval length ditibution, the algoithm detemine the maximum level, maxlevel. When computing the cove of an inteval o endpoint it only ue dyadic inteval fom level up to maxlevel. Note that the input team can have inteval of length geate than 2 maxlevel. The lowe maxlevel, the lowe the elf-join ize fo X E. On the othe hand, long inteval will equie moe dyadic inteval than befoe (ince the longe dyadic inteval on the uppe level above maxlevel can not be ued any moe). Notice that the moe mall inteval the data et contain, the lowe maxlevel, in the exteme the dyadic ketch tun into the tandad ketch fo maxlevel = EXPERIMENTS We evaluate the pefomance of ou patial join ize etimation technique (hencefoth efeed to a SKETCH) on both ynthetic and eal life 1D and 2D patial dataet. We compae ou algoithm with ecently popoed technique baed on genealized Eule Hitogam [26] (hencefoth efeed to a EH) and with the Geometic Hitogam technique [5] (hencefoth efeed to a GH). Thee ae cuently the bet known technique fo patial join electivity etimation. 3 The EH appoach patition the data pace uing a gid of a given level L. A gid of level L patition each dimenion into 2 L equi-width cell. An Eule hitogam allocate bucket not only fo gid cell, but alo fo gid edge and gid vetice. Beide toing object count in a cell, the genealized Eule hitogam [26] alo toe infomation uch a the aveage height, width and aea of the inteection egion between the object and the cell. In tem of toage pace, a genealized Eule hitogam of level L ue 9 2 2L 6 2 L +1 unit (wod) of memoy. Geometic Hitogam alo patition the data pace uing a gid of given level L, imila to the genealized Eule Hitogam. The infomation toed in each cell i the total numbe of cone point, the um of the aea of the object, the um of the length of the vetical edge and the um of the length of the hoizontal edge of object inteecting the cell. Thu a Geometic Hitogam of level L ue 4 L+1 unit of memoy. In the following, all efeence to memoy allocation efe to the memoy allocated pe dataet to the SKETCH, GH o EH technique. 7.1 Effect of Input Size and Skew None of the exiting technique povide guaanteed eo bound (pobabilitic o othewie) fo patial join ize etimation. Hence, fo compaion pupoe, we povide SKETCH, GH and EH with the ame pace and compae the actual elative eo on thei epective etimate a the ize of the patial data et inceae, fo dataet with diffeent degee of kew. We ue ynthetic two-dimenional dataet, with inteval along each dimenion i geneated independently accoding to a Zipfian ditibution with Zipf paamete z i. The aveage length of an object along a dimenion i O( d i) whee d i i the ize of the domain along the i th dimenion. We ued genealized Eule hitogam with gid level L = 6, which coepond to about 36K unit of memoy. The ame pace wa allocated to SKETCH and GH, and the actual elative eo ae hown in Figue 5 and 6. Since ou ketch technique ae baed on andomization, the elative eo epoted ae aveage ove multiple independent un. Figue 5 plot the elative eo a the ize of the dataet inceae fom 30K to 0.5 million. Both the joining dataet had the ame ize, a pecified on the x-axi. The dataet ued hee have ectangle geneated on the two team with Zipf paamete z = 0 (unifom). A can be een fom the figue, the eo fo SKETCH (and GH) emain faily table a the input ize inceae, if the ditibution of the dataet emain the ame. Fo unifom data (z = 0), the SKETCH and GH technique pefom imilaly, with an aveage el- 3 The autho would like to thank Chengyu Sun fo poviding the oiginal EH code and the data et ued in [26].

10 Relative Eo V Dataet ize [ zipf=0 (unifom)] Relative Eo V Dataet ize [zipf kew=1] Actual Relative Eo V Dataet ize fo epilon=0.3 phi=0.01 (1D) SK EH GH SK EH GH Tue Eo Guaanteed Eo Bound 0.3 Relative Eo Relative Eo Relative Eo Dataet Size (K) Dataet Size (K) Dataet Size (K) Figue 5: Zipf = 0 Figue 6: Zipf = 1 Figue 7: Relative eo guaantee ative eo which i much lowe than the eo of the EH technique. Simila eult wee obeved on othe dataet with low kew in the ditibution of the patial object ove the data pace. Note that EH i moe geneal than GH, and hence it pefomance i moe enitive to data ditibution and gid level, which explain the poo pefomance fo the given data et. We would like to point out that fo a given gid level, the pefomance of both EH and GH depend heavily on the domain ize, ince that detemine the ganulaity of the gid. Thu, fo example, if the domain ize i doubled along each dimenion, the elative eo fo both EH and GH inceae conideably, even if the input i not changed! Fo the SKETCH technique, on the othe hand, if the maximum dyadic level i not changed (ee Section 6.5), the elative eo emain the ame fo the ame input, even if the undelying domain i conceptually (i.e. without changing the input dataet) doubled. Figue 6 how the elative eo fo dataet with a faily high degee of kew in the ditibution of the patial object. Fo thee dataet, the pojection of the object along each dimenion i ditibuted with Zipf paamete z = 1. Hee the pefomance of EH i compaable to GH and SKETCH, with SKETCH faing maginally bette than the othe two technique. Fo othe data et we obeved the geneal tend that SKETCH doe compaatively bette than the gid-baed hitogam technique fo kewed input. 7.2 Eo Guaantee and Space Requiement Ou next et of expeiment how how the actual elative eo and pace equiement fo ou technique vaie fo a given guaanteed elative eo bound fo patial dataet of diffeent ize. In Figue 8 and 7 we conide inteval join of two dataet with inteval unifomly ditibuted ove domain of ize anging fom to Figue 8 how the pace equiement and Figue 7 how the actual elative eo fo a guaanteed elative eo bound of 0.3 at the 99% confidence level. A can be een fom the figue, the pace equied emain almot contant at aound 63K a the ize of the dataet inceae. The eaon i that the ditibution of the object doe not change ignificantly. Note that fo a d-dimenional dataet of ize N, the total pace equied to toe the dataet completely i 2d N. Thu the ize of the SKETCH ummay tuctue a a faction of dataet ize vaie fom aound 60% fo malle dataet, to about 6% fo lage dataet. A can be een fom Figue 7, the actual elative eo i much malle than the guaanteed eo bound. Sketch ize (1000 wod) Space Requiement V Dataet ize fo epilon=0.3 phi=0.01 (1D) SKETCH Dataet Size (1000 object) [1 object = 2 wod] Figue 8: Space equiement 7.3 Real Life Dataet In thi ection, we compae the pefomance of EH, GH and SKETCH on the thee eal life 2-dimenional patial dataet ued in [26]. The deciption of the dataet ae a follow: LANDO: Thi dataet contain land cove ownehip and management infomation fo the tate of Wyoming at a 1 : 10 6 cale. Numbe of object = LANDC: Thi dataet contain land cove infomation uch a vegetation type fo the tate of Wyoming at a 1 : 10 6 cale. Numbe of object = SOIL: Thi dataet epeent oil of Wyoming at a 1 : 10 6 cale. Numbe of object = We conide all the 3 combination of join on thee patial dataet. Figue 9, 10 and 11 how the elative eo obtained on LANDO o LANDC, LANDC o SOIL and LANDO o SOIL epectively, a the allocated pace i vaied. The thee gaph ae imila in natue. The mot upiing eult i the unpedictability of the EH etimate: Wheea the etimate povided by SKETCH impove a the pace allocated to it i inceaed, hitogam-baed method (with no quality guaantee) uch a EH might ometime poduce woe etimate. Simila to the eult in [26], EH povide good etimate with mall memoy allocated to it, but the elative eo inceae apidly with fine gid patitioning. Thi behavio i elated to the pobabilitic model ued fo etimation by EH it might poduce mall pebucket eo which add up if the actual data ditibution diffe fom the model aumption. The GH technique i moe obut than EH in thi egad ince it ue a imple model. Howeve, it fae well only fo highe amount

11 Relative Eo fo the LANDC+LANDO patial join SKETCH EH GH Relative Eo fo the LANDC+SOIL patial join SKETCH EH GH Relative Eo fo the LANDO+SOIL patial join SKETCH EH GH Relative Eo Relative Eo Relative Eo Space Allocated (K wod) Space Allocated (K wod) Space Allocated (K wod) Figue 9: LANDC o LANDO Figue 10: LANDC o SOIL Figue 11: LANDO o SOIL of memoy, and i motly outpefomed by SKETCH (by a mall magin). The elative eo of SKETCH how a teady decline with inceaing pace in all the thee gaph. Thi i expected, ince SKETCH i an unbiaed etimato and theefoe the moe independent intance of thi etimato ae ued, the bette the expected eult. 7.4 Dicuion of Expeimental Reult We peented a compaion of SKETCH, GH and EH on ynthetic and eal life dataet. Oveall, the pefomance of GH and SKETCH i compaable, with SKETCH pefoming lightly bette. In geneal the expeiment highlight the pacticality of ou andomized etimation technique. It not only povide quality guaantee, but alo ha pedictable behavio: The moe pace we allocate, the bette the etimate. The hitogam technique might ometime poduce bette etimate, ometime they do much woe. Without knowing the data ditibution in advance, thei behavio and bet toage allocation cannot be detemined. Fo intance, the EH technique doe vey well fo mall pace, but poduce high eo when the numbe of bucket i inceaed. Since in pactice obutne of an etimation technique i an impotant facto, we believe that SKETCH in geneal i pefeable ove technique whoe behavio cannot be pedicted a pioi. Thee ae eentially only two paamete that effect SKETCH pefomance: the elf-join ize of the data et, and the eult ize. A long a the elf-join ize ae not too lage compaed to the eult ize, SKETCH povide good etimate and it pefomance i independent of the actual object ditibution (kew) in the joining patial dataet. Thi dependence on eult ize i a chaacteitic of all pobabilitic etimation technique with povable eo bound. 8. RELATED WORK The bet known geneal technique fo electivity etimation fo join involving patial object ae baed on ampling o on hitogam. An et al. [5] examine diffeent ampling technique and popoe new hitogam-baed appoache. Thei eult indicate that fo achieving a compaable accuacy, hitogam will equie le toage and etimation time than ampling. Mamouli and Papadia [23] take a moe geneal appoach of analytically etimating the electivity of complex patial queie, i.e., queie that combine election and join opeato. Thei fomula ae baed on unifomity aumption. A kewed data et i patitioned into a egula gid of cell, and the fomula ae applied fo each cell. Sun et al. [26] impove on thee eult with a famewok baed on Eule hitogam [25]. They alo ue a egula gid patitioning of the pace, but the Eule hitogam togethe with adaptively elected pe-cell etimation technique povide moe accuate etimate fo patial join with geometic election. Falouto et al. [15] and Belui and Falouto [8] popoe paametic method fo etimating the electivity of ε-(elf- ) join of point-et. Fo elf-imila data et they achieve good appoximation uing powe law and factal dimenionality. The technique of Achaya et al. [2] etimate the electivity fo point and ange queie ove two-dimenional ectangula data. The baic idea i to patition the data pace into bucket and to ue model baed on unifomity aumption pe bucket. Cloely elated to electivity etimation ae appoache fo etimating the cot (in tem of CPU o I/O) of pecific opeato implementation. Example ae cot model fo ange queie and index-uppoted join fo patial data [9, 20, 21, 22, 29, 30]. None of the peviou appoache povide any eult quality guaantee. Sample ae difficult to maintain in the peence of update, epecially delete which could emove object fom the ample. Similaly, thee i no efficient algoithm fo maintaining factal dimenionality and powe law paamete in the peence of update. The hitogam technique, due to the egula gid patitioning, ae eay to maintain dynamically, howeve, thei utility i limited when the data i kewed. Othe hitogam that ae moe appopiate fo kewed data cannot be maintained efficiently and intoduce an additional eo when the bucket of the joined data et do not align. Ou technique ae baed on AMS ketche [4, 3] which ae dynamically maintainable and povide appoximation quality guaantee. Diffeent type of ketche o fa have been ued fo efficiently maintaining aggegate ove data team [6]. Example application ae complex aggegate [14], quantile [17], fequent item [10, 11], multidimenional hitogam [28], and gaph tatitic [7]. An upcoming pape by Tao et al. popoe to ue ketche to impove the eult quality fo aggegate queie ove patiotempoal point et [27]. 9. CONCLUSION AND FUTURE WORK In thi pape, we popoed a new famewok fo uing patial ketch technique fo appoximately anweing patial queie. To the bet of ou knowledge, ou technique i the fit method fo patial data that give pobabilitic quality guaantee. It pemit incemental contuction unde inetion and deletion of ecod while at the ame time be-