Queries. Key Intervals. level 0. Key Intervals. level 1. Key Intervals. level 2. Data items

Transcription

1 P-Grid: A slf-organizing accss structur for P2P information systms Karl Abrr Dpartmnt of Communication Systms Swiss Fdral Institut of Tchnology (EPFL) 1015 Lausann, Switzrland karl.abrr@pfl.ch Abstract Pr-To-Pr systms ar driving a major paradigm shift in th ra of gnuinly distributd computing. Gnutlla is a good xampl of a Pr-To-Pr succss story: a rathr simpl softwar nabls Intrnt usrs to frly xchang fils, such asmp3music fils. But it shows up also som of th limitations of currnt P2P information systms with rspct to thir ability to manag data fficintly. In this papr w introduc P-Grid, a scalabl accss structur that is spcifically dsignd for Pr-To-Pr information systms. P-Grids ar constructd and maintaind by using randomizd algorithms strictly basd on local intractions, provid rliabl data accss vn with unrliabl prs, and scal gracfully both in storag and communication cost. Kywords. Pr-To-Pr computing, Distributd Indxing, Distributd Databass, Randomizd Algorithms. 1 Introduction Pr-To-Pr (P2P) systms ar driving a major paradigm shift in th ra of gnuinly distributd computing. Major industrial playrs bliv P2P rflcts socity bttr than othr typs of computr architcturs. It is similar to whn in th 1980's th PC gav us a bttr rflction of th usr" ( In a P2P infrastructur, th traditional distinction btwn clints and back-nd (or middl tir application) srvrs is simply disapparing. Evry nod of th systm plays th rol of a clint and a srvr. Th nod pays its participation in th global xchang community byproviding accss to its computing rsourcs. Gnutlla is a good xampl of a P2P succss story: a rathr simpl softwar nabls Intrnt usrs to frly xchang fils, such as MP3 music fils. In th currnt P2P fil-sharing systms, lik Gnutlla [1], no indxing mchanisms ar supportd. Sarch rqusts ar broadcastd ovr th ntwork and ach nod rciving a sarch rqust scans its local databas (i.. its fil systm) for possibl answrs. This approach is xtrmly costly in trms of communication and lads to high sarch costs and rspons tims. For supporting fficint sarch, howvr, appropriat accss structurs ar prrquisit. Accss structurs in distributd information systms hav bn addrssd mostly in th ara of distributd and paralll databass. Diffrnt approachs to data accss in distributd nvironmnts hav bn takn thr. W mntion som of th principal approachs that can b found. ffl Th distribution of on copy ofasarch tr ovr multipl, distributd nods is a tchniqu that has bn invstigatd in [7]. Th sam authors hav shown that, undr crtain assumptions, in that with that approach balancd sarch trs do not xist [8].

2 ffl Th rplication of th complt sarch structur is an approach that undrlis th RP Λ -Trs proposd in [9]. In [11] a mchanisms is proposd that lads vntually to th rplication of th sarch structurs. ffl Scalabl rplication of a sarch tr (mor prcisly B-Tr) is proposd in [6] (db-tr) and [12] (Fat- BTr). With scalabl rplication ach nod stors a singl laf nod of th sarch tr, th root nod th sarch tr is rplicatd to vry nod, and th intrmdiat nods ar rplicatd such that ach nod maintains a path from th root to its own laf. ffl o sarch structurs: in ths approachs opration mssags ar broadcastd to all participating nods. E.g. with RP Λ [9] th data is rang partitiond as in B-Trs but no indx xists and a multicast mchanism is usd. In currnt P2P fil sharing systms lik Gnutlla th P2P ntwork is usd to propogat sarch rqusts to all rachabl prs. Most of ths approachs assum a rliabl xcution nvironmnt, rquir som cntralizd srvics, ar dsignd for a fairly small numbrs of nods (hundrds) or focus on dtrministic xcution guarants. In this papr w would lik to tak a diffrnt approach and invstigat th qustion of how an accss structur can b built in a community consisting of a vry larg numbr of unrliabl prs without any cntral authority, that can provid still a crtain lvl of rliability ofsarch and scals wll in th numbr of prs, both in storag and communication cost. In ordr to obtain scalability w us th approach of scalabl rplication of tr structurs, as proposd in [6, 12]. To build and us ths sarch structurs w us randomizd algorithms that ar xclusivly basd on local intractions among prs. Thidaisthat by randomly mting among ach othr th prs succssivly partition th sarch spac and rtain nough information in ordr to b abl to contact othr prs for fficintly answring futur sarch rqusts. Th rsulting distributd accss structur w call P-Grid (Pr-Grid). As this invstigation is intndd to clarify whthr such an approach is principally fasibl w mak th simplifying assumption that data distribution is not skwd. Thus th us of binary sarch trs ovr a totally ordr domain of kys is sufficint (as.g. also usd in [7]). Similarly, for th analysis and simulation of P-Grids, w mak uniformity assumptions on th bhavior of th prs. Though dfinitly in a nxt stp skwd data distributions and xtndd mthods for balancing th sarch trs (as wll known from B-Tr structurs) ar rquird, vn th basic mthods proposd in this papr could b xtrmly bnficial to improv th fficincy of currnt fil sharing applications. Th main charactristics of P-Grids ar ffl thy ar compltly dcntralizd, thr xists no cntral infrastructur, all prs can srv as ntry point to th ntwork and all intractions ar strictly local. ffl thy us randomizd algorithms for constructing th accss structur, updating th data and prforming sarch; probabilistic stimats can b givn for th succss of sarch rqusts, and sarch is robust against failurs of nods. ffl thy scal gracfully in th total numbr of nods and th total numbr of data itms prsnt in th ntwork, qually for all nods, both, with rspct to storag and communication cost. In th nxt sction w introduc our systm modl and th structur of P-Grids. In Sction 3 w dscrib th distributd, randomizd algorithm that is usd to construct P-Grids. In Sction 4 w giv som analysis on basic proprtis of P-Grids. Sction 5contains xtnsiv simulation rsults, that dmonstrat th fasibility of th P-Grid algorithms and xhibit important scalability proprtis of P-Grids. 2 Systm Modl and Accss Structur W assum that a community of prs P is givn that can b addrssd using a uniqu addrss addr : P! ADDR. For an addrss r 2 ADDR w dfinpr(r) =a iff addr(a) =r for a 2 P. Th prs ar onlin 2

3 with a probability onlin : P! [0; 1]. Prs that ar onlin can b rachd rliably using th undrlying communication infrastructur by mans of thir addrss. Evry pr stors information itms from a st DI that ar charactrizd by an indx trm from a st K. Th st of indx trms is totally ordrd, such that asarchtrcan b constructd in th usual way. In th following, w assum that th indx trms ar binary strings, P built from 0's and 1's and n that a ky k = p1 :::p n ; p i 2 f0; 1g corrsponds to a valu val(k) = i=1 2 ipi and an intrval I(k) = [val(k);val(k)+2 n [ [0; 1[. ow w dfin th accss structur. Th goal is to construct an accss structur, such that ffl Th sarch spac is subsquntly partitiond into intrvals of th form I(k); k 2 K. Evry pr taks ovr rsponsibitity for on intrval I(k). As ach ky corrsponds to a path in th binary sarch tr w will also say that th pr is rsponsibl for th path k. ffl Taking ovr rsponsibility for an intrval I(k) mans, that a pr should provid th addrsss of all prs that hav an information itm with a ky valu k qury that blongs to I(k), i.. val(k qury ) 2 I(k). ffl For ach prfix k l of k of lngth l; l =1; : : : lngth(k) apra maintains rfrncs to othr prs, that hav th sam prfix of lngth l, but a diffrnt valu at position l + 1, for th ky thy ar rsponsibl for. W will call ths rfrncs to othr prs, a's rfrncs at lvl l +1. Ths rfrncs ar usd to rout sarch rqusts for kys with th sam prfix k l,butacontinuation that dos not match th own ky, to othr agnts. ffl A sarch can b startd at ach pr. Bfor giving th formal dfinition of th accss structur, w show in Figur 1 a simpl xampl to illustrat th ida. Th diffrnt lvls rlat to th diffrnt lvls of th binary sarch tr. Th intrvals rlat to th nods of th sarch tr. W indicat th ky valus that corrspond to th intrvals (i.. 0 and 1 at lvl 1, 00, 01, 10, and 11 at lvl 2). At th lowst lvl w hav ntrd 6 prs, indicatd by th black circls, into th laf nods corrsponding to th kys for which thy ar rsponsibl for. On can s that multipl prs can b rsponsibl for th sam ky. W will call ths latr also rplicas. W also ntrd th agnts into all intrvals on th path from th root to thir laf nod. At th laf nods w show th pointrs to spcific data itms that hav aky of which thky rlatd to th laf nod is a prfix. On can s that at lvl zro vry pr is associatd with th whol intrval, in othr words, it stors a root nod of th sarch tr. At lvl 1 vry pr is associatd with xactly on of th two intrvals. At lvl 2 vry pr is associatd with xactly on intrval. Th connctors from on lvl to th nxt ar th rfrncs that ach pr maintains to covr th othr sid of th sarch tr. For xampl, at lvl 0 pr 1 has a rfrnc to pr 3, and at lvl1pr1hasa rfrnc to pr 2. Whn a sarch rqust is issud it is routd ovr th rsponsibl prs. Thr ar two possiblitis, ithr at th nxt lvl th pr itslf is rsponsibl, thn it can furthr procss th rqust itslf, or, th rqust nds to b forwardd to anothr pr. For illustration w hav includd into th Figur th procssing of two quris. In th first xampl th qury 00 is submittd to pr 1. As pr 1 is rponsibl for 00 it can procss th complt qury. In th scond xampl th qury 10 is submittd to pr 6. Using its rfrnc at lvl 0 it contacts pr 5, which in turn contacts at lvl 1 pr 4, who is rsponsibl for th ky corrsponding to th qury. W dfin now formally th data structur for prs that allows to rprsnt th P-Grid. Evry pr a 2 P maintains a squnc (p1;r1)(p2;r2) :::(p n ;R n ); whr p i 2f0; 1g and R i ρ ADDR. W dfin path(a) =p1 :::p n, prfix(i; a) =p1 :::p i for 0» i» n and rfs(i; a) =R i. In addition th numbr of rfrncs in R i will b limitd by avalu rfmax. Th sts R i,1» i» n ar rfrncs to othr prs and satisfy th following proprty: 3

4 Quris Ky Intrvals lvl Ky Intrvals 0 1 lvl Ky Intrvals lvl Data itms Figur 1: Exampl P-Grid r 2 rfs(i; a) :prfix(i; pr(r)) = prfix(i 1;a)p i whr path(a) =p1 :::p n and p is dfind as p =(p +1) MOD 2. In addition, ach pr maintains a st of rfrncs D ρ ADDR K to th prs that stor data itms indxd by kywords k for which path(a) is a prfix. In othr words, at th laf lvl th pr knows at which prs data itms corrsponding to th sarch kys that it is rsponsibl for, can b found. This givs ris to a straightforward dpth first sarch algorithm using th accss structur: qury(a, p, l) found = FALSE; rmpath = sub_path(path(a), l+1, k); compath = common_prfix_of(p, rmpath); IF lngth(compath) = lngth(p) THE rsult = a ELSE IF lngth(path(a)) > l + lngth(compath) THE qurypath = sub_path(p, lngth(compath) + 1, lngth(p)); rfs = rfs(l + lngth(compath) + 1, a); WHILE rfs > 1 AD OT found r = random_slct(rfs); IF onlin(pr(r)) found = qury(pr(r), qurypath, l + lngth(compath)); RETUR found; /* Commnt: sub_path(p1...pn, l, k) := pl...pk common_prfix_of(p1...pk pk+1...pn, p1...pk qk+1...ql = p1...pk) random_slct(rfs) rturns a random lmnt from rfs and rmovs it from rfs */ } 4

5 Givn a P-Grid, a qury p can b issud to ach pr a through a call qury(a; p; 0). 3 P-Grid Construction Having introducd th accss structur and th sarch algorithm, th main qustion is now, of how a P- Grid can b constructd. As thr xists no global control this has to b don by using xclusivly local intractions. Thy ida is that whnvr prs mt, thy us th opportunity to crat a rfinmnt of th accss structur. W do not car at this point why prs mt. Thy may mt randomly, bcaus thy ar involvd in othr oprations, or bcaus thy systmatically want to build th accss structur. But assuming that by som mchanisms thy mt frquntly, th procdur works as follows. Initially, all prs ar rsponsibl for th whol sarch spac, i.. all sarch kys. At that stag, whn two prs mt initially, thy dcid to split th sarch spac into two parts and tak ovr rsponsibility for on half ach. Thy also stor th rfrnc to th othr pr in ordr to covr th othr part of th sarch spac. Th sam happns whnvr two prs mt, that ar rsponsibl for th sam intrval at th sam lvl. Howvr, as soon as th P-Grid dvlops, also othr cass occur. amly, prs will mt 1. thir kys shar a common prfix, 2. thir kys ar in a prfix rlationship. In cas 1, what th prs can do is to initiat nw xchangs, by forwarding ach othr to prs thy ar thmslvs rfrncing. In cas 2 th pr with th shortr ky can spcializ by xtnding its ky. In ordr to obtain a balancd P-Grid it will spcializ in th opposit way th othr pr has alrady don at that lvl. Th othr pr rmains unchangd. Ths considrations giv ris to th following algorithm that two prs a1 and a2 xcut whn thy mt. xchang(a1, a2, r) commonpath = common_prfix_of(path(a1), path(a2)); lc = lngth(commonpath); IF lc > 0 (* xchang rfrncs at th lvl whr th paths agr *) commonrfs = union(rfs(lc, a1), rfs(lc, a2)); rfs(lc, a1) = random_slct(rfmax, commonrfs); rfs(lc, a2) = random_slct(rfmax, commonrfs); l1 = lngth(sub_path(path(a1), lc + 1, lngth(path(a1))); l2 = lngth(sub_path(path(a2), lc + 1, lngth(path(a2))); (* Cas 1: if both rmaining paths ar mpty introduc a nw lvl *) CASE l1 = 0 AD l2 = 0 AD lngth(commonpath) < maxlngth path(a1) = appnd(path(a1), 0); path(a2) = appnd(path(a2), 1); rfs(lc + 1, a1) = a2}; rfs(lc + 1, a2) = a1}; (* Cas 2: if on rmaining path is mpty split th shortr path *) CASE l1 = 0 AD l2 > 0 AD lngth(commonpath) < maxlngth path(a1) = appnd(path(a1), valu(lc+1, path(a2))^-; rfs(lc + 1, a1) = a2}; rfs(lc + 1, a2) = random_slct(rfmax, union( a1}, rfs(lc+1, a2)); (* Cas 3: analogous to cas 2 *) CASE l1 > 0 AD l2 = 0 AD lngth(commonpath) < maxlngth 5

6 path(a2) = appnd(path(a2), valu(lc+1, path(a1))^-; rfs(lc + 1, a2) = a1}; rfs(lc + 1, a1) = random_slct(rfmax, union( a2}, rfs(lc+1, a1)); (* Cas 4: rcursivly prform xchang with rfrncd prs *) CASE l1 > 0 AD l2 > 0 AD r < rcmax, rfs1 = rfs(lc+1, a1) a2}; rfs2 = rfs(lc+1, a2) a1}; FOR r1 I rfs1 DO IF onlin(pr(r1)) THE xchang(a2, pr(r1), r+1); FOR r2 I rfs2 DO IF onlin(pr(r2)) THE xchang(a1, pr(r2), r+1); /* Commnt: random_slct(k, rfs) rturns a st with k random lmnts from rfs } appnd(p1...pn, p) = p1...pn p valu(k, p1...pn) = pk p^- = 1+p MOD 2 */ A fw rmarks ar in plac. A masur to prvnt ovrspcialization of prs is to bound th maximal lngth of paths that can b constructd to maxlngth. Simulations show that this rsults in a mor uniform distribution of path lngths among prs and bttr convrgnc of th P-Grid. Also such a bound is ndd whn a crtain dgr of rplication at th lowst grid lvl is to b achivd. Th disadvantag is that som global knowldg is usd, namly th maximal path lngth, which not always might b locally availabl or asily drivabl. In practical applications, on possibl indication that a path has rachd maxlngth could b that th numbr of data itms blonging to th ky is falling blow a crtain thrshold. An altrnativ would b to avoid ovrspcialization by taking anothr approach in Cas 2 and Cas 3, whr on path is subpath of th othr and th pr with shortr path chooss to spcializ diffrntly than th othr pr. Hr on could shortn th longr path if th diffrnc in lngth is gratr than 1, such that both rsulting paths hav th sam lngth. Howvr, this would rquir additional mans to maintain consistncy of rfrncs as prs giv up rsponsibility for kys by gnralizing and could possibly spcializ at a latr stag diffrntly and slows down convrgnc. So w ommittd this possibility hr. Th rcursiv xcutions of th xchang function by using th locally availabl rfrncs hav an important influnc on th prformanc of th mthod. Ths xcutions ar mor promising of nding up in a succssful spcialization as thy ar alrady targttd to a mor spcific st of candidats. On th othr hand th rcursiv xcutions might lad to a quick ovrspcialization of th P-Grid in subparts of it. Thrfor, w bound th rcursion dpth up to which th xchang function is calld by thvalu rcmax. This valu has a vry strong influnc on th global prformanc of th algorithm, as w will s latr. So far, w hav only considrd th construction procss of th accss structur itslf. At th laf lvl th prs nd also to know th data itms, rspctivly th prs storing thos data itms, that corrspond to thir rsponsibility. As many prs can b rsponsibl for th sam ky th gnral problm is of how to find all thos prs in cas of an updat. Diffrnt stratgis ar possibl: ffl Randomly prforming dpth first sarchs for prs rsponsibl for th ky multipl tims and propagating th updat to thm ffl Prforming bradth first sarchs for prs rsponsibl for th ky onc and propagating th updat to thm ffl Crating a list of buddis for ach pr, i.. othr prs that shar th sam ky, and propagat th updat to all buddis. W will not giv th dtaild algorithms hr as thy ar quit obvious. But in Sction 5 w will idntify by using simulations, which is th most fficint mthod. 6

7 4 Analysis of Sarch Prformanc W want to analyz th qustion of how probabl it is to find a pr that is rsponsibl for a spcific sarch ky starting th sarch at an arbitrary pr. This analysis allows to giv rough stimats on th sizing of th P-Grid paramtrs that ar rquird to achiv a dsird sarch rliability inagivn stting. W prform th analysis for an idalizd situation, whr for all prs th paramtrs of th P-Grid, lik kylngth and numbr of prs rsponsibl for th sam ky, ar uniformly distributd. Though such a distribution will not occur in practic it givs a good stimation th quantitativ natur of a P-Grid. Th following paramtrs dtrmin th problm. Th numbr of prs and th numbr of data objcts ach pr can stor d pr dtrmin th total numbr of data objcts that can b stord in th ntwork as d global = Λ d pr. Th siz of a rfrnc r and th amount of spac ach pr is willing to mak availabl for indxing purposs s pr dtrmins th possibl numbr of rfrncs that can b stord at ach pr i pr = spr r. ow w dtrmin th numbr of ntris rquird for a crtain grid organisation. Th lngth of a ky rquird to diffrntiat data itms locatd at diffrnt prs is givn by k log2 d global i laf (1) whr i laf is th numbr of rfrncs to data itms ach pr stors at th laf lvl. Thn th total numbr of indx ntris stord at a pr is givn by i laf + k Λ rfmax, whr rfmax is th multiplicity of rfrncs usd to build th grid structur. Thus w obtain th constraint i laf + k Λ rfmax» i pr (2) which dtrmins th valu of i laf. In ordr to allow at th lowst lvl of th grid th support for rf max altrnativ prs, rfrncs to data itms nd to b rplicatd with a factor of rf max. This is only possibl if d global i laf Λ rfmax» (3) i.. thr must b sufficintly high numbrs of prs availabl, such that ach intrval at th lowst grid lvl is supportd by at last rfmax prs. Givn a constant probability p that a pr is onlin w arnowintrstd in th qustion what is th probability of prforming a succssful sarch for a pr that is rsponsibl for th qury ky. In th worst cas at ach lvl of th grid a nw pr nds to b contactd, that is slctd out of th availabl rfrncs. At achlvl of th grid, th probability ofraching a pr at th nxt lvl is 1 (1 p) rfmax, i.. on minus th probability that all rf max rfrncd prs ar offlin. Sinc th sarch tr is of dpth k, thn th probability of prforming a succssful sarch for a ky is (1 (1 p) rfmax ) k (4) W givnow an xampl, to illustrat what a P-Grid would cost in trms of spac for a practical stting. Exampl. Lt us considr th stting P2P fil sharing as it currntly is found with Gnutlla. W us som rough stimats of th actual paramtrs that ar obsrvd for this application. Assum that d global =10 7 data objcts (fils) xist, that a rfrnc costs at most r = 10 Byts of storag (th path plus th IP addrss) and that vry pr is willing to provid s local =10 5 Byts of storag for indxing (which is in fact far lss than th siz of an avrag mdia fil). Furthrmor w assum that prs ar onlin with probability 30%. Lt us now analyz how larg a community for supporting th 10 7 fils must b in ordr to nsur that sarch rqusts for fils ar answrd with a probability of99%. Each prcan stor at most i pr =10 5 rfrncs. If w "guss" a valu of i laf = , w s that inquality (1) is satisfid for a valu of k =10. For a valu of rfmax = 20 thn, according to (4), 7

8 th probability of sucssfully finding a pr for a givn ky is largr than 99%. Th storag rquird is du to our good initial guss xactly i pr =10 5. Th numbr of prs rquird to support this grid has to b according to inquality (3) largr than This is a vry rasonabl numbr compard to th siz of th actual Gnutlla usr community. 5 Simulation For prforming simulations w hav implmntd th algorithm for constructing P-Grids using th mathmatical computing packag Mathmatica ( With ths simulations w intnd to obtain rsults on th following qustions. ffl How many communications in trms of xcuting th xchang algorithm ar rquird for building a P-Grid? ffl What is th influnc of th rcursion factor rcmax in th xchang algorithm on th fficincy of th P-Grid construction? ffl Is th rsulting structur rasonably wll balancd with rspct to distribution path lngths and numbr of rplicas pr path? ffl How rliabl can data b found using th P-Grid? First w analyz th convrgnc spd whn th P-Grid is constructd. In this simulation w varid th numbr of prs from 200 to Th prs mt randomly pairwis and xcut th xchang function. Th path lngth was boundd by maxlngth = 6. As a critrion for a P-Grid to b built w assumd that th avrag lngth of th kys that th prs ar rsponsibl for rachs a crtain thrshhold t, i.. 1 P a2p lngth(path(a)) < t. In th following simulation th numbr of calls to th xchang function whr countd till an avrag path lngth of 5.94 (99% of maxlngth) was rachd. Th simulation was prformd with a rcursion dpth rcmax of 0 and 2. Th valu of rfmax was st to 1, i.. only on rfrnc to anothr pr ist stord. This influncs only th prformanc in th cas whr rcmax > 0. Th rsults indicat that a linar rlationship xists btwn th numbr of prs () and th total numbr of communications () ndd in building th P-Grid. As a consqunc, vry pr has to prform on avrag a constantnumbr of xchangs to rach its maximal path lngth indpndnt of th total numbr of prs involvd. rcmax = 0 rcmax= Th nxt simulation shows how th choic of a maximal path lngth maxlngth influncs th numbr of xchangs maxlngth rquird. Th simulation is don for = 500 prs. Th rsults indicat that th numbr of communication grows xponntially in th path lngth (to a basis 2) whn no rcursion is usd. With a rcursion bound rcmax = 2 th convrgnc spds up substantially. 8

9 rcmax =0 rcmax =2 maxlngth maxlngth maxlngth maxlngth maxlngth 1 maxlngth maxlngth maxlngth maxlngth Th following tabl shows that th rcursion dpth rcmax has substantial influnc on th convrgnc spd. Whn using rcursiv calls to th xchang function th probability that a random mting lads to a succssful xchang in th P-Grid incrass. Howvr, if rcursion is not constraind this can lad to ngativ ffcts as th prs tnd to ovrspcializ quickly. Th rsult shows that for 500 prs and maximal path lngth 6 th optimal rcursion dpth limit is 2. rcmax If rfmax > 1, i.. prs maintain mor than on rfrnc to othr prs at ach lvl, i.. thr xist mor possibilitis to mak rcursiv calls. Thus if rcmax > 0 this should hav an influnc on th numbr of xchangs that ar prformd whn constructing th P-Grid. ot that this additional ffort is rwardd by a highr dnsity of th P-Grid. W analyzd this with th following simulation with = 1000 prs, a rcursion dpth limit rcmax = 2 and using maximal path lngth of 6. rfmax As on can s th numbr of xchangs grows xponntially, which is undsirabl. In fact, this turnd out to b a waknss in th algorithm w proposd. Howvr, thr xists a simpl way tofix this. On limits th numbr of rfrncd prs with which xchangs ar mad throughout rcursion to a low numbr. Thn th rsults bcom vry stabl as th following simulation with th modifid algorithm shows, whr rcursiv calls ar only mad to 2 randomly slctd rfrncd prs. rfmax Th following simulations ar basd on a configuration that is similar to th on rsulting from our analysis in Sction 4. This confirms that th algorithms scal wll for ralistic paramtr sttings. W ar using prs that build a P-Grid with binary kys of maximal lngth 10. Th maximal numbr of rfrncs rfmax at ach lvl is limitd to 20. Th onlin probability of prs is 30%. Building a P-Grid of that siz within a simulation nvironmnt is starting to consum considrabl rsourcs. Within approximatly 10 hours of running tim on a Pntium III procssor th P-Grid built up 9

10 to an avrag dpth of 9:43. During that tim xchangs among prs wr taking plac, i.. 62 pr pr. Basd on th P-Grid constructd this way w mad th following analyss. A first qustion of intrst is, how balancd th P-Grid is with rspct to th distribution of kys, i.. how many prs ar rsponsibl for th sam kys. W will call such prs rplicas. A simpl, intuitiv argumnt shows that th xchang function inhrntly tnds to balanc th distribution of kys. Lt us look at th top lvl, i.. at th dcision whthr th ky of a pr starts with a 0 or a 1. A pr will dcid upon this whn it mts th first tim anothr pr that has alrady takn this dcision or also nds to dcid on th top lvl, and h will dcid for th opposit valu. Thus, if thr xists an imbalanc such that on of th two valus dominats this lads immdiatly to a compnsation ffct that th valus occuring with lss frquncy will mor likly b chosn. In that way th algorithm is slf-stabilizing. Th following Figur 2 shows th distribution of th numbr of prs rsponsibl for th sam ky. Th x-axis indicats th rplication factor and th y-axis th numbr of prs that hav this rplication factor. Th avrag numbr of rplicas for a pr is Figur 2: Rplica distribution xt w would lik to xprimntally confirm th analysis on th probability of finding a pr that is rsponsibl for a givn ky. W mad a simulation whr w sarchd tims for a random ky of lngth 9. Rmmbr that only 30% of th prs ar onlin. In 99.97% of th cass th sarch was succssful and a sarch rquird on avrag mssags among prs. W wr counting as mssags th succssful calls of th qury opration to anothr pr. This shows that sarchs can b prformd rliably. ow w turn to th qustion of how rliably updats can b prformd. Th problm with an updat, in contrast to a sarch, is that w hav to find all rplicas of a path, not just on. Thrfor w analyz how fficintly a larg fraction of all rplicas can b found. W compar thr approachs. 1. Rpatd dpth first sarchs 2. Rpatd dpth first sarchs including all buddis that hav bn idntifid throughout indx construction 3. Rpatd bradth first sarchs W rpatdly sarchd for a random ky of lngth 9 and thn computd th fraction of rplicas idntifid to th actual numbr of xisting rplicas. Figur 3 shows th rsult. Th x-axis shows th numbr of mssags usd in th insrtion procss, and th y-axis th prcntag of succssfully idntifid rplicas. Without giving th dtaild simulation stup, on can s that clarly th stratgy of using bradth first sarchs is by far suprior, whil th two othr mthods prform comparably. On can s also from ths simulations, that for achiving a high updat rliability a fairly larg numbr of mssags ar rquird (in th ordr of hundrds pr rplica to b updatd). So this approach is accptabl 10

11 1 0.8 sarch with buddis bradth first sarch dpth first sarch Figur 3: Finding all rplicas in th cas whr updats ar rar as compard to quris. Howvr, th rlvant problm is not so much to achiv high rliability in kping th rplicas consistnt, but rathr high rliability in obtaining corrct qury rsults. Thus w can follow a diffrnt approach. W updat a sufficintly high numbr of rplicas using fwr mssags, and thn us rpatd qury oprations, till a qury rsult is multiply confirmd. Obviously, if mor than half of th rplicas ar corrct by rpating quris arbitrarily high rliability can b achivd. Thus, by procding in this mannr w incras slightly th qury cost and in turn rduc drastically th updat cost. Thr is anothr factor that is hlpful in that contxt. ot all rplicas ar as likly to b found. This implis that rplicas that ar found during updats ar also mor likly to b found during quris. Th following simulations illustrat th tradoff among updat and qury cost and indicat what advantgous combinations of updat and qury stratgis xist. 100 updats wr prformd and ach updatd data itm was sarchd 10 tims, thus 1000 quris wr prformd in ach configuration. Each updat was prformd by a bradth first sarch whr at ach lvl rcbradth rfrncs ar followd. Th bradth first sarch was rpatd during updat rptition tims. Th succssrat is th fraction of succssfully answrd quris aftr updat. Th cost is again in trms of numbr of mssags. with rptitiv sarch rcbradth rptition succssrat qury cost insrtion cost without rptitiv sarch Th main rsult hr is that th approach of using rpatd sarchs to achiv qury rliability pays off dramatically. Th configuration (rcbradth = 3; rptition = 3) without using rpatd sarch, which achivs only 99,4% rliability, would rquir at last a ratio of 160 quris pr updat in ordr rach th 11

12 brak-vn point with th configuration (rcbradth = 2; rptition = 3) with using rpatd sarch, but which offrs practically 100% rliability. 6 Discussion This papr introducd P-Grid a first stp towards dvloping scalabl, robust and slf-organizing accss structur for Pr-To-Pr information systms. In ordr to bttr undrstand th ffctivnss of a P-Grid w can compar it to th us of cntralizd rplicatd srvrs. Assum D is th numbr of data itms and th numbr of prs. For storag w considr th numbr of rfrncs that nd to b stord at th nods ignoring any cost for local indxing. For qurying w considr th numbr of mssags xchangd at th nods assuming that ach nod crats a constant numbr of quris pr tim unit. Thn a solution using cntralizd (possibly rplicatd) srvrs compars to th P-Grid as follows. P-Grid Cntral Srvr Storag prs: O(log D) srvr: O(D) clint: constant Qury prs: O(log ) srvr: O() clint: constant On can s that both storag and communication cost scal wll for th P-Grid. For a srvr solution in particular th linar growth of communication cost in trms of numbr of clints is to b considrd ciritical as srvrs tnd to bcom bottlncks. This shows that, bsids othr considrations motivating th us of P-Grids lik dcntralization or robustnss, also scalability is on of th bnfits. Th mphasis of this papr was on distributd, randomizd algorithms which allow th prs to cooprativly partition th sarch spac in an fficint way. Th approach is in this papr is limitd to uniform data distributions. For uniformly distributd ky valus th P-Grid can b immdiatly applid. For prfix sarch on txt th algorithm can b adaptd by xtnding th f0; 1g alphabt. This would allow to support tri sarch structurs. Howvr a worthwhil invstigation would b to xtnd th mthod by adapting mor sophisticatd txt sarch structurs, lik in [3, 4]. An obvious continuation of this rsarch istodvlop P-Grids that can adapt to skwd data distributions. To that xtnt throughout th construction procss th actual data distribution nds to b takn into account and th structurs hav tocontinuously to adapt to updats. Howvr, w s no principal difficultis in implmnting such an approach. Anothr natural xtnsion of th approachwould b to tak othr known paramtrs, lik known rliability of prs, knowldg on th ntwork topologyorknowldg on qury distribution into account throughout P-Grid construction and updats. To achiv load balancing a computational conomy can b imposd, as alrady invstigatd for distributd data mangmnt in[10,5]. W s th P-Grid, as it is prsntd in this papr, as a first rprsntativ of accss structurs for Pr- To-Pr information systms, for which w xpct to s in th futur many variations, that ar adaptd to th spcific rquirmnts of various Pr-To-Pr application domains. 7 Acknowldgmnts I would lik to thank Manfrd Hauswirth for carfully rading and commnting th manuscript. I would also lik to thank Magdalna Puncva andrachid Gurraoui for many hlpful discussions. This work also gratly bnfitd from th inspiring working nvironmnt that is providd by th nwly foundd Communication Systms Dpartmnt at EPFL. 12

13 Rfrncs [1] E. Adar and B. A. Hubrman: Fr ridingongnutllatchnical rport, Xrox PARC, 10 Aug [2] D. Clark. Fac-to-Fac with Pr-to-Pr tworking. IEEE Computr, January [3] Y. Chn, K. Abrr: Layrd Indx Structurs in Documnt Databas Systms Procdings of th 1998 ACM CIKM Intrnational Confrnc on Information and Knowldg Managmnt, Bthsda, Maryland, USA, ovmbr 3-7, ACM Prss, p , [4] Y. Chn, K. Abrr: Combining Pat-Trs and Signatur Fils for Qury Evaluation in Documnt Databass DEXA 99, Flornc, Italy. [5] D.F. Frguson, C. ikolaou and Y. Ymini An Economy for Managing Rplicatd Data in Autonomous Dcntralizd Systms Proc. Int. Symp. on Autonomous Dcntralizd Sys. (ISADS'93), Kawasaki, Japan, [6] T. Johnson, P. Krishna Lazy Updats for Distributd Sarch Structur SIGMOD Confrnc 1993: [7] B. Kröll, P. Widmayr Distributing a Sarch Tr Among a Growing umbr of Procssors. ACM SIGMOD Confrnc 1994: [8] B. Kröll, P. Widmayr Balancd Distributd Sarch Trs Do ot Exist WADS 1995: [9] W. Litwin, M. imat, D. A. Schnidr RP*: A Family of Ordr Prsrving Scalabl Distributd Data Structurs. VLDB 1994: [10] M. Stonbrakr, P. M. Aoki, W. Litwin, A. Pfffr, A. Sah, Jff Sidll, Carl Stalin, Andrw Yu: Mariposa A Wid-Ara Distributd Databas Systm VLDB Journal 5(1): 48-63, [11] R. Vingralk, Y. Britbart, G. Wikum SOWBALL: Scalabl Storag on tworks of Workstations with Balancd Load Distributd and Paralll Databass Vol 6(2), Kluwr Acadmic Publishrs, 1998 [12] H. Yokota, Y. Kanmasa, J. Miyazaki Fat-Btr: An Updat-Conscious Paralll Dirctory Structur ICDE 1999: