Introduction to Algorithms 6.046J/18.401J

Transcription

1 3/4/28 Intoduton to Algothms 6.46J/8.4J Letue 8 - Hashng Pof. Manols Kells Hashng letue outlne Into and defnton Hashng n pate Unvesal hashng Pefet hashng Open Addessng (optonal) 3/4/28 L8.2 Data Stutues Role of data stutues: Enapsulate data Suppot etan opeatons (e.g., INSER, DELEE, SEARCH) Ou fous: effeny of the opeatons Algothms vs. data stutues Symbol-table poblem Symbol table holdng n eods: eod x key[x] Othe felds ontanng satellte data Opeatons on : INSER(, x) DELEE(, x) SEARCH(, k) How should the data stutue be oganzed? 3/4/28 L8.3 3/4/28 L8.4 Det-aess table IDEA: Suppose that the set of keys s K {,,, m }, and keys ae dstnt. Set up an aay [.. m ]: x f k K and key[x] k, [k] NIL othewse. hen, opeatons take Θ() tme. Poblem: he ange of keys an be lage. Eg.: 64-bt numbes (whh epesent 8,446,744,73,79,55,66 dffeent keys) Chaate stngs (even lage!). Hash funtons Soluton: Use a hash funton h to map the unvese U of all keys nto {,,, m }: K k 2 k k 4 k 5 U k3 When a eod to be nseted maps to an aleady ouped As eah key slot s n nseted,, a ollson h maps ous. t to a slot of. h(k ) h(k 4 ) h(k 2 ) h(k 5 ) h(k 3 ) m 3/4/28 L8.5 3/4/28 L8.6

2 3/4/28 Resolvng ollsons by hanng Reods n the same slot ae lnked nto a lst h(49) h(86) h(52) Run tme analyss eques key assumpton: 3/4/28 L8.7 Smple unfom hashng We make the assumpton of smple unfom hashng: Eah key k K of keys s equally lkely to be hashed to any slot of table, ndependent of whee othe keys ae hashed. Let n be the numbe of keys n the table, and let m be the numbe of slots. Defne the load fato of to be α n/m aveage numbe of keys pe slot. 3/4/28 L8.8 Seah ost fo hanng unde smple unfom hashng Expeted tme to seah fo a eod wth a gven key Θ( + α). apply hash funton and aess slot seah the lst Expeted seah tme Θ() f α O(), o equvalently, f n O(m). Into and defnton Hashng n pate Unvesal hashng Pefet hashng Open Addessng (optonal) 3/4/28 L8.9 3/4/28 L8. Choosng a hash funton he assumpton of smple unfom hashng s had to guaantee, but seveal ommon tehnques tend to wok well n pate as long as the defenes an be avoded. Desata: A good hash funton should dstbute the keys unfomly nto the slots of the table. Regulaty n the key dstbuton should not affet ths unfomty.. Dvson method Assume all keys ae nteges, and defne h(k) k mod m. Defeny: Don t pk an m that has a small dvso d. A pepondeane of keys that ae onguent modulo d an advesely affet unfomty. Exteme defeny: If m 2, then the hash doesn t even depend on all the bts of k: If k 2 and 6, then h(k) 2. h(k) 3/4/28 L8. 3/4/28 L8.2 2

3 3/4/28. Dvson method (ontnued) h(k) k mod m. Pk m to be a pme not too lose to a powe of 2 o and not othewse used pomnently n the omputng envonment. Annoyane: Sometmes, makng the table sze a pme s nonvenent. But, ths method s popula, although the next method we ll see s usually supeo. 2. Multplaton method Assume that all keys ae nteges, m 2, and ou ompute has w-bt wods. Defne h(k) (A k mod 2 w ) sh (w ), whee sh s the bt-wse ght-shft opeato and A s an odd ntege n the ange 2 w < A < 2 w. Don t pk A too lose to 2 w. Multplaton modulo 2 w s fast. he sh opeato s fast. 3/4/28 L8.3 3/4/28 L8.4 Suppose that m and that ou ompute has w 7-bt wods: 3A A. k h(k) A 2. Multplaton method example h(k) (A k mod 2 w ) sh (w ) Modula wheel Don t pk A too lose to 2 w. 2A 2. Multplaton method example h(k) (A k mod 2 w ) sh (w ) w k Neve omputed (mult. mod 2 w ) h(k) (exploe moe of the wheel spae) 3/4/28 L8.5 3/4/28 L8.6 A Don t pk A too lose to 2 w (have balane of s and s) 3. Dot-podut method Randomzed stategy: Eah value k < m, wth m pme. Let m be pme. Deompose k nto + k k k 2 k - k dgts, eah n {,,..m-}. Pk andom veto a, a a a 2 a - a smlaly deomposed (eah a hosen andomly m m m m m fom {,,..m-} ). Calulate dot podut k a, a k a k a 2 k 2 a k eah multplaton mod m Exellent n pate, but ha ( k) ak mod m expensve to ompute. Into and defnton Hashng n pate Unvesal hashng Pefet hashng Open Addessng (optonal) 3/4/28 L8.7 3/4/28 L8.8 3

4 3/4/28 A weakness of hashng Poblem: Fo any hash funton h, a set of keys exsts that an ause the aveage aess tme of a hash table to skyoket. Advesay an pk all keys st. {k U:h(k)} fo some slot (e.g. denal-of-seve attaks). IDEA: Choose the hash funton at andom, ndependently of the keys. Even f an advesay an see you ode, they annot fnd a bad set of keys, sne they don t know exatly whh hash funton wll be hosen. 3/4/28 L8.9 Unvesal hashng: Defnton Let U be a unvese of keys. Let H be a fnte olleton of hash funtons, eah mappng U to {,,, m }. H s unvesal f fo all x, y U, whee x y, we have {h H : h(x) h(y)} H /m,.e. only /m of hash funtons n H esult n x,y ollson. hus, f we hoose h andomly fom H, the hane of a ollson between x and y s /m {h : h(x) h(y)} H m H 3/4/28 L8.2 Unvesalty s good heoem. Let h be a hash funton hosen (unfomly) at andom fom a unvesal set H of hash funtons. Suppose h s used to hash n abtay keys nto the m slots of a table. hen, fo a gven key x, we have #ollsons wth x] < n/m. Poof of theoem Poof. Let C x be the andom vaable denotng the total numbe of ollsons of keys n wth x, and let f h(x) h(y), othewse. Note: ] /m and C. x 3/4/28 L8.2 3/4/28 L8.22 Poof (ontnued) Poof (ontnued) E [ C ] E y x ake expetaton of both sdes. C x ] E ] ake expetaton of both sdes. Lneaty of expetaton. 3/4/28 L8.23 3/4/28 L8.24 4

5 3/4/28 C x Poof (ontnued) ] E / m ] ake expetaton of both sdes. Lneaty of expetaton. ] /m. Poof (ontnued) Cx ] E n. m / m ] ake expetaton of both sdes. Lneaty of expetaton. ] /m. Algeba. 3/4/28 L8.25 3/4/28 L8.26 Constutng a set of unvesal hash funtons Let m be pme. Deompose key k nto + dgts, eah wth value n the set {,,, m }. hat s, let k k, k,, k, whee k < m. Randomzed stategy w/ dot-podut method: Pk a a, a,, a whee eah a s hosen andomly fom {,,, m } ndpt of nput. Defne h ( k) a k mod m. a How bg s H {h a }? H m +. Dot podut, modulo m REMEMBER HIS! Unvesalty of dot-podut hash funtons heoem. he set H {h a } s unvesal. Poof. Suppose that x x, x,, x and y y, y,, y ae dstnt keys. hus, they dffe n at least one dgt poston, w.l.o.g. poston (and possbly moe postons). Fo how many h a H do x and y ollde? ha( x) ha( b) a x a y (mod m) 3/4/28 L8.27 3/4/28 L8.28 Poof (ontnued) Equvalently, we have o a ( x a ( x y ) (mod m) y) + a ( x y ) (mod m whh mples that ), a( x y) a ( x y ) (mod m). Multply both sdes by nvese of ( x y ). 3/4/28 L8.29 Fat fom numbe theoy heoem. Let m be pme. Fo any z Z m suh that z, thee exsts a unque z Z m suh that z z (mod m). z s known as the multplatve nvese of z. z Example: m 7. z Explanaton. gd(z,m), zx+myl, zx (mod m), whee (z,x,y) s the output of EXENDED-EUCLID(z,m). Goup theoy. (Z n *, n) s a fnte abelan goup. (See Chapte 3, Poof: 3.3 and 3.26). 3/4/28 L8.3 5

6 3/4/28 We have Bak to the poof a( x y) a ( x y ) (mod m), and sne x y, an nvese (x y ) must exst, whh mples that a ( ) ( ) a x y x y (mod m). hus, fo any hoes of a, a 2,, a, exatly one hoe of a auses x and y to ollde. Poof (ompleted) Q. How many h a s ause x and y to ollde? A. hee ae m hoes fo eah of a, a 2,, a, but one these ae hosen, exatly one hoe fo a auses x and y to ollde, namely a a ( x y ) ( x y ) mod m. hus, the numbe of h a s that ause x and y to ollde s m m H /m. 3/4/28 L8.3 3/4/28 L8.32 Poof (ompleted) Q. Why sn t the answe just a? a( x y) a ( x y ) (mod m) A. Sue, f x y fo all, then the soluton to the equaton s a. Howeve, what the theoem shows s that fo any ombnaton of <x > and <y >, thee exsts an a that satsfes: a a ( x y ) ( x y ) mod m Into and defnton Hashng n pate Unvesal hashng Pefet hashng Open Addessng (optonal) 3/4/28 L8.33 3/4/28 L8.34 Pefet hashng Gven a set of n keys, onstut a stat hash table of sze m O(n) suh that SEARCH takes Θ() tme n the wost ase. IDEA: wolevel sheme wth unvesal 2 hashng at 3 both levels No ollsons at level 2! 86 S h S 3 (4) h 3 (27) 4 S m a Collsons at level 2 heoem. Let H be a lass of unvesal hash funtons fo a table of sze m n 2. hen, f we use a andom h H to hash n keys nto the table, the expeted numbe of ollsons s at most /2. Poof. By the defnton of unvesalty, the pobablty that 2 gven keys n the table ollde n unde h s /m /n 2. Sne thee ae ( ) pas 2 of keys that an possbly ollde, the expeted numbe of ollsons s n n( n ) E [ X ] < n 2 n 2 3/4/28 L8.35 3/4/28 L8.36 6

7 3/4/28 No ollsons at level 2 Coollay. he pobablty of no ollsons s at least /2. Poof. Makov s nequalty says that fo any nonnegatve andom vaable X, we have P{X t} X]/t. Applyng ths nequalty wth t, we fnd that the pobablty of o moe ollsons s at most /2. hus, just by testng andom hash funtons n H, we ll qukly fnd one that woks. Analyss of stoage Fo level- hash table, hoose m n. Rand. va. n # of keys that hash to slot n. If we use n 2 slots fo the level-2 hash table S, expeted total stoage equed fo the two-level sheme s: m ( n 2 E Θ ) Θ( n), (the analyss s dental to the analyss of buket sot expeted unnng tme fom etaton). Fo pobablty bound, apply Makov nequalty. 3/4/28 L8.37 3/4/28 L8.38 Into and defnton Hashng n pate Unvesal hashng Pefet hashng Open addessng (optonal) Resolvng ollsons by open addessng No stoage s used outsde of the hash table tself. Inseton systematally pobes the table untl an empty slot s found. he hash funton h(k,) depends on both the key k and the pobe numbe : h : U {,,, m } {,,, m }. he pobe sequene h(k,), h(k,),, h(k,m ) should be a pemutaton of {,,, m }. he table may fll up, and deleton s dffult (but not mpossble). 3/4/28 L8.39 3/4/28 L8.4 Example of open addessng Example of open addessng Inset key k 496:. Pobe h(496,) Inset key k 496:. Pobe h(496,). Pobe h(496,) 586 ollson ollson m m 3/4/28 L8.4 3/4/28 L8.42 7

8 3/4/28 Example of open addessng Example of open addessng Inset key k 496:. Pobe h(496,). Pobe h(496,) 2. Pobe h(496,2) nseton m Seah fo key k 496:. Pobe h(496,). Pobe h(496,) 2. Pobe h(496,2) Seah uses the same pobe sequene, temnatng suessfully f t fnds the key m and unsuessfully f t enountes an empty slot. 3/4/28 L8.43 3/4/28 L8.44 Pobng stateges Pobng stateges Lnea pobng: Gven an odnay hash funton h (k), lnea pobng uses the hash funton h(k,) (h (k) + ) mod m. hs method, though smple, suffes fom pmay lusteng, whee long uns of ouped slots buld up, neasng the aveage seah tme. Moeove, the long uns of ouped slots tend to get longe. Double hashng Gven two odnay hash funtons h (k) and h 2 (k), double hashng uses the hash funton h(k,) (h (k) + h 2 (k)) mod m. hs method geneally podues exellent esults, but h 2 (k) must be elatvely pme to m (othewse yle of <m elements, not all slots ae pobed). One way s to make m a powe of 2 and desgn h 2 (k) to podue only odd numbes. 3/4/28 L8.45 3/4/28 L8.46 Analyss of open addessng We make the assumpton of unfom hashng: Eah key s equally lkely to have any one of the m! pemutatons as ts pobe sequene. heoem. Gven an open-addessed hash table wth load fato α n/m <, the expeted numbe of pobes n an unsuessful seah s at most /( α). Poof of the theoem Poof. At least one pobe s always neessay. Wth pobablty n/m, the fst pobe hts an ouped slot, and a seond pobe s neessay. Wth pobablty (n )/(m ), the seond pobe hts an ouped slot, and a thd pobe s neessay. Wth pobablty (n 2)/(m 2), the thd pobe hts an ouped slot, et. Obseve that n < n α fo, 2,, n. m m 3/4/28 L8.47 3/4/28 L8.48 8

9 3/4/28 Poof (ontnued) heefoe, the expeted numbe of pobes s + n + n + n 2 L + L m m m 2 m n + + α( + α( + α( L( + α ) L) )) + α + α 2 + α 3 + L α. α he textbook has a moe goous poof. Implatons of the theoem If α s onstant, then aessng an openaddessed hash table takes onstant tme. If the table s half full, then the expeted numbe of pobes s /(.5) 2. If the table s 9% full, then the expeted numbe of pobes s /(.9). 3/4/28 L8.49 3/4/28 L8.5 Q: What about DELEE(k) Hashng letue outlne A: Use two speal values, NIL and DEL. NIL nothng was eve hee. DEL somethng was hee but was deleted. Seah ontnues past DEL slots, but an stoe elements thee. Hene, subsequent INSER opeatons and emove DEL values. Rebuld the table when too many DEL values aumulate. Into and defnton Hashng n pate Unvesal hashng Pefet hashng Open Addessng (optonal) 3/4/28 L8.5 3/4/28 L8.52 Summay (ehash) Hashng: map lage U n a small spae {,..,m}. Content-based addessng, onstant-tme lookup! Hash funtons n pate: dv, mult, dot-pod. Unvesal hashng: and h H, pob ollson /m. Pefet hashng: n fxed keys known n advane, wost-ase O() lookup n Θ(n) spae. 2-level h. Open addessng: h(k,) pobe untl empty slot, lnea pobng, double hashng (optonal). 3/4/28 L8.53 9