8.1 Hashing Algorithms

Transcription

1 CS787: Advaced Algorthms Scrbe: Mayak Maheshwar, Chrs Hrchs Lecturer: Shuch Chawla Topc: Hashg ad NP-Completeess Date: September Prevously we looked at applcatos of radomzed algorthms, ad bega to cover radomzed hashes, dervg several results whch wll be used today s lecture otes. Today we wll fsh coverg Radomzed hashg ad move o to the defto of the classes P ad NP, ad NP- Completeess. 8.1 Hashg Algorthms Sometmes t s desreable to store a set of tems S take from a uverse U usg a amout of storage space ad havg average lookup tme O(1). Oe system whch has these propertes s a hash table wth a completely radom hash fucto. Last tme we saw how to mplemet ths usg a 2-uversal hash famly. Ufortuately these hash famles ca have a large lookup tme the worst case. Sometmes we wat the worst case lookup tme to also be O(1). We wll ow see how to acheve ths perfect hashg Perfect hashg reducg worst case loopkup tme Gve a 2-uform hashg fucto h (formally, a fucto whch has a 1/ probablty of assgg 2 dfferet depedetly chose elemets of U to the same value, also called b ) ad that S = m, U = u ad S U we ca costruct a array [] of bts such that f a elemet S maps to h() the the array aswers that s S. Note that there s a certa probablty that ths wll be false because elemets ot S ca be mapped to the same locato by the hash fucto. If there are more tha oe elemet S mapped to the same bt the the sgle bt the array s replaced by a lked lst of elemets, whch ca be searched utl the desred elemet s foud. Uder ths arragemet, the worst case lookup tme s determed by the logest expected lked lst [], whch s equvalet to the expected umber of assgmets of elemets S to a sgle b,.e. collsos b. The expectato of the umber of collsos at b s gve by ( ) m E[X ] = 1/ 2 (8.1.1) where X s a radom varable represetg the umber of elemets S mapped to b. I order for the expected lookup tme to be O(1) the quatty o the rght had sde of (8.1.1) must be O(1) whch meas that must be O(m 2 ). I order to esure that E[X ] = O(1) we ca repeatedly choose ew hash fuctos h from a 2- Uversal famly of fuctos H : U [ 2 ] utl oe s foud whch produces o more tha O(1) collsos ay b. 1

2 8.1.2 Perfect hashg wth lear space A mprovemet o ths method of hashg elemets of S whch reduces ts space complexty s to choose m ad stead of usg a lked lst of elemets whch map to the same b, we ca use aother hash of sze to store the elemets the lst. Let b be the umber of elemets b. We ote that the expected maxmum umber of elemets a sgle b, s roughly because we kow from theorem that Pr[max b k] 1/k 2 ad thus Pr[ay b ] 1/, so we accpet wth hgh cofdece that o b has more tha tems t. The sze of ths hash s roughly the square of the umber of elemets to be stored t, makg ts expected lookup tme O(1) as dscussed above. If b s the umber of elemets placed b,.e. the umber elemets the sub-hash b, (f b 1), the the total amout of space used by ths method, T sp s gve by: T sp = m + E [ ] b 2 (8.1.2) The frst term s for the array tself, ad the secod s for the sub-hashes, each of whch stores b tems, ad requres b 2 space. We ca ote that E [ b 2 ] = 2 ( ) b + 2 b (8.1.3) because ( ) b = b (b 1) 2 2 The b term s equal to m, the umber of tems stored the ma hash array. We ca approxmate the summato term my sayg that T sp the becomes ( ) b 1 ( ) m 2 m 2 T sp 2m + 2 m(m 1) 3m (8.1.4) m 2 2

3 8.2 Bloom Flters Before we go o to the dea of bloom flters, let us defe the cocept of false postve. Defto (False Postve) A gve elemet e U ad S = m ad the elemets of S are mapped to a array of sze. If the elemet e / S but the hashg algorthm returs a 1 or yes as a aswer.e. the elemet e s mapped to the array, t s called a false postve. Now the probablty of a false postve usg perfect hashg s: Pr[a false postve] = 1 Pr[0 that posto] ( m) = 1 (1 1 )m 1 (1 m ) = m Ths probablty of gettg a false postve s cosderably hgh. So how ca we decrease ths? By usg 2-hash fuctos, we ca reduce ths probablty to ( m )2. So, geeral, by usg k-hash fuctos, we ca decrease the probablty of gettg a false postve to a sgfcatly low value. So, a fal-safe mechasm to get the hashg to retur a correct value by troducg redudacy the form of k-hash fuctos s the basc dea of a bloom flter. Problem:Gve a e U, we have to map S to [] usg k hash fuctos. So to solve ths problem, we Fd h 1 (e),h 2 (e),...,h k (e). If ay of them s 0, output e / S, else output e S. Now the Pr[a false postve] = 1 Pr[ay oe posto s 0]. So let s say the Pr[some gve posto s 0] = (1 1 )mk exp( mk ) = (say)p. (8.2.5) So Pr[ay gve posto s 1]= 1 p. Ad Pr[all postos that e maps to are 1 gve e / S]=(1 p) k. Ths s correct f all the hash fuctos are depedet ad k s small. Problem:The probablty of gettg a false postve usg k-hash fuctos s (1 p) k. So how ca we mmze t? Let us say the fucto to be mmzed s f(k) = (1 exp( km ))k. 3

4 By frst takg the log of both sdes ad the fdg the frst dervatve, we get d dk log(f(k)) = d k log(1 exp( km dk )) = log(1 exp( km )) + = log(1 p) p 1 p log(p) k 1 exp( km ) exp( km )m By substtutg p = exp( km ) ad km = log(p) To mmze ths fucto, we eed to solve the equato wth ts frst dervatve beg equal to 0. So to solve ths equato: log(1 p) p 1 p log p = 0 (1 p)log(1 p) = p log p Solvg ths equato gves p = 1 2. So we get exp( km ) = 1 2 k = m l 2 So wth that value of k, we ca deduce the value of f(k) as: f(k) = Pr[postve] = (1 p) k = 1 2 = c m. m l 2 If we choose = m log m where c < 1 s a costat, ths gves f = 1 m. So ths way, we ca reduce the probablty of gettg a false postve by usg k-hash fuctos. 8.3 NP-Completeess P ad NP Whe aalyzg the complexty of algorthms, t s ofte useful to recast the problem to a decso problem. By dog so, the problem ca be thought of as a problem of verfyg the membershp of a gve strg a laguage, rather tha the problem of geeratg strgs a laguage. The 4

5 complexty classes P ad NP dffer o whether a wtess s gve alog wth the strg to be verfed. P s the class of algorthms whch termate a amout of tme whch s O() where s the sze of the put to the algorthm, whle NP s the class of algorthms whch wll termate a amout of tme whch s O() f gve a wtess w whch correspods to the soluto beg verfed. More formally, L NP ff P-tme verfer V P s.t. x L, w, w = poly( x ), V(x, w) accepts x L, w, w = poly( x ), V(x, w) rejects The class Co-NP s defed smlarly: L Co-NP ff P-tme verfer V P s.t. x L, w, w = poly( x ), V(x, w) accepts x L, w, w = poly( x ), V(x, w) rejects A example of a problem whch s the class NP s Vertex Cover. The problem states, gve a graph G = (V,E) fd a set S V, the e E, e s cdet o a vertex S, ad S k for some umber k. There exsts a verfer V by costructo; V takes as put a graph G, ad as a wtess a set S V ad verfes that all edges e E are cdet o at least oe vertex S ad that S k. If G has o vertex cover of sze less tha or equal to k the there s o wtess w whch ca be gve to the verfer whch wll make t accept P-tme reducblty There exst some problems whch ca be used to solve other problems, as log as a way of solvg them exsts, ad a way of covertg staces of other problems to staces of the problem wth a kow soluto also exsts. Whe talkg about decso problems, a problem A s sad to reduce to problem B f there exsts a algorthm whch takes as put a stace of problem A, ad outputs a stace of problem B whch s guarateed to have the same result as the stace problem A,.e. f L A s the laguage of problem A, ad L B s the laguage of problem B, ad f there s a algorthm whch traslates all l L A to l B L B ad whch traslates all l L A to l B L B the problem A reduces to problem B. The practcal mplcato of ths s that f a effcet algorthm exsts for problem B, the problem A ca be solved by covertg staces of problem A to staces of problem B, ad applyg the effcet solver to them. However, the process of traslatg staces betwee problems must also be effcet, or the beeft of dog so s lost. We therefore defe P-tme reducblty as the process of traslatg staces of problem A to staces of problem B tme bouded by a polyomal the sze of the stace to be traslated, wrtte A P B. Ths reducto s also called Cook reducto. 5

6 8.3.3 NP-Completeess If t s possble to traslate staces of oe problem to staces of aother problem, the f there exsts a problem L such that L NP, L P L the a algorthm whch decdes L decdes every problem NP. Such problems are called NP- Hard. If a problem s NP-Hard ad NP, the t s called NP-Complete. If there exsts a P-tme algorthm whch decdes a NP-Complete problem, the all NP problems ca be solved P-tme, whch would mea that N P P. We already kow that P N P because every P-tme algortm ca be thought of as a NP algorthm whch takes a 0-legth wtess. Therefore, P = NP ff there exsts a P-tme algorthm whch decdes ay NP-Complete problem. Ths result was proved depedetly by Cook ad Lev, ad s called the Cook-Lev theorem. The frst problem proved to be NP-Complete was Boolea SAT, whch asks, gve a Boolea expresso, s there a settg of varables whch allows the etre expresso to evaluate to True? 6