Rcap: Maxmum 3-Sasfably Maxmum 3-Sasfably: Analyss CS 580: Algorhm Dsgn and Analyss Jrmah Block Purdu Unvrsy Sprng 2018 Announcmns: Homwork 6 dadln xndd o Aprl 24 h a 11:59 PM Cours Evaluaon Survy: Lv unl 4/29/2018 a 11:59PM. Your fdback s valud! xacly 3 dsnc lrals pr claus MAX-3SAT. Gvn 3-SAT formula, fnd a ruh assgnmn ha sasfs as many clauss as possbl. C 1 x 2 x 3 x 4 C 2 x 2 x 3 x 4 C 3 x 1 x 2 x 4 C 4 x 1 x 2 x 3 C 5 x 1 x 2 x 4 Smpl da. Flp a con, and s ach varabl ru wh probably ½, ndpndnly for ach varabl. Obsrvaon. Random assgnmn sasfs of h k clauss n xpcaon (proof: lnary of xpcaon) Q. Can w urn hs da no a 7/8-approxmaon algorhm? In gnral, a random varabl can almos always b blow s man. Lmma. Th probably ha a random assgnmn sasfs 7k/8 clauss s a las 1/(8k). Pf. L p j b probably ha xacly j clauss ar sasfd; l p b probably ha 7k/8 clauss ar sasfd. 7 8 7 8 1 8 7 8 1 1 8 3 5 Rarrangng rms ylds p 1 / (8k). 13.4 MAX 3-SAT Th Probablsc Mhod Corollary. For any nsanc of 3-SAT, hr xss a ruh assgnmn ha sasfs a las a 7/8 fracon of all clauss. Pf. Random varabl s a las s xpcaon som of h m. Maxmum 3-Sasfably: Analyss Johnson's algorhm. Rpadly gnra random ruh assgnmns unl on of hm sasfs 7k/8 clauss. Thorm. Johnson's algorhm s a 7/8-approxmaon algorhm. Pf. By prvous lmma, ach raon succds wh probably a las 1/(8k). By h wang-m bound, h xpcd numbr of rals o fnd h sasfyng assgnmn s a mos 8k. Probablsc mhod. W showd h xsnc of a nonobvous propry of 3-SAT by showng ha a random consrucon producs wh posv probably! 4 6 Copyrgh 2000, Kvn Wayn 1
Exnsons. Maxmum Sasfably Allow on, wo, or mor lrals pr claus. Fnd max wghd s of sasfd clauss. Thorm. [Asano-Wllamson 2000] Thr xss a 0.784-approxmaon algorhm for MAX-SAT. Thorm. [Karloff-Zwck 1997, Zwck+compur 2002] Thr xss a 7/8-approxmaon algorhm for vrson of MAX-3SAT whr ach claus has a mos 3 lrals. Thorm. [Håsad 1997] Unlss P = NP, no -approxmaon algorhm for MAX-3SAT (and hnc MAX-SAT) for any > 7/8. RP and ZPP RP. [Mon Carlo] Dcson problms solvabl wh on-sdd rror n poly-m. Can dcras probably of fals ngav On-sdd rror. o 2-100 by 100 ndpndn rpons If h corrc answr s no, always rurn no. If h corrc answr s ys, rurn ys wh probably ½. ZPP. [Las Vgas] Dcson problms solvabl n xpcd polym. runnng m can b unboundd, bu on avrag s fas Thorm. P ZPP RP NP. Qucksor Sorng. Gvn a s of n dsnc lmns S, rarrang hm n ascndng ordr. RandomzdQucksor(S) { f S = 0 rurn choos a splr a S unformly a random forach (a S) { f (a < a ) pu a n S - ls f (a > a ) pu a n S + } RandomzdQucksor(S - ) oupu a RandomzdQucksor(S + ) } vry unlkly o mprov ovr smpl randomzd algorhm for MAX-3SAT Fundamnal opn qusons. To wha xn dos randomzaon hlp? Dos P = ZPP? Dos ZPP = RP? Dos RP = NP? Rmark. Can mplmn n-plac. O(log n) xra spac 7 9 11 Mon Carlo vs. Las Vgas Algorhms Mon Carlo algorhm. Guarand o run n poly-m, lkly o fnd corrc answr. Ex: Conracon algorhm for global mn cu. Las Vgas algorhm. Guarand o fnd corrc answr, lkly o run n poly-m. Ex: Randomzd qucksor, Johnson's MAX-3SAT algorhm. sop algorhm afr a cran pon Rmark. Can always convr a Las Vgas algorhm no Mon Carlo, bu no known mhod o convr h ohr way. 13.5 Randomzd Dvd-and-Conqur Runnng m. Qucksor [Bs cas.] Slc h mdan lmn as h splr: qucksor maks (n log n) comparsons. [Wors cas.] Slc h smalls lmn as h splr: qucksor maks (n 2 ) comparsons. Randomz. Proc agans wors cas by choosng splr a random. Inuon. If w always slc an lmn ha s bggr han 25% of h lmns and smallr han 25% of h lmns, hn qucksor maks (n log n) comparsons. Noaon. Labl lmns so ha x 1 < x 2 < < x n. 8 12 Copyrgh 2000, Kvn Wayn 2
Qucksor: BST Rprsnaon of Splrs Qucksor: Expcd Numbr of Comparsons Chrnoff Bounds (abov man) BST rprsnaon. Draw rcursv BST of splrs. x 7 x 6 x 12 x 3 x 11 x 8 x 7 x 1 x 15 x 13 x 17 x 10 x 16 x 14 x 9 x 4 x 5 x 10 S - x 5 S + x 3 x 9 x 2 x 4 x 7 x 12 x 15 x 17 x 1 x 6 x 8 x 14 x 11 frs splr, chosn unformly a random x 13 x 16 Thorm. Expcd # of comparsons s O(n log n). Pf. 2 j 1 2 n 1 n 1 n 1 2n 2 n 1 j n 1 j2 j j 1 j x dx 2 n ln n x 1 probably ha and j ar compard Thorm. [Knuh 1973] Sddv of numbr of comparsons s ~ 0.65N. Ex. If n = 1 mllon, h probably ha randomzd qucksor aks lss han 4n ln n comparsons s a las 99.94%. Chbyshv's nqualy. Pr[ X - k] 1 / k 2. Thorm. Suppos X 1,, X n ar ndpndn 0-1 random varabls. L X = X 1 + + X n. Thn for any E[X] and for any > 0, w hav Pf. W apply a numbr of smpl ransformaons. For any > 0, Now Pr[ X (1 ) ] 1 (1 ) sum of ndpndn 0-1 random varabls s ghly cnrd on h man Pr[X (1)] Pr X (1) (1) E[ X ] f(x) = X s monoon n x E[ X ] E[ X Markov's nqualy: Pr[X > a] E[X] / a ] E[ X ] dfnon of X ndpndnc 13 15 17 Qucksor: BST Rprsnaon of Splrs Obsrvaon. Elmn only compard wh s ancsors and dscndans. x 2 and x 7 ar compard f hr lca = x 2 or x 7. x 2 and x 7 ar no compard f hr lca = x 3 or x 4 or x 5 or x 6. Clam. Pr[x and x j ar compard]. x 10 13.9 Chrnoff Bounds Chrnoff Bounds (abov man) Pf. (con) L p = Pr[X = 1]. Thn, E[ X ] for any 0, 1+ Combnng vryhng: Pr[ X (1 ) ] p (1 p ) (1 ) 0 1 p ( 1) E[ X ] (1 ) p ( 1) p ( 1) x 5 x 3 x 9 x 11 x 13 x 16 p = E[X] prvous sld nqualy abov ( 1 ) ( 1) Fnally, choos = ln(1 + ). x 2 x 4 x 7 x 12 x 15 x 17 14 x 1 x 6 x 8 x 14 18 Copyrgh 2000, Kvn Wayn 3
Chrnoff Bounds (blow man) Load Balancng Load Balancng: Many Jobs Thorm. Suppos X 1,, X n ar ndpndn 0-1 random varabls. L X = X 1 + + X n. Thn for any E[X] and for any 0 < < 1, w hav Pf da. Smlar. Pr[ X (1 ) ] 2 / 2 Rmark. No qu symmrc snc only maks sns o consdr < 1. Load balancng. Sysm n whch m jobs arrv n a sram and nd o b procssd mmdaly on n dncal procssors. Fnd an assgnmn ha balancs h workload across procssors. Cnralzd conrollr. Assgn jobs n round-robn mannr. Each procssor rcvs a mos m/n jobs. Dcnralzd conrollr. Assgn jobs o procssors unformly a random. How lkly s ha som procssor s assgnd "oo many" jobs? Thorm. Suppos h numbr of jobs m = 16n ln n. Thn on avrag, ach of h n procssors handls = 16 ln n jobs. Wh hgh probably vry procssor wll hav bwn half and wc h avrag load. Pf. L X, Y j b as bfor. Applyng Chrnoff bounds wh = 1 ylds 16nln n ln n 1 1 Pr[ X 2] 2 4 n Pr[ X 1 ] 1 1 2 22 2 (16nlnn) 1 n 2 Unon bound vry procssor has load bwn half and wc h avrag wh probably 1-2/n. 19 21 23 Load Balancng 13.10 Load Balancng Analyss. L X = numbr of jobs assgnd o procssor. L Y j = 1 f job j assgnd o procssor, and 0 ohrws. W hav E[Y j ] = 1/n Thus, X = j Y j,and = E[X ] = 1. c 1 Applyng Chrnoff bounds wh = c - 1 ylds Pr[ X c] c c L (n) b numbr x such ha x x = n, and choos c = (n). c 1 c ( n) 2 ( n) 1 1 1 Pr[ X c] c 2 c c ( n) ( n) n Unon bound wh probably 1-1/n no procssor rcvs mor han (n) = (logn / log log n) jobs. Fac: hs bound s asympocally gh: wh hgh probably, som procssor rcvs (logn / log log n) 13.6 Unvrsal Hashng 22 Copyrgh 2000, Kvn Wayn 4
Dconary Daa Typ Ad Hoc Hash Funcon Hashng Prformanc Dconary. Gvn a unvrs U of possbl lmns, manan a subs S U so ha nsrng, dlng, and sarchng n S s ffcn. Dconary nrfac. Cra(): Inalz a dconary wh S =. Insr(u): Add lmn u U o S. Dl(u): Dl u from S, f u s currnly n S. Lookup(u): Drmn whhr u s n S. Challng. Unvrs U can b xrmly larg so dfnng an array of sz U s nfasbl. Applcaons. Fl sysms, daabass, Googl, complrs, chcksums P2P nworks, assocav arrays, crypography, wb cachng, c. Ad hoc hash funcon. n h(srng s, n n) { n hash = 0; for (n = 0; < s.lngh(); ++) hash = (31 * hash) + s[]; rurn hash % n; } hash funcon ala Java srng lbrary Drmnsc hashng. If U n 2, hn for any fxd hash funcon h, hr s a subs S U of n lmns ha all hash o sam slo. Thus, (n) m pr sarch n wors-cas. Q. Bu sn' ad hoc hash funcon good nough n pracc? Idalsc hash funcon. Maps m lmns unformly a random o n hash slos. Runnng m dpnds on lngh of chans. Avrag lngh of chan = = m / n. Choos n m on avrag O(1) pr nsr, lookup, or dl. Challng. Achv dalzd randomzd guarans, bu wh a hash funcon whr you can asly fnd ms whr you pu hm. Approach. Us randomzaon n h choc of h. advrsary knows h randomzd algorhm you'r usng, bu dosn' know random chocs ha h algorhm maks 25 27 29 Hashng Algorhmc Complxy Aacks Unvrsal Hashng Hash funcon. h : U { 0, 1,, n-1 }. Hashng. Cra an array H of sz n. Whn procssng lmn u, accss array lmn H[h(u)]. Collson. Whn h(u) = h(v) bu u v. A collson s xpcd afr (n) random nsrons. Ths phnomnon s known as h "brhday paradox." Spara channg: H[] sors lnkd ls of lmns u wh h(u) =. H[1] H[2] H[3] H[n] jocularly null suburban browsng srously unravlld consdrang Whn can' w lv wh ad hoc hash funcon? Obvous suaons: arcraf conrol, nuclar racors. Surprsng suaons: dnal-of-srvc aacks. malcous advrsary larns your ad hoc hash funcon (.g., by radng Java API) and causs a bg pl-up n a sngl slo ha grnds prformanc o a hal Ral world xplos. [Crosby-Wallach 2003] Bro srvr: snd carfully chosn packs o DOS h srvr, usng lss bandwdh han a dal-up modm Prl 5.8.0: nsr carfully chosn srngs no assocav array. Lnux 2.4.20 krnl: sav fls wh carfully chosn nams. Unvrsal class of hash funcons. [Carr-Wgman 1980s] For any par of lmns u, v U, Pr h H h(u) h(v) 1/n Can slc random h ffcnly. chosn unformly a random Can compu h(u) ffcnly. Ex. U = { a, b, c, d,, f }, n = 2. a b c d f H = {h 1, h 2} Pr h H [h(a) = h(b)] = 1/2 h 1(x) 0 1 0 1 0 1 Pr h H [h(a) = h(c)] = 1 no unvrsal h 2(x) 0 0 0 1 1 1 Pr h H [h(a) = h(d)] = 0... a b c d f H = {h 1, h 2, h 3, h 4} Pr h H [h(a) = h(b)] = 1/2 h 1(x) 0 1 0 1 0 1 Pr h H [h(a) = h(c)] = 1/2 h 2(x) 0 0 0 1 1 1 Pr h H [h(a) = h(d)] = 1/2 unvrsal h 3(x) 0 0 1 0 1 1 Pr h H [h(a) = h()] = 1/2 Pr h H [h(a) = h(f)] = 0 h 4(x) 1 0 0 1 1 0... 26 28 30 Copyrgh 2000, Kvn Wayn 5
Unvrsal Hashng Dsgnng a Unvrsal Class of Hash Funcons Unvrsal hashng propry. L H b a unvrsal class of hash funcons; l h H b chosn unformly a random from H; and l u U. For any subs S U of sz a mos n, h xpcd numbr of ms n S ha colld wh u s a mos 1. Pf. For any lmn s S, dfn ndcaor random varabl X s = 1 f h(s) = h(u) and 0 ohrws. L X b a random varabl counng h oal numbr of collsons wh u. E hh [X] E[ ss X s ] ss E[X s ] ss Pr[X s 1] 1 ss n S 1 1 n lnary of xpcaon X s s a 0-1 random varabl unvrsal (assums u S) Thorm. H = { h a : a A } s a unvrsal class of hash funcons. Pf. L x = (x 1, x 2,, x r ) and y = (y 1, y 2,, y r ) b wo dsnc lmns of U. W nd o show ha Pr[h a (x) = h a (y)] 1/n. Snc x y, hr xss an ngr j such ha x j y j. W hav h a (x) = h a (y) ff a j ( y j x j ) a (x y ) mod p j z m Can assum a was chosn unformly a random by frs slcng all coordnas a whr j, hn slcng a j a random. Thus, w can assum a s fxd for all coordnas j. Snc p s prm, a j z = m mod p has a mos on soluon among p possbls. s lmma on nx sld Thus Pr[h a (x) = h a (y)] = 1/p 1/n. Exra Slds 31 33 Dsgnng a Unvrsal Famly of Hash Funcons Numbr Thory Facs Thorm. [Chbyshv 1850] Thr xss a prm bwn n and 2n. Modulus. Choos a prm numbr p n. Ingr ncodng. Idnfy ach lmn u U wh a bas-p ngr of r dgs: x = (x 1, x 2,, x r ). Hash funcon. L A = s of all r-dg, bas-p ngrs. For ach a = (a 1, a 2,, a r ) whr 0 a < p, dfn r h a (x) a x mod p 1 Hash funcon famly. H = { h a : a A }. no nd for randomnss hr Fac. L p b prm, and l z 0 mod p. Thn z = m mod p has a mos on soluon 0 < p. Pf. Suppos and ar wo dffrn soluons. Thn ( - )z = 0 mod p; hnc ( - )z s dvsbl by p. Snc z 0 mod p, w know ha z s no dvsbl by p; follows ha ( - ) s dvsbl by p. Ths mpls =. Bonus fac. Can rplac "a mos on" wh "xacly on" n abov fac. Pf da. Eucld's algorhm. 32 34 Copyrgh 2000, Kvn Wayn 6