Chapter 1 - Algorithms Computig CSE 5311 Notes 1: Mathematical Prelimiaries Last updated 1/20/18 12:56 PM) Relatioship betwee complexity classes, eg log,, log, 2, 2, etc Chapter 2 - Gettig Started Loop Ivariats ad Desig-by-Cotract especially Table 1 o p 13): http://dlacmorgezproxyutaedu/citatiocfm?doid=25787022506375 RAM Model Assumed Reality Memory Access Arithmetic Word size Chapter 3 - Asymptotic Notatio f) = Og)) g) bouds f) above by a costat factor) f) = Ωg)) g) bouds f) below by a costat factor) f) = θg)) f) = Og)) ad f) = Ωg)) Iterated Logarithms log k = log ) k log k) = loglog k-1) ) = log log log log 1) = log ) lg* = Cout the times that you ca puch a log key o your calculator before value is 1 Appedix A - Summatios Arises for algorithms that ru i practically or early costat time Review: Geometric Series p 1147) Harmoic Series p 1147) Approximatio by itegrals p 1154)
Chapter 4 - Recurreces 2 SUBSTITUTION METHOD - Review 1 Guess the boud 2 Prove usig strog) math iductio Exercise: ) = 4T 2 + Try Ο 3 ad cofirm by math iductio Assume T k) ck 3 for k < c 3 2 8 + ) = 4 2 4c 3 8 + = c 2 3 + = c 3 c 2 3 + c 2 3 + 0 if is sufficietly large c 3 Improve boud to Ο 2 ad cofirm Assume T k) ck 2 for k < c 2 2 4 ) = 4T 2 + 4c 2 4 + = c2 + STUCK!
Ω 3 as lower boud: 3 Assume T k) ck 3 for k < c 3 2 8 + ) = 4 2 4c 3 8 + = c 2 3 + Ω 2 as lower boud: = c 3 c 2 3 + c 2 3 + 0? c 3 DID NOT PROVE!!! Assume T k) ck 2 for k < c 2 2 4 ) = 4T 2 What s goig o T 1 + 4c 2 4 + = c2 + c 2 for 0 < c ) = d ) = 4T 1) + 2 = 4d + 2 ) = 4T 2) + 4 = 4 4d + 2) + 4 =16d + 8 + 4 =16d +12 ) = 4T 4) + 8 = 4 16d +12) + 8 = 64d + 48 + 8 = 64d + 56 ) = 4T 8) +16 = 4 64d + 56) +16 = 256d + 224 +16 = 256d + 240 T 2 T 4 T 8 T 16 Hypothesis : T ) = d 2 + 2 = d +1) 2 = c 2
Ο 2 : 4 Assume T k) ck 2 k for k < ) c 2 2 4 2 T ) = 4T 2 ) + 4 c 2 4 2 ) + = c2 2 + = c 2 Ω 2 : [This was already prove] Exercise: Assume T k) ck 2 k for k < ) c 2 2 4 2 T ) = 4T 2 ) + 4 c 2 4 2 ) + = c2 2 + = c 2 T ) = T ) +1 Ο log): Ο loglog ) Assume T k) c lgk for k < T ) c lg = c lg 2 T ) = ) +1 c lg c lg if c 2 2 +1= c lg c lg 2 +1 Assume T k) c lglg k for k < Note : log a log a Θ log b log b )) T ) c lglg = c lg lg 2 = c lglg c ) +1 c lglg c +1 c lglg if c 1 T ) =
Ω loglog ) 5 Assume T k) c lglg k for k < T ) c lglg = c lg lg 2 = c lglg c ) +1 c lglg c +1 c lglg if 0 < c 1 T ) = Substitutio Method is a geeral techique - See CSE 2320 Notes 08 for quicksort aalysis RECURSION-TREE METHOD - Review ad Prelude to Master Method Covert to summatio ad the evaluate Example: Mergesort, p 38 T ) = 2T 2) + case 2 for master method) T=2k)= T/2=2k-1)= T/2=2k-1)= T/4=2k-2)= T/4=2k-2)= T/4=2k-2)= T/4=2k-2)= T/8=2k-3)= T/16=2k-4)= T/2k=2k-k=1)=
Exercise: 6 ) = 4T 2 + T) T/2) /2 4/2=2 T/4) /4 16/4=4 lg + 1 levels T/8) /8 64/8=8 T/16) /16 T2)=T/2lg -1) 2 256/16=16 4lg -1 /2lg -1 =2lg -1 T1) 1 4lg =lg 4 =2 Usig defiite geometric sum formula: lg 1 c 2 k + c 2 = c 2lg 1 k=0 2 1 +c2 t Usig x k k=0 Case 1 of master method) = c 1) +c 2 = c 2 c +c 2 = Θ 2 ) + * = xt+1 1 x 1 x 1
Exercise: case 3 of master method) 7 ) = 2T 2 + 3 T) 3 3 T/2) 3/8 23/8=3/4 T/4) 3/64 43/64=3/16 lg + 1 levels T/8) 3/512 83/512=3/64 T/16) 3/4096 163/4096=3/256 T2)=T/2lg -1) 8 2lg -1/2lg -1)3=3/4lg -1 T1) 1 2lg = Usig idefiite geometric sum formula: c 3 lg 1 1 k=0 4 k + c c 3 1 k=0 4 k + c = c 3 1 1 1 + c From x k = 1 k=0 1 x 4 0 < x <1 = 4 3 c3 + c = O 3 ) Usig defiite geometric sum formula: ) lg 1 c 3 lg 1 1 k=0 4 k + c = c 3 1 t 4 1 4 1 + c Usig x k k=0 = c 3 2 1 3 4 = Θ 3 * ) + c = c 3 1 2 3 4 = xt+1 1 x 1 + c = 4 3 c3 1 1 2 * + c ) x 1
MASTER METHOD/THEOREM ew ) - CLRS 45 8 T ) = at b ) + f ) a 1 b > 1 Three mutually exclusive cases proof sketched i 461): 1 f ) = O log b a ε, ε > 0 ) = Θ log b a leaves domiate) 2 f ) = Θ log b a ) = Θ log b a log each level cotributes equally) 3 ) = Ω log b a ε f ), ε > 0, ad af b c <1 for all sufficietly large ) cf ), ) ) = Θ f ) root domiates) Problem 46-3 shows that ε > 0 i 3 follows from the existece of c ad 0 By takig = bk 0 ad the coditio o f, a c ε logb 1 c ) ) ) log b log b 0 f 0 ) f ) may be established Last step is Example: ) =10T + 10 a = 10 b = 10 f) = 5 log 10 10 = Case 1: 5 = O 1-ε ), 0 < ε 05 T ) = Θ )
Example: recursio tree give earlier - leaves domiate) 9 ) = 4T 2 + a = 4 b = 2 f) = log 2 4 = 2 Case 1: = O 2-ε ), 0 < ε 1 Example: ) = 9T 3 ) = Θ 2 + 2 2 a = 9 b = 3 f) = 2 2 log 3 9 = 2 Case 2: 2 2 = Θ 2 ) Recursio Tree ) ) = Θ 2 log ) T) 22 22 T/3) 22/9 9 22 T/9) 22/81 9 22 T/27) 22/729 T1) c 9 22 c9log 3 =clog 3 9=c2 ) = 2 2 log 3 + c 2 = Θ 2 log
Example: 10 ) = 2T 2 + 3 a = 2 b = 2 f) = 3 lg 2 = Case 3: 3 = Ω 1+ε ), 0 < ε 2 Example: ) = 2 3 af b ) = T 2 8 ) = Θ 3 + 2 c 3 = cf ), 1 4 c <1 ) a = 1 b = 2 f) = /2 lg 1 = 0 = 1 Case 3: /2 = Ω 0 ε ), 0 < ε 1 ) =1 22 af b From CLRS, p 95: ) = 2T 2 T ) = Θ ) + lg = 4 c 2 = cf ), 1 2 c <1 a = 2 b = 2 f) = lg lg 2 = 1 = f ) = ω ), but f ) Ω ε af b ) = 2 f 2 ) = 2 2 lg 2) * for ay ε > 0 ) = lg = clg + 1 c) lg cf ) = clg for c 1 So master method case 3) does ot support T ) = Θ lg )
11 Usig exercise 46-2 as a hit or a simple recursio tree with = 2 k ) Substitutio method to show Ο : Assume T k) ck lg 2 k for k < ) c 2 lg2 2 ) = c 2 ) + lg 2c 2 2 T ) = 2 2 Ω : clg 2 for c 1 2 + ε ) = Θ lg 2 lg 1 )2 Assume T k) ck lg 2 k for k < ) c 2 lg2 2 ) = c 2 ) + lg 2c 2 2 T ) = 2 2 clg 2 for 0 < c 1 2 lg 1 )2 + lg = c lg 2 2clg + c + lg lg 1 )2 lg 1 )2 + lg = c lg 2 2clg + c + lg PROBABILISTIC ANALYSIS Hirig Problem - Iterview potetial assistats, always hirig the best available Iput: Output: Permutatio of 1 The umber of sequece values that are larger tha all previous sequece elemets 3 1 2 4 1 2 3 4 4 3 2 1 Worst-case = Observatio: For ay k-prefix of sequece, the last elemet is the largest with probability Summig for all k-prefixes: l < 1 = k H l +1 k=1
Hirig m-of- Problem 12 Observatio: For ay k-prefix of sequece, the last elemet is oe of the m largest with probability if k m otherwise Sum for all k-prefixes expected umber that are hired) m + m k k=m+1 = m + mh mhm What data structure supports this applicatio? Coupo Collectig Kuth) types of coupo Oe coupo per cereal box How may boxes of cereal must be bought expected) to get at least oe of each coupo type? Collectig the coupos is decomposed ito steps: Step 0 = get first coupo Step 1 = get secod coupo Step m = get m+1st coupo Step - 1 = get last coupo Number of boxes for step m Let p i = probability of eedig exactly i boxes difficult) p i = m ) i 1 m, so the expected umber of boxes for coupo m + 1 is ipi i=1 Let q i = probability of eedig at least i boxes = probability that previous i - 1 boxes are failures much easier to use) So, pi = qi qi+1
ip i = i q i q i+1 ) i=1 i=1 = iq i iq i+1 i=1 i=1 = iq i i 1)q i i=1 i=2 = q 1 + iq i iq i + q i i=2 i=2 i=2 = q i i=1 13 q 1 =1 q2 = m q 3 = m ) 2 ) k 1 q k = m q i = m ) i = 1 1 i=1 i=0 m = m = Expected umber of boxes for coupo m + 1 Summig over all steps gives 1 i=0 i Aalyze the followig game: = 1+ 1 2 + 1 3 + + 1 ) l +1 ) sides o each die, umbered 1 through Roll dice oe at a time Keep a die oly if its umber > umbers o dice you already have a What is the expected umber of rolls icludig those ot kept) to get a die umbered? i=0 ) 1 i = 1 1 1 = 1) =
b What umber of dice do you expect mathematically) to keep? 14 Keep goig util a is ecoutered Repeats of a umber are discarded Results i hirig problem H l +1) Suppose a gamblig game ivolves a sequece of rolls from a stadard six-sided die A player wis 1 whe the value rolled is the same as the previous roll If a sequece has 1201 rolls, what is the expected amout paid out? Probability of a roll payig 1 = 1/6 1200/6 = 200 Suppose a gamblig game ivolves a sequece of rolls from a stadard six-sided die A player wis 1 whe the value rolled is larger tha the previous roll If a sequece has 1201 rolls, what is the expected amout paid out? Payout for ext roll = 1 6 Expected payout after roll of i 6 i=1 1200*5/12 = 500 ) = 1 6 5 6 + 4 6 + 3 6 + 2 6 + 1 6 + 0 = 15 36 = 5 12 Suppose a gamblig game ivolves a sequece of rolls from a stadard six-sided die A player wis k dollars whe the value k rolled is smaller tha the previous roll If a sequece has 601 rolls, what is the expected amout paid out? 1 6 1 5 6 + 2 4 6 + 3 6 3 + 4 2 6 + 5 1 6 + 6 0 6) = 5+8+9+8+5 = 36 35 36 600*35/36 = 58333 Suppose a gamblig game ivolves a sequece of rolls from a stadard six-sided die What is the expected umber of rolls util the first pair of cosecutive sixes appears? Expected rolls to get first six = 6 = i=0 ) 6 1 6 i = 1 1 6 1 6 = 6 6 6 1 ) * * ) Probability that ext roll is ot a six = 5 6
Expected rolls for cosecutive sixes = 6 +1 k=0 ) ) 5 6 k = 6 +1) 1 1 5 6 = 42 15 Also, see: http://dlacmorgezproxyutaedu/citatiocfm?doid=24087762408800 ad http://dlacmorgezproxyutaedu/citatiocfm?doid=24285562428578 alog with the book by Gristead Sell https://mathdartmouthedu/~prob/prob/probpdf ), especially the first four chapters ad Example 46 o p 136, http://marilyvossavatcom/game-show-problem/ ) GENERATING RANDOM PERMUTATIONS PERMUTE-BY-SORTING p 125, skim) Geerates radoms i 1 3 ad the sorts to get permutatio i Θ log ) time Ca use radix/coutig sort CSE 2320 Notes 8) to perform i Θ) time RANDOMIZE-IN-PLACE Array A must iitially cotai a permutatio Could simply be idetity permutatio: A[i] = i for i=1 to swap A[i] ad A[RANDOMi,)] Code is equivalet to reachig i a bag ad choosig a umber to lock ito each slot Uiform - all! permutatios are equally likely to occur Problem 53-3 PERMUTE-WITH-ALL for i=1 to swap A[i] ad A[RANDOM1,)] Produces outcomes, but! does ot divide ito evely Not uiform - some permutatios are produced more ofte tha others Assume =3 ad A iitially cotais idetity permutatio RANDOM choices that give each permutatio 1 2 3: 1 2 3 1 3 2 2 1 3 3 2 1 1 3 2: 1 2 2 1 3 3 2 1 2 2 3 1 3 1 1 2 1 3: 1 1 3 2 2 3 2 3 2 3 1 2 3 3 1 2 3 1: 1 1 2 1 3 1 2 2 2 2 3 3 3 1 3 3 1 2: 1 1 1 2 2 1 3 2 2 3 3 3 3 2 1: 1 2 1 2 1 1 3 2 3 3 3 2
RANDOMIZED ALGORITHMS 16 Las Vegas Output is always correct Time varies depedig o radom choices or radomess i iput) Challege: Determie expected time Classic Examples: Radomized pivot) quicksort takes expected time i Θ log) Uiversal hashig This Semester: Treaps Perfect Hashig Smallest Eclosig Disk Rabi-Karp Text Search Asides: Mote Carlo Etheret: http://ewikipediaorg/wiki/expoetial_backoff List Rakig - Shared Memory ad Distributed Versios http://ragerutaedu/~weems/notes4351/09otespdf) Correct solutio with some large) probability Use repetitio to improve odds Usually has oe-sided error - oe of the outcomes ca be wrog Example - testig primality without factorig To check N for beig prime: Radomly geerate some a with 1 < a < N ) If N moda = 0, the report composite else report prime Oe-sided error - prime could be wrog Several observatios from umber theory are eeded to make this robust for avoidig Carmichael umbers) Cocept: Repeated applicatio of Mote Carlo techique ca lead to: Improved reliability Las Vegas algorithm that gives correct result with high probability