Model of Computation and Runtime Analysis

Transcription

1 Model of Computatio ad Rutime Aalysis

2 Model of Computatio

3 Model of Computatio Specifies Set of operatios Cost of operatios (ot ecessarily time) Examples Turig Machie Radom Access Machie (RAM) PRAM Map Reduce(?) 3 / 6

4 Radom Access Machie Word Group of costat umber of bits (e. g. byte) log(iput size) Usually itegers or floats Memory Big array of words Access by address Operatios Read ad write a word from or ito memory Arithmetic +,,, /, mod, Logic (ca be bitwise),, xor, Compariso based decisios Each operatio cost 1 uit of time. / 6

5 Asymptotic Complexity

6 Asymptotic Complexity Goal Determie rutime of a algorithm. Depeds o Iput Hardware Programmig laguage, compiler, ad rutime eviromet Solutio Asymptotic Complexity How does the rutime behave based o the iput size? 6 / 6

7 Asymptotic Complexity Hardware Raspberry Pi B 0.9 GHz Nexus 5.3 GHz Itel i7.0 GHz Same for other compoets (e.g. memory) Rutime Eviromet Machie code (e. g. C++) Maaged code (e. g. C#/Java) Iterpreted code (e. g. Pytho) Virtual Machies (e. g. VirtualBox) Coclusio Igore costat factors. 7 / 6

8 Asymptotic Complexity Cosider two algorithms T 1 () = T () = T 1 () 1 31,1 66,81 1,053,701,797,701 T () 56,096 65,536 1,08,576,777, T 1 /T Coclusio Oly keep strogest part. ( i this case) 8 / 6

9 Example Cosider two algorithms ad two computers Fast computer ad slow algorithm 10 7 operatios per secod T 1 () = Slow computer ad fast algorithm 10 operatios per secod T () = log Iput size: 10 6 Rutime T 1 = (106 ) 10 7 s = 10 5 s 7.8 h T = 106 log s =,000 s 33.3 mi Coclusio First lower complexity, the costat factors. 9 / 6

10 Big-O Notatio Based o complexity, 3 log, ad are the same as. How do we write this? Big-O Notatio O(g) = { f : N N c > 0 0 > 0 0 : f () c g()} f O(g) meas g is a upper boud for f. O(3 log ) = O( ) = O( ) If a algorithm has rutime , we say it rus i O( ) time. Note that O() O( log ) O( ) 10 / 6

11 Big-O Notatio f Ω(g): g is a lower boud for f. c > 0 0 > 0 0 : f () c g() f Ω(g) g O(f ) f Θ(g) c 1, c > 0 0 > 0 0 : c 1 g() f () c g() Θ(g) = O(g) Ω(g) f o(g): f is domiated by g. c > 0 0 > 0 0 : f () c g() (This icludes c 1.) 11 / 6

12 Commo Examples Complexity Example O(1) costat basic operatios O(log ) logarithmic biary search O() liear coutig, liear search, Euleria cycle O( log ) sortig, fidig doubles, covex hull O( ) quadratic checkig all pairs O( ) expoetial SAT O(!) checkig all permutatios 1 / 6

13 Questios True or False? Explai your aswer. a) f O(g) implies g O(f ) b) f + g Θ(mi(f, g)) c) f O(g) implies log f O(log g) d) f O(g) implies f O( g ) e) f O(f ) f) f O(g) implies g Ω(f ) g) f () Θ(f (/)) h) g o(f ) implies f + g Θ(f ) 13 / 6

14 Rutime Aalysis for Recurreces

15 Divide ad Coquer Idea Split problem ito smaller sub-problems. Solve sub-problems recursively. Combie solutios of sub-problems to solve origial problem. Examples Biary Search Merge sort, Quicksort Matrix multiplicatio Drawig biary trees 15 / 6

16 Rutime of Divide ad Coquer Geeral Formula T() = { O(1) if = 1 a T ( b ) + f () if > 1 For simplicity, we igore the case = 1. Biary Search T() = T ( ) + 1 Merge sort T() = T ( ) + / 6

17 Solvig Recurrece Substitutio Method Guess a (upper or lower) boud Prove it usig iductio Recursio Tree Covert recurrece to tree. Each ode represets a fuctio call. Add cost of each layer ad of all layers. Master Theorem Geeral solutio (for some cases) 17 / 6

18 Substitutio Method Example: T() = T ( ) + Hypothesis: T() O( log ), i. e. T() c log T() = T(/) + c (/) log(/) + = c log c log + = c log (c log 1) c log 18 / 6

19 Substitutio Method Example: T() = T ( ) + Hypothesis: T() O( ), i. e. T() c T() = T(/) + c ( /) + = c + Does ot work. Geeral advise for iductio: Make your hypothesis stroger. 19 / 6

20 Substitutio Method Example: T() = T ( ) + Hypothesis: T() c T() = T(/) + c ( /) (/) + = c + = c 19 / 6

21 Recursio Tree Example: T() = T ( ) + log.... log 0 / 6

22 Recursio Tree Example: T() = T ( ) + T ( ) + T ( 8 ) log ( 7 ) log ( 7 ) i i=0 8 8 i=0 xi = 1 1 x for 0 < x < 1 1 / 6

23 Recursio Tree Example: T() = 3T ( ) + 3 log log ( 3 ) i i=0 / 6

24 Example: T() = 3T(/) + We kow, Thus, m i=0 r i = rm+1 1 r 1 log i=0 ( ) i 3 = log = log = 3 ( log 1.5 ) log = 3 ( log ) log = Θ ( 1.58) 3 / 6

25 Master Theorem Cosider a recurrece i the form (with a 1, b > 1) ( ) T() = a T + f () b (1) f () O( log b a ε ) T() Θ( log b a ) () f () Θ( log b a ) T() Θ( log b a log ) (3) f () Ω( log b a+ε ) T() Θ(f ()) For (1) ad (3), ε > 0. For (3), 0 < c < 1 ad af ( ) cf (). b / 6

26 Master Theorem T() = 3T ( ) + (from recursio tree: Θ( 1.58 )) a = 3, b = log b a = log f () =, f () O( log b a ε ) (Case 1) T() Θ( log b a ) = Θ( 1.58 ) T() = T ( ) + (from recursio tree: Θ( log )) a =, b = log b a = log = f () =, f () Θ( log b a ) (Case ) T() Θ( log b a log ) = Θ( log ) 5 / 6

27 Master Theorem T() = T ( ) + a =, b = log b a = log = 1 f () =, f () Ω( log b a+ε ) (Case 3) T() Θ(f ()) = Θ( ) 6 / 6