Algorithm Analysis. Algorithms that are equally correct can vary in their utilization of computational resources

Transcription

1 Algorithm Aalysis Algorithms that are equally correct ca vary i their utilizatio of computatioal resources time ad memory a slow program it is likely ot to be used a program that demads too much memory may ot be executable o the machies that are available E.G.M. Petrakis Algorithm Aalysis 1

2 Memory - Time It is importat to be able to predict how a algorithm behaves uder a rage of possible coditios usually differet iputs The emphasis is o the time rather tha o the space efficiecy memory is cheap! E.G.M. Petrakis Algorithm Aalysis 2

3 Ruig Time The time to solve a problem icreases with the size of the iput measure the rate at which the time icreases e.g., liearly, quadratically, expoetially This measure focuses o itrisic characteristics of the algorithm rather tha icidetal factors e.g., computer speed, codig tricks, optimizatios, quality of compiler E.G.M. Petrakis Algorithm Aalysis 3

4 Additioal Cosideratio Sometimes, simple but less efficiet algorithms are preferred over more efficiet oes the more efficiet algorithms are ot always easy to implemet the program is to be chaged soo time may ot be critical the program will ru oly few times (e.g., oce a year) the size of the iput is always very small E.G.M. Petrakis Algorithm Aalysis 4

5 Bubble sort Cout the umber of steps executed by the algorithm step: basic operatio igore operatios that are executed a costat umber of times for ( i = 1; i < ( 1); i ++) for (j = ; j < (i + 1); j --) if ( A[ j 1 ] > A[ j ] ) T() = c 2 swap(a[ j ], A[ j 1 ]); //step E.G.M. Petrakis Algorithm Aalysis 5

6 Asymptotic Algorithm Aalysis Measures the efficiecy of a program as the size of the iput becomes large focuses o growth rate : the rate at which the ruig time of a algorithm grows as the size of the iput becomes big Complexity => Growth rate focuses o the upper ad lower bouds of the growth rates igores all costat factors E.G.M. Petrakis Algorithm Aalysis 6

7 Growth Rate Depedig o the order of a algorithm ca beof Liear time Complexity: T() = c Quadratic Complexity: T() = c 2 Expoetial Complexity: T() = c k The differece betwee a algorithm whose ruig time is T() = 10 ad aother with ruig time is T() = 2 2 is tremedous for > 5 the liear algorithm is must faster despite the fact that it has larger costat it is slower oly for small but, for such, both algorithms are very fast E.G.M. Petrakis Algorithm Aalysis 7

8 E.G.M. Petrakis Algorithm Aalysis 8

9 Upper Lower Bouds Several terms are used to describe the ruig time of a algorithm: The upper boud to its growth rate which is expressed by the Big-Oh otatio f() is upper boud of T() T() is i O(f()) The lower boud to it growth rate which is expressed by the Omega otatio g() is lower boud of T() T() is i Ω(g()) If the upper ad lower bouds are the same T() is i Θ(g()) E.G.M. Petrakis Algorithm Aalysis 9

10 Big O (upper boud) Defiitio: T() T() Ο(g()) if 0 cg() 0, c 1 Example: T() = 3 T() = 6 2 T() Ο( T() O( 3 ) 2 ) T() = T() O( 3 ) E.G.M. Petrakis Algorithm Aalysis 10

11 Properties of O O is trasitive: if f Ο(g) g Ο(h) f Ο(h) If f O(g) f + g O(g) If If f O(f ) f g O(f g ) g O(g ) f O(f ) f + g O(f + g ) g O(g ) E.G.M. Petrakis Algorithm Aalysis 11

12 More o Upper Bouds T() O(g()) g() :upper boud for T() There exist may upper bouds e.g., T() = 2 O( 2 ), O( 3 ) O( 100 )... Fid the smallest upper boud!! O otatio hides the costats e.g., c 2, c 2 2, c! 2, c k 2 O( 2 ) E.G.M. Petrakis Algorithm Aalysis 12

13 O i Practice Amog differet algorithms solvig the same problem, prefer the algorithm with the smaller rate of growth O it will be faster for large Ο(1) < Ο(log ) < Ο() < Ο( log ) < Ο( 2 ) < Ο( 3 ) < < Ο(2 )< O( ) Ο(), Ο( log ), Ο( 2 ) < Ο( 3 ): polyomial Ο(log ): logarithmic O( k ), O(2 ), (!), O( ): expoetial!! E.G.M. Petrakis Algorithm Aalysis 13

14 10-6 sec/commad Complexity log mi mi 21mi 27hrs log / da y 2.7hrs 125cet cet cet yrs cet 58mi cet E.G.M. Petrakis Algorithm Aalysis 14

15 Omega Ω (lower boud) Defiitio: f() Ω(g()) if c 1 f() cg() g() is a lower boud of the rate of growth of f() f() icreases faster tha g() as icreases e.g: T() = =>T() i Ω( 2 ) E.G.M. Petrakis Algorithm Aalysis 15

16 More o Lower Bouds Ω provides lower boud of the growth rate of T() if there exist may lower bouds T() = 4 Ω(), Ω( 2 ), Ω( 3 ), Ω( 4 ) fid the larger oe => T() Ω( 4 ) E.G.M. Petrakis Algorithm Aalysis 16

17 Thetα Θ If the lower ad upper bouds are the same: f O(g) f Ω(g) f Θ(g) e.g.: T() = Θ( 3 ) E.G.M. Petrakis Algorithm Aalysis 17

18 Theorem: T()=Σ α i i Θ( m ) T() = α m +α -1 m-1 + +α 1 + α 0 O( m ) Proof: T() T() α m + α m-1 m-1 + α 1 + α 0 m { α m + 1/ α m / m-1 α 1 +1/ m α 0 } m { α m + α m α 1 + α 0 } T() O( m ) E.G.M. Petrakis Algorithm Aalysis 18

19 Theorem (cot.) b) T() = α m m + α m-1 m-1 + α 1 + α 0 >= c m + c m-1 + c + c >= c m where c = mi{α m, α m-1, α 1, α 0 } => T() Ω( m ) (a), (b) => Τ() Θ( m ) E.G.M. Petrakis Algorithm Aalysis 19

20 Theorem: (a) (b) T() 2 T() i= 1 (i k = = i k T() i= 1 i= ( -i + 1) i= 1 i= 1 k i= 1 Θ( 2 (either i or ( - i + 1) ) T() i k 2 ) = k ( -i + 1) i= 1 i k = ( ) 2 k k = 1 k+ 1 T() Ω( k+ 1 2 k k+ 1 ), k T() O( k+ 1 ) k+ 1 ) 0 E.G.M. Petrakis Algorithm Aalysis 20

21 Recursive Relatios (1) T() = 1 T( -1) + if if = 1 > 1 T() = T( -1) + = = T( - 2) + ( -1) + = =... = T(1) ( -1) + = = i= 1 ( + 1) i = 2 T() Θ( 2 ) E.G.M. Petrakis Algorithm Aalysis 21

22 F(0) = 0 F(1) = 1 Fiboacci Relatio F() = F( - 1) + F( - 2) if 2 (a) F() = F( - 1) + F( - 2) (b) 2F( - 1) 4F( - 2) 2 F() 2 F() 2-1 F(1) = 2-1 F() O(2 E.G.M. Petrakis Algorithm Aalysis F() = F( - 1) + F( - 2) 2F( - 2) 4F( - 4) K (-1)/2 (-2)/2 if odd F() Ω(2 if eve /2 ) ) = Ω( 2 )

23 Fiboacci relatio (cot.) From (a), (b): F() Ω( 2 ) O(2 ) It ca be prove that F() Θ(φ ) where φ = 2 = golde ratio E.G.M. Petrakis Algorithm Aalysis 23

24 Biary Search it BSearch(it table[a..b], key, ) { if (a > b) retur ( 1) ; it middle = (a + b)/2 ; if (T[middle] == key) retur (middle); else if ( key < T[middle] ) retur BSearch(T[a..middle 1], key); else retur BSearch(T[middle + 1..b], key); } E.G.M. Petrakis Algorithm Aalysis 24

25 Aalysis of Biary Search Let = b a + 1: size of array ad = 2 k 1 for some k (e.g., 1, 3, 5 etc) The size of each sub-array is 2 k-1 1 k c k = 0 T() = T(2-1) = k 1 d + T(2 1) c: executio time of first commad d: executio time outside recursio T: executio time k > 0 E.G.M. Petrakis Algorithm Aalysis 25

26 Biary Search (cot.) T() = T(2 k 1) = d + T(2 = kd + T(0) = kd + c = d log( + 1) + c => T() O(log) k -1-1) = 2d + T(2 k -2-1) =... E.G.M. Petrakis Algorithm Aalysis 26

27 Biary Search for Radom c T() = d + max{t(( 1)/2 ), T( ( 1)/2 )} k = 0 k > 0 T() <= T(+1) <=...T(u()) where u()=2 k 1 For some k: < u() < 2 The, T() <= T(u()) = d log(u() + 1) + c => d log(2 + 1) + c => T() O(log ) E.G.M. Petrakis Algorithm Aalysis 27

28 Merge sort list mergesort(list L, it ) { if ( == 1) retur (L); L 1 = lower half of L; L 2 = upper half of L; retur merge (mergesort(l 1,/2), mergesort(l 2,/2) ); } : size of array L (assume L some power of 2) merge: merges the sorted L 1, L 2 i a sorted array E.G.M. Petrakis Algorithm Aalysis 28

29 E.G.M. Petrakis Algorithm Aalysis 29

30 Aalysis of Merge sort [25] [57] [48] [37] [12] [92] [86] [33] log 2 merges [25 57] [37 48] [12 92] [33 86] [ ] [ ] [ ] N steps per merge, log 2 steps/merge => O( log 2 ) E.G.M. Petrakis Algorithm Aalysis 30

31 Aalysis of Merge sort T() = 2T(/2) + c 2 c1 > 1 = 1 Two ways: recursio aalysis iductio E.G.M. Petrakis Algorithm Aalysis 31

32 Iductio Let T() = O( log ) the T() ca be writte as T() <= a log + b, a, b > 0 Proof: (1) for = 1, true if b >= c 1 (2) let T(k) = a k log k + b for k=1,2,..-1 (3) for k= prove that T() <= a log +b For k = /2 : T(/2) <= a /2 log /2 + b therefore, T() ca be writte as T() = 2 {a /2 log /2 + b} + c 2 = a (log 1)+ 2b + c 2 = E.G.M. Petrakis Algorithm Aalysis 32

33 Iductio (cot.) T() = a log a + 2 b + c 2 Let b = c 1 ad a = c 1 + c 2 => T() <= (c 1 + c 2 ) log + c 1 => T() <= C log + B E.G.M. Petrakis Algorithm Aalysis 33

34 Recursio aalysis of mergesort T() = 2T(/2) + c 2 <= <= 4T(/4) + 2 c 2 <= <= 8T(/8) + 3 c 2 <= T() <= 2 i T(/2 i ) + ic 2 where = 2 i T() <= T(1) + c 2 log =>T() < = c 1 + c 2 log => T() <= (c 1 + c 2 ) log => T() <= C log E.G.M. Petrakis Algorithm Aalysis 34

35 Best, Worst, Average Cases For some algorithms, differet iputs require differet amouts of time e.g., sequetial search i a array of size... x = pi xi = ( + 1)/2 comparisos worst-case: elemet ot i array => T() = best-case: elemet is first i array =>T() = 1 average-case: elemet betwee 1, =>T() = /2 E.G.M. Petrakis Algorithm Aalysis 35

36 Compariso The best-case is ot represetative happes oly rarely useful whe it has high probability The worst-case is very useful: the algorithm performs at least that well very importat for real-time applicatios The average-case represets the typical behavior of the algorithm ot always possible kowledge about data distributio is ecessary E.G.M. Petrakis Algorithm Aalysis 36