Analysis of Algorithms

Transcription

1 Analysis of Algorithms Section 4.3 Prof. Nathan Wodarz Math Fall 2008 Contents 1 Analysis of Algorithms Analysis of Algorithms Complexity Analysis Notation and Terminology Best, Worst and Average-Cases Complexity of Example Algorithms P = NP? The Class P The Class NP NP-complete Algorithms P = NP?

2 1 Analysis of Algorithms 1.1 Analysis of Algorithms Analysis of Algorithms Want to estimate time and space needed to run a program. Exact amounts depend On computer used... On language used... On compiler used... On implementation used... On input... Will use size of input n to stand in for input Use steps of algorithm rather than actual time Actually, only count important steps Look at: Best-case time Worst-case time Average case-time 2

3 Analysis of Algorithms Sequential search: (sequence of length n) Best-case: Find item in first try, takes 1 time through loop. Worst-case: Don t find item, takes n times through loop. Input: s, n, x Output: The index of the first occurrence of x in s, otherwise 0 (indicating an unsuccessful search) 1. sequential search(s,n,x){ 2. for i = 1 to n 3. if (x = s i ) 4. return i 5. return 0 6. } Analysis of Algorithms Binary search: (sequence of length n) Best-case: Item in middle, takes 1 time through loop. Worst-case: May take lg n + 1 trips through loop. Input: s, n, x Output: The index of the first occurrence of x in s, or 0 1. binary search(s,n,x){ 2. lo = 1 3. hi = n 4. while (lo hi) { 5. mid = (lo + hi)/2 6. if (s mid = x) 7. return mid 8. else if (x < s mid ) 9. hi = mid else 11. lo = mid } 13. return } 3

4 2 Complexity Analysis 2.1 Notation and Terminology Notation and Terminology Let f and g be functions with domain N = {1, 2, 3,...} f (n) = O(g(n)) Say: f (n) is of order at most g(n) or f (n) is big-o of g(n) Means there is a positive constant C 1 with f (n) C 1 g(n) for n large enough f (n) = Ω(g(n)) Say: f (n) is of order at least g(n) or f (n) is omega of g(n) Means there is a positive constant C 2 with f (n) C 2 g(n) for n large enough f (n) = Θ(g(n)) Say: f (n) is of order g(n) or f (n) is theta of g(n) Means f (n) = Ω(g(n)) and f (n) = O(g(n)) Also, C 2 g(n) f (n) C 1 g(n) for some constants Notation and Terminology Example. 1. n 2 + 4n = Θ(n 2 ) n = Θ(n 2 ) 3. 1 k + 2 k + 3 k + + n k = Θ(n k+1 ) 4. n = O(n 2 ) and n 2 = Ω(n) 5. n lg n = O(n 2 ) 6. a k n k + a k 1 n k a 1 n + a 0 = Θ(n k ) 4

5 Notation and Terminology Θ(1): constant Θ(lg lg n): log log Θ(lg n): log Θ(n): linear Θ(n lg n): n log n Θ(n 2 ): quadratic Θ(n 3 ): cubic Θ(n k ), k 1: polynomial Θ(c n ), c > 1: exponential Θ(n!): factorial (n! = (n 1) n) 2.2 Best, Worst and Average-Cases Best, Worst and Average-Cases Best-case time is O(g(n)) Algorithm takes t(n) steps in best-case t(n) = O(g(n)) Also, define worst-case time and average-case time similarly 5

6 2.3 Complexity of Example Algorithms Complexity of Sequential Search Sequential search: (sequence of length n) Best-case: Find item in first try, takes 1 time through loop. Worst-case: Don t find item, takes n times through loop. Best-case time: O(1) (Actually Θ(1)) Worst-case time: O(n) (Actually Θ(n)) Input: s, n, x Output: The index of the first occurrence of x in s, otherwise 0 (indicating an unsuccessful search) 1. sequential search(s,n,x){ 2. for i = 1 to n 3. if (x = s i ) 4. return i 5. return 0 6. } 6

7 Complexity of Binary Search Binary search: (sequence of length n) Best case: Item in middle, takes 1 time through loop. Worst case: May take lg n + 1 trips through loop. Best-case time: O(1) Worst-case time: O(lg n) Output: The index of the first occurrence of x in s, or 0 1. binary search(s,n,x){ 2. lo = 1 3. hi = n 4. while (lo hi) { 5. mid = (lo + hi)/2 6. if (s mid = x) 7. return mid 8. else if (x < s mid ) 9. hi = mid else 11. lo = mid } 13. return } 7

8 Complexity of Insertion Sort Insertion Sort: (sequence of length n) Best-case time: O(n) Worst-case time: O(n 2 ) Input: s, n Output: s (with elements arranged in nondecreasing order) 1. insertion sort(s,n){ 2. for i = 2 to n { 3. val = s i 4. j = i while (( j 1) and (val < s j )) { 6. s j+1 = s j 7. j = j } 9. s j+1 = val 10. } 11. } Complexity of Bubble Sort Bubble Sort: (sequence of length n) Best-case time: O(n 2 ) Worst-case time: O(n 2 ) Input: s, n Output: s (with elements arranged in nondecreasing order) 1. bubble sort(s,n){ 2. for i = 1 to n-1 { 3. for j = 1 to n-i { 4. if (s j > s j+1 ) { 5. temp = s j 6. s j = s j+1 7. s j+1 = temp 8. } 9. } 10. } 11. } 8

9 3 P = NP? 3.1 The Class P The Class P If an algorithm has worst-case time O(n k ), it is said to run in polynomial time Such an algorithm is often called tractable All problems solvable by polynomial time algorithms make up the complexity class P Integer multiplication is in P 3.2 The Class NP The Class NP We consider algorithms to be deterministic What if we weaken this assumption? Ideas: Don t need to find an answer, just need to check one Oracle guides the algorithm We can consider every deterministic algorithm to be non-deterministic as well If a non-deterministic algorithm has worst-case time O(n k ), it is said to run in non-deterministic polynomial time The complexity class NP is all problems solvable by algorithms running in non-deterministic polynomial time. Does not mean non-polynomial time Integer factorization is in NP 9

10 3.3 NP-complete Algorithms NP-complete Algorithms It is easy to see that P NP. Is it true that NP P? If so, then P = NP Don t know if integer factorization is P An NP-complete algorithm is one of the most difficult NP-algorithms Can quickly transform any NP problem to an NP-complete one (Multiplication vs. squaring) An algorithm to solve an NP-complete problem can be adapted to solve any NP problem. Sudoku is NP-complete 3.4 P = NP? P = NP? Not known if P = NP Summary Summary You should be able to: Understand and use big-o, theta and omega notation Be able to determine complexity of simple algorithms Know the most common growth functions 10