Time complexity analysis

Transcription

1 Time complexity analysis Let s start with selection sort C. Seshadhri University of California, Santa Cruz sesh@ucsc.edu March 30, 2016 C. Seshadhri (UCSC) CMPS101 1 / 40

2 Sorting Sorting Input An array e.g., [1, 5, 6, 7, 3, 2, 1] Output Sorted array [1, 1, 2, 3, 5, 6, 7] C. Seshadhri (UCSC) CMPS101 2 / 40

3 Sorting Formally Input Output An array e.g., [a 1, a 2, a 3,..., a n ] A permuted version of the array [a 1, a 2, a 3,..., a n] such that a 1 a 2 a 3... a n C. Seshadhri (UCSC) CMPS101 3 / 40

4 Sorting Comes up everywhere Give me webpages on Data Structures, in decreasing order of relevance Give me a list of coffee shops, in decreasing order of ratings Give me a list of grocery stores, in increasing order of distance C. Seshadhri (UCSC) CMPS101 4 / 40

5 Sorting Sorting with keys Input Output An array of key-value pairs e.g., [(k 1, v 1 ), (k 2, v 2 ), (k 3, v 3 ),..., (k n, v n )] A permuted version of the array [(k 1, v 1 ), (k 2, v 2 ), (k 3, v 3 ),..., (k n, v n)] such that v 1 v 2 v 3... v n C. Seshadhri (UCSC) CMPS101 5 / 40

6 Sorting What is an Algorithm? A sequence of well defined computational steps that takes some input transforms it into an output (with a certain property) Alternatively An algorithm describes a specific computational procedure for achieving a specified input/output relationship C. Seshadhri (UCSC) CMPS101 6 / 40

7 Selection Sort (Let length of array A be n.) SELECTION-SORT(A) 1 For i = 1 to n: 1 Find minimum element b in A[i... n]. 2 Set b to be ith element in sorted order C. Seshadhri (UCSC) CMPS101 7 / 40

8 Selection Sort (Let length of array A be n.) Selection-Sort(A) 1 for i = 1 to n 2 min = A[i] / Initialize min 3 minind = i / Initialize min index 4 for j = i to n 5 if A[j] < min // Aha! Min needs to be changed 6 min = A[j] // So change min and min index 7 minind = j 8 // Swap A[i] with A[minInd] 9 A[minInd] = A[i] 10 A[i] = min C. Seshadhri (UCSC) CMPS101 8 / 40

9 How many comparisons? Selection-Sort on an array of n items makes at most how many comparisons? (Give the best answer.) (R) n (B) n 1 (G) (n 1)(n 2)/2 (O) n 2 2n C. Seshadhri (UCSC) CMPS101 9 / 40

10 How many comparisons? Selection-Sort on an array of n items makes at most how many comparisons? (Give the best answer.) (R) n (B) n 1 (G) (n 1)(n 2)/2 (O) n 2 2n C. Seshadhri (UCSC) CMPS / 40

11 How long does Selection-Sort take? It really depends on your machine! How can we mathematically talk about the running time of Selection-Sort? C. Seshadhri (UCSC) CMPS / 40

12 Analyzing Selection-Sort Definition The running time of Selection-Sort (or your favorite algorithm) on input A is the number of elementary operations that Selection-Sort takes on A. Beats wall clock time. Challenges: What is elementary? Ok, for sorting, any assignment or comparison counts as one operation. Running time is not one number; it depends heavily on input (especially size). C. Seshadhri (UCSC) CMPS / 40

13 Analyzing Selection-Sort Definition The running time of Selection-Sort (or your favorite algorithm) on input A is the number of elementary operations that Selection-Sort takes on A. Beats wall clock time. Challenges: What is elementary? Ok, for sorting, any assignment or comparison counts as one operation. Running time is not one number; it depends heavily on input (especially size). C. Seshadhri (UCSC) CMPS / 40

14 Worst-case Analysis Definition The worst-case running time or worst-case time complexity of Selection-Sort (or your favorite algorithm) is the function T (n) where: T (n) is the maximum running time over all inputs of size n. C. Seshadhri (UCSC) CMPS / 40

15 Back to Selection Sort (Let length of array A be n.) Selection-Sort(A) 1 for i = 1 to n 2 min = A[i] / Initialize min 3 minind = i / Initialize min index 4 for j = i to n 5 if A[j] < min // Aha! Min needs to be changed 6 min = A[j] // So change min and min index 7 minind = j 8 // Swap A[i] with A[minInd] 9 A[minInd] = A[i] 10 A[i] = min C. Seshadhri (UCSC) CMPS / 40

16 Exactly finding T (n) It s a real pain. Once you get the basic idea, it s boring to figure out exactly. C. Seshadhri (UCSC) CMPS / 40

17 The solution? C. Seshadhri (UCSC) CMPS / 40

18 The solution? C. Seshadhri (UCSC) CMPS / 40

19 Basic Idea Focus on the asymptotic growth of T (n) Asymptotic: Fancy way of saying that we will focus on how the function grows as n. C. Seshadhri (UCSC) CMPS / 40

20 Oh Notation Given time complexity T (n) and convenient/simple function f (n) we say that T (n) O(f (n)) if there exists constants c and n 0 such that 0 T (n) c f (n) for all n n 0. Notational convention: T (n) = O(f (n)), T (n) is O(f (n)), T (n) is in O(f (n)). We re providing a convenient upper bound for T (n). C. Seshadhri (UCSC) CMPS / 40

21 In Pictures C. Seshadhri (UCSC) CMPS / 40

22 Is 2n + 30 = O(n)? (R) Yes (B) No C. Seshadhri (UCSC) CMPS / 40

23 Example - 1 2n + 30 = O(n) Need to show that there exists c and n 0 such that 0 2n + 30 c n for all n n 0 Let c = 3 and n 0 = 30. C. Seshadhri (UCSC) CMPS / 40

24 Is 2n + 30 = O(n 2 )? (R) Yes (B) No C. Seshadhri (UCSC) CMPS / 40

25 Example - 2 2n + 30 = O(n 2 ) Need to show that there exists c and n 0 such that 0 2n + 30 c n 2 for all n n 0 Let c = 2 and n 0 = 6. C. Seshadhri (UCSC) CMPS / 40

26 Is 5n n = O(n 2 )? (R) Yes (B) No C. Seshadhri (UCSC) CMPS / 40

27 Example - 3 Reductio ad absurdum 5n n / O(n 2 ) C. Seshadhri (UCSC) CMPS / 40

28 Example - 3 Reductio ad absurdum 5n n / O(n 2 ) C. Seshadhri (UCSC) CMPS / 40

29 Example - 3 5n n / O(n 2 ) Proof by contradiction. Suppose c and n 0 existed. 0 5n n c n 2 for all n n 0 0 5n + 10 n c for all n n 0 But as n we have 5n + 10 n. C. Seshadhri (UCSC) CMPS / 40

30 Selection Sort (Let length of array A be n.) Selection-Sort(A) 1 for i = 1 to n 2 min = A[i] / Initialize min 3 minind = i / Initialize min index 4 for j = i to n 5 if A[j] < min // Aha! Min needs to be changed 6 min = A[j] // So change min and min index 7 minind = j 8 // Swap A[i] with A[minInd] 9 A[minInd] = A[i] 10 A[i] = min Selection-Sort time complexity is O(n 2 ). C. Seshadhri (UCSC) CMPS / 40

31 Omega Notation Given functions T (n) and f (n) we say that T (n) Ω(f (n)) if there exists constants c and n 0 such that 0 cf (n) T (n) for all n n 0. Notational convention: T (n) = Ω(f (n)) C. Seshadhri (UCSC) CMPS / 40

32 Theta Notation Given functions T (n) and f (n) we say that T (n) Θ(f (n)) if there exists constants c 1, c 2 and n 0 such that 0 c 1 f (n) T (n) c 2 f (n) for all n n 0. Notational convention: T (n) = Θ(f (n)) C. Seshadhri (UCSC) CMPS / 40

33 An Alternate View Point T (n) T (n) = O(f (n)) = lim n f (n) const T (n) T (n) = Ω(f (n)) = lim n f (n) const T (n) T (n) = Θ(f (n)) = const 1 lim n f (n) const 2 C. Seshadhri (UCSC) CMPS / 40

34 In Pictures C. Seshadhri (UCSC) CMPS / 40

35 Example - 4 5n n = Θ(n 2 ) Need to show that there exists c 1, c 2 and n 0 such that Rewrite as 0 c 1 n 2 5n n c 2 n 2 for all n n 0 0 c n c 2 for all n n 0 But as n we have n 5. So select c 1 = 4 and c 2 = 6, with n 0 = 10. C. Seshadhri (UCSC) CMPS / 40

36 Prove For any two functions f (n) and g(n), we have f (n) = Θ(g(n)) f (n) = O(g(n)) and f (n) = Ω(g(n)). C. Seshadhri (UCSC) CMPS / 40

37 The time complexity of Selection-Sort is: (R) Θ(n 2 ) (B) O(n 2 ) but not Θ(n 2 ). (G) Ω(n 2 ) but not Θ(n 2 ). (O) I have no idea what s going on. C. Seshadhri (UCSC) CMPS / 40

38 Selection Sort (Let length of array A be n.) Selection-Sort(A) 1 for i = 1 to n 2 min = A[i] / Initialize min 3 minind = i / Initialize min index 4 for j = i to n 5 if A[j] < min // Aha! Min needs to be changed 6 min = A[j] // So change min and min index 7 minind = j 8 // Swap A[i] with A[minInd] 9 A[minInd] = A[i] 10 A[i] = min Selection-Sort time complexity is Θ(n 2 ). C. Seshadhri (UCSC) CMPS / 40

39 Little o and ω Notation o Notation O notation is not asymptotically tight. For instance n = O(n) and also n = O(n 2 ). Given functions g(n) and f (n) we say that f (n) o(g(n)) if there exists a constant n 0 such that 0 f (n) cg(n) for all c > 0 and n n 0. Carefully note the order of the qualifiers. Intuitively, f (n) becomes insignificant compared to g(n) f (n) lim n g(n) = 0. C. Seshadhri (UCSC) CMPS / 40

40 Little o and ω Notation Example - 5 5n n / o(n 2 ) lim n n = 5 0 5n n = o(n 3 ) 5 lim n n + 10 = 0 n 2 C. Seshadhri (UCSC) CMPS / 40

41 Little o and ω Notation ω Notation Given functions g(n) and f (n) we say that f (n) ω(g(n)) if there exists a constant n 0 such that 0 cg(n) f (n) for all c > 0 and n n 0. Carefully note the order of the qualifiers. Intuitively, f (n) grows much faster as compared to g(n) f (n) lim n g(n) =. C. Seshadhri (UCSC) CMPS / 40

42 Little o and ω Notation Questions? C. Seshadhri (UCSC) CMPS / 40