Time Analysis of Sorting and Searching Algorithms

Transcription

1 Time Analysis of Sorting and Searching Algorithms CSE21 Winter 2017, Day 5 (B00), Day 3 (A00) January 20,

2 Big-O : How to determine? Is f(n) big-o of g(n)? i.e. is? Approach 1: Look for constants C and k. Approach 2: Use properties Domination If f(n) <= g(n) for all n then f(n) is big-o of g(n). Transitivity If f(n) is big-o of g(n), and g(n) is big-o of h(n), then f(n) is big-o of h(n) Additivity/ Multiplicativity If f(n) is big-o of g(n), and if h(n) is nonnegative, then f(n) * h(n) is big-o of g(n) * h(n) where * is either addition or multiplication. Sum is maximum f(n)+g(n) is big-o of the max(f(n), g(n)) Ignoring constants For any constant c, cf(n) is big-o of f(n) Rosen p

3 Big O : How to determine? Is f(n) big-o of g(n)? i.e. is? Approach 1: Look for constants C and k. Approach 2: Use properties Domination If f(n) <= g(n) for all n then f(n) is big-o of g(n). Transitivity If f(n) is big-o of g(n), and g(n) is big-o of h(n), then f(n) is big-o of Look h(n) at terms one-by-one Additivity/ Multiplicativity and drop if constants. f(n) is big-o of Then g(n), and if h(n) is nonnegative, only then keep f(n) * maximum. h(n) is big-o of g(n) * h(n) where * is either addition or multiplication. Sum is maximum f(n)+g(n) is big-o of the max(f(n), g(n)) Ignoring constants for any constant c, cf(n) is big-o of f(n) Rosen p

4 Big O : How to determine? Is f(n) big-o of g(n)? i.e. is? Approach 3. The limit method. Consider the limit I. If this limit exists and is 0: then f(n) grows strictly slower than g(n).. In which cases can we conclude A. I, II, III B. I, III C. I, II D. None of the above? II. If this limit exists and is a constant c > 0: then f(n), g(n), grow at the same rate. III. If the limit tends to infinity: then f(n) grows strictly faster than g(n). IV. if the limit doesn't exist for a different reason use another approach!

5

6 Other asymptotic classes and notation Rosen p means there are constants, C and k such that for all n > k. means means and What functions are in the family?

7 Selection Sort (MinSort) Performance (Again) Rosen page 210, example 5 Number of comparisons of list elements (n-1) + (n-2) + + (1) = n(n-1)/2 Sum of positive integers up to (n-1) Rewrite this expression using big-o notation: A. O(n) B. O(n(n-1)) C. O(n 2 ) D. O(1/2) E. None of the above

8 Linear Search: HOW Starting at the beginning of the list, compare items one by one with x until find it or reach the end procedure linear search (x: integer, a 1, a 2,..., a n : distinct integers ) i := 1 while (i <= n and x a i ) i := i+1 if i <=n then location := i else location := 0 return location { location is the subscript of the term that equals x, or is 0 if x is not found }

9 How fast is Linear Search: WHEN Rosen page 220, part of example 2 The time it takes to find x (or determine it is not present) depends on the number of probes, that is the number of list entries we have to retrieve and compare to x How many probes do we make when doing Linear Search on a list of size n if x happens to equal the first element in the list? if x happens to equal the last element in the list? if x happens to equal an element somewhere in the middle of the list? if x doesn't equal any element in the list?

10 How fast is Linear Search: WHEN Best case: 1 probe target appears first Worst case: n probes target appears last or not at all Average case: n/2 probes target appears in the middle, (expect to have to search about Running time depends on more than size of the input! half of the array... more on expected value later in the course) Rosen p. 220

11 Binary Search: WHEN procedure binary search (x: integer, a 1, a 2,..., a n : increasing integers ) i := 1 j := n while i<j m := floor( (i+j)/2 ) if x > a m then i := m+1 else j := m if x=a i else location := 0 return location then location := i Each time divide size of "list in play" by 2 { location is the subscript of the term that equals x, or is 0 if x is not found }

12 Rosen page 220, example 3 How fast is Binary Search: WHEN Number of comparisons (probes) depends on number of iterations of loop, hence size of set. After iterations of loop (Max) size of list "in play" 0 n 1 n/2 2 n/4 3 n/8?? 1

13 Rosen page 220, example 3 How fast is Binary Search: WHEN Number of comparisons (probes) depends on number of iterations of loop, hence size of set. After iterations of loop (Max) size of list "in play" 0 n 1 n/2 2 n/4 3 n/8 ceil(log 2 n) 1 Rewrite this formula in order notation: A. B. C. D. E. None of the above

14

15 Comparing linear search and binary search Rosen pages Linear Search Binary Search Assumptions None Sorted list # probes in * best case or * worst case * average case?? Best case analysis depends on whether we check if midpoint agrees with target right away or wait until list size gets to 1

16 Comparing linear search and binary search Rosen pages Linear Search Binary Search Assumptions None Sorted list # probes in * best case or * worst case * average case??

17 Comparing linear search and binary search Rosen pages Linear Search Binary Search Assumptions None Sorted list # probes in * best case or * worst case * average case?? Is it worth it to sort our list first?

18 Determining the big-o class of algorithms How to deal with Basic operations Consecutive (non-nested) code Loops (simple and nested) Subroutines

19 Determining the big-o class of algorithms How to deal with Basic operations : operation whose time doesn t depend on input Consecutive (non-nested) code : one operation followed by another Loops (simple and nested) : while loops, for loops Subroutines : method calls

20 Determining the big-o class of algorithms Consecutive (non-nested) code : Run Prog 1 followed by Prog 2 If Prog 1 takes O( f(n) ) time and Prog 2 takes O( g(n) ) time, what's the big-o class of runtime for running them consecutively? A. O( f(n) + g(n) ) [[sum]] B. O( f(n) g(n) ) [[ multiplication ]] C. O( g(f(n)) ) [[ function composition ]] D. O( max (f(n), g(n)) ) E. None of the above.

21 Determining the big-o class of algorithms Simple loops: What's the runtime? (assuming th Guard Condition is O(1))? A. Constant B. Same order as the number of iterations through the loop. C. Same order as the runtime of the guard condition D. Same order as the runtime of the body of the loop. E. None of the above.

22 Determining the big-o class of algorithms Simple loops: If Guard Condition uses basic operations and body of the loop is constant time, then runtime is of the same order as the number of iterations.

23 Determining the big-o class of algorithms Nested code: If Guard Condition uses basic operations and body of the loop has runtime O( T 2 ) in the worst case, then runtime is O( T 1 T 2 ) Runtime O(T 2 ) in the worst case where T 1 is the bound on the number of iterations through the loop. Product rule

24 Determining the big-o class of algorithms Subroutine Call method S on (some part of) the input. If subroutine S has runtime T S (n) and we call S at most T 1 times, A. Total time for all uses of S is T 1 +T S (n) B. Total time for all uses of S is max(t 1,T S (n)) C. Total time for all uses of S is T 1 T S (n) D. None of the above

25 Selection Sort (MinSort) Pseudocode Before, we counted comparisons, and then went to big-o procedure selection sort(a 1, a 2,..., a n : real numbers with n >=2 ) for i := 1 to n-1 m := i for j:= i+1 to n if ( a j < a m ) then m := j (n-1) + (n-2) + + (1) interchange a i and a m = n(n-1)/2 { a 1,..., a n is in increasing order}

26 Now, straight to big O Selection Sort (MinSort) Pseudocode procedure selection sort(a 1, a 2,..., a n : real numbers with n >=2 ) for i := 1 to n-1 m := i for j:= i+1 to n if ( a j < a m ) then m := j interchange a i and a m { a 1,..., a n is in increasing order} Strategy: work from the inside out

27 Now, straight to big O Selection Sort (MinSort) Pseudocode procedure selection sort(a 1, a 2,..., a n : real numbers with n >=2 ) for i := 1 to n-1 m := i for j:= i+1 to n if ( a j < a m ) then m := j O(1) interchange a i and a m { a 1,..., a n is in increasing order} Strategy: work from the inside out

28 Now, straight to big O Selection Sort (MinSort) Pseudocode procedure selection sort(a 1, a 2,..., a n : real numbers with n >=2 ) for i := 1 to n-1 m := i for j:= i+1 to n if ( a j < a m ) then m := j interchange a i and a m O(1) { a 1,..., a n is in increasing order} Simple for loop, repeats n-i times Strategy: work from the inside out

29 Now, straight to big O Selection Sort (MinSort) Pseudocode procedure selection sort(a 1, a 2,..., a n : real numbers with n >=2 ) for i := 1 to n-1 m := i for j:= i+1 to n if ( a j < a m ) then m := j interchange a i and a m { a 1,..., a n is in increasing order} O(n-i), but i ranges from 1 to n-1 Strategy: work from the inside out

30 Now, straight to big O Selection Sort (MinSort) Pseudocode procedure selection sort(a 1, a 2,..., a n : real numbers with n >=2 ) for i := 1 to n-1 m := i for j:= i+1 to n if ( a j < a m ) then m := j interchange a i and a m { a 1,..., a n is in increasing order} Worst case: when i =1, O(n) Strategy: work from the inside out

31 Now, straight to big O Selection Sort (MinSort) Pseudocode procedure selection sort(a 1, a 2,..., a n : real numbers with n >=2 ) for i := 1 to n-1 O(1) m := i for j:= i+1 to n O(n) if ( a j < a i ) then m := j O(1) interchange a i and a m { a 1,..., a n is in increasing order} Strategy: work from the inside out

32 Now, straight to big O Selection Sort (MinSort) Pseudocode procedure selection sort(a 1, a 2,..., a n : real numbers with n >=2 ) for i := 1 to n-1 m := i for j:= i+1 to n if ( a j < a i ) then m := j interchange a i O(n) and a m { a 1,..., a n is in increasing order} Strategy: work from the inside out

33 Now, straight to big O Selection Sort (MinSort) Pseudocode procedure selection sort(a 1, a 2,..., a n : real numbers with n >=2 ) for i := 1 to n-1 m := i for j:= i+1 to n if ( a j < a i ) then m := j Nested for loop, interchange a i O(n) and a m repeats O(n) times { a 1,..., a n is in increasing order} Strategy: work from the inside out Total: O(n 2 )

34 Next Time Analyzing algorithms that solve other problems (besides sorting and searching) Designing better algorithms pre-processing re-use of computation