Mat Week 6. Fall Mat Week 6. Algorithms. Properties. Examples. Searching. Sorting. Time Complexity. Example. Properties.

Transcription

1 Fall 2013

2 Student Responsibilities Reading: Textbook, Section Assignments: 1. for sections 3.1 and Worksheet #4 on Execution s 3. Worksheet #5 on Growth Rates Attendance: Strongly Encouraged Week 8 Overview Growth of Functions

3 3.1 Algorithm: a finite set of unambiguous instructions for performing a computation or for solving a problem. : Shampoo Instructions: Lather, Rinse, Repeat Recipe for making Italian Beef: Place beef roast 1 pkg Au Jus dry gravy mix 1 pkg dry Italian dressing mix and 1 C water in slow cooker cook all day or over night shred beef and serve with French Bread or rolls The instructions that come with a sewing pattern The instructions for a model airplane or rocket kit Insert disk and press any key to continue

4 of Input an algorithm usually has input from a specified set Output the solution to the problem, also from a specified set Definiteness steps of an algorithm must be defined precisely Correctness an algorithm must produce the correct values for each of the input values

5 of Continued Finiteness an algorithm must produce the desired output after a finite (but perhaps large) number of steps for any input in the set Effectiveness it must be possible to perform each step of an algorithm exactly and in a finite amount of time Generality an algorithm should be applicable for all problems of the desired form, not just for a particular set of input values

6 Finding the Maximum Value in a Finite Sequence Pseudocode integer max(a1, a2,..., an : integers) currmax := a1 for i := 2 to n if currmax < ai then currmax := ai {currmax is the largest element} return currmax;

7 C++ Implementation of Max template <class T> T Max (const vector<t> & L){ // PRE: L not empty, type T is comparable // POST: returns the largest value in vector L T mymax = L[0]; for (int i = 1; i < L.size; i++) if (mymax < L[i]) mymax = L[i]; return mymax; }

8 Search The problem: locate a particular element (target) in a list Distinguish between unordered and ordered (sorted) lists Two primary algorithms: Linear or Sequential Search look at each item in the list, first to last, comparing them to target until target is found or we reach end of list Binary Search (only used on ordered lists) compare target to middle element; discard low or high half of list and repeat on remaining half of list until found or list is empty

9 The Linear Search Algorithm integer LinearSearch ( x: integer, a1, a2,..., an: distinct integers) i := 1 while (i <= n and x!= ai) i := i + 1 if i <= n then return i else return 0 { the value returned is the subscript of term that equals x, or is 0 if x is not found } Note: it doesn t matter if the list is ordered or not, this algorithm will still work

10 The Binary Search Algorithm integer BinarySearch( x: integer, a1, a2,..., an: increasing integers) i := 1 {left endpoint of search interval} j := n {right endpoint of search interval} while i < j begin m := floor[(i + j) / 2] if x > am then i := m + 1 else j := m end if x = ai then return i else return 0 Notes: the value returned is the subscript of term that equals x, or is 0 if x is not found; can only be used on sorted list

11 Bubble Sort Algorithm procedure Bubblesort (a1,..., an: real numbers with n >= 2) for i := 1 to n-1 for j := 1 to n - i if a(j) > a(j+1) then swap a(j) and a(j+1) { a1,..., an is in increasing order } Other Sorts: Selection Sort Insertion Sort Merge Sort Quick Sort Bucket Sort Radix Sort

12 3.2 Growth of Functions is a measure of the computational steps of an algorithm relative to the size of input are analyzed to see how the number of computational steps grows in relation to the size of input, n Once we have a function to compute the time complexity of algorithms which solve the same problem, we can compare them to determine which is more efficient For example, if the time it takes one sorting algorithm to sort n values is T 1 (n) = 3 2 n2 + 3n and another takes time T 2 (n) = 5n log n + 29 which algorithm should we implement?

13 Comparing Function Growth Quantify the concept: g grows at least as fast as f What really matters when comparing the complexity of algorithms? We mostly care about the behavior for large problems (i.e., what happens for sufficiently large input sizes) Even bad algorithms can be used to solve sufficiently small problems We can ignore some implementation details such as loop counter incrementation we can straight line any loop, etc. Remember, the functions we re discussing represent the time complexities of algorithms.

14 f O(g) time g(x) f(x) k size of input

15 Big Oh Notation g asymptotically dominates f : Let f and g be functions from N to R. Then f O(g) f is Big Oh of g or f is order g iff k C n[n > k f (n) C g(n), k, C > 0] In English: for sufficiently large n, if the function f is bounded from above by a positive, constant multiple of the function g, then we say f is Big Oh of g. If lim n f (n) g(n) = 0 then f o(g) (f is Little Oh of g, or f is strictly bounded by g)

16 Proving Asymptotic Domination time g(x) f(x) k size of input To prove f O(g), given f and g: Determine / choose some positive k Determine / choose a positive C (which may depend upon choice of k) Once k and C are chosen, the implication must be proven true

17 Three Important Classes Bounds from above Big Oh Growth rates equivalent Theta Bounds from below Big Omega Omega Man

18 Classes The sets O(g), o(g), Ω(g), ω(g), and Θ(g) are called complexity classes. O(g) is a set which contains all the functions which g dominates. f is O(g) means f O(g) We say f Ω(g) if there are positive constants k and C such that f (n) Cg(n) whenever n > k If f O(g) and f Ω(g), then f Θ(g) We use little-oh and little omega when we have strict inequality

19 Let f (n) = 4n + 5 and g(n) = n 2. We wish to show that f O(g) We need to find constants k and C, then show the implication n > k, f (n) Cg(n) is true for the values we chose.

20 To find k, we can set the functions equal, and solve for n: 4n + 5 = n 2 0 = n 2 4n 5 0 = (n 5)(n + 1) So, n = 5 or n = 1, but n is the size of input and therefore cannot be negative. Thus, k must be at least 5. If we choose k = 6, then C can be any positive number greater than or equal to 1. All that is left is the proof that n > k, f (n) Cg(n), which we shall revisit when we discuss induction proofs.

21 Big Oh f is O(g) iff O(f ) O(g) If f O(g) and g O(f ), then O(f ) = O(g) The set O(g) is closed under addition: If f O(g) and h O(g), then f + h O(g)

22 O(g) is closed under multiplication by a scalar a R: If f O(g) then af O(g) I.e., O(g) is a vector space Also, as you would expect, If f O(g) and g O(h), then f O(h) In particular, O(f ) O(g) O(h)

23 Theorem If f 1 O(g 1 ) and f 2 O(g 2 ), then: 1. f 1 f 2 O(g 1 g 2 ) 2. f 1 + f 2 O(max{g 1, g 2 })

24 Functional Values for Small n log 2 n n n n 2 2 n n! n n , , , ,

25 Approximate Functional Values for Powers of n log 2 n n n n 2 2 n n! n n (10 6 ) (10 30 ) 9.3( ) ( ) 4( ) ( ) 2.9(10 35,659 ) 10 40, , (10 456,573 ) , , (10 5,565,708 ) 10 60,000, BIG LARGE HUGE

26 Important Classes Theorem. The hierarchy of several familiar sequences in the sense that each sequence is Big Oh of any sequence to its right: 1, log 2 n,..., 4 n, 3 n, n, n, n log 2 n, n n, n 2, n 3, n 4,..., 2 n, n!, n n Similarly, stated in set notation: O(1) O(log n) O(n) O(n log n) O(n 2 ) O(n j ) O(c n ) O(n!) where j > 2 and c > 1

27 Equivalences Second Milliseconds 1, Microseconds 1,000, Nanoseconds 1,000,000,

28 Largest Problem Sizes Let f (n) be the time complexity of an algorithm in microseconds. The largest problem of size n that can be solved in: 1 second is f (n)/ minute is f (n)/( ) 1 hour is f (n)/( ) 1 day is f (n)/( ) 1 month is f (n)/( ) 1 year is f (n)/( ) 1 century is f (n)/( )

29 Largest Problems Do able in 1 Second 1. Let f (n) = n. Then the largest problem for which we can compute an answer in one second is: n/10 6 = 1 n = Let f (n) = n 2. Then the largest problem for which we can compute an answer in one second is: n 2 /10 6 = 1 n = 10 6 = Let f (n) = 2 n. Then the largest problem for which we can compute an answer in one second is: 2 n /10 6 = 1 2 n = n 19

30 Let f (n) = n!. Then the largest problem for which we can compute an answer in one second is: n!/10 6 = 1 n! = 10 6 Here, it helps to use a calculator..., and we find 9! = 362, 880 too small 10! = 3, 628, 880 too large n 9 (Recall that n is input size)

31 Exponents x a x b = x a+b = 2 12 x a a b 38 = x x b 3 = (x a ) b = x ab (5 2 ) 3 = 5 6 Notes x n + x n = 2x n not x 2n = 2(3 2 ) = = 2 n + 2 n = = = 2 n + 3 n =

32 Logarithms Logarithm: x a = b iff log x b = a 2 3 = 8 iff log 2 8 = = 625 iff log Theorem. log ab = = iff log 3 81 = 4 log a + log b log 32 = log(2 5 ) = 5 = log(8 4) = log 8 + log 4 = log log 2 2 = = 5

33 Other Formulae You Should Know log a b = log a log b log 32 4 = log = log = log 2 3 = 3 and log 32 4 = log 32 log 4 = log 2 5 log 2 2 = 5 2 = 3 Determine log both ways shown above log a b = b log a log 16 3 = log(2 4 ) 3 = log 2 12 = 12 and log 16 3 = 3 log 16 = 3 log 2 4 = 3 4 = 12 Determine log both ways shown above

34 Other General Knowledge x > 0, log x < x n > 0, log n < n log 1 = 0, log 2 = 1, log 1024 = 10, log 1, 048, 576 = 20 n i=0 2i = 2 n+1 1 Thus, if n = 3: 3 i=0 2i = = = 15 and 2 n+1 1 = = = 16 1 = 15

35 In general: n i=0 ai = an+1 1 a 1 n i=1 i = n(n+1) 2 n2 2 Thus, 5 i=1 i = = 15 and 5 i=1 i = 5(5+1) 2 = = 30 2 = 15