The maximum-subarray problem. Given an array of integers, find a contiguous subarray with the maximum sum. Very naïve algorithm:

Transcription

1 The maximum-subarray problem Given an array of integers, find a contiguous subarray with the maximum sum. Very naïve algorithm: Brute force algorithm: At best, θ(n 2 ) time complexity 129

2 Can we do divide and conquer? Want to use answers from left and right half subarrays. Problem: The answer may not lie in either! Key question: What information do we need from (smaller) subproblems to solve the big problem? Related question: how do we get this information? 130

3 A divide and conquer algorithm Algorithm in Ch 4.1: Recurrence: T(1) = C, and for n>1 T(n) = 2T(n/2) + θ(n) T(n) = θ(n log n) 131

4 More divide and conquer : Merge Sort Divide: If S has at least two elements (nothing needs to be done if S has zero or one elements), remove all the elements from S and put them into two sequences, S 1 and S 2, each containing about half of the elements of S. (i.e. S 1 contains the first n/2 elements and S 2 contains the remaining n/2 elements). Conquer: Sort sequences S 1 and S 2 using Merge Sort. Combine: Put back the elements into S by merging the sorted sequences S 1 and S 2 into one sorted sequence 132

5 Merge Sort: Algorithm Merge-Sort(A, p, p, r) r) if if p < r then then q(p+r)/2 Merge-Sort(A, p, p, q) q) Merge-Sort(A, q+1, q+1, r) r) Merge(A, p, p, q, q, r) r) Merge(A, p, p, q, q, r) r) Take Take the the smallest of of the the two two topmost elements of of sequences A[p..q] and and A[q+1..r] and and put put into into the the resulting sequence. Repeat this, this, until until both both sequences are are empty. Copy Copy the the resulting sequence into into A[p..r]. 133

6 Merge Sort: example 134

7 Merge Sort: example 135

8 Merge Sort: example 136

9 Merge Sort: example 137

10 Merge Sort: example 138

11 Merge Sort: example 139

12 Merge Sort: example 140

13 Merge Sort: example 141

14 Merge Sort: example 142

15 Merge Sort: example 143

16 Merge Sort: example 144

17 Merge Sort: example 145

18 Merge Sort: example 146

19 Merge Sort: example 147

20 Merge Sort: example 148

21 Merge Sort: example 149

22 Merge Sort: example 150

23 Merge Sort: example 151

24 Merge Sort: example 152

25 Merge Sort: example 153

26 Merge Sort: example 154

27 Merge Sort: example 155

28 To sort n numbers Merge Sort: summary if n=1 done! recursively sort 2 lists of numbers n/2 and n/2 elements merge 2 sorted lists in (n) time Strategy break problem into similar (smaller) subproblems recursively solve subproblems combine solutions to answer 156

29 Recurrences Running times of algorithms with Recursive calls can be described using recurrences A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs Example: Merge Sort Tn ( ) solving_trivial_problem if n 1 num_pieces Tn ( / subproblem_size_factor) dividing combining if n1 Tn ( ) (1) if n 1 2 Tn ( /2) ( n) if n1 157

30 Solving recurrences Repeated substitution method Expanding the recurrence by substitution and noticing patterns Substitution method guessing the solutions verifying the solution by the mathematical induction Recursion-trees Master method templates for different classes of recurrences 158

31 Repeated Substitution Method Let s find the running time of merge sort (let s assume that n=2 b, for some b). T( n) 1 if n 1 2 T( n/2) n if n 1 T( n) 2 T n/ 2 n substitute 2 2 T n/ 4 n/ 2 n expand 2 2 T( n/ 4) 2 n substitute 2 2 (2 T( n/8) n/ 4) 2 n expand 3 2 T( n/8) 3 n observe the pattern T( n) i i 2 T( n/2 ) in lg n 2 T( n/ n) nlgn n nlgn 159

32 Repeated Substitution Method The procedure is straightforward: Substitute Expand Substitute Expand Observe a pattern and write how your expression looks after the i-th substitution Find out what the value of i (e.g., lg n) should be to get the base case of the recurrence (say T(1)) Insert the value of T(1) and the expression of i into your expression 160

33 Solve T( n) 4 T( n/ 2) n Substitution method 1) Guess that T( n) O( n ), i.e., that T of the form cn T k ck k n 3 2) Assume ( ) for / 2 and ) Prove T( n) cn by induction T( n) 4 T( n/ 2) n (recurrence) 3 4c(n/2) n (ind. hypoth.) c n 3 n (simplify) 2 3 c 3 cn n n (rearrange) 2 3 cn if c 2 and n 1 (satisfy) 3 Thus T( n) O( n )! Subtlety: Must choose c big enough to handle T( n) (1) for n n for some n

34 Achieving tighter bounds Substitution method Tn 2 Try to show ( ) ( ) Assume Tk ( ) ck Tn ( ) 4 Tn ( /2) n On 4( cn/2) cn cn 2 n n for no choice of c

35 Substitution method The problem: We could not rewrite the equality as: Tn 2 ( ) cn+ (something positive) Tn ( ) in order to show the inequality we wanted Sometimes to prove inductive step, try to strengthen your hypothesis cn T(n) (answer you want) - (something > 0) 2 163

36 Substitution method Corrected proof: the idea is to strengthen the inductive hypothesis by subtracting lower-order terms! 2 Assume Tk ( ) ck 1 ck 2 for kn Tn ( ) 4 Tn ( /2) n 4( c ( n/ 2) c ( n/ 2)) n cn cnn cn cn( cnn) cn cnif c

37 Recursion Tree A recursion tree is a convenient way to visualize what happens when a recurrence is iterated Construction of a recursion tree Tn ( ) Tn ( /4) Tn ( /2) n 2 165

38 Recursion Tree 166

39 Recursion Tree Tn ( ) Tn ( /3) T(2 n/3) n 167

40 Master Method The idea is to solve a class of recurrences that have the form Tn ( ) atnb ( / ) fn ( ) a 1 and b > 1, and f is asymptotically positive! Abstractly speaking, T(n) is the runtime for an algorithm and we know that a subproblems of size n/b are solved recursively, each in time T(n/b) f(n) is the cost of dividing the problem and combining the results. In merge-sort Tn ( ) 2 Tn ( /2) ( n) 168

41 Master method Split problem into a parts at log b n logbn logba levels. There are a n leaves 169

42 Master method Number of leaves: Iterating the recurrence, expanding the tree yields a log b n n log Tn ( ) fn ( ) atnb ( / ) Thus, b a f n af n b a T n b 2 2 ( ) ( / ) ( / ) f n af n b a T n b 2 2 ( ) ( / ) ( / )... logbn1 logbn1 logbn a f n b a T logb n1 j j logb a ( ) ( / ) ( ) Tn afnb n j0 ( / ) (1) The first term is a division/recombination cost (totaled across all levels of the tree) log b a The second term is the cost of doing all n subproblems of size 1 (total of all work pushed to leaves) 170

43 Master method intuition Three common cases: Running time dominated by cost at leaves Running time evenly distributed throughout the tree Running time dominated by cost at root Consequently, to solve the recurrence, we need only to characterize the dominant term logb a In each case compare f( n) with On ( ) 171

44 Master method Case 1 logb a for some constant f( n) O( n ) f(n) grows polynomially (by factor n ) slower than log b a n The work at the leaf level dominates logb a Summation of recursion-tree levels On ( ) logb a Cost of all the leaves ( n ) logb a ( n ) Thus, the overall cost 0 172

45 logb a f ( n) ( n lg n) log f ( n) b a n Master method Case 2 and are asymptotically the same The work is distributed equally logb a throughout the tree Tn ( ) ( n lg n) (level cost) (number of levels) 173

46 Master method Case 3 logb a for some constant f( n) ( n ) Inverse of Case 1 f(n) grows polynomially faster than Also need a regularity condition The work at the root dominates log b a n c1 and n 0 such that af( n/ b) cf( n) nn Tn ( ) ( fn ( ))

47 Master Theorem Summarized Given a recurrence of the form 1. f( n) O n Tn ( ) 2. f( n) n log log log logb a Tn ( ) n lgn Tn ( ) atnb ( / ) fn ( ) logb a 3. ( ) and ( / ) ( ), for some 1, Tn ( ) fn ( ) b n b a a b a f n n af n b cf n c nn The master method cannot solve every recurrence of this form; there is a gap between cases 1 and 2, as well as cases 2 and

48 Using the Master Theorem Extract a, b, and f(n) from a given recurrence Determine log b a n Compare f(n) and asymptotically Determine appropriate MT case, and apply Example merge sort log b a n Tn ( ) 2 Tn ( /2) ( n) a b n n n n Also f( n) ( n) log b a log 2 2, 2; 2 ( ) log b a Case 2 : Tn ( ) n lgn nlgn 176

49 Examples Tn () Tn (/2) 1 a b n log 21 1, 2; 1 also fn ( ) 1, fn ( ) (1) Case 2: Tn () (lg) n Tn ( ) 9 Tn ( /3) n a9, b3; fnnfnon log 39 ( ), ( ) ( ) with 1 Cas e 1: Tn () n 2 Binary-search(A, p, p, r, r, s): s): q(p+r)/2 if if A[q]=s then then return q else else if if A[q]>s then then Binary-search(A, p, p, q-1, q-1, s) s) else else Binary-search(A, q+1, q+1, r, r, s) s) 177

50 Examples Tn ( ) 3 Tn ( /4) nlgn 3, 4; 4 a b n n log log4 3 ( ) lg, ( ) ( ) with 0.2 f n n n f n n Case 3: Regularity condition af ( n / b) 3( n /4)lg( n /4) (3/4) n lg n cf ( n) for c 3/ 4 Tn ( ) ( nlg n) Tn ( ) 2 Tn ( /2) nlgn log 2 1 2, 2; 1 f ( n) nlg n, f( n) ( n ) with? 2 a b n n 1 also nlg n/ n lgn neither Case 3 nor Case 2! 178

51 Examples T( n) 4 T( n/2) n 4, 2; 2 a b n n 3 log ( ) ; ( ) ( ) f n n f n n Case 3: T( n) Checking the regularity condition 4 f( n/ 2) cf( n) 4 n /8 cn n /2 cn c 3/41 n 3 179

52 A quick review of logarithms Properties to remember 1. log (ab) = log a + log b 2. log (a/b) = log a - log b 3. log (1/a) = - log a 4. log a n = n log a 5. a = 2 log 2 a It follows that : 1. n n = 2 n log 2 n 2. 2 n n = 2 n + log 2 n 3. n log 2 n = 2 (log 2 n)2 180

53 Next 1. Covered basics of a simple design technique (Divideand-conquer) Ch. 4 of the text. 2. Next, more sorting algorithms. 181