The maximum-subarray problem Given an array of integers, find a contiguous subarray with the maximum sum. Very naïve algorithm: Brute force algorithm: At best, θ(n 2 ) time complexity 129
Can we do divide and conquer? Want to use answers from left and right half subarrays. Problem: The answer may not lie in either! Key question: What information do we need from (smaller) subproblems to solve the big problem? Related question: how do we get this information? 130
A divide and conquer algorithm Algorithm in Ch 4.1: Recurrence: T(1) = C, and for n>1 T(n) = 2T(n/2) + θ(n) T(n) = θ(n log n) 131
More divide and conquer : Merge Sort Divide: If S has at least two elements (nothing needs to be done if S has zero or one elements), remove all the elements from S and put them into two sequences, S 1 and S 2, each containing about half of the elements of S. (i.e. S 1 contains the first n/2 elements and S 2 contains the remaining n/2 elements). Conquer: Sort sequences S 1 and S 2 using Merge Sort. Combine: Put back the elements into S by merging the sorted sequences S 1 and S 2 into one sorted sequence 132
Merge Sort: Algorithm Merge-Sort(A, p, p, r) r) if if p < r then then q(p+r)/2 Merge-Sort(A, p, p, q) q) Merge-Sort(A, q+1, q+1, r) r) Merge(A, p, p, q, q, r) r) Merge(A, p, p, q, q, r) r) Take Take the the smallest of of the the two two topmost elements of of sequences A[p..q] and and A[q+1..r] and and put put into into the the resulting sequence. Repeat this, this, until until both both sequences are are empty. Copy Copy the the resulting sequence into into A[p..r]. 133
Merge Sort: example 134
Merge Sort: example 135
Merge Sort: example 136
Merge Sort: example 137
Merge Sort: example 138
Merge Sort: example 139
Merge Sort: example 140
Merge Sort: example 141
Merge Sort: example 142
Merge Sort: example 143
Merge Sort: example 144
Merge Sort: example 145
Merge Sort: example 146
Merge Sort: example 147
Merge Sort: example 148
Merge Sort: example 149
Merge Sort: example 150
Merge Sort: example 151
Merge Sort: example 152
Merge Sort: example 153
Merge Sort: example 154
Merge Sort: example 155
To sort n numbers Merge Sort: summary if n=1 done! recursively sort 2 lists of numbers n/2 and n/2 elements merge 2 sorted lists in (n) time Strategy break problem into similar (smaller) subproblems recursively solve subproblems combine solutions to answer 156
Recurrences Running times of algorithms with Recursive calls can be described using recurrences A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs Example: Merge Sort Tn ( ) solving_trivial_problem if n 1 num_pieces Tn ( / subproblem_size_factor) dividing combining if n1 Tn ( ) (1) if n 1 2 Tn ( /2) ( n) if n1 157
Solving recurrences Repeated substitution method Expanding the recurrence by substitution and noticing patterns Substitution method guessing the solutions verifying the solution by the mathematical induction Recursion-trees Master method templates for different classes of recurrences 158
Repeated Substitution Method Let s find the running time of merge sort (let s assume that n=2 b, for some b). T( n) 1 if n 1 2 T( n/2) n if n 1 T( n) 2 T n/ 2 n substitute 2 2 T n/ 4 n/ 2 n expand 2 2 T( n/ 4) 2 n substitute 2 2 (2 T( n/8) n/ 4) 2 n expand 3 2 T( n/8) 3 n observe the pattern T( n) i i 2 T( n/2 ) in lg n 2 T( n/ n) nlgn n nlgn 159
Repeated Substitution Method The procedure is straightforward: Substitute Expand Substitute Expand Observe a pattern and write how your expression looks after the i-th substitution Find out what the value of i (e.g., lg n) should be to get the base case of the recurrence (say T(1)) Insert the value of T(1) and the expression of i into your expression 160
Solve T( n) 4 T( n/ 2) n Substitution method 1) Guess that T( n) O( n ), i.e., that T of the form cn T k ck k n 3 2) Assume ( ) for / 2 and 3 3 3 3) Prove T( n) cn by induction T( n) 4 T( n/ 2) n (recurrence) 3 4c(n/2) n (ind. hypoth.) c n 3 n (simplify) 2 3 c 3 cn n n (rearrange) 2 3 cn if c 2 and n 1 (satisfy) 3 Thus T( n) O( n )! Subtlety: Must choose c big enough to handle T( n) (1) for n n for some n 0 0 161
Achieving tighter bounds Substitution method Tn 2 Try to show ( ) ( ) Assume Tk ( ) ck Tn ( ) 4 Tn ( /2) n 2 2 2 On 4( cn/2) cn cn 2 n n for no choice of c 0. 162
Substitution method The problem: We could not rewrite the equality as: Tn 2 ( ) cn+ (something positive) Tn ( ) in order to show the inequality we wanted Sometimes to prove inductive step, try to strengthen your hypothesis cn T(n) (answer you want) - (something > 0) 2 163
Substitution method Corrected proof: the idea is to strengthen the inductive hypothesis by subtracting lower-order terms! 2 Assume Tk ( ) ck 1 ck 2 for kn Tn ( ) 4 Tn ( /2) n 4( c ( n/ 2) c ( n/ 2)) n cn cnn cn cn( cnn) cn cnif c 1 2 1 2 2 1 2 2 2 1 2 2 2 1 2 2 164
Recursion Tree A recursion tree is a convenient way to visualize what happens when a recurrence is iterated Construction of a recursion tree Tn ( ) Tn ( /4) Tn ( /2) n 2 165
Recursion Tree 166
Recursion Tree Tn ( ) Tn ( /3) T(2 n/3) n 167
Master Method The idea is to solve a class of recurrences that have the form Tn ( ) atnb ( / ) fn ( ) a 1 and b > 1, and f is asymptotically positive! Abstractly speaking, T(n) is the runtime for an algorithm and we know that a subproblems of size n/b are solved recursively, each in time T(n/b) f(n) is the cost of dividing the problem and combining the results. In merge-sort Tn ( ) 2 Tn ( /2) ( n) 168
Master method Split problem into a parts at log b n logbn logba levels. There are a n leaves 169
Master method Number of leaves: Iterating the recurrence, expanding the tree yields a log b n n log Tn ( ) fn ( ) atnb ( / ) Thus, b a f n af n b a T n b 2 2 ( ) ( / ) ( / ) f n af n b a T n b 2 2 ( ) ( / ) ( / )... logbn1 logbn1 logbn a f n b a T logb n1 j j logb a ( ) ( / ) ( ) Tn afnb n j0 ( / ) (1) The first term is a division/recombination cost (totaled across all levels of the tree) log b a The second term is the cost of doing all n subproblems of size 1 (total of all work pushed to leaves) 170
Master method intuition Three common cases: Running time dominated by cost at leaves Running time evenly distributed throughout the tree Running time dominated by cost at root Consequently, to solve the recurrence, we need only to characterize the dominant term logb a In each case compare f( n) with On ( ) 171
Master method Case 1 logb a for some constant f( n) O( n ) f(n) grows polynomially (by factor n ) slower than log b a n The work at the leaf level dominates logb a Summation of recursion-tree levels On ( ) logb a Cost of all the leaves ( n ) logb a ( n ) Thus, the overall cost 0 172
logb a f ( n) ( n lg n) log f ( n) b a n Master method Case 2 and are asymptotically the same The work is distributed equally logb a throughout the tree Tn ( ) ( n lg n) (level cost) (number of levels) 173
Master method Case 3 logb a for some constant f( n) ( n ) Inverse of Case 1 f(n) grows polynomially faster than Also need a regularity condition The work at the root dominates log b a n c1 and n 0 such that af( n/ b) cf( n) nn Tn ( ) ( fn ( )) 0 0 0 174
Master Theorem Summarized Given a recurrence of the form 1. f( n) O n Tn ( ) 2. f( n) n log log log logb a Tn ( ) n lgn Tn ( ) atnb ( / ) fn ( ) logb a 3. ( ) and ( / ) ( ), for some 1, Tn ( ) fn ( ) b n b a a b a f n n af n b cf n c nn The master method cannot solve every recurrence of this form; there is a gap between cases 1 and 2, as well as cases 2 and 3 0 175
Using the Master Theorem Extract a, b, and f(n) from a given recurrence Determine log b a n Compare f(n) and asymptotically Determine appropriate MT case, and apply Example merge sort log b a n Tn ( ) 2 Tn ( /2) ( n) a b n n n n Also f( n) ( n) log b a log 2 2, 2; 2 ( ) log b a Case 2 : Tn ( ) n lgn nlgn 176
Examples Tn () Tn (/2) 1 a b n log 21 1, 2; 1 also fn ( ) 1, fn ( ) (1) Case 2: Tn () (lg) n Tn ( ) 9 Tn ( /3) n a9, b3; fnnfnon log 39 ( ), ( ) ( ) with 1 Cas e 1: Tn () n 2 Binary-search(A, p, p, r, r, s): s): q(p+r)/2 if if A[q]=s then then return q else else if if A[q]>s then then Binary-search(A, p, p, q-1, q-1, s) s) else else Binary-search(A, q+1, q+1, r, r, s) s) 177
Examples Tn ( ) 3 Tn ( /4) nlgn 3, 4; 4 a b n n log 3 0.793 log4 3 ( ) lg, ( ) ( ) with 0.2 f n n n f n n Case 3: Regularity condition af ( n / b) 3( n /4)lg( n /4) (3/4) n lg n cf ( n) for c 3/ 4 Tn ( ) ( nlg n) Tn ( ) 2 Tn ( /2) nlgn log 2 1 2, 2; 1 f ( n) nlg n, f( n) ( n ) with? 2 a b n n 1 also nlg n/ n lgn neither Case 3 nor Case 2! 178
Examples T( n) 4 T( n/2) n 4, 2; 2 a b n n 3 log 4 2 3 2 ( ) ; ( ) ( ) f n n f n n Case 3: 3 3 3 3 T( n) Checking the regularity condition 4 f( n/ 2) cf( n) 4 n /8 cn n /2 cn c 3/41 n 3 179
A quick review of logarithms Properties to remember 1. log (ab) = log a + log b 2. log (a/b) = log a - log b 3. log (1/a) = - log a 4. log a n = n log a 5. a = 2 log 2 a It follows that : 1. n n = 2 n log 2 n 2. 2 n n = 2 n + log 2 n 3. n log 2 n = 2 (log 2 n)2 180
Next 1. Covered basics of a simple design technique (Divideand-conquer) Ch. 4 of the text. 2. Next, more sorting algorithms. 181