Data Structures and Algorithms CSE 465 LECTURE 3 Asymptotic Notation O-, Ω-, Θ-, o-, ω-notation Divide and Conquer Merge Sort Binary Search Sofya Raskhodnikova and Adam Smith /5/0
Review Questions If input length doubles, by roughly how much does the worst-case running time of Insertion Sort increase? (Answer: 4) How long does Insertion Sort run on the input,,4,3,6,5,,n, n (as a function of n)? (Answer: Θ(n). Inner insertion loop never has to look more than one position down the list to find the right spot.) /5/0
Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there exist constants c > 0, n 0 > 0 such that 0 f(n) cg(n) for all n n 0. EXAMPLE: n = O(n 3 ) functions, not values (c =, n 0 = ) funny, one-way equality /5/0
Set definition of O-notation O(g(n)) = { f(n) : there exist constants c > 0, n 0 > 0 such that 0 f(n) cg(n) for all n n 0 } EXAMPLE: n O(n 3 ) (Logicians: λn.n O(λn.n 3 ), but it s convenient to be sloppy, as long as we understand what s really going on.) /5/0
Macro substitution Convention: A set in a formula represents an anonymous function in the set. EXAMPLE: (right-hand side) f(n) = n 3 + O(n ) means f(n) = n 3 + h(n) for some h(n) O(n ). /5/0
Macro substitution Convention: A set in a formula represents an anonymous function in the set. EXAMPLE: (left-hand side) n + O(n) = O(n ) means for any f(n) O(n): n + f(n) = h(n) for some h(n) O(n ). /5/0
Ω-notation (lower bounds) O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n ). Ω(g(n)) = { f(n) : there exist constants c > 0, n 0 > 0 such that 0 cg(n) f(n) for all n n 0 } EXAMPLE: n = Ω(lg n) (c =, n 0 = 6) /5/0
Θ-notation (tight bounds) Θ(g(n)) = O(g(n)) Ω(g(n)) EXAMPLE: n n = Θ( n ) Polynomials are simple: a d n d + a d n d + + a n + a 0 = Θ(n d ) /5/0
ο-notation and ω-notation O-notation and Ω-notation are like and. o-notation and ω-notation are like < and >. ο(g(n)) = { f(n) : for any constant c > 0, there is a constant n 0 > 0 such that 0 f(n) < cg(n) for all n n 0 } EXAMPLE: n = o(n 3 ) (n 0 = /c) /5/0
ο-notation and ω-notation O-notation and Ω-notation are like and. o-notation and ω-notation are like < and >. ω(g(n)) = { f(n) : for any constant c > 0, there is a constant n 0 > 0 such that 0 cg(n) < f(n) for all n n 0 } EXAMPLE: n = ω (lg n) (n 0 = +/c) /5/0
Summary Notation means Think E.g. Lim f(n)/g(n) f(n)=o(n) c>0, n 0 >0, n > n 0 : 0 f(n) < cg(n) Upper bound 00n = O(n 3 ) If it exists, it is < f(n)=ω(g(n)) c>0, n 0 >0, n > n 0 : 0 cg(n) < f(n) Lower bound n 00 = Ω( n ) If it exists, it is > 0 f(n)=θ(g(n)) both of the above: f=ω(g) and f = O(g) Tight bound = log(n!) = Θ(n log n) If it exists, it is > 0 and < f(n)=o(g(n)) c>0, n 0 >0, n > n 0 : 0 f(n) < cg(n) < n = o( n ) Limit exists, =0 f(n)=ω(g(n)) c>0, n 0 >0, n > n 0 : 0 cg(n) < f(n) > n = ω(log n) Limit exists, = /5/0
log(n) n n n log(n) n n,000,000 Some functions sorted by asymptotic growth n (beats n k for any fixed k) n! /5/0
The divide-and-conquer design paradigm. Divide the problem (instance) into subproblems.. Conquer the subproblems by solving them recursively. 3. Combine subproblem solutions. We ll see lots of examples /5/0
A faster sort: Merge Sort MERGE-SORT A[.. n]. If n =, done.. Recursively sort A[.. n/ ] and A[ n/+.. n ]. 3. Merge the sorted lists. Key subroutine: MERGE /5/0
A faster sort: Merge Sort input A[.. n] A[.. n/ ] A[n/+.. n] MERGE SORT MERGE SORT Sorted A[.. n/ ] Sorted A[n/+.. n] MERGE output /5/0
Merging two sorted arrays /5/0
Merging two sorted arrays /5/0
Merging two sorted arrays /5/0
Merging two sorted arrays /5/0
Merging two sorted arrays /5/0
Merging two sorted arrays /5/0
Merging two sorted arrays /5/0
Merging two sorted arrays /5/0
Merging two sorted arrays /5/0
Merging two sorted arrays /5/0
Merging two sorted arrays /5/0
Merging two sorted arrays /5/0
Merging two sorted arrays /5/0 Time = one pass through each array = Θ(n) to merge a total of n elements (linear time).
Analyzing Merge Sort Abuse T(n) Θ() T(n/) Θ(n) MERGE-SORT A[.. n]. If n =, done.. Recursively sort A[.. n/ ] and A[ n/+.. n ]. 3. Merge the sorted lists Sloppiness: Should be T( n/ ) + T( n/ ), but it turns out not to matter asymptotically. /5/0
Recurrence for Merge Sort T(n) = Θ() if n = ; T(n/) + Θ(n) if n >. We usually omit stating the base case because our algorithms always run in time Θ() when n is a small constant. CLRS and next lectures provide several ways to find a good upper bound on T(n). /5/0
Recursion tree Solve T(n) = T(n/) + cn, where c > 0 is constant. /5/0
Recursion tree Solve T(n) = T(n/) + cn, where c > 0 is constant. T(n) /5/0
Recursion tree Solve T(n) = T(n/) + cn, where c > 0 is constant. cn T(n/) T(n/) /5/0
Recursion tree Solve T(n) = T(n/) + cn, where c > 0 is constant. cn cn/ cn/ T(n/4) T(n/4) T(n/4) T(n/4) /5/0
Recursion tree Solve T(n) = T(n/) + cn, where c > 0 is constant. cn cn cn/ cn/ cn h = lg n cn/4 cn/4 cn/4 cn/4 cn Θ() #leaves = n Θ(n) Total = Θ(n lg n) /5/0
Merge Sort vs Insertion Sort Θ(n lg n) grows more slowly than Θ(n ). Therefore, Merge Sort asymptotically beats Insertion Sort in the worst case. In practice, Merge Sort beats Insertion Sort for n > 30 or so But how does Merge Sort do on nearly sorted inputs? Go test it out for yourself! /5/0