18 Algorthms Exercses 0.1. In each of the followng stuatons, ndcate whether f = O(g), or f = Ω(g), or both (n whch case f = Θ(g)). f(n) g(n) (a) n 100 n 200 (b) n 1/2 n 2/3 (c) 100n + log n n + (log n) 2 (d) n log n 10n log 10n (e) log 2n log 3n (f) 10 log n log(n 2 ) (g) n 1.01 n log 2 n (h) n 2 / log n n(log n) 2 () n 0.1 (log n) 10 (j) (log n) log n n/ log n (k) n (log n) 3 (l) n 1/2 5 log 2 n (m) n2 n 3 n (n) 2 n 2 n+1 (o) n! 2 n (p) (log n) log n 2 (log 2 n)2 (q) n =1 k n k+1 0.2. Show that, f c s a postve real number, then g(n) = 1 + c + c 2 + + c n s: (a) Θ(1) f c < 1. (b) Θ(n) f c = 1. (c) Θ(c n ) f c > 1. The moral: n bg-θ terms, the sum of a geometrc seres s smply the frst term f the seres s strctly decreasng, the last term f the seres s strctly ncreasng, or the number of terms f the seres s unchangng. 0.3. The Fbonacc numbers F 0, F 1, F 2,..., are defned by the rule F 0 = 0, F 1 = 1, F n = F n 1 + F n 2. In ths problem we wll confrm that ths sequence grows exponentally fast and obtan some bounds on ts growth. (a) Use nducton to prove that F n 2 0.5n for n 6. (b) Fnd a constant c < 1 such that F n 2 cn for all n 0. Show that your answer s correct. (c) What s the largest c you can fnd for whch F n = Ω(2 cn )? 0.4. Is there a faster way to compute the nth Fbonacc number than by fb2 (page 13)? One dea nvolves matrces. We start by wrtng the equatons F 1 = F 1 and F 2 = F 0 + F 1 n matrx notaton: ( ) ( ) ( ) F1 0 1 F0 =. F 2 1 1 F 1
S. Dasgupta, C.H. Papadmtrou, and U.V. Vazran 83 Exercses 2.1. Use the dvde-and-conquer nteger multplcaton algorthm to multply the two bnary ntegers 10011011 and 10111010. 2.2. Show that for any postve ntegers n and any base b, there must some power of b lyng n the range [n, bn]. 2.3. Secton 2.2 descrbes a method for solvng recurrence relatons whch s based on analyzng the recurson tree and dervng a formula for the work done at each level. Another (closely related) method s to expand out the recurrence a few tmes, untl a pattern emerges. For nstance, let s start wth the famlar T (n) = 2T (n/2) + O(n). Thnk of O(n) as beng cn for some constant c, so: T (n) 2T (n/2) + cn. By repeatedly applyng ths rule, we can bound T (n) n terms of T (n/2), then T (n/4), then T (n/8), and so on, at each step gettng closer to the value of T ( ) we do know, namely T (1) = O(1). T (n) 2T (n/2) + cn A pattern s emergng... the general term s 2[2T (n/4) + cn/2] + cn = 4T (n/4) + 2cn 4[2T (n/8) + cn/4] + 2cn = 8T (n/8) + 3cn 8[2T (n/16) + cn/8] + 3cn = 16T (n/16) + 4cn. T (n) 2 k T (n/2 k ) + kcn. Pluggng n k = log 2 n, we get T (n) nt (1) + cn log 2 n = O(n log n). (a) Do the same thng for the recurrence T (n) = 3T (n/2) + O(n). What s the general kth term n ths case? And what value of k should be plugged n to get the answer? (b) Now try the recurrence T (n) = T (n 1) + O(1), a case whch s not covered by the master theorem. Can you solve ths too? 2.4. Suppose you are choosng between the followng three algorthms: Algorthm A solves problems by dvdng them nto fve subproblems of half the sze, recursvely solvng each subproblem, and then combnng the solutons n lnear tme. Algorthm B solves problems of sze n by recursvely solvng two subproblems of sze n 1 and then combnng the solutons n constant tme. Algorthm C solves problems of sze n by dvdng them nto nne subproblems of sze n/3, recursvely solvng each subproblem, and then combnng the solutons n O(n 2 ) tme. What are the runnng tmes of each of these algorthms (n bg-o notaton), and whch would you choose? 2.5. Solve the followng recurrence relatons and gve a Θ bound for each of them. (a) T (n) = 2T (n/3) + 1 (b) T (n) = 5T (n/4) + n
84 Algorthms (c) T (n) = 7T (n/7) + n (d) T (n) = 9T (n/3) + n 2 (e) T (n) = 8T (n/2) + n 3 (f) T (n) = 49T (n/25) + n 3/2 log n (g) T (n) = T (n 1) + 2 (h) T (n) = T (n 1) + n c, where c 1 s a constant () T (n) = T (n 1) + c n, where c > 1 s some constant (j) T (n) = 2T (n 1) + 1 (k) T (n) = T ( n) + 1 2.6. A lnear, tme-nvarant system has the followng mpulse response: b(t) 1/t 0 t 0 t (a) Descrbe n words the effect of ths system. (b) What s the correspondng polynomal? 2.7. What s the sum of the nth roots of unty? What s ther product f n s odd? If n s even? 2.8. Practce wth the fast Fourer transform. (a) What s the FFT of (1, 0, 0, 0)? What s the approprate value of ω n ths case? And of whch sequence s (1, 0, 0, 0) the FFT? (b) Repeat for (1, 0, 1, 1). 2.9. Practce wth polynomal multplcaton by FFT. (a) Suppose that you want to multply the two polynomals x + 1 and x 2 + 1 usng the FFT. Choose an approprate power of two, fnd the FFT of the two sequences, multply the results componentwse, and compute the nverse FFT to get the fnal result. (b) Repeat for the par of polynomals 1 + x + 2x 2 and 2 + 3x. 2.10. Fnd the unque polynomal of degree 4 that takes on values p(1) = 2, p(2) = 1, p(3) = 0, p(4) = 4, and p(5) = 0. Wrte your answer n the coeffcent representaton. 2.11. In justfyng our matrx multplcaton algorthm (Secton 2.5), we clamed the followng blockwse property: f X and Y are n n matrces, and [ ] [ ] A B E F X =, Y =. C D G H
S. Dasgupta, C.H. Papadmtrou, and U.V. Vazran 85 where A, B, C, D, E, F, G, and H are n/2 n/2 submatrces, then the product XY can be expressed n terms of these blocks: [ ] [ ] [ ] A B E F AE + BG AF + BH XY = = C D G H CE + DG CF + DH Prove ths property. 2.12. How many lnes, as a functon of n (n Θ( ) form), does the followng program prnt? Wrte a recurrence and solve t. You may assume n s a power of 2. functon f(n) f n > 1: prnt_lne( stll gong ) f(n/2) f(n/2) 2.13. A bnary tree s full f all of ts vertces have ether zero or two chldren. Let B n denote the number of full bnary trees wth n vertces. (a) By drawng out all full bnary trees wth 3, 5, or 7 vertces, determne the exact values of B 3, B 5, and B 7. Why have we left out even numbers of vertces, lke B 4? (b) For general n, derve a recurrence relaton for B n. (c) Show by nducton that B n s Ω(2 n ). 2.14. You are gven an array of n elements, and you notce that some of the elements are duplcates; that s, they appear more than once n the array. Show how to remove all duplcates from the array n tme O(n log n). 2.15. In our medan-fndng algorthm (Secton 2.4), a basc prmtve s the splt operaton, whch takes as nput an array S and a value v and then dvdes S nto three sets: the elements less than v, the elements equal to v, and the elements greater than v. Show how to mplement ths splt operaton n place, that s, wthout allocatng new memory. 2.16. You are gven an nfnte array A[ ] n whch the frst n cells contan ntegers n sorted order and the rest of the cells are flled wth. You are not gven the value of n. Descrbe an algorthm that takes an nteger x as nput and fnds a poston n the array contanng x, f such a poston exsts, n O(log n) tme. (If you are dsturbed by the fact that the array A has nfnte length, assume nstead that t s of length n, but that you don t know ths length, and that the mplementaton of the array data type n your programmng language returns the error message whenever elements A[] wth > n are accessed.) 2.17. Gven a sorted array of dstnct ntegers A[1,..., n], you want to fnd out whether there s an ndex for whch A[] =. Gve a dvde-and-conquer algorthm that runs n tme O(log n). 2.18. Consder the task of searchng a sorted array A[1... n] for a gven element x: a task we usually perform by bnary search n tme O(log n). Show that any algorthm that accesses the array only va comparsons (that s, by askng questons of the form s A[] z? ), must take Ω(log n) steps. 2.19. A k-way merge operaton. Suppose you have k sorted arrays, each wth n elements, and you want to combne them nto a sngle sorted array of kn elements.
86 Algorthms (a) Here s one strategy: Usng the merge procedure from Secton 2.3, merge the frst two arrays, then merge n the thrd, then merge n the fourth, and so on. What s the tme complexty of ths algorthm, n terms of k and n? (b) Gve a more effcent soluton to ths problem, usng dvde-and-conquer. 2.20. Show that any array of ntegers x[1... n] can be sorted n O(n + M) tme, where M = max x mn x. For small M, ths s lnear tme: why doesn t the Ω(n log n) lower bound apply n ths case? 2.21. Mean and medan. One of the most basc tasks n statstcs s to summarze a set of observatons {x 1, x 2,..., x n } R by a sngle number. Two popular choces for ths summary statstc are: The medan, whch we ll call µ 1 The mean, whch we ll call µ 2 (a) Show that the medan s the value of µ that mnmzes the functon x µ. You can assume for smplcty that n s odd. (Hnt: Show that for any µ µ 1, the functon decreases f you move µ ether slghtly to the left or slghtly to the rght.) (b) Show that the mean s the value of µ that mnmzes the functon (x µ) 2. One way to do ths s by calculus. Another method s to prove that for any µ R, (x µ) 2 = (x µ 2 ) 2 + n(µ µ 2 ) 2. Notce how the functon for µ 2 penalzes ponts that are far from µ much more heavly than the functon for µ 1. Thus µ 2 tres much harder to be close to all the observatons. Ths mght sound lke a good thng at some level, but t s statstcally undesrable because just a few outlers can severely throw off the estmate of µ 2. It s therefore sometmes sad that µ 1 s a more robust estmator than µ 2. Worse than ether of them, however, s µ, the value of µ that mnmzes the functon max x µ. (c) Show that µ can be computed n O(n) tme (assumng the numbers x are small enough that basc arthmetc operatons on them take unt tme). 2.22. You are gven two sorted lsts of sze m and n. Gve an O(log m + log n) tme algorthm for computng the kth smallest element n the unon of the two lsts. 2.23. An array A[1... n] s sad to have a majorty element f more than half of ts entres are the same. Gven an array, the task s to desgn an effcent algorthm to tell whether the array has a majorty element, and, f so, to fnd that element. The elements of the array are not necessarly from some ordered doman lke the ntegers, and so there can be no comparsons of the form s A[] > A[j]?. (Thnk of the array elements as GIF fles, say.) However you can answer questons of the form: s A[] = A[j]? n constant tme.
88 Algorthms 2.26. Professor F. Lake tells hs class that t s asymptotcally faster to square an n-bt nteger than to multply two n-bt ntegers. Should they beleve hm? 2.27. The square of a matrx A s ts product wth tself, AA. (a) Show that fve multplcatons are suffcent to compute the square of a 2 2 matrx. (b) What s wrong wth the followng algorthm for computng the square of an n n matrx? Use a dvde-and-conquer approach as n Strassen s algorthm, except that nstead of gettng 7 subproblems of sze n/2, we now get 5 subproblems of sze n/2 thanks to part (a). Usng the same analyss as n Strassen s algorthm, we can conclude that the algorthm runs n tme O(n log 2 5 ). (c) In fact, squarng matrces s no easer than matrx multplcaton. In ths part, you wll show that f n n matrces can be squared n tme S(n) = O(n c ), then any two n n matrces can be multpled n tme O(n c ).. Gven two n n matrces A and B, show that the matrx AB + BA can be computed n tme 3S(n) + O(n 2 ).. Gven two n n matrces X and Y, defne the 2n 2n matrces A and B as follows: A = [ X 0 0 0 ] and B = [ 0 Y 0 0 What s AB + BA, n terms of X and Y?. Usng () and (), argue that the product XY can be computed n tme 3S(2n) + O(n 2 ). Conclude that matrx multplcaton takes tme O(n c ). 2.28. The Hadamard matrces H 0, H 1, H 2,... are defned as follows: H 0 s the 1 1 matrx [ 1 ] For k > 0, H k s the 2 k 2 k matrx [ ] Hk 1 H H k = k 1 H k 1 H k 1 Show that f v s a column vector of length n = 2 k, then the matrx-vector product H k v can be calculated usng O(n log n) operatons. Assume that all the numbers nvolved are small enough that basc arthmetc operatons lke addton and multplcaton take unt tme. 2.29. Suppose we want to evaluate the polynomal p(x) = a 0 + a 1 x + a 2 x 2 + + a n x n at pont x. (a) Show that the followng smple routne, known as Horner s rule, does the job and leaves the answer n z. z = a n for = n 1 downto 0: z = zx + a (b) How many addtons and multplcatons does ths routne use, as a functon of n? Can you fnd a polynomal for whch an alternatve method s substantally better? 2.30. Ths problem llustrates how to do the Fourer Transform (FT) n modular arthmetc, for example, modulo 7. ].