CS161: Algorithm Desig ad Aalysis Hadout #10 Staford Uiversity Wedesday, 10 February 2016 Lecture #11: Wedesday, 10 February 2016 Topics: Example midterm problems ad solutios from a log time ago Sprig 1998 Problem 1. [15 poits] Recurreces ad Asymptotics Give asymptotic solutios for the followig recurreces; argue that your solutio is correct. Assume T (1)=1. (a) T 1 ()=6T 1 ( /3 )+ 2 lg [2 poits] Usig the Master theorem we get a = 6 b = 3 f () = 2 lg log b a = log 3 6 Sice log 3 6 < 1.64, there exists > 0 (for example, = 0.1) such that f () = 2 lg = ( log 3 6+ ), ad so case 3 of the Master theorem seems to apply. But we must also check the regularity coditio: ca we fid a c < 1 such that ( af b) ( ) 2 = 6 lg 3 3 = 6 9 2 (lg lg3) = 2 3 2 lg 2 3 2 lg3 is o greater tha cf()=c 2 lg for all sufficietly large? Yes. If c = 2/3, the the regularity coditio is satisfied as log as 1. We coclude that T ()= ( 2 lg). Commo error: Settig log 3 6 = 2 Note that textbook refs here are to the first editio.
2 CS161: : Hadout #10 (b) T 2 ()=3T 2 ( /3 )+lg 3 [2 poits] Assumig is a power of 3, but without compromisig the geerality of our fial result as we observed i class, we use the iteratio method to get T 2 ()= 3T 2 (/3)+lg 3 = lg 3 + 3[ 3T 2 (/9)+/3 lg 3 (/3) ] = lg 3 + lg 3 (/3)+9[ 3T 2 (/27)+/9 lg 3 (/9) ] = lg 3 + lg 3 (/3)+lg 3 (/9)+ + lg 3 (/(/3)) + T 2 (1). The last sum i the above expressio cotais (lg) terms, each o greater tha lg 3, ad so T 2 ()=O(lg 4 ). O the other had, the first half of the lg 3 terms or, to be precise, the first m = log 3 2 terms are each at least as big as lg 3 =(lg 3 )/8 or, to be precise agai, at least as big as lg 3 3 m 1. So T 2 ()= (lg 4 ). Together we get T 2 ()= (lg 4 ). We ca also get this result directly usig the versio of the Master theorem give i class, ad appearig as exercise 4.4-2 o page 72 of CLR. It is also possible to solve this problem usig the tree method. However, we should be careful to show both a lower ad a upper asymptotic boud of lg 4. That is, arguig that the tree has depth at least log 3, with each level costig at most lg 3 successfully proves that T 2 ()=O(lg 4 ) but we must also show that T 2 ()= (lg 4 ) to coclude that T 2 ()= (lg 4 ). This ca be doe by cosiderig the cummulative cost of just half the tree levels, i a maer similar to our previous derivatio usig the iteratio method. Commo error: We caot apply the basic form of the Master theorem (page 63 of CLR) sice a = 3 b = 3 f () = lg 3 log b a = log 3 3 = ad, eve though lg 3 = (), it is ot ( 1+ ) for ay > 0. Why? Because if lg 3 = ( 1+ ), we could deduce that lg 3 = ( ), which cotradicts the formula o page 35 of CLR. So case 3 does ot apply. Commo error 2: Whe we obtai the summatio lg 3 + lg 3 (/3)+lg 3 (/9)+ + lg 3 (/(/3)) i the recurrece, this caot be approximated as O(lg 3 ). A simple way to aalyse this would be to observe that each term is O(lg 3 ) ad there are O(lg) such terms, yieldig a upper boud of O(lg 4 ).
CS161: : Hadout #10 3 (c) T 3 ()=T 3 ( /2 )+T 3 ( /4 )+ [2 poits] Usig the tree method we get the followig tree /2 /4 3/4 /4 /8 /8 /16 9/16 (3/4) i Note that the tree will be ubalaced ad that the subtree correspodig to T 3 ( /2 ) will be deeper (i.e. it will have more levels). At all levels after the shallow subtree the oe correspodig to T 3 ( /4 ) becomes exhausted, each level will cost o more tha ( 3 ) i 4. Sowehave T 3 () Now the top level costs, ad hece i=0 ( ) 3 i = O(). 4 T 3 () = (). Combiig the two, we cocude that T 3 ()= (). Commo error: Sice there are O(lg) levels, ad each level costs O(), the total cost is O(lg). The problem with this argumet is that it is loose. There are O(lg) levels, but the cost at each level decreases. I fact, the cost at level i is ( 3 i. 4) Summig up, we obtai the above boud of O(). (d) For each fuctios f () alog the left side of the table below ad each fuctio g() across the top, write O,,or i the appropriate box, depedig o whether f () =O(g()), f () = (g()), or f () = (g()). If there is more tha oe relatio betwee f () ad g(), write oly the strogest oe. The first lie is a demo solutio. [1 poit per etry]
4 CS161: : Hadout #10 f () g() lg lg 4 2 O T 1 () T 2 () T 3 () The aswers are: f () g() lg lg 4 T 1 ()= ( 2 lg) O T 2 ()= (lg 4 ) O T 3 ()= () O O Commo error: lg 3 is ot O(lg ). Problem 2. [18 (3 poits per questio)] Potpourri For each of the statemets below: circle whether is TRUE or FALSE Explai your aswer i oe or two seteces (a) A adversary places a bet that he ca select a iput so that whe you ru RANDOMIZED QUICK-SORT oce o his iput, the algorithm will take ( 2 ) steps. There is a positive (i.e, ozero) probability that the adversary will wi his bet. TRUE FALSE
CS161: : Hadout #10 5 The aswer is TRUE. I fact, the adversary does ot eve have to thik hard: ay iput s/he gives to radomized QUICK-SORT has a o-zero probability of forcig the algorithm through ( 2 ) steps. The reaso is that the ruig time of radomized QUICK-SORT depeds o the iteral, radom selectio of pivots; hece, it is ot impossible (i.e. there is a positive probability) that the selectio will be bad at every sigle recursive step, partitioig the array ito subproblems of sizes 1 ad 1. Also, if the iput cosists of repetitios of the same umber, the a QUICK-SORT which uses the PARTITION procedure show i class will defiitely take ( 2 ) steps, whether it s determiistic or radomized. (b) A max-based heap ca be trasformed ito a mi-based heap i liear time. The aswer is TRUE. I fact, ay array (i.e. ot ecessarily a heap) ca be tured ito a mi-based heap i liear time. The way to do this is to simply ru BUILD-HEAP o the array, which takes liear time. Here, of course, we are usig a BUILD-HEAP procedure based o a ew HEAPIFY which, i tur, guaratees that every paret is o greater tha its childre. Although it is possible to build a mi-based heap by extractig the elemets of the maxbased heap i reverse sorted order, this algorithm takes (lg ) steps. It is ot possible to covert a max-based heap ito a mi-based heap by reversig the elemets i the heap array. As a couterexample, cosider the heap stored as 5,1,4 This heap appears o the right side i the figure below. Reversig the cotets of the array produces 4,1,5 which is ot a mi-based heap sice the root, 4, is bigger that its left child, 1. (c) A preorder tree walk o a (max-)heap will output the values stored i the heap i reverse sorted order. The aswer is FALSE. Or, to be more precise, ot ecessarily. For example, the left heap i the figure below produces 5,4,1 durig a preorder walk, while the right heap produces 5,1,4.
6 CS161: : Hadout #10 5 5 4 1 1 4 Oe couterexample suffices to show the fallacy of the claim. Aother, more geeral way to prove that o tree walk (whether iorder, preorder, or postorder) ca output the odes i reverse sorted order is to simply observe that a heap guaratees oly that a paret is o smaller that its childre; there is o guaratee as to the relative orderig of the two childre. (d) Hashig with chaiig is preferable to ope addressig, if the load factor of the hash table is likely to get greater tha 1. The aswer is TRUE. If we have > 1 the the umber of elemets to be hashed is greater tha the umber of slots available, so we ca t use ope addressig. Eve whe does t exceed 1 but it is close to 1, chaiig is much better sice clusterig will make ope addressig very iefficiet. Studets who argued the iefficiecy of ope addressig for close to 1 but did t say that ope addressig fails for > 1 lost oly 1 poit. (e) If we kow the ordered lists of keys produced by a preorder ad a iorder traversal of a biary search tree T, the we have eough iformatio to fully recostruct T. The aswer is TRUE. I fact we do t eve eed the iorder output because we have all the iformatio we eed to recostruct the tree from the preorder output. (Actually, we could get the iorder output by sortig the preorder output, if we really wated to.) Now, sice a preorder traversal outputs the root of the tree before either subtree, the to recostruct the tree we oly eed to do the followig: 1. Iitialize the tree to be empty. 2. Isert the elemets i the order give by the preorder output. We ll get the origial tree because the root of the origial tree (ad every subtree recursively) is iserted ito the ew tree before ay of its origial subtrees. Thus the structure of the ew tree is exactly the same as that of the origial tree. Aother way to thik about it is the followig: imagie that every elemet we visit i a preorder traversal is deleted from the tree ad iserted i a secod (origially empty) tree. It will go exactly at the same (correspodig) positio from which it was deleted.
CS161: : Hadout #10 7 (f) Durig a Red/Black tree INSERT, (log) tree liks are modified i the worstcase. The aswer is FALSE. I the worst case may we eed to move all the way up the tree from the ew leaf to the root; however, the oly operatio that propagates this way is the COLOR-FLIP operatio that does ot modify tree liks. Balace ca always be restored at the ed by oe or two rotatios, each of which modifies oly three liks. Problem 3. [21 poits] No-comparative Sorts Let = 2 k 1. A array A[1,...,] cotais all k-bit strigs except oe. The oly operatio we ca use to access the array A is fetch the j-th bit of A[i], which takes costat time. Show that we ca determie the missig strig i O() time. I the first phase of the algorithm, we look at the least sigificat bit of each umber we are give (i.e. FETCH(x,1) for each umber A[i] a total of probes). Let S b, b {0,1}, be the set of umbers with b as their least sigificat bit. If S 1 < /2 the there must be at least oe umber i the rage [0,] which is ot i the sequece of give umbers ad which has a 1 as its least sigificat bit. Similarly, if S 0 < /2, the there must be at least oe umber i the rage [0,] which is ot i the sequece of give umbers ad which has a 0 as its least sigificat bit. We ow elimiate the umbers i the larger set sice these umbers all differ from the umber we are lookig for i the least sigificat bit, ad we record the value of the least sigificat bit. We ow proceed to probe the secod least sigificat bit of the remaiig set of umbers (the size of this set is less tha /2), form S 0 ad S 1 as above, record the secod least sigificat bit ad agai elimiate the larger set. We cotiue this process util we determie all the bits of the missig umber. That is, i the j-th phase of this algorithm, we look at the j-th least sigificat bit of the remaiig umbers (at most probes), ad elimiate at least half of these umbers for the ext phase. I short, we go through a total of k phases ad deduce the missig strig bit-by-bit; alteratively, we ca go through k 1 phases ad determie the missig strig by flippig the most sigificat bit of the sigle member of S 0 S 1 (i.e. the oe ad oly strig that survived all k 1 phases). Sice this algorithm elimiates at least half the sequece of umbers it looks at i each phase, it takes at most lg phases to fiish. I the j-th phase, the algorithm makes at most calls to the procedure 2 j 1 FETCH. Thus the total umber of probes is at most: lg j=0 lg 2 j = 1 2 j 2 j=0 Several other correct but (asymptotically) slower solutios are possible. They all ed up lookig at every bit of every strig, thus takig (k)= (lg ) time to complete; recall that, as = 2 k 1, k is by o meas a costat istead, k = (lg ). Examples iclude: Create a array B[0,...,] of booleas, all iitialized to FALSE. Go through all elemets of A, extractig each A[i] bit by bit ad iterpretig it as a biary umber; the set B[A[i]] to TRUE. 2 j 1
8 CS161: : Hadout #10 Fially, go through all elemets of IA, ad fid the oe, say it s B[ j], which is still FALSE; the missig strig is the biary represetatio of j. Perform k passes over A. Durig pass j, extract the jth bit of all the elemets of A, coutig separately the zeros ad the oes. If there are more (less) zeros tha oes, coclude the pass by settig the jth bit of the missig strig to oe (zero). Set the missig strig M to 0. Go through all elemets of A, extractig each A[i] bit by bit ad iterpretig it as a biary umber; the XOR this umber with M, storig the result back i M. Whe you are doe, M cotais the missig strig. Use a o-comparative sort to order the elemets of A, iterpreted as biary umbers, i icreasig order. The look at the least sigificat bits of adjacet elemets i the sorted array A. If you ever fid two idetical bits i adjacet elemets A[i] ad A[i + 1], this meas that the missig strig is A[i]+1. To cover the cases where either the strig 00...00 or the strig 11...11 is missig, simply compare the first ad last elemets of A agaist 00...01 ad 11...10, respectively. Commo errors: 1. If = 2 k 1, thek = lg( + 1) O(1). This is a error that early everyoe made. It is essetial to ote here that k is ot a idepedet parameter. It is defied i terms of, ad sice we ca solve for k = lg( + 1), we caot claim that k is a costat. All algorithms that ru i time O(k) ru i time O(lg), which is ot liear. RADIX-SORT does ot ru i liear time Give iput umbers that are d digits log, radix sort rus i time O(d+kd),wherek = O(2 d ) is the rage of each digit. I our case, d = lg, hece RADIX sort, rus i time O(lg).