CS / MCS 401 Homework 3 grader solutios assigmet due July 6, 016 writte by Jāis Lazovskis maximum poits: 33 Some questios from CLRS. Questios marked with a asterisk were ot graded. 1 Use the defiitio of a biomial radom variable X ad the defiitio of the expected value E[X] = all x x P (x) to prove E[X] = p. Do ot use idicator radom variables. Let X be a biomial radom variable with probability p of success, ad suppose that X is evaluated times. Recall the probability of k successes over trials for X is ( k) p k (1 p) k. The expected value of X is the E[X] = all x x P (x) (defiitio) = k P (k successes) (sice trials, reamig x to k) k=0 ( ) = k p k (1 p) k k k=0 (defiitio)! = k k!( k)! pk (1 p) k k=1 (defiitio, first term is 0) ( 1)! = p (k 1)!( k)! pk 1 (1 p) k (reducig ad factorig) k=1 ( ) 1 = p p k 1 (1 p) k (rearragig) k 1 k=1 1 ( ) 1 = p p k (1 p) ( 1) k (reidexig) k k=0 = p (p + (1 p)) 1 (biomial theorem) = p. (simplifyig) Give X is a biomial radom variable, use idicator radom vairables to prove E[X] = p. Let X k, for k = 0, 1,..., be the idicator radom variable evaluatig to 1 if there are k successes, ad evaluatig to 0 otherwise. By Lemma 5.1 (o page 118 of CLRS), we have that E[X k ] = p, so the [ ] E[X] = E X k = E[X k ] = p = p. k=0 k=0 k=0 5.-3 (p.1) Use idicator radom variables to compute the expected value of the sum of dice. For each die, let X k, for k = 1,, 3, 4, 5, 6 be the idicator radom variable evaluatig to 1 if the umber k is rolled, ad evaluatig to 0 otherwise. Let X be the radom variable evaluatig to the umber rolled o the die. It is clear that E[X k ] = 1/6 for all k, assumig the dice are fair. For every roll, the expected value is E[X] = 1 E[X 1 ] + + 6 E[X 6 ] = 1 + + 6 6 Hece the expected value for rolls is E[X] = E[X] = 7. = 1 6 = 7. 1
.3-1 (p.37) Usig Figure.4 as a model, illustrate the operatio of merge sort o the array A = 3, 41, 5, 6, 38, 57, 9, 49. We follow the diagram i Figure.4 with the give sequece. sorted sequece 3, 9, 6, 38, 41, 49, 5, 57 merge 3, 6, 41, 5 9, 38, 49, 57 merge merge 3, 41 6, 5 38, 57 9, 49 merge merge merge merge 3 41 5 6 38 57 9 49 iitial sequece.3- (p.37) Rewrite the Merge procedure so that it does ot use setiels, istead stoppig oce either array L or R has had all its elemets copied back to A ad the copyig the remaider of the other array back ito A. The followig is oe versio of such a rewrite. adjustmet of the for loop o lie 1. The mai differece is the removal of lies 8 ad 9, ad the 1 1 = q p + 1 = r q 3 let L[1,..., 1 ] ad R[1,..., ] be ew arrays 4 for i = 1 to 1 5 L[i] = A[p + i 1] 6 for j = 1 to 7 R[j] = A[q + j] 8 i = 1 9 j = 1 10 k = p 11 while i 1 + 1 ad j + 1 1 if L[i] R[j] 13 A[k] = L[i] 14 i = i + 1 15 else 16 A[k] = R[j] 17 j = j + 1 18 k = k + 1 19 if i = 1 + 1 ad j 1 + 1 0 A[k,..., r] = R[j,..., ] 1 else if j = + 1 ad i + 1 A[k,..., r] = L[i,..., 1 ] Note that L ad R may be the same legth, so to ot raise a error, a else if commad has to be used istead of else. The differece is that else followig a if is evaluated if the if fails, whereas to coditio of else if followig a if is checked before evaluated. We could have used two if commads, but the secod oe will be checked eve if the first oe evaluates to true, icreasig ruig time.
.3-3 (p.39) Use mathematical iductio to show that whe is a exact power of, the solutio of the recurrece is T () = log (). T () = { if =, T (/) + if = k, for k > 1 I the base case = we see that T () = by what is give, ad T () = log () = by what is expected. Hece the base case holds. For the iductive case, whe = k for k, we have that T ( k ) = T ( k 1 ) + k = k 1 log ( k 1 ) + k = k (k 1) + k = k k = log (), as desired..3-6 (p.39) Observe that the while loop of lies 5-7 of the Isertio-sort procedure i Sectio.1 uses a liear search to sca (backward) through the sorted subarray A[1.. j 1]. Ca we use a biary search (see Exercise.3-5) istead to improve the overall worst-case ruig time of isertio sort to Θ( log ())? No, because eve if we use biary search, the search must be doe for every elemet i the list, so the umber of times lie 5 is executed is still j= s j. While it may be true that s j t j, the sum is still up to, so we will get a quadratic factor of i the ruig time calculatio..4 a,b,c (p.41) Let A[1.. ] be a array of distict umbers. If i < j ad A[i] > A[j], the the pair (i, j) is called a iversio of A. a. List the five iversios of the array, 3, 8, 6, 1. b. What array with elemets from the set {1,,..., } has the most iversios? How may does it have? c. What is the relatioship betwee the ruig time of isertio sort ad the umber of iversios i the iput array? Justify your aswer. a. The five iversios of A are (1, 5), (, 5), (3, 4), (3, 5), ad (4, 5). b. The reverse ordered array, 1,...,, 1 has the most iversios, sice every pair (i, j) with i < j is a iversio. The umber of iversios are ( 1) + ( ) + + + 1 = 1 k=1 k = ( 1)/. c. For each j, the umber of iversios (i, j) is oe less tha t j from lie 5 i the code of isertio sort. Sice the oly lies to cotribute a term are oes with t j i the sums, if there are (for example) log () iversios, the the ruig time of ierstio sort will be O( log ()). However, sice terms cotributig to the ruig time come from lies ot cotaiig t j, o matter how small the umber of iversios, the ruig time will ever be less tha O(). 3
6 Fid the best-case ad worst-case time for the Bubble-Sort ad Better-Bubble-Sort algorithms give i class slides. The slides are posted o Piazza. Iclude the algorithms with your work. First we aalyze the algorithms, similarly to how it is doe i the book. Bubble-sort cost times 1 for m = A.legth dowto c 1 for i = 1 to m 1 c m= m 3 if A[i] > A[i + 1] c 3 m= m 1 4 swap(a[i], A[i + 1]) c 4 m= m 1 Better-Bubble-sort cost times 1 for m = A.legth dowto c 1 sorted = true c 1 3 for i = 1 to m 1 c 3 m= m 4 if A[i] > A[i + 1] c 4 m= m 1 5 swap(a[i], A[i + 1]) c 5 m= m 1 6 sorted = false c 6 m= m 1 7 if sorted = true c 7 1 8 retur c 8 1 For bubble-sort, the best case is if the if loop is ever evaluated, so we get c 1 + c m= m + c 3 m= ( ( + 1) (m 1) = c 1 + c ) ( ) ( 1) 1 + c 3 = O( ). The worst case happes if if is true every time, ad lie 4 is evaluated every time. The the ruig time is c 1 +c m= m+c 3 m= m 1+c 4 m= ( ( + 1) (m 1) = c 1 +c ) ( ) ( ) ( 1) ( 1) 1 +c 3 +c 4 = O( ). For better-bubble-sort, the best case is if we hit lie 8 as soo as possible (i the first go through the for loop i lie 1) ad the if i lie 4 is ever evaluated. That would be a ruig time of c 1 + c + c 3 + c 4 ( 1) + c 7 + c 8 = O(). The worst case is similar to the worst case for bubble-sort, i fact it must be at least O( ), sice the code for bubble-sort is cotaied withi the code for better-bubble-sort. Sice oe of the times are cubic i, it must be that the ruig time for the worst case is also O( ). 7 Fid the best-case, worst-case ad average-case ruig ties for Isertio-Sort. Show all work ad explai ay assumptios that you use. Iclude the algorithm with your work. 8 Prove that x = x ad that x = x. From the defiitio of the floor ad ceilig fuctios, we have that x 1 < x x ad x x < x + 1 x x > x 1. Sice there is exactly oe iteger i the iterval ( x 1, x], ad both x ad x are itegers i this rage, it must be that x = x. A aalogous argummet works for the secod part. 4
7.1-1 (p.173) Usig Figure 7.1 as a model, illustrate the operatio of Partitio o the array A = 13, 19, 9, 5, 1, 8, 7, 4, 1,, 6, 11. We follow Figure 7.1 with the give array. Step 0 has the array i the give order, ad step 11 has the array i the fial order. i p, j r step 0 13 19 9 5 1 8 7 4 1 6 11 i p j r step 1 13 19 9 5 1 8 7 4 1 6 11 i p j r step 13 19 9 5 1 8 7 4 1 6 11 i, p j r step 3 9 19 13 5 1 8 7 4 1 6 11 step 4 9 5 13 19 1 8 7 4 1 6 11 step 5 9 5 13 19 1 8 7 4 1 6 11 step 6 9 5 8 19 1 13 7 4 1 6 11 step 7 9 5 8 7 1 13 19 4 1 6 11 step 8 9 5 8 7 4 13 19 1 1 6 11 step 9 9 5 8 7 4 13 19 1 1 6 11 step 10 9 5 8 7 4 19 1 1 13 6 11 p i r step 10 9 5 8 7 4 6 1 1 13 19 11 p i r step 11 9 5 8 7 4 6 11 1 13 19 1 The procedure returs the idex value i + 1 = p + 7. 5
7.1- (p.174) What value of q does Partitio retur whe all elemets i the array A[p.. r] have the same value? Modify Partitio so that q = (p + r)/ whe all elemets i the array A[p.. r] have the same value. If all the elemets i the array have the same value, the Partitio will retur the idex r, because the if coditio o lie 4 will always be true, ad so the value of i will be icreased r p times, ad sice i starts out at p 1, we have that p 1 + r p = r 1. Out of the for loop the value of i + 1 = r 1 + 1 = r is retured. Here is oe way of modifyig the Partitio algorithm - by keepig a variable that is chaged oly if a differet value from A[r] is ecoutered. 1 x = A[r] i = p 1 3 same = true 4 for j = p to r 1 5 if A[j] x 6 i = i + 1 7 exchage A[i] with A[j] 8 if A[j] < x 9 same = false 10 exchage A[i + 1] with A[r] 11 if same 1 retur (p + r)/ 13 else 14 retur i + 1 7.1-3 (p.174) Give a brief argumet that the ruig time of Partitio o a subarray of size is Θ(). Every for ad while loop (usually) icreases ruig time by a factor of or log() (either way, a o-costat fuctio of ), ad i Partitio we oly have oe for loop. The lie o which the for loop appears is executed r p times, which is the legth of the subarray beig cosidered. The lies outside the for loop are executed eactly oce ad the lies iside the for loop are also executed at most r p 1 = 1 times. This meas the ruig time is a + b = Θ() for some appropriate costats a ad b. 7.1-4 (p.174) How would you modify Quicksort to sort ito oicreasig order? Here is oe way of modifyig the Quicksort algorithm - by switchig the iequality i lie 5 of Partitio the other way: 5 if A[j] x 5 if A[j] > x Leavig the rest of the code aloe, this puts all values greater tha A[r] o the left of A[q] at the ed, ad all values less tha or equal to A[r] o the right of A[q]. Aother method (which oly chages Quicksort ad ot Partitio) is swappig the two groups after Partitio is doe with them: 1 if p < r q = Partitio(A, p, r) 3 exchage A[p,..., q 1] with A[q + 1,..., r] 4 q = r (q p) 5 Quicksort(A, p, q 1) 6 Quicksort(A, q + 1, r) Note it is ot ecessary to swap the parts after the last executio of the ested Quicksort, because the there is oly oe elemet i the subarray, so there is othig to swap. 6
7-3 (p.187) A alterative aalysis of the ruig time of radomized quicksort focuses o the expected ruig time of each idividual recursive call to Radomized-Quicksort, rather tha o the umber of comparisos performed. a. Argue that, give a array of size, the probability that ay particular elemet is chose as the pivot is 1/. Use this to defie idicator radom variables X i = I{ith smallest elemet is chose as the pivot}. What is E[X i ]? b. Let T () be a radom variable deotig the ruig time of quicksort o a array of size. Argue that [ ] E[T ()] = E X q (T (q 1) + T ( q) + Θ()). c. Show that we ca rewrite equatio (7.5) as d. Show that E[T ()] = 1 E[T (q)] + Θ(). q= 1 k log (k) 1 log () 1 8. k= Hit: Split the summatio ito two parts, oe for k =, 3,..., / 1 ad oe for k = /,..., 1. e. Usig the boud from equatio (7.7), show that the recurrece i equatio (7.6) has the solutio E[T ()] = Θ( log ()). Hit: Show, by substitutio, that E[T ()] a log () for sufficietly large ad for some positive costat a. a. Sice the pivot is chose at radom from the array, every elemet has the same probability of beig chose. This meas each elemet has probability 1/ of beig chose. By Lemma 5.1, we kow that for a idicator variable X A evaluatig to 1 if evet A occurs ad 0 otherwise, E[X] = Pr(A). Sice the ith smallest elemet of the array is just oe specific elemet, we have that E[X i ] = 1/ for all i. b. Previously we argued that Partitio has ruig time Θ(). If idex q is chose as the pivot, the we immediately see that E[T ()] = E[Θ() + T (q 1) + T ( q)] by the code of Quicksort. Sice the idicator vairbale X q is 1 oly if q is chose as the pivot, we may cover all the possibilities by summig over all q ad multiplyig each term by X q, givig the desired form. c. Equatio (7.5) is the equatio from part b. above, ad simplifyig it is just algebra. [ ] E[T ()] = E X q (T (q 1) + T ( q) + Θ()) = = = 1 = 1 (give) E [X q (T (q 1) + T ( q) + Θ())] (liearity of E) E [X q ] E [T (q 1) + T ( q) + Θ())] E [T (q 1) + T ( q) + Θ())] E[Θ()] + 1 = Θ() + 1 = Θ() + = Θ() + (idepedece of X q ad T ()) (part a. above) (E [T (q 1)] + E [T ( q)]) (liearity of Σ ad E) (E [T (q 1)] + E [T ( q)]) (Θ is ot a radom variable) E [T (q 1)] (sice q 1 = ( (q 1))) E[T (0)] + E[T (1)] + E [T (q 1)] q=3 7
= Θ() + E [T (q 1)] q=3 = Θ() + 1 E [T (q)] q= (T (0), T (1) have liear ruig time) (reidexig) I the secod lie from the bottom the ruig times T (0) ad T (1) are absorbed ito the Θ(), ad as followig the algorithm we do approximately executios. d. This part is also just algebra. We follow the hit to get 1 k log (k) = k= / 1 k= / 1 k= k log (k) + k log (/) + 1 k= / 1 k= / / 1 = log (/) k + log () k= k log (k) 1 / 1 = log () k log () k k= ) 1 ( ( 1) = log () ( ( 1) log () 1 k= k log () 1 k= / ( ( 1 ) ( ( 1) ) 1 k 1 ) = log () 8 8 = 1 log () 1 ( 8 log () + log () + ) 4 + 1 ) (takig the hit) (boudig above) (factorig) (laws of logarithms) (sum formula) (boudig above) 1 log () 1 8. Note i the secod-to-last lie the last term is positive for all positive itegers, so addig it simply bouds the origial expressio from above. e. Applyig substitutio, we guess that E[T ()] a log () for sufficietly large ad for some positive costat a. Pluggig this ito the results from parts c. ad d. above, we get E[T ()] = 1 E[T (q)] + Θ() q= (part c. above) = 1 aq log (q) + Θ() (guessig the result) q= a ( 1 log () 1 ) 8 + Θ() (part d. above) = a log () a 4 + Θ() a log () = Θ( log ()). (for a large eough) Note that Θ() Θ( log ()), but by makig a large eough, the term a/4 cacels the effect of ay costat fuctio b Θ(). 8
11 Fid a big-o for Quicksort for the followig three cases: a. a pre-sorted array; b. a reverse pre-sorted array; c. a array with all equal elemets. Cosider a situatio where the recursive calls to Quicksort alterate good case, bad case, good case, bad case, etc. Uder what coditios could this possibly happe? 9