Fall 05 Homework : Solutos Problem : (Practce wth Asymptotc Notato) A essetal requremet for uderstadg scalg behavor s comfort wth asymptotc (or bg-o ) otato. I ths problem, you wll prove some basc facts about such asymptotcs. Gve ay two fuctos f( ) ad g( ), show that f() + g() = Θ(max{f(), g()}). Soluto: Note Uless metoed otherwse, we wll always cosder fuctos from the postve tegers to the o-egatve real umbers. To show f()+g() = Θ(max{f(), g()}), we eed to show f()+g() = Ω(max{f(), g()}) ad f() + g() = O(max{f(), g()}). Frst, sce the fuctos are o-egatve, we have that f() + g() f() ad f() + g() g() combg these, we get that f() + g() max{f(), g()} for all ; thus f() + g() = Ω(max{f(), g()}). O the other had, we also have that f() + g() max{f(), g()} for all ; thus f() + g() = O(max{f(), g()}). Ths completes the proof. A algorthm ALG cossts of two tuable sub-algorthms ALG A ad ALG B, whch have to be executed serally (.e., oe ru of ALG volves frst executg ALG A followed by ALG B ). Moreover, gve ay fucto f(), we ca tue the two algorthms such that oe ru of ALG A takes tme O(f()) ad ALG B takes tme O(/f()). How should we choose f to mmze the overall rutme of ALG (.e., to esure the rutme of ALG s O(h()) for the smallest-growg fucto h)? How would your aswer chage f ALG A ad ALG B could be executed parallel, ad we have to wat for both to fsh? Soluto: Sce the two algorthms are ru sequetally, the total rutme s O(f() + /f()) from the prevous part, we have that ths s same as O(max{f(), /f()}). Now, order to mmze ths, t s clear we eed to set f() such that both parts are equal. Thus, we should choose f() = ad thus h() =. I case the two ra parallel, the rutme would ow be O(max{f(), /f()}) clearly ths would have the same optmal rutme! Part (c) We are gve a recursve algorthm whch, gve a put of sze, splts t to problems of sze /, solves each recursvely, ad the combes the two parts tme O(). Thus, f T () deotes the rutme for the algorthm o a put of sze, the we have: Prove that T () = O( log ). T () = T (/) + O()
Fall 05 Homework : Solutos Ht: Note that for a costat sze put, the algorthm takes O() tme. How may recursos does t requre to reduce a problem of sze to costat sze subproblems? What s the total rutme overhead at each recursve level? Soluto: We wll solve ths va a explct coutg argumet, whch I fd structve uderstadg how rutmes accumulate a recurso. Let k = {,,..., K} deote the levels of the recurso tree here k = s the orgal problem of sze, k = s the frst level of recurso wth two subproblems of sze /, ad extedg ths, at level k, we have k subproblems, each of sze / k, ad at level K the subproblems are of sze. Now observe the followg: The subproblems are of sze after K log recursve levels. Moreover, the tme take to solve a subproblem of sze s O(). The overhead from a subproblem of sze s O() thus the total overhead at level k s k O(/ k ) = O() Puttg ths together, we get that T () = O( log ). Problem : (Some mportat asymptotes) I class, we defed the harmoc umber H = = /. Argue that: + x dx H + x dx Thus, prove that H = Θ(l ). Ht: Boud the /x fucto from above ad below by a step fucto. Soluto: The dea s to represet the harmoc umber as the area of a curve (see Fgure ). Essetally, we have that H s the area uder a set of rectagles of legth ad heght /, {,,..., }. Now suppose we have a step fucto such that f u (x) = /, x [, + ) (.e., the rectagles above the /x curve). The we have: H + dx = l( + ) x O the other had, f we defe f l (x) = /, x (, ], ad cosder x [, ], we have that: H = + + = dx = + l() x Thus we have that l( + ) H + l ad hece H = Θ(log ).
Fall 05 Homework : Solutos Fgure : Itegral Test by Jm.belk. Lcesed uder Publc Doma va Commos, https: //commos.wkmeda.org/wk/fle:itegral_test.svg#/meda/fle:itegral_test.svg Next, we try to fd the asymptotc growth!. As the prevous part, argue that: Thus, prove that! = Θ( l ). l xdx l! + l xdx Soluto: Ths proceeds a very smlar fasho as above, except that ow log x s a creasg fucto. We frst compare the fucto f l (x) = log, x [, + ) for x [, + ] to get: log! + l xdx = x log x x + = ( + ) log( + ) To lower boud, we use the fucto f u (x) = log, x (, ] for x [, ] to get: log! = log = l xdx = x log x x = log + Combg ad usg the fact that = O( log ), we get that log! = Θ( log ). Part (c) (Strlg s approxmato) We ow mprove the estmate the prevous part to get the famlar form of Strlg s approxmato. Frst, argue that for ay teger, we have: + log xdx log + log( + ) 3
Fall 05 Homework : Solutos Usg ths, show that:! e ( ) e Ht: Gve ay >, where does the le jog the two pots (, l ) ad ( +, l( + )) le wth respect to the fucto log x? Soluto: We aga wat to use the tegral boudg trck here however, we use the added property that log x s a cocave fucto, ad hece for ay postve teger, the le segmet jog (, log ) ad ( +, log( + )) les below the curve log x for x [, + ]. Moreover, the area of the trapezod bouded by the le segmet jog (, log ) ad ( +, log( + )), ad log +log(+) the les x = 0, y = ad y = + s gve by (recall the area of a trapezod s / (heght) (sum of legths of parallel sdes)). Thus we have that: + log xdx log + log( + ) Now summg up over {,,..., }, we have: = + log xdx log! log However the left had sde s just log xdx = log +. Thus we get: Expoetatg both sdes, we get: log! log + + log! e ( ) e Problem 3: (The Geometrc Dstrbuto) A radom varable X s sad to have a Geometrc(p) dstrbuto f for ay teger k, we have P[X = k] = p( p) k. Suppose we repeatedly toss a co whch gves HEADS wth probablty p. Argue that the umber of tosses utl we see the frst HEADS s dstrbuted as Geometrc(p). Soluto: Our sample space Ω cossts of all sequeces over the alphabet {H, T } that ed wth H (HEADS) ad cota o other H s,.e. Ω = {H, T H, T T H,...}. The umber of falures k before the frst success (HEADS) wth a probablty of success p s gve by: P[X = k] = p( p) k wth k beg the total umber of tosses cludg the frst HEADS that termates the expermet. Therefore, the umber of tosses utl we see the frst HEADS s dstrbuted as Geometrc(p). 4
Fall 05 Homework : Solutos (Memoryless property) Usg the defto of codtoal probablty, prove that for ay tegers, k, the radom varable X obeys: P[X = k + X > k] = P[X = ] Also covce yourself that ths follows mmedately from the characterzato of the Geometrc r.v.. Soluto: By the defto of codtoal probablty, P[X = k+ X > k] = P[X = k + X > k] P[X > k] = P[X = k + ] P[X > k] = p( p)k+ ( p) k = p( p) = P[X = ]. Note that here we used that P[X > k] = ( p) k. The evet X > k meas that at least k + tosses are requred. Ths s exactly equvalet to sayg that the frst k tosses are all TAILS ad the probablty of ths evet s precsely ( p) k. Part (c) Show that: ()E[X] = p p, ad ()V ar[x] = p Ht: Note that by the memoryless property, a Geometrc(p) radom varable X s wth probablty p, ad +Y wth probablty ( p), where Y also has a Geometrc(p) dstrbuto. Now try wrtg the expectato ad varace recursvely. Soluto: Note that by the memoryless property, a Geometrc(p) radom varable X s wth probablty p, ad + Y wth probablty ( p), where Y also has a Geometrc(p) dstrbuto. Therefore, E[X] = E[p + ( p) ( + Y )] = p + ( p)e[ + Y ] = + ( p)e[y ] = + ( p)e[x]. Solvg for E[X], we get E[X] = p. Next, recall that V ar[x] = E[X ] (E[X]) = E[X ] p. So, frst we eed to calculate E[X ]. E[X ] = E[p + ( p) ( + Y ) ] = p + ( p)( + E[Y ] + E[Y ]) = p + ( p)( + E[X] + E[X ]) = p + ( p)( + p + E[X ]). Smplfyg, we get E[X ] = p, ad hece V ar(x) = E[X ] = p p p p 5
Fall 05 Homework : Solutos Problem 4: (Upper Bouds o Collso Probabltes) Let X m, deote the umber of collsos whe m balls are dropped u.a.r. to bs. I class, we showed that the the expected umber of collsos s ( ) m /. We ow upper boud the probablty that o collso occurs. Assume that > m (clearly ths s requred for o collsos!). Frst, usg the law of total probablty, argue that: P[No collsos whe m balls dropped u.a.r. bs] = Next, usg the equalty e x ( x), smplfy the above to show: m = ( ) P[No collsos whe m balls dropped u.a.r. bs] e E[Xm,] Soluto: Let D be the evet that there s o collso after havg throw the -th ball. If there s o collso after throwg balls the they must all be occupyg dfferet slots, so the probablty of o collso upo throwg the ( + )-st ball s exactly ( )/. That s, Also ote that P[D ] =. computed va P[D + D ] =. The probablty of o collso at the ed of the game ca ow be P[D m ] = P[D m D m ] P[D m ] = = = m = P[D + D ] = m = ( ). Note that /. So we ca use the equalty x e x for each term of the above expresso. Ths meas that: m ( ) P[D m ] e = e m(m ) = e E[Xm,]. Problem 5: (Posteror Cofdece Verfyg Matrx Multplcato) I class, we saw Frevald s algorthm for checkg matrx multplcato, whch, gve matrces A, B ad C, retured the followg: If AB = C, the the algorthm always retured TRUE If AB C, the the algorthm retured TRUE wth probablty at most / Gve ay ɛ > 0, how may tmes do we eed to ru Frevald s algorthm to be sure that {AB = C} wth probablty greater tha ɛ? 6
Fall 05 Homework : Solutos Suppose we started wth the belef that the evets {AB = C} ad {AB C} were equally lkely (.e., P[AB = C] = P[AB C] = /). Moreover, suppose k depedet rus of Frevald s algorthm all retured TRUE. The what s our ew (or posteror) belef that {AB = C}? Soluto: We kow that P[AB = C], therefore, we wat ɛ. Whch meas, k log ɛ, ad hece = log ɛ. Let I be our formato that k depedet rus of Frevald s algorthm all retured TRUE. Now we smply eed to use Bayes Theorem to fd the posteror: P[AB = C I] = P[I AB = C]P[AB = C] P[I AB = C]P[AB = C] + P[I AB C]P[AB C] = + k = + k. 7