A Theorist s Toolit CMU 8-89T, Fall ) Lecture : Asymptotics September 9, Lecturer: Rya O Doell Scribe: Misha Lavrov I additio to the boo refereces provided at the ed of this documet, two chapters of lecture otes o asymptotics by A.J. Hildebrad ca be foud at: http://www.math.uiuc.edu/~hildebr/9ama/ama-ch.pdf http://www.math.uiuc.edu/~hildebr/9ama/ama-ch.pdf Asymptotic Notatio We all ow that i = i= ) =. If we wat to describe the behavior of this fuctio for large, the quadratic term is the more importat. We ca write this as i = O ). i= Formally, the otatio fx) = Ogx)) big-oh of gx) ) meas that for x sufficietly large, there is a costat C > such that fx) Cgx). This describes the asymptotic behavior of fx) as x. Sometimes we will also tal about asymptotics of some fuctio fx) as x. I that case, fx) = Ogx)) meas that there is some x > such that for x x, fx) Cgx). It will probably be clear from cotext which oe is meat; geerally, a variable amed goes to, while a variable amed ɛ goes to. If we use Ogx)) i a expressio, such as fx) = Ogx)), what we mea is that Ogx)) ca be replaced with some aoymous fuctio of x which is Ogx)). For example, we could write i = O) = )) O. i= Oe use of this that deserves special metio is O): this is a fuctio of x that is evetually bouded above by some costat. I additio to O, there are several other symbols that ca be used to say slightly differet thigs: This is ot etirely stadard: some people allow Cgx) fx) Cgx).
fx) = Ωgx)) meas that gx) = Ofx)): i other words, for x sufficietly large/small, there is a costat C > such that fx) Cgx). fx) = Θgx)) meas that fx) = Ωgx)) ad fx) = Ogx)) simultaeously: for x sufficietly large/small, there are costats C, C > such that C gx) fx) C gx). fx) gx) meas that fx) gx) i the limit. For example, we could write i. i= fx) = ogx)) meas that fx) gx) example, we could write, ad fx) = ωgx)) meas that fx) gx) i = o)). i=. For fx) polygx))) meas that fx) = gx) O) : fx) is bouded by some polyomial fuctio of gx). fx) = Õgx)) meas that fx) gx) polylog gx)): we forget about some polyomial i log gx), which is isigificat compared to g itself. For example, we ca write = Õ ). Note that Õ ): the differece betwee ad is too big. We ca put a tilde o other thigs as well, such as Θgx)). If x, the fx) = Õgx)) istead meas fx) gx) polylog /gx)), sice gx) is probably somethig small. For example, ɛ log /ɛ) = Õɛ). The Harmoic Number Let H =. This is called the -th harmoic umber. It comes up i may applicatios icludig, for example, the coupo collector problem: if there are differet coupos, ad you pic coupos uiformly at radom with replacemet, you will eed to loo at H coupos i expectatio) before you have see all of them at least oce. We ca get a simple upper boud for H as follows: if = log ), the H =
............. a). Z b) dx x. c) Figure : Comparig the harmoic series to a itegral {z } =. 7 Therefore H dlog )e log. A similar techique ca give us a lower boud o H : if = blog c, the H = 8 8 =. {z } 7 8 8 8 Therefore H blog c log. This is already eough to coclude H = Θlog ). Note that the exact base of the log is irrelevat i big-θ otatio.) I this class, this may be the last time you see the base i log writte explicitly; we will assume log is log which is sometimes also writte as lg ); for the atural log, we will write l proouced lo e ). What should we do if we wat to out how big H is more precisely? Pfigure the sum i= i by a itegral. A lower boud for the itegral R We ca approximate dx is = H, ad a upper boud is = H see x Figure ).
Sice the atiderivative of is l x, the itegral is l l = l, so we have x H l H l ) H l. We ca write dow a slightly less accurate but prettier estimate: l H l. I particular, H l. What is the error whe we replace l ) by l? l ) = l )) = l l ) ) = l Θ. The last step follows from the Taylor expasio of l x: for < x. l x) = x x x. Sice x i our case is, which goes to, the first term is most sigificat, ad we ca approximate l ) by. The error boud i Taylor s theorem states that l ) = ξ for some ξ, so actually we ca say l ) = l O ). This is related to oe of the most useful asymptotic approximatios you will use: e x is approoximately x for small x. To be more precise, e x = x x! x! = x Ox ). The Taylor expasio holds for all x, but oly for small x is x less sigificat tha x. However, it s true for all x that e x x. Although we will ot prove this, H ca actually be described eve more precisely: ) H = l γ O, where γ.77 is ow as the Euler-Mascheroi costat. The Birthday Paradox Suppose we have m bis which ca hold balls, ad we chuc balls ito the bis at radom to be precise, each ball chooses a bi uiformly at radom, ad the choices are idepedet). What is the probability that o two balls lad i the same bi?
I the case m =, we get the birthday paradox: if you have people, what is the probability that two will share a birthday? The reaso it s a paradox is that the probability is surprisigly large: although oly = 7 guaratees that two people share a birthday, i a group of people the probability is already over 7%. I geeral, the probability of o collisios betwee balls throw ito m bis is p,m = m ) m ) m A reasoable questio to as is: for what, as a fuctio of m, is p,m? By usig the iequality e x x from the previous sectio, we ca get a simple upper boud: p,m e /m e /m e )/m = exp m m ) m ) ) = exp. m ) The is somewhat aoyig: we d lie to write p,m exp, but this does t m quite follow from the above. I ay case, if we believe that this iequality is close to beig the truth, we could aswer the questio whe is p,m by solvig this for, obtaiig l m. We also wat a lower boud. The iequality x e x came from l x) x, or l x x. By extedig the Taylor series for l x, we ca get a matchig lower boud that loos lie x expx Ox )). It follows that: p,m exp m )) m O exp m )) O exp )) ) m m O m = exp m m ) exp m m O ) ) ) ) )) ) = exp exp O. m m We get the same thig we got i the upper boud, multiplied by a error factor. But whe = Θ m), this error factor loos lie exp O m )), which is O m ). The implied costat i = Θ m) does t matter too much. But it ofte helps to ow that whe there are O m) balls ad m bis, you probably get a collisio. I our class, Alex Kazachov shares a birthday with me Misha Lavrov). ).
.............. a) l l b) R. l x dx c) l Figure : Comparig the sum l l l to a itegral Stirlig s Formula Recall that! = ) ). How large is!, asymptotically? A very simple upper boud for!: we ca replace every factor by, gettig!. For a lower boud, we could try several thigs: All factors except are at least, so!. d/e The first factors are at least, so! /. Because all of these are very large products, we ca get a better sese of how large they are by taig logs. We ca the coclude that /! because l l!) l. This is already eough to say that l!) = Θ l ), ad therefore! = Θ log ). To get a better estimate, we ca expad ) l l!) = l l l. Just lie we did for harmoic umbers, we ca approximate this sum R by a itegral. The sum l l i Figure a) is a over-estimate of the itegral l x dx i Figure b). The atiderivative of l x is x l x x, so we coclude l!) l ) l ) = l. To get a upper boud o l!), we ca subtract off the triagles i Figure c, which will leave a uder-estimate of the itegral. Looed at sideways, the triagles have height ad their bases sum to l, so their area is l, ad therefore l!) l l.
If we tae these bouds ad expoetiate, we get: ) ) e! e. e e I particular,! = Θ ) ) e. We ca still hope to do better. The error i our upper boud is the area of the slivers that form the overlap betwee Figure b ad Figure c. It ca be show that the total area of these slivers is O). As a cosequece,! = Θ ) ). e This is ow as Stirlig s formula, of which a more precise variat states that! = ) )) π ± O. e We will see where the π comes from whe we tal about the Cetral Limit Theorem. We ca use Stirlig s formula to estimate the biomial coefficiets ) =! whe! )! = p to simplify otatio, let q = p, so that = q). As for costat p, )! =! )! =! p)!q)! π ) e ± O = )) πp p ) p e πq q ) q e ± O )) ± O )) p q ) π pq p p q q = Hp) πpq where Hp) = p log q log. This quatity is of idepedet iterest ad is ow as the p q biary etropy of p. I particular, H/) =, so we ca coclude / ) ) π = Θ. Whe is eve, this is the probability that if fair cois are flipped, exactly half will come up heads. Cut the -th triagle by the taget lie to l x at. This bouds the -th sliver i a smaller triagle with height ad base l ) l. The sum of the areas of the first triagles is therefore l ) H ), which coverges to γ) as. 7
Whe = o), thigs are slightly differet: the error factor depedig o p is o loger comparable to the oe depedig o. I this case, it s more accurate to approximate ) by. The error aalysis is as follows:! ) = ) ) )! ) =! ) ). The error factor is exactly the probability p, we looed at i the Birthday Paradox sectio. We ow that p, has a costat value for = Θ ), so for = o ), p, = o), ad we have )!. Sice ) ) ), this is also a upper boud for all. Refereces [DB7] Nicolaas Govert De Bruij. Asymptotic Methods i Aalysis, volume. Dover Publicatios, 97. [GKP9] Roald L Graham, Doald E Kuth, ad Ore Patashi. Cocrete Mathematics: A Foudatio for Computer Sciece. Addiso-Wesley Logma Publishig Co., Ic., 99. 8