Sums, Approximations, and Asymptotics II

Transcription

1 6.04/8.06J Mathmatics for Computr Scic March 5, 005 Srii Dvadas ad Eric Lhma Lctur Nots Sums, Approximatios, ad Asymptotics II Block Stackig How far ca a stack of idtical blocks ovrhag th d of a tabl without topplig ovr? Ca a block b suspdd tirly byod th tabl s dg? Tabl Physics imposs som costraits o th arragmt of th blocks. I particular, th stack falls off th dsk if its ctr of mass lis byod th dsk s dg. Morovr, th ctr of mass of th top k blocks must li abov th (k + ) st block; othrwis, th top k would fall ovr. I ordr to fid th bst cofiguratio of blocks satisfyig ths costraits, w ll d a fact about ctrs of mass. Fact. If two objcts hav masss m ad m ad ctrs of mass at positios z ad z, th th ctr of mass of th two objcts togthr is at positio: z m + z m m + m Dfi th offst of a particular cofiguratio of blocks to b th horizoal distac from its ctr of mass to its rightmost dg.? offst ctr of mass - s? Tabl

2 Sums, Approximatios, ad Asymptotics II Th offst dtrmis prcisly how far that cofiguratio ca xtd byod th dsk sic at bst th ctr of mass lis right abov th dsk s dg. So hraftr w ca focus o maximizig th offst of a stack of blocks rathr tha th ovrhag. As a rsult, w d oly b cocrd with whthr th stack is itrally stabl ad ot with whthr th whol cofiguratio is too far to th right ad falls off th tabl. W ca stablish th gratst possibl offst of a stack of blocks with a iductiv argumt. This is a istac whr iductio ot oly srvs as a proof tchiqu, but also turs out to b a ic tool for rasoig about th problm. Thorm. Th gratst possibl offst of a stack of blocks is: X = Proof. W us iductio o, th umbr of blocks. Lt P () b th propositio that th gratst possibl offst of a stack of blocks is / + / /(). Bas cas: For a sigl block, th ctr of mass is distac X = / from its rightmost dg. So th offst is / ad P () is tru. Iductiv stp: For, w assum that P ( ) is tru i ordr to prov P (). A stack of blocks cosists of th bottom block togthr with a stack of blocks o top. I ordr to achiv th gratst possibl offst with blocks, th top blocks should thmslvs hav th gratst possibl offst, which is X ; othrwis, w could do bttr by rplacig th top blocks with a diffrt cofiguratio that has gratr offst. Furthrmor, th ctr of mass of th top blocks should li dirctly abov th right dg of th bottom block; othrwis, w could do bttr by slidig th top blocks farthr to th right. blocks s X - s X + - W v ow idtifid th cofiguratio of blocks with th gratst possibl offst. What rmais is to dtrmi what that offst is. To that d, w ll apply Fact whr positios ar masurd rlativ to th rightmost dg of th block stack. I particular, w hav blocks with ctr of mass at positio X, ad block with ctr of mass at positio X +. So Fact implis that th maximum possibl offst of a stack of

3 Sums, Approximatios, ad Asymptotics II 3 blocks is: ) X = X ( ) + (X + = X + = W us th assumptio P ( ) i th last stp. This provs P (). Th thorm follows by th pricipl of iductio.. Harmoic Numbrs Sums similar to th o i Thorm com up all th tim i computr scic. I particular, is calld a harmoic sum. Its valu is calld th th harmoic umbr ad is dotd H. I ths trms, th gratst possibl offst of a stack of blocks is H. W ca tabulat th maximum achivabl ovrhag with =,, 3 ad 4 blocks by computig th first fw harmoic umbrs: # of blocks maximum ovrhag H = ( ) = H = ( + ) = H 3 ( ) = + + = H 4 = ( ) = > Th last li rvals that w ca suspd th fourth block byod th dg of th tabl! Hr s th cofiguratio that dos th trick: 8 6 4

4 4 Sums, Approximatios, ad Asymptotics II. Boudig a Sum with a Itgral W d to kow mor about harmoic sums to dtrmi what ca b do with a larg umbr of blocks. Ufortuatly, thr is o closd form for H. But, o th bright sid, w ca gt good lowr ad uppr bouds o harmoic sums usig a gral tchiqu ivolvig itgratio that applis to may othr sums as wll. Hr s how it works. First, w imagi a bar graph whr th ara of th k th bar is qual to th k th trm i th sum. I particular, ach bar has width ad hight qual to th valu of th k th trm. For xampl, th bar graph corrspodig to th harmoic sum H = is show blow. 6 / Now th valu of th sum is qual to th ara of th bars, ad w ca stimat th ara of th bars usig itgratio. Suppos w draw a smooth curv that rus just blow th bars; i fact, thr s a atural choic: th curv dscribd by th fuctio y = /(x + ). 6 / 3 4 y = /(x+ )

5 Sums, Approximatios, ad Asymptotics II 5 Th ara of th bars is at last th ara udr this curv, so w hav a lowr boud o th th harmoic sum: H = dx 0 x + = l(+ ) Rmmbr that blocks ca ovrhag th dg of a tabl by H block widths. So if w had, say, a millio blocks, th this lowr boud implis that w could achiv a ovrhag of at last l(, 000, ) = block widths! I fact, sic th lowr boud of l( + ) grows arbitrarily larg, thr is o limit o how far th stack ca ovrhag. Of cours, this assums o brz, dformatio of th blocks, or gravitatioal variatio as our stack grows thousads of mils high. W ca gt a uppr boud o th th harmoic umbr by playig sstially th sam gam. Now w d a curv that skims just abov th bar graph. Th curv dfid by y = /x fits th bill. 6 y = /x / Th ara udr this curv is a uppr boud o th ara of th bar graph ad thus o th th harmoic sum. But thr s a problm: th ara udr th curv is ifiit bcaus y = /x has a bad asymptot at x = 0. This is a commo problm wh boudig sums with itgrals ad thr s a simpl solutio: tak th xact valu of th first trm (/) ad th boud th rmaiig trms (/ + / /) with a itgral. I this cas, w

6 6 Sums, Approximatios, ad Asymptotics II gt th uppr boud: H = dx x = + l So v though thr is o closd form for th th harmoic sum, w ow kow that th harmoic umbrs grow logarithmically: l( + ) H + l Thr is a rfimt to th itgratio mthod w v just mployd, calld Eulr Maclauri summatio, which yilds a squc of trms that corrct rrors i th iital stimat. This tchiqu givs a absurdly accurat formula for harmoic umbrs: ɛ() H = l + γ Th scod trm is Eulr s costat, γ = This is probably th third most importat mathmatical costat bhid ad π. It is somwhat mystrious; for xampl, o o kows whthr γ is ratioal or irratioal, though if it quals a ratio of itgrs, th th domiator must b at last 0 4,080. I th fial trm of th formula abov, ɛ() dots som umbr btw 0 ad. You ll vr d this kid of prcisio i this cours or, probably, aywhr ls. Th Factorial Fuctio O of th most commo lmts i mssy mathmatical xprssios is th factorial fuctio:! = 3 ( ) Factorial coms up particularly oft i combiatorics ad probability, which happ to b th major topics i th rmaidr of Withi a fw wks, you ar goig to hav factorials comig out your ars. A good way to dal with ay product is to covrt it ito a sum by takig a logarithm: ( ) l f(k) = l f(k) k= k= Th w ca apply all our summig tools ad xpotiat at th d to udo th ffct of th log. Lt s us this stratgy to gt bouds o!. First, w tak a logarithm to mak

7 Sums, Approximatios, ad Asymptotics II 7 th product ito a sum: l! = l( 3 ) = l + l + l l This sum is rathr asty, but w ca still gt bouds by th itgratio mthod. A pictur ca b xtrmly hlpful i workig out th propr boudig fuctios ad limits of itgratio. 6 y = l(x + ) y = l x l 3 l 4 l I this cas, w gt th bouds: l x dx l! l(x + )dx x l x x l + Fially, w xpotiat to gt bouds o!. l! (x + ) l(x + ) (x + ) ( ) ( )+ +! This givs som idicatio how big! is: about (/). This stimat is oft good ough. If you d a littl mor accuracy, you ca add o mor trm to gt Stirlig s Formula: ( )! π Stirlig s formula is worth committig to mmory. W ll oft us it to rwrit xprssios ivolvig!. Now, o might objct that th xprssio o th lft actually looks a lot bttr tha th xprssio o th right. But that s just a artifact of otatio. If you actually watd to comput!, you d d multiplicatios. Howvr, th xprssio o th right is a closd form; valuatig it rquirs oly a hadful of basic opratios, rgardlss of th valu of. Furthrmor, wh! appars isid a largr xprssio, 0 l! ( + ) l( + ) ( + ) + 0

8 8 Sums, Approximatios, ad Asymptotics II you usually ca t do much with it. It dos t radily cacl or combi with othr trms. I cotrast, th xprssio o th right looks scary, but mlds icly ito largr formulas. So do t b put off: Stirlig s Formula is your frid. (Stppig back a bit, Stirlig s formula is fairly amazig. Who would guss that th product of th first positiv itgrs could b so prcisly dscribd by a formula ivolvig both ad π?) If you r flig a littl crazy, you might pull out ths v mor prcis bouds: π ( ) ( )! π /(+) ( ) ! 00π } /0 {{} ( ) ! 00π } /00 {{} /() Ths bouds ar ridiculously clos. For xampl, if = 00, th w gt: = = Th uppr boud is lss tha 7 hudrd thousadths of % gratr tha th lowr boud! 3 Asymptotic Notatio Our fial topic is a spcial otatio calld asymptotic otatio dsigd to swp mathmatical msss udr th rug. Asymptotic otatio is sort of lik th Forc from Star Wars. It s vrywhr. Tt s a sstial part of th laguag of computr scic. Ad mostly it is a tool for good. At bst, asymptotic otatio supprsss irrlvat dtailss, whil allowig th sstial faturs of a mathmatical aalysis to shi through. Howvr, as w ll s, asymptotic otatio also has a allurig dark sid. 3. Asymptotic Notatio: Th Jdi Prspctiv Suppos you wat to kow how log a computr taks to multiply two matrics. You could tally up all th multiplicatios ad additios ad loop variabl icrmts ad comparisos ad prhaps factor i hardwar spcific cosidratios such as pag faults ad cach misss ad brach misprdicts ad floatig poit uit availability ad all this would giv you o sort of aswr. (Whw!) Such a aswr would b vry accurat, but ot vry isightful. Giv a vry complicatd formula for th ruig tim, w would b hard prssd to aswr basic qustios: How would doublig th siz of th matrics altr th ruig tim? What is th biggst matrix w ca hadl? Furthrmor, a mior chag i th procdur or hardwar would ivalidat th aswr.

9 Sums, Approximatios, ad Asymptotics II 9 O th othr had, ach of th tris i th product matrix taks about stps to comput. So th ruig tim is roughly = 3. This aswr is crtaily lss prcis, but it was asy to driv ad is asy to itrprt. For xampl, w ca s that doublig th siz of th matrics from to would icras th ruig tim from about 3 to about () 3 = 8 3 a factor of 8. Ad, assumig a computr prforms billios of opratios i a rasoabl tim (as opposd to millios or trillios), th largst matrics w could hadl would b roughly Furthrmor, this approximat aswr is idpdt of tiy implmtatio ad hardwar dtails. It rmais valid v aftr you upgrad your computr. So approximat aswrs ar quit attractiv. Ad asymptotic otatio allows o to formulat vagu statmts lik roughly 3 i a vry prcis way. 3. Six Fuy Looki Symbols Asymptotic otatio ivolvs six wird littl symbols: O Ω Θ o ω oh omga thta littl oh littl omga tild W ll focus o O, which is th most widly usd. Th othrs ar about qually popular, xcpt for ω, which is th Jar Jar of th lot. Suppos that f ad g ar fuctios. Th th statmt f(x) = O(g(x)) mas that thr xist costats x 0 ad c > 0 such that f(x) c g(x) for all x > x 0. Now this dfiitio is prtty hairy. But what it s tryig to say, with all its cryptic littl costats, is that f grows o fastr tha g. A bit mor prcisly, it says that f is at most a costat tims gratr tha g, xcpt mayb for small valus of x. For xampl, hr s a tru statmt: 5x + 00 = O(x) This holds bcaus th lft sid is oly about 5 tims largr tha th right. Of cours, for small valus of x (lik x = ) th lft sid is may tims largr tha th right, but th dfiitio of O is clvrly dsigd to swp asid such icovit dtails. Lt s work carfully through a squc of xampls ivolvig O i ordr to bttr udrstad this dfiitio. Claim. 5x + 00 = O(x)

10 0 Sums, Approximatios, ad Asymptotics II Proof. W must show that thr xist costats x 0 ad c > 0 such that 5x + 00 c x for all x > x 0. Lt c = 0 ad x 0 = 0 ad ot that: 5x x + 5x = 0x for all x > 0 Hr s aothr claim that poits out a vry commo rror ivolvig O otatio. Claim 3. x = O(x ) Proof. W must show that thr xist costats x 0 ad c > 0 such that x c x for all x > x 0. Lt c = ad x 0 = ad ot that x x for all x > May popl fail to raliz that a statmt of th form f(x) = O(g(x)) oly placs a uppr boud o th growth of th fuctio f. So, for xampl, you ll oft har popl say thigs lik, I ca t us a algorithm that rus i O(x ) stps bcaus that s too slow. Popl who say thigs lik that ar dumb ad you should mak fu of thm util thy cry. Okay, mayb ot. But thy ar icorrct; w just provd that a fast algorithm that rquirs oly x stps also rus i tim O(x )! O proprly xprsss a lowr boud usig th Ω otatio, which w ll com to prstly. What about th rvrs of th prvious claim? Is x = O(x)? O a iformal basis, this mas x grows o fastr tha x, which is fals. Lt s prov this formally. Claim 4. x = O(x) Proof. W argu by cotradictio; suppos that thr xist costats x 0 ad c such that: x c x for all x > x 0 Dividig both sids of th iquality by x givs: But this is fals wh x = + max(x 0, c). x c for all x > x 0 W ca show that x = O(00x) by sstially th sam argumt; ituitivly, x grows quadratically, whil 00x grows oly liarly. Grally, chags i multiplicativ costats do ot affct th validity of a assrtio ivolvig O. Howvr, costats i xpotials ca b critical: Claim 5. 4 x = O( x )

11 Sums, Approximatios, ad Asymptotics II Proof. W argu by cotradictio; suppos that thr xist costats x 0 ad c > 0 such that: 4 x c x for all x > x 0 Dividig both sids by x givs: But this is fals wh x = + max(x 0, log c). x c for all x > x Asymptotic Notatio ad Limits Th rmaiig symbols, Θ, Ω, o, ω, ad, all hav dfiitios similar to O: Thr xist costats blah ad blah such that for all blah, such ad such holds. This may prompt a ss of dja vu: ths dfiitios ar all quit similar to th dfiitio of a limit. I fact, this similarity has a happy cosquc: all ths symbols hav simpl, ituitiv itrprtatios i trms of limits. Ths ar summarizd i th tabl blow. Notatio Ituitio lim f/g Exampl x f = O(g) f grows o fastr tha g = 4x + 7 = O(x ) f = Ω(g) f grows at last as fast as g = 0 9x = Ω(x) f = Θ(g) f ad g grow at about th sam rat = 0, 8x + x = Θ(x ) f g f ad g grow at th sam rat = x + x x f = o(g) f grows slowr tha g = 0 / = o() f = ω(g) f grows fastr tha g = = ω() This summary of asymptotic otatio is valid for sstially all fuctios coutrd i computr scic. Howvr, i th rar cas wh lim f/g dos ot xist or is gativ, o must cosult th formal, it picky dfiitios of ths symbols. Ths dfiitios ar providd i th ots for th rcitatio followig this lctur. 3.4 Etr th Dark Sid So wh somo talks asymptotics, why should you rach for your light sabr? Asymptotic otatio is so widly usd to aalyz algorithm ruig tims that thr is a tmptatio to dsig for th otatio rathr tha for tru usfulss. Rmmbr that asymptotic otatio cocals som iformatio about a algorithm s prformac whil highlightig othr aspcts. So wh a rsarchr dsigs a algorithm, h or sh might mak tradoffs so that th rvald part look good v though th cocald parts mak th algorithm udsirabl.

12 Sums, Approximatios, ad Asymptotics II Idd, thr ar som otabl xampls of algorithms whr asymptotic otatio obscurs Dathstar sizd problms. For xampl, thr is a igious algorithm that multiplis two matrics i O(.376 ) stps istad of th obvious O( 3 ). Howvr, th O symbol cocals a costat so ormous that th aiv multiplicatio algorithm is prfrabl up to at last = 0 0. No o sms to hav figurd out xactly whr th crossovr poit is, but w r ulikly to com aywhr ar it i this ctury at last. Aothr xampl ivolvs a algorithm that fids a approximat solutio to th Travlig Salsma Problm i arly liar tim. Th ruig tim to fid a approximatio withi 0% of optimal is aroud O( log 400 ). Now, i a ss, th author s claim that this is arly liar is corrct; for xampl: But this is just a tad misladig sic.0 log 400 = O( ).0 log 400 for all < 0 00,000. Ths xtrm xampls ar wll kow, but whthr misus of asymptotic otatio drivs a wdg btw algorithmic thory ad practic mor grally is a qustio you might podr. Th moral is: us asymptotic otatio to clarify, ot cocal. Avoid th Dark Sid, you must.