A New Method to Order Functions by Asymptotic Growth Rates Charlie Obimbo Dept. of Computing and Information Science University of Guelph

Transcription

1 A New Method to Order Fuctios by Asymptotic Growth Rates Charlie Obimbo Dept. of Computig ad Iformatio Sciece Uiversity of Guelph ABSTRACT A ew method is described to determie the complexity classes of fuctios by compariso. This method is a similar to that of L'Hospital. It applies the logarithmic fuctio to the give fuctios to ad the fid the limits. The ew rule ca compare the complexity classes of most fuctios, givig more iformatio tha L'Hospital's Rule, ad i some cases, solvig some that are difficult to solve, by purely usig L'Hospital's Rule. There are some fuctios, however, which oe has to revert to applyig L'Hospital's Rule. Keywords: Algorithms, Complexity, Big-Oh, Asymptotic Growth, L'Hospital's Rule. 1. Itroductio I this paper, we discuss the use of a ew approach i dealig with complexity rakigs of fuctios i a Algorithm Aalysis Course. We itroduce a ew method ad attempt to aalyze some obvious, ad some ot so obvious fuctios, i terms of their growth. The method gives a better aalytical basis, rather tha the sometimes popular ad ituitioal (amog studets), plug ' play (pluggig large umbers i the fuctios ad hopig for the best) method. Aother method is to either use Maple or Matlab, to plot graphs or write a program that would tell you the comparative orders of complexity. The method itroduced i this paper is still preferable, i that, oe -- it would take a very brief time to use it to evaluate the relative complexity orders of two fuctios, thus quickly assessig whether oe algorithm is better tha aother. Secodly, by cotiual use of it, oe becomes adept to statig whether oe fuctio grows faster tha aother. 1

2 . Defiitios This sectio gives first a iformal remider of the symbols used i complexity, ad the some formal defiitios of these terms. These symbols (O, Ω, o, θ, ω) also called Ladau symbols, have precise mathematical defiitios. They have may uses, such as i the aalysis of algorithms. These symbols are used to evaluate ad to cocisely describe the performace of a algorithm i time ad space. Sice the properties related to these symbols hold for asymptotic otatios, oe ca draw a aalogy betwee the asymptotic compariso of two fuctios f ad g ad the compariso of two real umbers a ad b. We will use this aalogy, i the table below to give a brief iformal remider of the symbols ames ad their use: Table.1 Ladau Symbols Symbol Name Usage Aalogy O Big Oh Upper Boud f =O(g) a b Ω Big Omega Lower Boud f =Ω (g) a b θ Theta Same Order f =θ(g) a = b o Little Oh Strict Upper Boud f =o(g) a < b ω Little Omega Strict Lower Boud f =ω(g) a > b A illustratio of O ad Ω are give i the Figures.1 ad. below: Figure.1 f() = Ω(g()) Figure. f() = O(g())

3 We ow formally defie Big Oh, Theta ad Big Omega. The reader ca skip this part, as it is ot madatory to uderstad the rest of the paper. We defie f ad g to be fuctios from the set of itegers to the set of positive real umbers. Defiitio.1 [Big Oh ] [Upper Boud] f() = O(g()) iff there exist positive costats c ad 0 such that f() c g() for all 0. Example: + = O() as + 4 for. Defiitio. [Big Omega ] [Lower Boud] f() = Ω(g()) iff there exist positive costats c ad 0 such that f() c g() for all 0. Example: + = Ω() as + for 1 Defiitio. [Theta] [Tight Boud] f() = θ(g()) iff there exist positive costats c 1, c ad 0 such that c 1 g() f() c g() for all 0. Example: + = θ() as + 4 for..1 Calculatio of Big Oh Relatios f ( ) Let lim = L. g( ) The if L = 0, f() = O(g()). I particular, f() = o(g()). If L =, f() = Ω(g()) ad i particular, f() = ω(g()). If 0 < L <, f() = θ(g()). (.1)

4 . L Hospital (l Hôpital) s Rule f ( ) lim is ot always easy to calculate. For example take /. Sice both g ( ) ad go to as goes to ad there is o apparet factor commo to both, the calculatio of the limit is ot immediate. Oe tool we may be able to use i such cases is L'Hospital's Rule, which is give as a theorem below. Theorem.1 [L'Hospital's Rule] Let lim f ( ) = ad lim g( ) =, ad let f() ad g() both have their first f ( ) f '( ) derivatives, f () ad g (), respectively, the lim = lim. g ( ) g '( ) Example.1: Let f() = ad g() =. The ( )' ( )' ( ) ' f ( ) lim = lim = lim = lim = lim = lim = 0 ( ) l g l ( l )' Thus f() = O(g()) by Equatio (.1). f() is i fact o(g()), sice is a loose upper boud of. Note that this rule ca be applied repeatedly as log as the coditios are satisfied. Example.: Now cosider the fuctios f() = ad g() =. It is easy to see that the first is a loose upper-boud of the secod, i.e. f() = ω(g()). However, if we attempt to apply L Hôpital s Rule, the first few steps would be as follows. We first calculate the derivatives of the fuctios f() ad g(). f () =?. Let y = the, by Logarithmic Differetiatio, l y = l thus l y = l. 1 y 1 y' = + l = 1+ l y' = y( 1+ l ) = ( 1+ l ) 4

5 This meas f '( ) ( 1+ l ) = ad g '( ) = l f ( ) ( )' Thus = lim = lim ( ) g ( )' (1 + l ) lim = lim =... l At this poit it seems to be a good time to stop doig it aalytically, sice the expressio appears to get oly worse. Below are other pairs of fuctios which oe may fid it difficult to compare by usig L Hôpital s Rule directly, without usig MatLab, Maple, or tryig to plot. 1. lg ad lglg. ad 1 +. lg ad lg! The last oe is compouded by the fact that the derivative of! caot be foud directly. Oe may use Stirlig s Approximatio. This, however, does ot make the determiatio of complexity orderig usig L Hôpital s Rule ay easier. Aside from the use of Mathematical ad Graphig packages, aalytical determiatio becomes a ightmare. I the followig sectio, we look at a rule that is a modificatio of L Hôpital s Rule, which facilitates the aalytical determiatio of the complexity orders of most fuctios. 4. Charlie s Rule The mai differece betwee L Hôpital s Rule ad Charlie s Rule is that i Charlie s Rule, we take the logs of the fuctios ad the differetiate, if ecessary. This makes sese i that if the log of a positive icreasig mootoe grows faster tha aother, i.e. the limit of their quotiet, as approaches ifiity, is greater tha 1, the the former ecessarily grows faster tha the later. This is evidet due to the facts that, were their expoets of the same base (i.e. say for example for ad the base is ), the i takig the logs, what we are actually comparig are the expoets. [This ideed is a rudimetary proof.] 5

6 Before we state the Theorem, let us demostrate a example or two. Ideed we ca look at Examples.1 ad. which have bee doe (ad i the secod case, attempted to be doe) usig L Hôpital s Rule. Example 4.1: (Similar to Example.1) Let f() = ad g() =. The By L Hôpital s Rule: ( )' ( )' ( ) ' f ( ) lim = lim = lim = lim = lim = lim = 0 ( ) l g l By Charlie s Rule: ( l )' f ( ) lim g( ) = lim CR lg lim lg lg = lim = lg lg lim ( 1/ ) () 1 = 0 1 The otatio CR is just merely to show that we are applyig Charlie s Rule, ad thus the left had side may ot ecessarily be equal to the right had side. However, if the RHS teds to L > 1, the the left had limit teds to ifiity, ad the coverse, RHS teds to 0 L < 1, the the left had limit teds to zero. Also i step 1 (with the arrow) with prior kowledge that the log fuctio (lg ) is subliear (the i the deomiator), we ca already tell that f() = o(g()). I step we applied L Hôpital s Rule. Example 4.: (Similar to Example.) Now let us recosider the fuctios f() = ad g() =. Recall that i Example. we fially got to the statemet: ( )' ( )' f ( ) (1 + l ) lim = lim = lim = lim =... ( ) g l which was ot exactly desirable, usig L Hôpital s Rule. Usig Charlie s Rule we get: f ( ) lim = lim g( ) lg lim lg lg lg = lim = lim = lg lg Thus: f() = ω(g()) (f() grows much faster tha g() ). 6

7 We ow state the ew rule as follows: Theorem 4.1 [Charlie's Rule] Let f() ad g() be positive mootoes over the iterval [a, ), for some a>0. Also let lim f ( ) = ad lim g( ) =. (I other words we have a idetermiate form ). Also let lg f ( ) lim lg g( ) = L, the if if L < 1 L > 1 the the f ( ) = o f ( ) = ω ( g( ) ) ( g( ) ) Note that for this rule, the limit (L) does ot have to be. If it is greater tha 1, the the fuctio i the umerator grows faster. If it is betwee 0 ad 1, icludig 0, the the umerator grows slower. For example ad : lim lg lim lg = < 1 Thus = O( ), or i particular = o( ) ( grows slower tha ). 4.1 What more do we get with Charlie s Method Ideed there is ew iformatio obtaied by usig Charlie s Method, which may be ot as evidet whe usig L Hôpital s Rule. Oe is that,where as i L Hôpital s Rule we are just cocered with the limit of the Quotiet attaiig ifiity, a costat or zero, ad it just tells us whether the fuctio grow faster, slower, or is of the same order, i Charlie s method we may get a umber betwee 0 ad 1, or 1, or a umber betwee 1 ad ifiity. We ca therefore tell, depedig o the quotiet obtaied by the logarithmic limit, whether a fuctio belogs to the same class. For example, i the last example discussed i Sectio 4, ( ad ), we ca tell that they belog to the same family of fuctios (polyomials), sice their logarithmic limit is /. I Example 4.1, however, ad, do ot belog to the same class sice the limit of their logarithmic quotiet is 0. Ideed belogs to the Polyomial class, whereas, belogs to the Expoetial Class. 7

8 Secodly, we ca ow easily solve ew problems aalytically that could t be easily solved before usig L Hôpital s Rule (without other aids). The example below illustrates just oe of the may examples. Example 4.: Cosider the fuctios f lg ( ) = ad ( ) lg! g =. The lg lg f ( ) = lg = lg lg = lg ad lg g ( ) = lglg! [Recall that lg = log ] However, sice lg! = θ ( lg ), the ( lg( lg ) ) = θ ( lg + lglg ) ) θ ( lg ). lg g( ) = lglg! = θ = lg f ( ) lg Therefore lim = lim = lim = 0 lg g( ) lg lg Hece lg = o(lg!), or lg grows much slower tha lg!. 5. Discussios ad Coclusios 5.1 Problems that caot be solved with Charlie s Method There are problems that caot be solved usig Charlie s Method. These are the oes whose logarithmic quotiets give a result of 1 (i.e. L =1 i Theorem 4.1). Example 4.4: Let f ( ) = ad g( ) = lg. The i applyig Charlie s Rule, we get: lg f ( ) lg lg lim = lim = lim = 1 lg g( ) lg lg + lg lg ( lg ) The oly iformatio obtaied i this case is that they belog to the same class of algorithms (polyomials i this case), but we ca t tell whether oe grows faster or slower by usig Charlie s Method, as depicted i Theorem 4.1. These problems are, however trivial to solve i most cases, ad do ot eve eed L Hôpital s Rule. However, i cases that are slightly more complex, oe ca apply L Hôpital s Rule. 8

9 5.1 Studets View of the New Method The studets i the Algorithms: Aalysis ad Desig Courses (CIS490) at the Uiversity of Guelph ad (CS5) at the Uiversity of Price Edward Islad, have bee taught this method. Oe of their tasks was to aalyze a group of fuctios i icreasig asymptotic order. The example we took was from the CLR (ow CLRS) book [], problem - i Chapter (page 58). The problem asks to rak 0 fuctios (we oly iclude the first two lies of them) by order of growth; i.e. to fid a arragemet g 1, g, L, g of fuctios satisfyig g1 = Ω( g ) = Ω( g ) = L = Ω( g ). Some of the fuctios give are: lg* lg* ( lg* ) ( )! ( lg ) lg lg lg lg (! )! For may studets comig to Computer Sciece owadays, ad for may of us, this is a itimidatig ordeal. However, there was a great respose from the studets, oce they applied this ew method. They foud it faster, less agoizig, ad oce they had their heads aroud logs, they foud they did t have to remember much to be able to do the problems with ease, ad thus apply it to other parts of the course to compare the differet algorithms. 6. Refereces 1. T. Corme, C. Leiserso, R. Rivest, C. Stei. Itroductio to Algorithms, d Editio. MIT Press (McGraw Hill), G. Brassard ad P. Bratley. Fudametals of Algorithmics. Pretice Hall, R. Neapolita ad K. Naimipour. Foudatios of Algorithms: with C++ pseudocode, d Editio. Joes ad Bartlett Publishers, U. Maber. Itroductio to Algorithms: A Creative Approach. Addiso Wesley, K. Rose. Discrete Mathematics Ad Its Applicatios, 5th Editio. McGraw Hill, S. Epp. Discrete Mathematics With Applicatios, d Editio. Brooks/Cole Publishers Compay,

10 Author Charlie F. Obimbo Dept. of Computig ad Iformatio Sciece Uiversity of Guelph Guelph, ON N1G W1 10