Analysis of Algorithms. Growth of Functions

Transcription

1 Aalysis of Algorithms Growth of Fuctios Growth of Fuctios Asymptotic Notatio : Ο, Ω, Θ, ο, ω Asymptotic Notatio Properties Growth of Fuctios Growth Rates log Liear

2 lim f () / g() = Growth Rates 0 f () grows slower tha g() f () grows faster tha g() otherwise f () ad g() have the same growth rate f () g() lim f () / g() = 0 f () g() : f () grows slower tha g() log log ! l'hôpital's Rule lg vs. If f () ad g() are differetiable, lim f () =, lim g() =, ad lim f ' () / g ' () exists, the lim f () / g() = lim f ' () / g ' () f ( ) = lg g( ) = lg lim = 0 l = lim l 2 1 l = lim l = lim l = lim l 2 ( ) Every subliear fuctio grows faster tha ay polylogarithmic fuctio e.g., 0.01 vs. ( log ) 100 d l d d d = l / = 1/(2 )

3 Asymptotic y = 1 + x 2 asymptotes Asymptotic : ay approximatio value that gets closer ad closer to the truth, whe some parameter approaches a limitig value. Asymptotic Notatios Deal with the behaviour of fuctios i the limit (for sufficietly large value of its parameters) Permit substatial simplificatio (api mathematic, rough order of magitude) Classify fuctios by their growth rates Same Growth Rates No Faster Tha Growth Rates log log log log + 7 log! log log log log + 7 log! Θ (1) Θ (log ) Θ ( 0.5 ) Θ ( log ) Θ ( 5 ) Ο (1) Ο (log ) Ο ( 0.5 ) Ο ( log ) Ο ( 5 )

4 Slower Tha Growth Rates No Slower Tha Growth Rates log 2 2 log log + 9 6log + 7 log! ο (2 ) ο (log ) ο ( 0.5 ) ο () ο ( 3 ) log log + 9 Ω (1) Ω (log ) log log + 7 log! Ω ( 0.5 ) Ω ( log ) Ω ( 5 ) Faster Tha Growth Rates Special Orders of Growth log log + 9 ω (1) ω ( 0.1 ) log ω () 6log + 7 log! ω ( 4 ) costat : Θ( 1 ) logarithmic : Θ( log ) polylogarithmic : Θ( log c ), c 1 subliear : Θ( a ), 0 < a < 1 liear : Θ( ) quadratic : Θ( 2 ) polyomial : Θ( c ), c 1 expoetial : Θ( c ), c > 1

5 Aalogy O-otatio f () ad g() f ad g (real umbers) f () = Θ(g()) f = g f () = Ο(g()) f g f () = ο(g()) f < g f () = Ω(g()) f g f () = ω(g()) f > g Not all fuctios are asymptotically comparable vs. 1+ si O( g()) = { f () :there exist positive costats c ad 0 such that 0 f () c g() for all 0 } O( 3 )? ??? 3 for all 03? Asymptotic Upper Boud Ω-otatio 0 f () = O( g() ) c g() f () Ω( g()) = { f () :there exist positive costats c ad 0 such that 0 c g() f () for all 0 } Ω( )? 100??? for all 4? 0

6 Asymptotic Lower Boud Θ-otatio 0 f () = Ω( g() ) f () c g() Θ( g()) = { f () : there exist positive costats c 1, c 2, ad 0 such that 0 c 1 g() f () c 2 g() for all 0 } Θ( 2 )???? ??? 28 2 for all 0? Asymptotic Tight Boud ο-otatio ad ω-otatio c 2 g() c 1 g() f () f () = ο( g() ) lim f () g() = 0 f () = ω( g() ) lim f () = g() 0 f () = Θ( g() )

7 Asymptotic Notatio Properties Trasitivity Reflexivity Symmetry Traspose symmetry Trasitivity f () = Θ(g()) ad g() = Θ(h()) imply f () = Θ(h()) f () = Ο(g()) ad g() = Ο(h()) imply f () = Ο(h()) f () = Ω(g()) ad g() = Ω(h()) imply f () = Ω(h()) f () = ο(g()) ad g() = ο(h()) imply f () = ο(h()) f () = ω(g()) ad g() = ω(h()) imply f () = ω(h()) f () = Θ( f () ) f () = Ο( f () ) f () = Ω( f () ) Reflexivity Symmetry ad Traspose Symmetry f () = Θ( g () ) if ad oly if g() = Θ( f () ) f () = Ο( g () ) if ad oly if g() = Ω( f () ) f () = ο( g () ) if ad oly if g() = ω( f () )

8 Θ, Ω, Ο Example f () = Θ( g () ) if ad oly if f () = Ω( g () ) ad f () = Ο( g () ) Ο 1 i 1 = +1 = Ο 1 +1 ( ) i = Θ + 1 ( ) 1 i / 2 i / = = Ω ( ) Ω Ο Ω Example log! = Θ( log )! = (-1) 2 1 = log! log = O( log )! = (-1) (/2) (/2-1) 2 1 /2 /2 / (/2) /2 log! (/2) log (/2) = Ω( log ) Logarithms Asymptotically, i logarithm, the base of the log does ot matter log b = ( log c ) / ( log c b ) log 10 = ( log 2 ) / ( log 2 10 ) = Θ( log ) ay polyomial fuctio of does ot matter log 30 = 30 log = Θ( log ) lg, l, log (CS) (math) (asymptotic)

9 Polylog, Polyomial, Expoetial Ay positive expoetial fuctio grows faster tha ay polyomial fuctio f () : mootically growig fuctio, ( f () ) c = o( a f () ), c > 0, a > 1 Ay positive polyomial fuctio grows faster tha ay polylogarithmic fuctio log b = o( c ) Oe-Way Equality = O( 2 ) O( 2 ) O ( 2 ) = Why do t we use? (GKP) traditio : the practice stuc traditio : CS is used to abuse equal sig traditio : read = as is, is is oe-way atural : whe we do asymptotic calculatio Asymptotic Notato i Equatios Maipulatig Asymptotic Notatios H = 1/1 + 1/2 ++ 1/ = l + γ + O(1/) = Θ() = f (), f () = Θ() Elimiate iessetial detail e.g., MergeSort : T() = 2T(/2) +? T() = 2T(/2) + (-1)? T() = 2T(/2) + Θ() = Θ(log ) c O ( f () ) = O( f () ) O(O ( f () )) = O( f () ) O ( f () )O( g () ) = O( f () g() ) O ( f () g() ) = f () O( g() ) O ( f () + g() ) = O( max( f (), g() ) ) Σ = 1 Σ = 1 O( f () ) = O( f () )

10 Example : BuildHeap Coclusio h= 0 = lg h 2 O( h) = h O h= h 0 2 = h O h h=0 2 = O(2) = O() Growth of fuctios give a simple characterizatio of fuctio s behavior allow us to compare the relative growth rates of fuctios Use asymptotic otatio to classify fuctios by their growth rates Asymptotics is the art of owig where to be sloppy ad where to be precise Do Kuth Father of aalysis of algorithm Author of The Art of Computer Programmig Programmer of the TEX ad METAFONT