Differet complexity measures Advaced Course of Algorithm Desig ad Aalysis Asymptotic complexity Big-Oh otatio Properties of O otatio Aalysis of simple algorithms A algorithm may may have differet executio time depedig o the iput data eve if the amout of data is the same Worst case complexity W() Average case complexity A() Best case complexity B() Uavoidable complexity T() 5 ms 4 ms 3 ms 2 ms 1 ms worst-case average-case? best-case 1/30 A B C D E F G Iput 2/30 Differet complexity measures Usually we talk about the worst-case complexity It may be more difficult to fid the average case complexity because we must assume some statistical properties about the iput ad do the aalysis how this affects the average complexity of the algorithm. It is importat to kow the worst case complexity of the software i the safety-critical applicatios where the good performace i average is ot eough. Sometimes it is importat to estimate the space complexity - how much memory (storage) is eeded Measurig the complexity How to measure the ruig time of a algorithm? We ca do experimets write a program implemetig the algorithm ru it o data of differet size ad with differet properties measure the time of executio. (eg. System.curretTimeMillis() i Java) t (ms) 60 50 40 30 20 10 3/30 0 50 100 4/30 Shortcomigs of the experimetal complexity To aalyse the complexity of a algorithm, we have to implemet a program ad ru it. compariso ca be doe oly o the fixed software/hardware eviromet It is possible to experimet with oly a fiite umber of iputs ad this may ot represet the behaviour o all possible iput datasets. Geeral methodology We will look at some geeral methodology, which uses the abstract represetatio of a algorithm take accout of all possible iputs does ot deped o software/hardware eviromet 5/30 6/30 1
Complexity of algorithms Complexity of a algorithm is a fuctio C() of depedece betwee umber of operatios ad size of iput It is ot clear how to defie a operatio geerally somethig doe i hardware i some costat (ited) time arithmetical operatio, assigmet, compariso operatio umber of lies Iput size ca be defied differetly amout of iput data (array, file, database) value of iput parameter size of iput parameter (umber of bits/bytes) Asymptotical complexity Goal: simplify aalysis to discard the uimportat details (aalogous to roudig 1,000,001 1,000,000) We wat to say 3 2 2 3 F() - 2 F() We will use big-o otatio 3 2 o O( 2 ) 3 F() - 2 o O(F()) 4687 o O() 1,76*10 25 o O(1) 7/30 8/30 Asymptotical complexity Asymptotical complexity There are costats c ad N, so that for fuctios f() ad g() holds: f() c g(), if N c g() = 4 f() = 2 + 6 o the other side 2 is ot O() because there is o costats c ad N, so that: 2 c, if N coclusio: 2+6 is O(). g() = (We ca choose arbitrary c, but there is some, so that 2 >c ). 9/30 10/30 Big Oh otatio Eables you to talk about the complexity of the algorithms i rough terms Tool to compare the complexity of algorithms Basis for complexity classes time() = 3 F() - 2 time() = 2 time() = 4687 time() = 1,76*10 25 is O(F()) is O() is O() is O(1) Formally: A Geeral Portable Performace Metric O( g() ) is the set of fuctios, f, such that f() < c g() for some costat, c > 0, ad > N Alteratively, we may write ad say f() g() c ie for sufficietly large g is a upper boud for f 11/30 12/30 2
A Geeral Portable Performace Metric O( g ) - the set of fuctios that grow o faster tha g. A Geeral Portable Performace Metric O( g ) - the set of fuctios that grow o faster tha g. Two additioal otatios Ω( g ) - the set of fuctios, f, such that f() > c g() for some costat, c, ad > N g is a lower boud for f f() g() c 13/30 14/30 A Geeral Portable Performace Metric O( g ) - the set of fuctios that grow o faster tha g. Two additioal otatios Ω( g ) - the set of fuctios, f, such that f() > c g() for some costat, c, ad > N Θ( g ) = O( g ) Ω( g ) g is a lower boud for f Set of fuctios growig at the same rate as g 15/30 16/30 A Geeral Portable Performace Metric O( g ) - Big-O the set of fuctios that grow o faster tha g. o( g ) - little-o the set of fuctios f that grow slower tha g. f() = 0 g() ω( g ) - little-omega the set of fuctios f that grow faster tha g. f() = g() Properties of the O otatio Costat factors may be igored k > 0, kf is O( f) Higher powers grow faster r is O( s ) if 0 r s Fastest growig term domiates a sum If f is O(g), the f + g is O(g) eg a 4 + b 3 is O( 4 ) Polyomial s growth rate is determied by leadig term If f is a polyomial of degree d, the f is O( d ) If f o( g ) or f ω( g ) the f Θ( g ) 17/30 18/30 3
Properties of the O otatio Properties of the O otatio f is O(g) is trasitive If f is O(g) ad g is O(h) the f is O(h) Product of upper bouds is upper boud for the product If f is O(g) ad h is O(r) the fh is O(gr) Expoetial fuctios grow faster tha powers k is O( b ) b > 1 ad k 0 eg 20 is O( 1.05 ) Logarithms grow more slowly tha powers log b is O( k ) b > 1 ad k > 0 eg log 2 is O( 0.5 ) Importat! All logarithms grow at the same rate log b is O(log d ) b, d > 1 19/30 20/30 Properties of the O otatio All logarithms grow at the same rate log b is O(log d ) b, d > 1 Sum of first r th powers grows as the (r+1) th power Σ k r k=1 is Θ ( r+1 ) eg Σ i = (+1) is Θ ( 2 ) k=1 2 Summary Ο, Ω, Θ Ο, Ω, Θ, o, ω are the sets of fuctios O upper boud of the set Ω lower boud of the set Θ exact complexity class o, ω O\Θ, Ω\Θ O is ofte used whe Θ is cosidered Abstracts away a costat factor ad lower rak members O( 1000 3 + 34 + 1 ) = O( 3 ) Iclusio relatio of the commo complexity classes O(log ) O() O( 2 ) O( k ) O(k ) O(!) O( ) 21/30 22/30 Polyomial ad Itractable Algorithms Polyomial Time complexity A algorithm is said to be polyomial if it is O( d ) for some iteger d Polyomial algorithms are said to be efficiet They solve problems i reasoable times! Itractable algorithms Algorithms for which there is o kow polyomial time algorithm We will come back to this importat class later i the course Aalysig a Algorithm Simple statemet sequece s 1 ; s 2 ;. ; s k O(1) as log as k is costat Simple loops for(i=0;i<;i++) { s; where s is O(1) Time complexity is O(1) or O() Nested loops for(i=0;i<;i++) for(j=0;j<;j++) { s; Complexity is O() or O( 2 ) This part is O() 23/30 24/30 4
Aalysig a Algorithm Loop idex does t vary liearly h = 1; while ( h <= ) { s; h = 2 * h; h takes values 1, 2, 4, util it exceeds There are 1 + log 2 iteratios Complexity O(log ) Aalysig a Algorithm Loop idex depeds o outer loop idex for(j=0;j<;j++) for(k=0;k<j;k++){ s; Ier loop executed 1, 2, 3,., times Complexity O( 2 ) Σ i = i=1 (+1) 2 Distiguish this case - where the iteratio cout icreases (decreases) by a costat O( k ) from the previous oe - where it chages by a factor O(log ) 25/30 26/30 Aalysis of programs Sequetial (liear) search Worst-case complexity W() = Best-case complexity B() = 1 Average-case complexity A() = ( + 1) / 2 If x is i the list A() = (p/) (+1)/2 + (1-p) If x is i the list with probability p No every-case complexity sequetial search(list L,item x) { for (each item y i the list) if (y matches x) retur y retur o match 27/30 28/30 Bubble (exchage) sort Some mathematical otatios Every-case complexity T() = (-1)/2 O( 2 ) If there is a every-case complexity the it is equal to worst, best, ad average-case complexity This sortig algorithm is simple but ueffective for (i=0; i<-1; i++) { for (j=0; j<-1-i; j++) /*compare eighbors*/ if (a[j+1] < a[j]) { /*swap a[j]. a[j+1]*/ tmp = a[j]; a[j] = a[j+1]; a[j+1] = tmp; x floor of x 7/2 = 3 x ceilig of x 7/2 = 4 sum i = 1 + 2 + 3 +... + i=1 29/30 30/30 5
Logarithms A commo logarithm of a umber x is the power to which 10 must be raised to produce the umber A logarithm of a umber x is the power to which the base a must be raised to produce the umber log 10 = 1 10 1 = 10 log 1000 = 3 10 3 = 1000 log 0.01 = -2 10-2 = 0.01 log 1 = 0 10 0 = 1 Properties of logarithms log 1 = 0 a log x = x log(xy) = log x + log y log x/y = log x - log y log x y = y log x y log x = x log y log a x = log b x / log b a log 2 8 = 3 2 3 = 8 log 2 7 2.807 2 2.807 7 lg x log 2 x l x log e x (e 2.7182818) 31/30 32/30 Home Assigmets Home assigmets at the course web page! Solve the exercises ad ad submit the solutios at the ext exercise lesso (ext week), Get the textbook from the library if you do t have oe. We will discuss the solutios i the exercise lesso. Write the programs of Fiboacci umbers! 33/30 6