Chapter Developig Efficiet Algorithms 1
Executig Time Suppose two algorithms perform the same task such as search (liear search vs. biary search). Which oe is better? Oe possible approach to aswer this questio is to implemet these algorithms i Java ad ru the programs to get executio time. But there are two problems for this approach: First, there are may tasks ruig cocurretly o a computer. The executio time of a particular program is depedet o the system load. Secod, the executio time is depedet o specific iput. Cosider liear search ad biary search for example. If a elemet to be searched happes to be the first i the list, liear search will fid the elemet quicker tha biary search.
Growth Rate It is very difficult to compare algorithms by measurig their executio time. To overcome these problems, a theoretical approach was developed to aalyze algorithms idepedet of computers ad specific iput. This approach approximates the effect of a chage o the size of the iput. I this way, you ca see how fast a algorithm s executio time icreases as the iput size icreases, so you ca compare two algorithms by examiig their growth rates. 3
Big O Notatio Cosider liear search. The liear search algorithm compares the key with the elemets i the array sequetially util the key is foud or the array is exhausted. If the key is ot i the array, it requires comparisos for a array of size. If the key is i the array, it requires / comparisos o average. The algorithm s executio time is proportioal to the size of the array. If you double the size of the array, you will expect the umber of comparisos to double. The algorithm grows at a liear rate. The growth rate has a order of magitude of. Computer scietists use the Big O otatio to abbreviate for order of magitude. Usig this otatio, the complexity of the liear search algorithm is O(), proouced as order of. 4
Best, Worst, ad Average Cases For the same iput size, a algorithm s executio time may vary, depedig o the iput. A iput that results i the shortest executio time is called the best-case iput ad a iput that results i the logest executio time is called the worst-case iput. Best-case ad worst-case are ot represetative, but worst-case aalysis is very useful. You ca show that the algorithm will ever be slower tha the worst-case. A average-case aalysis attempts to determie the average amout of time amog all possible iput of the same size. Average-case aalysis is ideal, but difficult to perform, because it is hard to determie the relative probabilities ad distributios of various iput istaces for may problems. Worst-case aalysis is easier to obtai ad is thus commo. So, the aalysis is geerally coducted for the worst-case. 5
Igorig Multiplicative Costats The liear search algorithm requires comparisos i the worst-case ad / comparisos i the average-case. Usig the Big O otatio, both cases require O() time. The multiplicative costat (1/) ca be omitted. Algorithm aalysis is focused o growth rate. The multiplicative costats have o impact o growth rates. The growth rate for / or 100 is the same as, i.e., O() = O(/) = O(100). f() / 100 100 100 50 10000 00 00 100 0000 f(00) / f(100) 6
Igorig No-Domiatig Terms Cosider the algorithm for fidig the maximum umber i a array of elemets. If is, it takes oe compariso to fid the maximum umber. If is 3, it takes two comparisos to fid the maximum umber. I geeral, it takes -1 times of comparisos to fid maximum umber i a list of elemets. Algorithm aalysis is for large iput size. If the iput size is small, there is o sigificace to estimate a algorithm s efficiecy. As grows larger, the part i the expressio -1 domiates the complexity. The Big O otatio allows you to igore the o-domiatig part (e.g., -1 i the expressio -1) ad highlight the importat part (e.g., i the expressio -1). So, the complexity of this algorithm is O(). 7
8 Useful Mathematic Summatios 1 1... 1 1... 1) ( 1) (... 3 1 1 1) ( 3 1 0 1 1) ( 3 1 0 = = = a a a a a a a a
Examples: Determiig Big-O Repetitio Sequece Selectio Logarithm 9
Repetitio: Simple Loops executed times for (i = 1; i <= ; i) { k = k 5; costat time Time Complexity T() = (a costat c) * = c = O() Igore multiplicative costats (e.g., c ). 10
Repetitio: Nested Loops executed times for (i = 1; i <= ; i) { for (j = 1; j <= ; j) { k = k i j; costat time ier loop executed times Time Complexity T() = (a costat c) * * = c = O( ) Igore multiplicative costats (e.g., c ). 11
Repetitio: Nested Loops executed times Time Complexity for (i = 1; i <= ; i) { for (j = 1; j <= i; j) { k = k i j; costat time T() = c c 3c 4c c = c(1)/ = (c/) (c/) = O( ) Igore o-domiatig terms Igore multiplicative costats ier loop executed i times 1
Repetitio: Nested Loops executed times for (i = 1; i <= ; i) { for (j = 1; j <= 0; j) { k = k i j; costat time ier loop executed 0 times Time Complexity T() = 0 * c * = O() Igore multiplicative costats (e.g., 0*c) 13
Sequece executed 10 times executed times for (j = 1; j <= 10; j) { k = k 4; for (i = 1; i <= ; i) { for (j = 1; j <= 0; j) { k = k i j; ier loop executed 0 times Time Complexity T() = c *10 0 * c * = O() 14
O() Selectio if (list.cotais(e)) { System.out.pritl(e); else for (Object t: list) { System.out.pritl(t); Let be list.size(). Executed times. Time Complexity T() = test time worst-case (if, else) = O() O() = O() 15
Costat Time The Big O otatio estimates the executio time of a algorithm i relatio to the iput size. If the time is ot related to the iput size, the algorithm is said to take costat time with the otatio O(1). For example, a method that retrieves a elemet at a give idex i a array takes costat time, because it does ot grow as the size of the array icreases. 16
Commo Recurrece Relatios Recurrece Relatio Result Example = T( / ) O(1) = O(log ) Biary search, Euclid s GCD = T ( 1) O(1) = O( ) Liear search = T( / ) O(1) = O( ) = T( / ) O( ) = O( log) Merge sort (Chapter 4) = T( / ) O( log) = O( log ) = T ( 1) O( ) = O( ) Selectio sort, isertio sort = T( 1) O(1) = O( ) Towers of Haoi = T( 1) T( ) O(1) = O( ) Recursive Fiboacci algorithm 17
Comparig Commo Growth Fuctios 3 O (1) < O(log) < O( ) < O( log ) < O( ) < O( ) < O( ) O(1) O(log) O() O( log) O( ) O( 3 ) O( ) Costat time Logarithmic time Liear time Log-liear time Quadratic time Cubic time Expoetial time 18
Comparig Commo Growth Fuctios 3 O (1) < O(log) < O( ) < O( log ) < O( ) < O( ) < O( ) O( ) O( ) O(log) O() O(log) O(1) 19
Practical Cosideratios The big O otatio provides a good theoretical estimate of algorithm efficiecy. However, two algorithms of the same time complexity are ot ecessarily equally efficiet. Liag, Itroductio to Java Programmig, Teth Editio, (c) 013 Pearso Educatio, 0 Ic. All