Big O Notation for Time Complexity of Algorithms

Transcription

1 BRONX COMMUNITY COLLEGE of he Ciy Uiversiy of New York DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE CSI 33 Secio E01 Hadou 1 Fall 2014 Sepember 3, 2014 Big O Noaio for Time Complexiy of Algorihms Time depeds o size of he daa Whe workig wih a sigifica amou of daa, he ime i akes o process i will of course deped o how much daa here is. As he daa size icreases, some algorihms will eveually be faser ha ohers (eve hough hey may be performig he same ask, such as sorig a array). Here are some simple examples of Pyho fucios which have ruig ime which depeds o, he size of he daa se beig processed (he umber of idividual daa iems). We ca express his relaio bewee ruig ime ad daa size, usig fucio oaio, as = f(). I all cases, he fucio akes a sigle parameer a which is a Pyho lis. A Pyho lis is implemeed as a array of daa refereces; if he legh of he lis is, he here are daa iems i he lis. A cosa ime algorihm For our firs example, he sigle lie of code i he body of he fucio is execued oce, ad he fucio reurs. fucio0(a) a[0] = 0; The ruig does o deped o he array size here; if he ime i akes o ru he fixed block of code is k 0, he he formula for he ruig ime is = k 0. I a graph of he fucio f() = k 0, we ca see ha if he size icreases arbirarily, he ruig ime remais cosa; oly he firs eleme of he array is ever used. 1

2 A liear ime algorihm For our secod example, he body of he fucio cosiss of a loop which execues imes. fucio1(a) = le(a) for i = 0; i < ; i++ a[i] = 0; So he fucio will ake loger if he size of he array is larger, i direc proporio: = k 1, where k 1 is he ime i akes o ru he code i he loop oce. The graph of he fucio f() = k 1 is a sraigh lie hrough he origi whose slope is k 1. Comparig ay liear fucio wih a cosa fucio shows ha he cosa fucio is eveually faser, sice he value of says cosa, while i he liear fucio i always icreases as ges larger. 2

3 A quadraic ( 2 ) ime algorihm For our hird example, he body of he fucio cosiss of a loop which execues imes. Bu esed wihi his loop is aoher loop which also execues imes. The code i he ier loop rus imes for each ime he ouer loop rus. Sice he ouer loop also has ieraios, he ier code mus execue 2 imes. So he fucio will ake 2 imes he legh of ime i akes o ru he ier loop oce. If he ime o ru he ier loop oce is k 2, he formula for ruig ime is = k 2 2. fucio2(i a) for (i = 0; i < ; i++) for (j = 0; i < ; j++) a[i][j] = 0; The graph of his quadraic fucio akes he shape of a parabola. Comparig his graph wih a liear fucio, we see ha he quadraic fucio is o as good as liear: eveually, ay liear fucio will lie below he parabola, meaig ime is smaller for large eough daa size. 3

4 A cubic ( 3 ) ime algorihm For our ex example, he body of he fucio cosiss of a loop which execues imes, wih wo levels of esig of ier loops, each ieraig imes. Bu esed wihi his loop is aoher loop which also execues imes. The fucio will ake 3 imes he legh of ime i akes o ru he ier loop oce, ha is, if he ime o ru he ier loop oce is k 3, he formula for ruig ime is = k 3 3. fucio3(i a) for (i = 0; i < ; i++) for (j = 0; i < ; j++) for (k = 0; i < ; k++) a[i][j][k] = 0; If we were o graph his fucio, i would have he ypical shape of a cubic fucio, icreasig more rapidly ha ay quadraic fucio. I fac, i comparig ay polyomial fucios, i is oly he erm of highes degree which maer. The graph of ay quadraic fucio (wih degree 2) will always be a parabola, so he fucio will eveually be less ha ay polyomial of degree 3. A logarihmic ime algorihm The ex example is a recursive algorihm o perform a biary search of a sored array o fid a specific value (from he ex, page 580, bu wih he parameer size chaged o ) o follow he oher examples: # bsearch.py def search(iems, arge): pre: iems is a **sored** lis of umbers pos: reurs o-egaive x where iems[x] == arge, if arge i iems; reurs -1, oherwise low = 0 high = le(iems) - 1 while low <= high: # There is sill a rage o search mid = (low + high) // 2 # posiio of middle iem iem = iems[mid] if arge == iem : # Foud i! Reur he idex reur mid elif arge < iem: # x is i lower half of rage high = mid - 1 # move op marker dow else: # x is i upper half low = mid + 1 # move boom marker up reur -1 # o rage lef o search, # x is o here To simplify he aalysis, assume ha he size,, is always a power of 2. Each recursive call o search divides by wo, reducig by oe power of wo. I oher words, he logarihm o base wo of m decreases by oe. So he maximum umber of calls will be he umber eeded o reduce he logarihm of o zero. This will ake log 2 () recursive calls, wih each call akig a cosa ime. This aalysis reveals ha he ime o perform his fucio is, o average, proporioal o he logarihm (base 2) of : = klog 2 (). From precalculus, recall ha chagig he base of he logarihm chages he cosa, bu i is sill some cosa. So he impora par of he fucio is ha i is a logarihm, ad all such 4

5 fucios have he usual shape, muliplied by some cosa: = k log 2 (). Comparig his graph wih ay liear fucio, we see ha O log() is a improveme over O(), sice i is eveually faser. Big O Noaio All cosa fucios have he form f() = k 0 = k 0 1. They all have he propery ha hey are eveually faser ha liear fucios, which have he form f() = k 1 + k 0. We express his by sayig ha O(1), he class of cosa fucios, is eveually less ha he class of O(), he class of liear fucios. We wrie his as O(1) < O(). There is a hierarchy, or orderig, of he ruig imes of differe kids of algorihms based o wha big-o class hey are i: O(1) < O() < O( 2 ) < O( 3 ) ad so o; here a < b meas ba is eveually faser ha b. We have also see ha O(1) < O(log()) < O(). This meas ha ay O(1) fucio f 0 () = k 0 is eveually less ha ay O(log()) fucio f() = k 1 log(), which is eveually less (faser) ha ay O() fucio f() = k 1 + k 0. Aoher way of sayig his is ha here is some such ha, for all m >, k 0 < k 1 log(m) < k 2 m. Playig wih hese iequaliies, we muliply hem by m, which is greaer ha zero, so i does o chage he direcio of he iequaliies: eveually, k 0 m < k 1 m log(m) < k 2 m 2. So we kow ha O() < O( log()) < O( 2 ). This gives more 5

6 iformaio abou which algorihms are fases: O(1) < O(log()) < O() < O( log()) < O( 2 ), ad so o. Differe sorig algorihms are kow o have differe complexiies. Bubblesor, iserio sor ad selecio sor all have ime complexiy of O( 2 ), sice hese are loops wihi loops. Quicksor ad heapsor have complexiy O( log()), because hey are recursive, ad divide he sorig problem roughly i half for each sage of he ieraio. Oher cosideraios whe comparig algorihms Whe oe algorihm has he same big-o as aoher, you mus look a oher issues, like wha are he cosas o he fucios. A smaller cosa (a smaller or faser ier body of code) will make he algorihm wih he smaller cosa he beer choice. If a member fucio for a absrac daa ype is goig o be used wih more frequecy ha aoher fucio, he he ime complexiy (big-o) of he algorihm which i uses is more impora ha he big-o of he oher fucio. Coversely, if a member fucio is rarely used, i ca use a less ime-efficie algorihm ha a fucio which is used more ofe. If he size of he daa will be kow o be below a cerai value, he he cosas i he ime formulas become more impora ha he expoes, sice he applicaio may be i he par of he graph where he higher big-o fucio is sill below he lesser big-o fucio. The laer fucio oly wis eveually, bu for small values of his may o happe. Differe sorig algorihms are chose o his basis. If you are sorig a lis of less ha, say, 10 iems, a bubble sor (O( 2 )) may work as well as quicksor (O( log()). Bu if you are sorig a daabase of he eire U.S. populaio (over 3 millio), he bes opio is he mos efficie algorihm possible: here, heapsor or quicksor would be used isead of iserio sor or selecio sor, for example. 6