COMP26120: Introducing Complexity Analysis (2018/19) Lucas Cordeiro

Transcription

1 COMP60: Itroduig Complexity Aalysis (08/9) Luas Cordeiro

2 Itroduig Complexity Aalysis Textbook: Algorithm Desig ad Appliatios, Goodrih, Mihael T. ad Roberto Tamassia (hapter ) Itrodutio to Algorithms, Corme, Leiserso, Rivest, Stei (hapters ad 3)

3 Motivatig Example (COMP) What does this ode fragmet represet? What is its omplexity?! for(i = 0; i < N-; i) { for(j = 0; j < N-; j) { if (a[j]> a[j]) { t = a[j]; a[j] = a[j]; a[j] = t;! This is bubble sort There are two loops Both loops make - iteratios so we have (-)*(-) The omplexity is O( ) Perform worst ase aalysis ad igore ostats

4 Iteded Learig Outomes Defie asymptoti otatio, futios, ad ruig times Aalyse the ruig time used by a algorithm via asymptoti aalysis Provide examples of asymptoti aalysis usig the isertio sortig algorithm

5 Asymptoti Performae I this ourse, we are most about asymptoti performae We fous o the ifiite set of large igorig small values of o The best hoie for all, but very small iputs How does the algorithm behave as the problem size gets very large? Ruig time Memory/storage requiremets Badwidth/power requiremets/logi gates/et.

6 Asymptoti Notatio By ow you should have a ituitive feel for asymptoti (big-o) otatio: What does O() ruig time mea? O( )? O(lg )? How does asymptoti ruig time relate to asymptoti memory usage? Our first task is to defie this otatio more formally ad ompletely

7 Searh Problem (Arbitrary Sequee) Iput sequee of umbers (a, a ) searh for a speifi umber (q) a, a, a 3,.,a ; q Output idex or NIL j ; ; 9 NIL

8 Liear Searh j= while j<=legth(a) ad A[j]!=q do j if j<=legth(a) the retur j else retur NIL Worst ase: f()=, average ase: / Ca we do better usig this approah? this is a lower boud for the searh problem i a arbitrary sequee

9 A Searh Problem (Sorted Sequee) Iput sequee of umbers (a <=a,, a - <=a ) searh for a speifi umber (q) a, a, a 3,.,a ; q Output idex or NIL j ; ; 8 NIL Did the sorted sequee help i the searh?

10 Biary Searh Assume that the array is sorted ad the perform suessive divisios left= right=legth(a) do j=(leftright)/ if A[j]==q the retur j else if A[j]>q the right=j- else left=j while left<=right retur NIL

11 Aalysis of the Biary Searh How may times the loop is exeuted? At eah iteratio the umber of positios is ut i half How may times do we ut i half to reah? o lg lg = x = lg 8 = 3 x

12 Aalysis of Algorithms (COMP5) Aalysis is performed with respet to a omputatioal model We will usually use a geeri uiproessor radom-aess mahie (RAM) All memory equally expesive to aess Istrutios exeuted oe after aother (o ourret operatios) All reasoable istrutios take uit time o Exept, of ourse, futio alls Costat word size o Uless we are expliitly maipulatig bits

13 Iput Size Time ad spae omplexity This is geerally a futio of the iput size o E.g., sortig, multipliatio How we haraterize iput size depeds: o Sortig: umber of iput items o Multipliatio: total umber of bits o Graph algorithms: umber of odes & edges o Et

14 Ruig Time Number of primitive steps that are exeuted Exept for time of exeutig a futio all most statemets roughly require the same amout of time o f = (g h) (i j) o y = m * x b o = 5 / 9 * (t - 3 ) o z = f(x) g(y) We a be more exat if eed be add t0, g, h # temp t0 = g h add t, i, j # temp t = i j sub f, t0, t # f = t0 - t

15 Aalysis Worst ase Provides a upper boud o ruig time A absolute guaratee Average ase Provides the expeted ruig time Very useful, but treat with are: what is average? o Radom (equally likely) iputs o Real-life iputs

16 A Example: Isertio Sort

17 A Example: Isertio Sort i = j = key = A[j] = A[j] =

18 A Example: Isertio Sort i = j = key = 0 A[j] = 30 A[j] = 0

19 A Example: Isertio Sort i = j = key = 0 A[j] = 30 A[j] = 30

20 A Example: Isertio Sort i = j = key = 0 A[j] = 30 A[j] = 30

21 A Example: Isertio Sort i = j = 0 key = 0 A[j] = A[j] = 30

22 A Example: Isertio Sort i = j = 0 key = 0 A[j] = A[j] = 30

23 A Example: Isertio Sort i = j = 0 key = 0 A[j] = A[j] = 0

24 A Example: Isertio Sort i = 3 j = 0 key = 0 A[j] = A[j] = 0

25 A Example: Isertio Sort i = 3 j = 0 key = 40 A[j] = A[j] = 0

26 A Example: Isertio Sort i = 3 j = 0 key = 40 A[j] = A[j] = 0

27 A Example: Isertio Sort i = 3 j = key = 40 A[j] = 30 A[j] = 40

28 A Example: Isertio Sort i = 3 j = key = 40 A[j] = 30 A[j] = 40

29 A Example: Isertio Sort i = 3 j = key = 40 A[j] = 30 A[j] = 40

30 A Example: Isertio Sort i = 4 j = key = 40 A[j] = 30 A[j] = 40

31 A Example: Isertio Sort i = 4 j = key = 0 A[j] = 30 A[j] = 40

32 A Example: Isertio Sort i = 4 j = key = 0 A[j] = 30 A[j] = 40

33 A Example: Isertio Sort i = 4 j = 3 key = 0 A[j] = 40 A[j] = 0

34 A Example: Isertio Sort i = 4 j = 3 key = 0 A[j] = 40 A[j] = 0

35 A Example: Isertio Sort i = 4 j = 3 key = 0 A[j] = 40 A[j] = 40

36 A Example: Isertio Sort i = 4 j = 3 key = 0 A[j] = 40 A[j] = 40

37 A Example: Isertio Sort i = 4 j = 3 key = 0 A[j] = 40 A[j] = 40

38 A Example: Isertio Sort i = 4 j = key = 0 A[j] = 30 A[j] = 40

39 A Example: Isertio Sort i = 4 j = key = 0 A[j] = 30 A[j] = 40

40 A Example: Isertio Sort i = 4 j = key = 0 A[j] = 30 A[j] = 30

41 A Example: Isertio Sort i = 4 j = key = 0 A[j] = 30 A[j] = 30

42 A Example: Isertio Sort i = 4 j = key = 0 A[j] = 0 A[j] = 30

43 A Example: Isertio Sort i = 4 j = key = 0 A[j] = 0 A[j] = 30

44 A Example: Isertio Sort i = 4 j = key = 0 A[j] = 0 A[j] = 0

45 A Example: Isertio Sort i = 4 j = key = 0 A[j] = 0 A[j] = 0 Doe!

46 Isertio Sort What is the preoditio for this loop?

47 Isertio Sort How may times will this loop exeute?

48 Isertio Sort What is the post-oditio for this loop?

49 Isertio Sort Ivariat: A [..i-] osists of the elemets origially i A [..i-], but i sorted order

50 Isertio Sort Termiatio: whe i == we have A[..i-] whih leads to A[..]

51 Isertio Sort Statemet Effort (-) 3 (-) 4 T 5 (T-(-)) 6 (T-(-)) 0 7 (-) 0 T = t t 3 t where t i is umber of while expressio evaluatios for the i th for loop iteratio

52 Aalysig Isertio Sort T() = (-) 3 (-) 4 T 5 (T - (-)) 6 (T - (-)) 7 (-) What a T be? Best ase -- ier loop body ever exeuted T = t t 3 t T() = a - b 3 = = = = t t t t j j ( ) ( ) ( ) ( ) ) ( = T ( ) ( ) ) ( T =

53 ( ) ( ) ( ) ( ) ( ) ( )... 3 = = = = = = j = j = j j= j= j=! Gaussia Closed Form a be defied as: Thus, we have: Similarly, we obtai: Sum Review

54 Aalysig Isertio Sort Worst ase -- ier loop body exeuted for all previous elemets o T() = a b - Average ase o??? ( ) = j j= ( ) ( ) = j j= ( ) ( ) ( ) ( ) ( ) ( ) ( ) ) ( T = =

55 Simplifiatios Abstrat statemet osts Order of growth Worst ase Ruig time average-ase Average ase Best best-ase Iputs size

56 Growth Futios,00E55,00E43,00E3 T(),00E9,00E07,00E95,00E83,00E7,00E59 log sqrt log 00 ^ ^3 ^,00E47,00E35,00E3,00E,00E

57 Upper Boud Notatio IsertioSort s ru time is O( ) ru time is i O( ) Read O as Big-O I geeral a futio f() is O(g()) if there exist positive ostats ad 0 suh that f() g() for all 0 Ruig time 0 Iput size g( ) f ( )

58 Isertio Sort Is O( ) Proof: Use the formal defiitio of O to demostrate that a b = O( ) If ay of a, b, ad are less tha 0 replae the ostat with its absolute value o 0 f() k g() for all 0 (k ad 0 must be positive) o 0 a b <= k o 0 a b/ / <= k Questio Is IsertioSort O()?

59 Lower Boud Notatio IsertioSort s ru time is Ω() I geeral a futio f() is Ω(g()) if there exist positive ostats ad 0 suh that 0 g() f() 0 Proof: Suppose rutime is a b o 0 a b o 0 a b/ Ruig time 0 Iput size f ( ) g( )

60 Asymptoti Tight Boud A futio f() is Θ(g()) if there exist positive ostats,, ad 0 suh that 0 g() f() g() for all 0 Theorem f() is Θ(g()) iff f() is both O(g()) ad Ω(g()) Ruig time 0 Iput size g ( ) f ( ) g ( )

61 Exerise: Asymptoti Notatio Use the formal defiitio of Θ to demostrate that ( 3 = Θ ) Solutio: 0 ½ -3 for all 0 ½ - 3/ ½ ½ - 3/ /4 for 7 Note that e must be positive ostats For suffiietly large, the term ½ is kept =7 is the smallest value for to be a positive ostat Withi set otatio, a olo meas suh that

62 Exerise: Asymptoti Notatio Use the formal defiitio of Θ to demostrate that ( ) 3 6 Θ Solutio: para for all todo 0 0. This a ot be true for suffiietly large sie must be a ostat

63 Other Asymptoti Notatios A futio f() is o(g()) if positive ostats ad 0 suh that f() < g() 0 A futio f() is ω(g()) if positive ostats ad 0 suh that g() < f() 0 Ituitively, o() is like < O() is like ω() is like > Ω() is like Θ() is like =

64 Asymptoti Comparisos We a draw a aalogy betwee the asymptoti ompariso of two futios f ad g ad the ompariso of two real umbers a ad b f() = O(g()) is like a b f() = Ω(g()) is like a b f() = Θ(g()) is like a = b f() = o(g()) is like a < b f() = ω(g()) is like a > b Abuse of otatio: f() = O(g()) idia que f() O(g())

65 Exerise: Asymptoti Notatio Chek whether these statemets are true: a) I the worst ase, the isertio sort is b) ) d) e) f) Θ ( ) Θ( ) = Θ( ) O( = O( = O( )O( )= O( O() O()= O() ) ) ) Θ( )

66 Summary Aalyse the ruig time used by a algorithm via asymptoti aalysis asymptoti (O, Ω, Θ, o, ω) otatios use a geeri uiproessor radom-aess mahie Time ad spae omplexity (iput size) Best, average ad worst ase