Data Structures and Algorithm. Xiaoqing Zheng

Transcription

1 Data Structures ad Algorithm Xiaoqig Zheg

2 What are algorithms? A sequece of computatioal steps that trasform the iput ito the output Sortig problem: Iput: A sequece of umbers <a 1, a,, a > Output: A permutatio (reorderig) <a' 1, a',, a' > such that a' 1 a' a' Istace of sortig problem: Iput: <3, 45, 64, 8, 45, 58> Output: <8, 3, 45, 45, 58, 64>

3 Example of isert sort doe

4 Isertio sort INSERTION-SORT(A) 1 for j to legth[a] do key A[j] 3 // Isert A[j] ito the sorted sequece A[1 j 1] 4 i j 1 5 while i > 0 ad A[i] > key 6 do A[i + 1] A[i] 7 i i 1 8 A[i + 1] key A: 1 i j Sorted key

5 Ruig time The ruig time depeds o the iput: a already sorted sequece is easier to sort Parameterize the ruig time by the size of the iput, sice short sequeces are easier to sort tha log oes Geerally, we seek upper bouds o the ruig time, because everybody likes a guaratee

6 Kids of aalyses Worst-case: (usually) T() = maximum time of algorithm o ay iput of size Average-case: (sometimes) T() = expected time of algorithm over all iputs of size Need assumptio of statistical distributio of iputs Best-case: (bogus) Cheat with a slow algorithm that works fast o some iput

7 Aalysis of isertio sort INSERTION-SORT(A) 1 for j to legth[a] do key A[j] 3 // Isert A[j] ito the sorted sequece A[1 j 1] 4 i j 1 5 while i > 0 ad A[i] > key 6 do A[i + 1] A[i] 7 i i 1 8 A[i + 1] key cost c 1 c 0 c 4 c 5 c 6 c 7 c 8 times j = j= j= t 1 j ( t 1) j ( t 1) j T( ) = c+ c ( 1) + c ( 1) + c t + c ( t 1) + c ( t 1) + c ( 1) j= j 6 j= j 7 j= j 8

8 Aalysis of isertio sort: best ad worst T( ) = c+ c ( 1) + c ( 1) + c t + c ( t 1) + c ( t 1) + c ( 1) j= j 6 j= j 7 j= j 8 Best case: The best case occurs if the array is already sorted T( ) = c+ c ( 1) + c ( 1) + c ( 1) + c ( 1) = ( c + c + c + c + c ) ( c + c + c + c ) a + b (liear fuctio) Worst case: The worst case occurs if the array is i reverse sorted order ( + 1) ( 1) ( 1) T( ) = c 1 + c( 1) + c4( 1) + c5( 1) + c6( ) + c7( ) + c8( 1) c c c c c c = ( + + ) + ( c c + c4 + + c8) ( c + c4 + c5 + c8) a + b + c (quadratic fuctio)

9 Machie-idepedet time What is isertio sort's worst-case? It depeds o the speed of our computer: Relative speed (o the same machie), Absolute speed (o differet machies) BIG IDEA: Igore machie-depedet costats Look at growth of T() as "Asymptotic Aalysis"

10 Θ-otatio Math: Θ(g()) = { f() : there exist positive costats c 1, c, ad 0 such that 0 c 1 g() f() c g() for all 0 } Egieerig: Drop low-order terms; igore leadig costats Example: = Θ( 3 )

11 Asymptotic performace Whe gets large eough, a Θ( ) algorithm always beats a Θ( 3 ) algorithm T() 0

12 Merge sort MERGE-SORT A[1 ] 1 If = 1, doe Recursively sort A[1 / ] ad A[ / + 1 ] 3 "Merge" the sorted lists Key subroutie: MERGE

13 Mergig two sorted arrays ,, 7, 9, 11, 1, 13, 0 Time = Θ() to merge a total of elemets (liear time)

14 Operatio of merge sort sorted sequece merge merge merge merge merge merge merge iitial sequece

15 Aalyzig merge sort T() Θ(1) T(/) Θ() MERGE-SORT A[1 ] 1 If = 1, doe Recursively sort A[1 / ] ad A[ / + 1 ] 3 "Merge" the sorted lists Sloppiess: Should be T( / ) + T( / ), but it turs out ot to matter asymptotically

16 Recurrece for merge sort T ( ) Θ (1) if = 1, = T ( /) + Θ ( ) if > 1

17 Recurrece for merge sort Θ (1) if = 1, T ( ) = T ( /) + Θ ( ) if > 1 Solve T() = T(/) + c, where c > 0 is costat

18 Recurrece for merge sort Θ (1) if = 1, T ( ) = T ( /) + Θ ( ) if > 1 Solve T() = T(/) + c, where c > 0 is costat T()

19 Recurrece for merge sort Θ (1) if = 1, T ( ) = T ( /) + Θ ( ) if > 1 Solve T() = T(/) + c, where c > 0 is costat c T(c/) T(c/)

20 Recurrece for merge sort Θ (1) if = 1, T ( ) = T ( /) + Θ ( ) if > 1 Solve T() = T(/) + c, where c > 0 is costat c c/ c/ T(c/4) T(c/4) T(c/4) T(c/4)

21 Recurrece for merge sort Θ (1) if = 1, T ( ) = T ( /) + Θ ( ) if > 1 Solve T() = T(/) + c, where c > 0 is costat c c/ c/ c/4 c/4 c/4 c/4 c c c c c c c

22 Recurrece for merge sort Θ (1) if = 1, T ( ) = T ( /) + Θ ( ) if > 1 Solve T() = T(/) + c, where c > 0 is costat c c c/ c/ c h = lg c/4 c/4 c/4 c/4 c c c c c c c c c Total: c lg + c

23 Isertio sort ad merge sort Isertio sort: c 1 Computer A executes oe billio istructios per secod Merge sort: c lg Computer B executes te millio istructios per secod c 1 = c = 50 Isertio sort: Merge sort: 50lg To sort oe millio umbers: (10 6 ) istructios 10 9 istructios/secod = 000 secods (te millio, 3 day) To sort oe millio umbers: lg10 6 istructios 10 7 istructios/secod 100 secods (te millio, 0 miutes)

24 Aalysis of algorithms The theoretical study of computer-program performace ad resource usage What's more importat tha performace? Modularity Correctess Maitaiability Fuctioality Robustess User-friedliess Programmer time Simplicity Extesibility Reliability

25 Compariso of ruig times For each fuctio f() ad time t i the followig table, determie the largest size of a problem that ca be solved i time t, assumig that the algorithm to solve the problem takes f() microsecods 1 secod 1 miute 1 hour 1 day 1 moth 1 year 1 cetury lg 1/ lg 3!

26 Asymptotically tight boud Θ(g()) = { f() : there exist positive costats c 1, c, ad 0 such that 0 c 1 g() f() c g() for all 0 } c g() f() c 1 g() f() = Θ(g()) 0

27 Asymptotically upper boud O(g()) = { f() : there exist positive costats c ad 0 such that 0 f() cg() for all 0 } cg() f() f() = O(g()) 0

28 Asymptotically upper boud Ω(g()) = { f() : there exist positive costats c ad 0 such that 0 cg() f() for all 0 } f() cg() f() = Ω(g()) 0

29 Asymptotic otatios A aalogy betwee the asymptotic compariso of two fuctios f ad g the compariso of two real umbers a ad b: f() = O(g()) a b, f() = Ω(g()) a b, f() = Θ(g()) a = b Θ(g()) = O(g()) Ω(g())

30 Recurreces Substitutio method Recursio-tree method Master method

31 Substitutio method The most geeral method: 1 Guess the form of the solutio Verify by iductio 3 Solve for costats EXAMPLE: T() = 4T(/) + [Assume that T(1) = Θ(1)] Guess O( 3 ) Assume that T(k) ck 3 for k < Prove T() c 3 by iductio

32 Example of substitutio T() = 4T(/) + 4c(/) 3 + = (c/) 3 + = c 3 ((c/) 3 ) desired residual c 3 desired wheever (c/) 3 0, for example, if c ad 1 residual

33 Example of substitutio (cotiued) We must also hadle the iitial coditios, that is, groud the iductio with base cases Base: T() = Θ(1) for all < 0, where 0 is a suitable costat For 1 < 0, we have "Θ(1)" c 3, if we pick c big eough This boud is ot tight!

34 A tighter upper boud? We shall prove that T() = O( ) Assume that T(k) ck for k < : T() = 4T(/) + 4c(/) + = c + = O( ) Wrog! = c ( ) [desired residual] c for o choice of c > 0 Lose! We must prove the iductive hypothesis

35 A tighter upper boud! IDEA: Stregthe the iductive hypothesis Subtract a low-order term Iductive hypothesis: T(k) c 1 k c k for k < T() = 4T(/) + 4c 1 (/) 4c (/) + = c 1 c + = c 1 c (c ) c 1 c if c 1 Pick c 1 big eough to hadle the iitial coditios

36 Substitutio: chagig variables T() = T( ) + lg Reamig m = lg yields T( m ) = T( m/ ) + m Reame S(m) = T( m ) to produce the ew recurrece S(m) = S(m/) + m S(m) = O(mlgm) Chagig back from S(m) to T() T() = T( m ) = S(m) = O(mlgm) = O(lg lglg)

37 Recursio-tree method A recursio tree models the costs (time) of a recursive executio of a algorithm The recursio-tree method promotes ituitio The recursio tree method is good for geeratig guesses for the substitutio method

38 Example of recursio tree Solve T() = 3T( /4 ) + T() = 3T(/4) +

39 Example of recursio tree Solve T() = 3T( /4 ) + T() = 3T(/4) + T()

40 Example of recursio tree Solve T() = 3T( /4 ) + T() = 3T(/4) + c T ( ) T ( ) T ( ) 4 4 4

41 Example of recursio tree Solve T() = 3T( /4 ) + T() = 3T(/4) + c c( ) 4 c( ) 4 c( ) 4 T ( ) T ( ) T ( ) T ( ) T ( ) T ( ) T ( ) T ( ) T ( )

42 Example of recursio tree Solve T() = 3T( /4 ) + T() = 3T(/4) + c c c( ) 4 c( ) 4 c( ) c lg 4 c( ) 16 c( ) 16 c( ) 16 c( ) 16 c( ) 16 c( ) 16 c( ) 16 c( ) 16 c( ) 16 3 ( ) 16 c T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) Θ log4 3 ( ) log4 3 Total: O( )

43 Cost for the etire tree T = c + c + c + + c +Θ log4 1 3 i log4 3 = ( ) c +Θ( ) i= i log 4 < ( ) c +Θ( 3 ) i= log4 3 = c +Θ( ) 1 (3/16) 16 log4 3 = c +Θ( ) 13 log 4 1 log 4 ( ) ( ) ( ) ( 3 ) = O ( )

44 Substitutio method to verify T T 3 T( /4 ) + c ( ) = 3 ( /4 ) +Θ( ) 3 d /4 + c 3 d ( /4) + c 3 = d + c 16 d Where the last step holds as log as d (16 /13) c

45 The master method The master method applies to recurreces of the form T() = at(/b) + f (), where a 1, b > 1, ad f is asymptotically positive

46 Three commo cases Compare f() with log b a

47 Three commo cases Compare f() with log b a ε > 0 log 1 f() = O( b a ε ) for some costat f() grows polyomially slower tha log b a (by a factor) ε T logb ( ) =Θ( a ) Solutio: ε

48 Three commo cases Compare f() with log b a ε > 0 log 1 f() = O( b a ε ) for some costat f() grows polyomially slower tha log b a (by a factor) ε T logb ( ) =Θ( a ) Solutio: log lg f() = O( ) b a log f() ad b a grow at similar rates logb T ( ) =Θ( a lg ) Solutio: ε

49 Three commo cases (cotiued) Compare f() with log b a logb ( a +ε Ω ) ε > 0 3 f() = for some costat f() grows polyomially faster tha log b a +ε (by a factor) ε ad f() satisfies the regularity coditio that af(/b) cf() for some costat c < 1 T ( ) = Θ( f ( )) Solutio:

50 Idea of master theorem lg b f ( ) b f ( ) b f ( ) b f ( ) b f() a f ( ) b f ( ) b a a a f ( ) b f ( ) b f ( ) b f ( ) b f ( ) b a a a a a a a a a f ( ) b T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) f() af ( / b) a f( / b ) Θ logb ( a ) log b a Total: Θ logb 1 logb a j j ( ) a f( / b ) + j= 0

51 Examples EX T() = 4T(/) + a =4, b = log b a = ; f() =

52 Examples EX T() = 4T(/) + a =4, b = log b a = ; f() = CASE 1: f( ) = O( ε ) for ε = 1 T ( ) =Θ( )

53 Examples EX T() = 4T(/) + a =4, b = log b a = ; f() = CASE 1: f( ) = O( ε ) for ε = 1 T ( ) =Θ( ) EX T() = 4T(/) + a =4, b = log b a = ; f() =

54 Examples EX T() = 4T(/) + a =4, b = log b a = ; f() = CASE 1: f( ) = O( ε ) for ε = 1 T ( ) =Θ( ) EX T() = 4T(/) + a =4, b = log b a = ; f() = CASE : f ( ) =Θ( ) T ( ) =Θ( lg )

55 Examples EX T() = 4T(/) + 3 a =4, b = log b a = ; f() = 3

56 Examples EX T() = 4T(/) + 3 a =4, b = log b a = ; f() = 3 CASE 3: f( ) =Ω( + ε ) for ε = 1 ad 4(/) 3 c 3 (reg cod) for c = 1/ 3 T ( ) =Θ( )

57 Examples EX T() = 4T(/) + 3 a =4, b = log b a = ; f() = 3 CASE 3: f( ) =Ω( + ε ) for ε = 1 ad 4(/) 3 c 3 (reg cod) for c = 1/ 3 T ( ) =Θ( ) EX T() = 4T(/) + /lg a =4, b = log b a = ; f() = /lg

58 Examples EX T() = 4T(/) + 3 a =4, b = log b a = ; f() = 3 CASE 3: f( ) =Ω( + ε ) for ε = 1 ad 4(/) 3 c 3 (reg cod) for c = 1/ 3 T ( ) =Θ( ) EX T() = 4T(/) + /lg a =4, b = log b a = ; f() = /lg Master method does ot apply

59 Ay questio? Xiaoqig Zheg Fuda Uiversity