CSI 5163 (95.573) ALGORITHM ANALYSIS AND DESIGN

Transcription

1 CSI 5163 (95.573) ALGORITHM ANALYSIS AND DESIGN CSI 5163 ( ) ALGORITHM ANALYSIS AND DESIGN (3 cr.) (T) Topics of curret iterest i the desig ad aalysis of computer algorithms for graphtheoretical applicatios; e.g. shortest paths, chromatic umber, etc. Lower bouds, upper bouds, ad average performace of algorithms. Complexity theory. PROFESSOR: SITE 5-031, ext.:6782 zaguia@site.uottawa.ca Office Hours: Friday11:30-13:00 LECTURES: Thursday, 18:00-21:00, LMX 339 REFERENCES Textbook: Kleiberg ad Tardos, Algorithm Desig, Addiso Wesley, 7th editio. Itroductio to algorithms, T.H. Corme, C.E. Leiserso ad R. L. Rivest, McGraw-Hill Book Compay, The desig ad aalysis of computer algorithms, A.V.Aha, M.R.Garey ad J.D.Ullma, Addiso-Wesley Combiatorial Optimizatio: algorithms ad complexity, C.H. Papadimitriou ad K. Steiglitz, Pretice Hall CSI 5163 (95.573) ALGORITHM ANALYSIS AND DESIGN COURSE OBJECTIVES This is a course o the desig ad aalysis of algorithms. Core results ad techiques are itroduced, which are useful to those plaig to specialize i other areas i computer sciece. Moreover, some fairly advaced topics will be covered. This will provide a idea of the curret research for the beefit of those who might wish to specialize i this area. I the last few lectures we will study a umber of specific applicatios. COURSE OUTLINE Itroductio: Itroductio of the course, asymptotic otatios, basic data structure Basic Graph algorithms ad Basic algorithms desig (greedy, divide ad coquer, dyamic programmig, ) More Graph algorithms: Chromatic umber, Network flow,.. 2 CSI 5163 (95.573) ALGORITHM ANALYSIS AND DESIGN Markig scheme: Report o the topic ad Research paper : 30% Oral presetatio o the Research paper : 15% Participatio durig presetatios 5% 2 Midterm Exams i Class (February 13, march 13) 50% Geeral Iformatio A readig project is required to fulfill the class requiremets. A list of papers ad topics o algorithms will be posted by Jauary 20th. You must write a report of about 15 pages about your topic ad also explaiig the results of the research paper ad their proofs. You must read you paper i depth. Your report should ot just be a codesed versio of the origial paper, but should explai why the paper is iterestig ad preset the mai ideas i the way you uderstad it best, perhaps usig examples or special cases, or perhaps reworkig a proof. Or you might fill i missig details of the origial paper. You must give a talk of about miutes explaiig the most importat ad major poits i the research paper. 3 1

2 CSI 5163 (95.573) ALGORITHM ANALYSIS AND DESIGN Evaluatio of reports. Evaluatio will be based o the followig criterio: (for the purpose of markig these criterio will be cosidered approximately equally, except where oted.) Writig quality Relevace ad icorporatio of the ideas preseted durig the lectures of this course Depth of the aalysis ad uderstadig of the paper Evaluatio of presetatios. Presetatios will be evaluated based o the format ad quality of the presetatio (orgaizatio, quality of overheads, colors, graphics, speakig clearly, maagig the allotted time of the presetatio, ) It should ot be a repetitio of materials from the paper, lecture otes or textbooks. Importat: All writig reports should be retured o March 20th before the begiig of the lecture. A copy of your presetatios (hardcopy) should be retured at the begiig of your talk. Please sed me a soft copy of the reports ad the presetatios by . 4 Asymptotic Performace Asymptotic performace How does the algorithm behave as the problem size gets very large? Ruig time Memory/storage requiremets Badwidth/power requiremets/logic gates/etc. 5 Aalysis of Algorithms Computatioal model The assumptio: geeric uiprocessor radom-access machie (RAM) All memory equally expesive to access No cocurret operatios All reasoable istructios take uit time Except, of course, fuctio calls Costat word size Uless we are explicitly maipulatig bits 6 2

3 Iput Size Time ad space complexity This is geerally a fuctio of the iput size e.g., sortig, multiplicatio How we characterize iput size depeds: Sortig: umber of iput items Multiplicatio: total umber of bits Graph algorithms: umber of odes & edges etc Number of primitive steps that are executed Except for time of executig a fuctio call, most statemets roughly require the same amout of time 7 Aalysis Worst case Provides a upper boud o ruig time A absolute guaratee Average case Provides the expected ruig time Very useful, but treat with care: what is average? Radom (equally likely) iputs Real-life iputs 8 A Example: Isertio Sort IsertioSort(A, { for i = 2 to { key = A[i] j = i - 1; while (j > 0) ad (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } 9 3

4 Isertio Sort IsertioSort(A, { for i = 2 to { -1 key = A[i] 1 j = i - 1; 1 while (j > 0) ad (A[j] > key) { i-1 A[j+1] = A[j] 1 j = j } 0 A[j+1] = key 1 } 0 Average case??? } Best case: while loop body ever executed is a liear fuctio = -1 Worst case: while loop body executed for all previous elemets = 2 i (2*(i-1)+1+1+1)= 2 i (2i+1)= 2( 2 i i) + (-1) = [(+1)/2] 1 + (-1) = [ ]/2 is a quadratic fuctio 10 Simplificatios Aalysis Igore actual ad abstract statemet costs Order of growth is the iterestig measure: Highest-order term is what couts Remember, we are doig asymptotic aalysis As the iput size grows larger it is the high order term that domiates A fuctio f( is O(g() if there exist positive costats c ad 0 such that f( c g( for all 0 11 Big O Fact We say Isertio Sort s ru time is O( 2 ) A polyomial of degree k is O( k ) Suppose f( = b k k + b k-1 k b 1 + b 0 Let a i = b i f( a k k + a k-1 k a 1 + a 0 k i ai k k a i c k 12 4

5 Lower Boud Notatio A fuctio f( is (g() if there exists c 0 ad 0 such that 0 cg( f( 0 A fuctio f( is (g() if there exists c 1 0, c 2 0, ad 0 0 such that c 1 g( f( c 2 g( 0 f( is (g() iff f( is both O(g() ad (g() 13 Practical Complexity 14 Practical Complexity 500 f( = f( = log( f( = log( f( = ^2 f( = ^3 f( = 2^

6 Practical Complexity 1000 f( = f( = log( f( = log( f( = ^2 f( = ^3 f( = 2^ Practical Complexity f( = f( = log( f( = log( f( = ^2 f( = ^3 f( = 2^ Practical Complexity Order of complexity miutes 5.2 miutes 13.0 miutes secod 17.9 miutes 12.7 days 35.7 years 366 ceturies miutes 6.5 years 3855 ceturies 2x10 8 ceturies 1.3x10 13 ceturies 18 6

7 Other Asymptotic Notatios A fuctio f( is o(g() if there exists positive costats c ad 0 such that f( < c g( 0 A fuctio f( is (g() if there exists positive costats c ad 0 such that c g( < f( 0 Ituitively, o() is like < O() is like () is like > () is like () is like = 19 Aalysis of Merge Sort Solvig recurreces MergeSort(A, left, right) { if (left < right) { mid = floor((left + right) / 2); MergeSort(A, left, mid); MergeSort(A, mid+1, right); Merge(A, left, mid, right); } } (1) (1) /2) /2) ( So = (1) whe = 1, ad 2/2) + ( whe > 1 20 Review: Solvig Recurreces The expressio: c T ( 2T c is a recurrece. There are basic techiques to solve recurrece relatios. Substitutio method Iteratio method Master method 21 7

8 Review: Solvig Recurreces The substitutio method : Guess the form of the aswer, the use iductio to fid the costats ad show that guessed solutio works Example: merge sort = 2/2) + c We guess that the aswer is O( lg Prove it by iductio Ca similarly show = Ω( lg, thus Θ( lg 22 Solvig Recurreces The Iteratio method 1. Expad the recurrece 2. Work some algebra to express as a summatio 3. Evaluate the summatio 23 Review: Solvig Recurreces s( = c + s(-1) = c + c + s(-2) = 2c + s(-2) = 2c + c + s(-3) = 3c + s(-3) = c + s(0) = c s( = c 0 s( c s( 1)

9 Review: Solvig Recurreces s( 0 0 s( = + s(-1) s( 1) 0 = s(-2) = (k-1) + s(-k) = s(0) 1 = i s(0) s( i Review: Solvig Recurreces = 2/2) + c c 2T c = 2(2/2/2) + c) + c = 2 2 /2 2 ) + 2c + c = 2 2 (2/2 2 /2) + c) + 3c = 2 3 /2 3 ) + 4c + 3c = 2 k /2 k ) + (2 k - 1)c = 2 lg /2 lg ) + (2 lg - 1)c = / + ( - 1)c = 1) + (-1)c = c + (-1)c = (2-1)c 26 Review: Solvig Recurreces What about this relatio: c at c b

10 Review: Solvig Recurreces = c 1 a/b) + c at c 1 a(a/b/b) + c/b) + c b a 2 /b 2 ) + c(a/b + 1) a 2 (a/b 2 /b) + c/b 2 ) + c(a/b + 1) a 3 /b 3 ) + c(a 2 /b 2 + a/b + 1) a k /b k ) + c(a k-1 /b k-1 + a k-2 /b k a 2 /b 2 +a/b +1) For k = log b, that is = b k = a k 1) + c(a k-1 /b k a 2 /b 2 + a/b + 1) = ca k /b k + c(a k-1 /b k a 2 /b 2 + a/b + 1) = c(a k /b k a 2 /b 2 + a/b + 1) 28 = c(a k /b k a 2 /b 2 + a/b + 1, (k=log b, =b k ) Recall that (x k + x k x + 1) = (x k+1-1)/(x-1) ad a log = log a k a b a b k a b 1 a b k k1 a a a k k1 b b b 1 If a = b the = c(k + 1) =c(log b + 1)=( log If a b the Review: Solvig Recurreces k1 a b a b k1 1 1 a b 1 1 If a b the a b1 1 a b = ( log a ) =c (a k / b k ) =c (a log / b log ) = c (a log / = c ( log a / = (c log a / = ( log a ) k a b 1 = c (1) = ( 1 29 Review: Solvig Recurreces So c at c b 1 1 log b log b a a b a b a b 30 10

11 The Master Theorem The Master Theorem is a very useful geeralizatio of the previous theorem: Cosider ay divide ad coquer algorithm Suppose the algorithm divides the problem of size ito a subproblems, each of size /b f( is the cost of subdividig the problem ito the subroblems plus the cost of combiig the solutios of subproblems = a/b) + f( 31 The Master Theorem if = a/b) + f( the logb a logb a log f ( f ( O f ( f ( logb a logb a af ( / b) cf ( for large 32 logb a AND 0 c 1 Usig The Master Method = 9/3) + a=9, b=3, f( = log b a = log 3 9 = ( 2 ) Need to compare f(= with log b a = ( 2 ), Sice f(= O( 2- ), where =1, case 1 applies: logb a logb a whe f ( O Thus the solutio is = ( 2 ) 33 11

12 Usig The Master Method = 3/4) + 1 ad 1)= (1) a=1, b=4/3, f( = 1 log b a = log 4/3 1 = ( 0 ) =1 Need to compare f(=1 with log b a =1, Sice f(= O( log b a ), case 2 applies: Thus the solutio is = (log 34 Usig The Master Method = 3/3) + log ad 1)= (1) a=3, b=3, f( = log log b a = log 3 3 = ( Need to compare f(= log with log b a = (, Sice log = (, case 3 is the best possibility. However there is o such that log = ( 1+ ). Thus the Master theorem caot be applied to this recurrece