Classification of problem & problem solving strategies. classification of time complexities (linear, logarithmic etc)

Transcription

1

2 Classificatio of problem & problem solvig strategies classificatio of time complexities (liear, arithmic etc) Problem subdivisio Divide ad Coquer strategy. Asymptotic otatios, lower boud ad upper boud: Best case, worst case, average case aalysis, amortized aalysis. Performace aalysis of basic programmig costructs. Recurreces: Formulatio ad solvig recurrece equatios usig Master Theorem.

3 Problem solvig is the applicatio of ideas, skills, or factual iformatio to achieve the solutio to a problem or to reach a desired outcome. Let's talk about differet types of problems ad differet types of solutios.

4 A well-defied problem is oe that has a clear goal or solutio, ad problem solvig strategies are easily developed. I cotrast, poorly-defied problem is the opposite. It's oe that is uclear, abstract, or cofusig, ad that does ot have a clear problem solvig strategy. routie problem is oe that is typical ad has a simple solutio. I cotrast, o-routie problem is more abstract or subjective ad requires a strategy to solve.

5 The first strategy you might try whe solvig a routie problem is called a algorithm. Algorithms are step-by-step strategies or processes for how to solve a problem or achieve a goal. Aother solutio that may people use to solve problems is called heuristics. Heuristics are geeral strategies used to make quick, short-cut solutios to problems that sometimes lead to solutios but sometimes lead to errors. Heuristics are based o past experieces.

6 Idetify a problem Uderstad the problem Idetify alterative ways to solve a problem Select beat way to solve a problem from the list of alterative solutios Evaluate the solutio

7

8 What is a algorithm? A algorithm is a fiite set of precise istructios for performig a computatio or for solvig a problem. This is a rather vague defiitio. You will get to kow a more precise ad mathematically useful defiitio whe you atted CS420. But this oe is good eough for ow CMSC Discrete Structures Fall

9 Properties of algorithms: Iput from a specified set, Output from a specified set (solutio), Defiiteess of every step i the computatio, Correctess of output for every possible iput, Fiiteess of the umber of calculatio steps, Effectiveess of each calculatio step ad CMSC Discrete Structures Fall

10 Why we should aalyze algorithms? Predict the resources that the algorithm requires Computatioal time (CPU cosumptio) Memory space (RAM cosumptio) Commuicatio badwidth cosumptio The ruig time of a algorithm is: The total umber of operatios executed Also kow as algorithm complexity 10

11 Iteral Factors Exteral Factors Space Complexity Processor Quality Time Complexity CPU Speed 11

12 Time complexity of a algorithm sigifies the total time required by the program to ru to completio. Time complexity of a algorithm is measured by its rate of growth relative to the stadard fuctio. Cases of time complexity are: Worst-case A upper boud o the ruig time for ay iput of give size Average-case Assume all iputs of a give size are equally likely Best-case The lower boud o the ruig time 12

13 Sequetial search i a list of size Worst-case: comparisos Best-case: 1 compariso Average-case: /2 comparisos The algorithm rus i liear time Liear umber of operatios 13

14 Notatio Complexity Descriptio Example O(1) Costat Simple statemet Additio O(()) Logarithmic Divide i half Biary search O() Liear loop Liear search O(*()) Liearithmic Divide & Coquer Merge sort O( 2 ) Quadratic Double loop Check all pairs O( 3 ) Cubic Triple loop Check all triples O(2 ) Expoetial Exhaustive search Check all subsets O(!) Factorial Recursive fuctio Factorial 14

15

16 Statemet 1 Statemet 2... Statemet k Total time=time(statemet 1)+time(statemet 2). +time(statemet k) Each statemet is simple so takes oly o(1) ie costat time

17 For (i=1;i<;i++) { Pritf (i) } Rus i O() where is the size of the array

18 It i=1 While(i<10) { pritf(i); i=i*2; } Rus i arithmic time O( )

19 algo Sum { for (it x=0; x<; x++) for (it y=0; y<; y++) sum += x*y; } Rus i quadratic time O( 2 )

20 algo Sum { it sum = 0; for (it a=0; a<; a++) for (it b=0; b<; b++) for (it c=0; c<; c++) sum += a*b*c; } Rus i cubic time O( 3 )

21 decimal Fiboacci(it ) { if ( == 0) retur 1; else if ( == 1) retur 1; else retur Fiboacci(-1) + Fiboacci(-2); } Rus i expoetial time O(2 ) The umber of elemetary steps is ~ Fib(+1) where Fib(k) is the k-th Fiboacci's umber

22

23 Ruig time of a algorithm as a fuctio of iput size. Expressed usig oly the highest-order term i the expressio for the exact ruig time. Describes behavior of fuctio i the limit. Writte usig Asymptotic Notatio. Comp 122

24 Big Oh Notatio: O Big theta Notatio : Q Big Omega Notatio W Small Oh Notatio : o Small Omega Notatio :w Defied for fuctios over the atural umbers. Defie a set of fuctios; i practice used to compare two fuctio sizes.the otatios describe differet rate-of-growth. Comp 122

25 Let f() ad g() be two fuctios the we ca say that f() = O(g()) if ad oly if there exists positive costats c ad 0, such that f() cg() for all 0

26 As: I the give Problem

27 As: I the give Problem

28 Let f() ad g() be two fuctios the we ca say that f() = W(g()) if ad oly if there exists positive costats c ad 0, such that f() cg() for all 0

29 As:

30 As:

31 Let f() ad g() be two fuctios the we ca say that f() = Q(g()) if ad oly if there exists positive costats c1,c2 ad 0, such that 0 c1g() f() c2g() for all 0

32 As:

33

34

35 Key poit: The time required to perform a sequece of data structure operatios is averaged over all operatios performed Amortized aalysis ca be used to show that The average cost of a operatio is small If oe averages over a sequece of operatios eve though a sigle operatio might be expesive

36 The most commo three techiques The aggregate method The accoutig method The potetial method If there are several types of operatios i a sequece The aggregate method assigs The same amortized cost to each operatio The accoutig method ad the potetial method may assig Differet amortized costs to differet types of operatios

37 Show that sequece of operatios takes Worst case time T() i total for all The amortized cost (average cost i the worst case) per operatio is therefore T() This amortized cost applies to each operatio Eve whe there are several types of operatios i the sequece

38 PUSH(S, x): pushed object x oto stack POP(S): pops the top of the stack S ad returs the popped object MULTIPOP(S, k): removes the k top objects of the stack S or pops the etire stack if S k PUSH ad POP rus i Q(1) time The total cost of a sequece of PUSH ad POP operatios is therefore Q() The ruig time of MULTIPOP(S, k) is Q(mi(s, k)) where s S

39 Let us aalyze a sequece of POP, PUSH, ad MULTIPOP operatios o a iitially empty stack The worst case of a MULTIPOP operatio i the sequece is O() Hece, a sequece of operatios costs O( 2 ) we may have MULTIPOP operatios each costig O() The aalysis is correct, however, Cosiderig worst-case cost of each operatio, it is ot tight We ca obtai a better boud by usig aggregate method of amortized aalysis

40 Aggregate method cosiders the etire sequece of operatios Although a sigle MULTIPOP ca be expesive Ay sequece of POP, PUSH, ad MULTIPOP operatios o a iitially empty sequece ca cost at most O() Proof: each object ca be popped oce for each time it is pushed. Hece the umber of times that POP ca be called o a oempty stack icludig the calls withi MULTIPOP is at most the umber of PUSH operatios, which is at most The amortized cost of a operatio is the average O() O(1)

41 The Accoutig Method: We assig differet charges to differet operatios with some operatios charged more or less tha they actually cost The amout we charge a operatio is called its amortized cost Whe the amortized cost of a operatio exceeds its actual cost the differece is assiged to specific objects i the data structure as credit Credit ca be used later to help pay for operatios whose amortized cost is less tha their actual cost That is, amortized cost of a operatio ca be cosidered as beig split betwee its actual cost ad credit (either deposited or used)

42 The Accoutig Method: Stack Operatios Assig the followig amortized costs: Push: 2 Pop: 0 Multipop: 0 We start with a empty stack of plates Whe we push a plate o the stack we use $1 to pay the actual cost of the push operatio we put a credit of $1 o top of the pushed plate At ay time poit, every plate o the stack has a $1 of credit o it. The $1 stored o the plate is a prepaymet for the cost of poppig it. I order to pop a plate from the stack we take $1 of credit off the plate ad use it to pay the actual cost of the pop operatio

43 The Accoutig Method: Stack Operatios Thus by chargig the push operatio a little bit more we do t eed to charge aythig from the pop & multipop operatios We have esured that the amout of credits is always oegative Thus, for ay sequece of push, pop, multipop operatios the total amortized cost is a upper boud o the total actual cost

44 The Potetial Method: Potetial method represets the prepaid work as potetial eergy that ca be released to pay for the future operatios The potetial is associated with the data structure as a whole rather tha with specific objects withi the data structure

45 The Potetial Method D 0 : Iitial Datastructure o which we perform operatios C i : the actual cost of the i-th operatio D i : data structure that results after applyig i-th operatio to D i1 : potetial fuctio that maps each data structure D i to a real umber (D i ) (D i ): the potetial associated with data structure D i Ĉ i : amortized cost of the i-th operatio

46 The Potetial Method actual icrease i potetial cost due to the operatio The total amortized cost of operatios is: ) ( ) ( ˆ 1 i i i i D D C C i i i i D D C C ) ( ) ( ˆ

47 Defie (S) S, the umber of objects i the stack For the iitial empty stack, we have (D 0 ) 0 Sice S 0, stack D i that results after i th operatio has oegative potetial total amortized cost is a upper boud o total actual cost Let us compute the amortized costs of stack operatios where i th operatio is performed o a stack with s objects

48 PUSH (S): ˆ C i C i (D ) i (D i1 ) 1 i -(i-1) 2 MULTIPOP(S, k): suppose k elemets are popped ˆ ˆ C i C i (D ) i (D i1 ) k' +(i-k' )-j 0 POP (S): C i C i (D i ) (D i1 ) 1 (i -1)-i 0 The amortized cost of each operatio is O(1), ad thus the total amortized cost of a sequece of operatios is O()

49

50 Divide-ad coquer is a geeral algorithm desig paradigm: Divide: divide the iput data S i two or more disjoit subsets S 1, S 2, Coquer : solve the subproblems recursively Combie: combie the solutios for S 1, S 2,, ito a solutio for S Aalysis ca be doe usig recurrece equatios

51 Merge-sort o a iput sequece S with elemets cosists of three steps: Divide: partitio S ito two sequeces S 1 ad S 2 of about 2 elemets each Recur: recursively sort S 1 ad S 2 Coquer: merge S 1 ad S 2 ito a uique sorted sequece Algorithm mergesort(s, C) Iput sequece S with elemets Output sequece S sorted if S.size() > 1 (S 1, S 2 ) partitio(s, /2) mergesort(s 1, C) mergesort(s 2, C) S merge(s 1, S 2 )

52 The coquer step of merge-sort cosists of mergig two sorted sequeces, each with 2 elemets ad implemeted by meas of a doubly liked list, takes at most b steps, for some costat b. Likewise, the basis case ( < 2) will take at b most steps. Therefore, if we let T() deote the ruig time of mergesort: T( ) b 2T ( / 2) b We ca therefore aalyze the ruig time of merge-sort by fidig a closed form solutio to the above equatio. That is, a solutio that has T() oly o the left-had side. if if 2 2

53 Draw the recursio tree for the recurrece relatio ad look for a patter: b if 2 T( ) 2T ( / 2) b if 2 depth T s size 0 1 time b i 2 i 2 i b b Total time = b + b Divide-ad-Coquer 53

54 Comp 122, Sprig Jue 2015

55 Give: a divide ad coquer algorithm A algorithm that divides the problem of size ito a subproblems, each of size /b Let the cost of each stage (i.e., the work to divide the problem + combie solved subproblems) be described by the fuctio f() The, the Master Theorem gives us a cookbook for the algorithm s ruig time:

56 Divide-ad-Coquer 56 May divide-ad-coquer recurrece equatios have the form: The Master Theorem: d f b at d c T if ) ( ) / ( if ) ( 1. for some ) ( ) / ( provided )), ( ( is ) ( the ), ( is ) ( if 3. ) ( is ) ( the ), ( is ) ( if 2. ) ( is ) ( the ), ( is ) ( if 1. 1 Q W Q Q Q f b af f T f T f T O f a k a k a a a b b b b b for all d

57 if T() = at(/b) + f() the W Q Q Q Q 1 0 large for ) ( ) / ( AND ) ( ) ( ) ( ) ( ) ( c cf b af f f O f f T a a a a a b b b b b

58 T() = 9T(/3) + a=9, b=3, f() = b a = 3 9 = 2 Sice f() = O( ), where =1, case 1 applies: T( ) Q a a b b whe f ( ) O Thus the solutio is T() = Q( 2 )

59 The Master Theorem: 1. if f 2. if f 3. if f ( ) is O( ( ) is ( ) is Q( W( b a b a b a ), the T ( ) is Q( ), the T ( ) is Q( ), the T ( ) is Q( f ( )), provided af ( / b) f ( ) for some 1. k b a ) b a for all d k 1 ) Example: T( ) 4T ( / 2) Solutio: b a = 2 4 = 2 so case 1 says T() is Q 2 ) Divide-ad-Coquer 59

60 The Master Theorem: 1. if f 2. if f 3. if f ( ) is O( ( ) is ( ) is Q( W( a a provided af ( / b) f ( ) b b b a ), the T ( ) is k Q( ), the T ( ) is Q( ), the T ( ) is Q( f ( )), for some 1. b a ) b a for all d k 1 ) Example: T( ) 2T ( / 2) Solutio: b a = 2 2 =, so case 2 says: T() = Q( 2 ). 60

61 The Master Theorem: 1. if f 2. if f 3. if f ( ) is O( ( ) is Q( ( ) is W( b b b a a a ), the T ( ) is Q( ), the T ( ) is Q( ), the T ( ) is Q( f ( )), provided af ( / b) f ( ) for some 1. k b a ) b a k 1 ) Example: T( ) T( / 3) Solutio: b a = 3 1 = 0, so case 3 says T() is Q( ). Divide-ad-Coquer 61

62 Example: T( ) 8T ( / 2) 2 Solutio: b a=3, so case 1 says T() is Q( 3 ). T( ) 9T ( /3) Solutio: b a=2, so case 3 says T() is Q( 3 ). T( ) T( / 2) 1 Solutio: b a=0, so case 2 says T() is Q( ). 3

63