Constraint Satisfaction. Algorithm Design (3) Constraint Satisfaction and Optimization. Formalization of Constraint Satisfaction Problems

Transcription

1 Algorithm Desig (3) Costrait Satisfactio ad Optimizatio Taashi Chiayama School of Egieerig The Uiversity of Toyo Costrait Satisfactio Fidig a way that satisfies give coditios e.g. How to obtai eough credits without attedig Moday classes Ways are selectios of classes here Without attedig Moday classes A subsettig costrait o possible ways To obtai eough credits A iequality costrait o obtaied credits Implicit costraits: Two classes held i the same time slot caot be tae, etc Formalizatio of Costrait Satisfactio Problems Fidig a set of values < 1,, >, D that satisfies a coditio C( 1,, ) Solutios are ot ecessarily uique e.g. Iteger is greater tha 3 ad less tha 6 Domai of the variable is iteger Costraits: 3 < < 6 Solutios: = 4 ad = 5 No ewly added iformatio i the solutio The problem is to modify the costraits ito a simpler form The defiitio of beig simple may vary Classes of Costrait Satisfactio Problems Variable domais Fiite domais (oe of fiite umber of cadidates) Ifiite domais (iteger, real, comple, ) Structures (trees, graphs, ) usually decomposed ito compoets Kids of costraits Equality (liear, oliear, ) Iequality (total or partial order) Logical combiatio of multiple costraits 1

2 Iteger Domai, Equality Costraits Craes ad Turtles There are 5 aimals, craes ad/or turtles, ad the sum of their legs is 14. How may craes ad turtles are there? Variables ad y i iteger domai Costraits: + y = y = 14 Solutio: = 2, y = 3 Comple Domai, Equality Costraits Solvig quadratic equatios Variable domai is comple umbers Costrait is a quadratic equatio Costrait : a 2 b c 0 2 b b 4ac Solutio : 2a I geeral, solutios to costrait satisfactio problems restates the give iformatio i some other forms Real Domai, Iequality Costraits y Below this lie Iteger Domai, Iequality Costraits y The regio satisfyig the costrait Above this lie 2

3 Fiite Domai, Combiatorial Costraits Thesis defese: selectig slots, rooms, judges Domais of slots, rooms, judges are fiite Costraits: Each judge has his/her epertise area At oe time slot ad oe room, oly oe defese ca be doe Judges ca atted oe defese at a time All the defeses should be doe Most of combiatorial costrait satisfactio problems are NP-hard with epoetial compleity P, NP, NP-hard, NP-complete P: Problems that ca be solved withi time epressed as a polyomial of iput size NP: Problems for which whether a give value is a solutio or ot ca be told i polyomial time NP is from Nodetermiistic Polyomial time, meaig that odetermiistic Turig machies ca solve them i polyomial time NP-hard: Problems to which ay NP problems ca be trasformed i polyomial time NP-complete: NP-hard problems that are NP I short, the hardest subset of NP Whether P=NP or ot is still a ope problem P, NP, NP-hard, ad NP-complete Visually Eight Quees Problem Placig eight quees o a chess board so that oe of them offed oe aother P NP NP-hard NP-complete 3

4 A Simple Strategy for Combiatorial Costraits Geerate ad Test Geerate all possible combiatios systematically Test all of the geerated combiatios whether they satisfy the costraits Whe the umber of variables icreases, the umber of combiatios becomes huge Combiatorial Eplosio e.g. Ways to put 8 quees o a 8 by 8 board is 64 C 8 = Tree Structure of Geeratig Combiatios First selectio Secod selectio Third selectio Brach factor b # of selectios d # of combiatios = b d Solvig Combiatorial Costraits More Efficietly Reductio of tree geeratio: brach pruig Geeratio scheme that reflects the costraits Costrait chec before complete geeratio: bactracig Checig the possibility of satisfactio durig geeratio: forward checig If the costraits eforce a selectio, mae that decisio earlier: determiacy detectio Geeralizatio: Mae decisios earlier o selectios with less alteratives I Case of Eight Quees Geeratio scheme Oly oe quee i a row C ,777,216 No more quees o colums already with oe 8 8 8! 40,320 Bactracig Costrait checs made each time a quee is placed, rather tha after placig all the quees More devices are eeded for larger-scale problems 4

5 Forward Checig After several quees are placed, chec whether all the remaiig rows still have safe colums Irrespective of quees positios o these rows Determiacy Detectio After several quees are placed, if there is a row with a uique safe positio, place a quee there first Oly oe safe place left o this row No quees ca be placed o this row Optimizatio Problems Multiple solutios satisfyig the costraits may have relative merits Should choose the best solutio with the highest merits Merits should have a total order With a partial order, the best might ot eist Whe more tha oe aspects are to be cosidered, they should be combied ito oe sigle value with total order Formalizatio of Costrait Optimizatio Miimize (or maimize) the objective fuctio f( 1,, ) with values < 1,, >, D that satisfies a coditio C( 1,, ) A objective fuctio to be miimized is also called a cost fuctio A set of values < 1,, > that satisfies the costrait but may or may ot give the miimum (or maimum) of the objective fuctio is called a feasible solutio 5

6 Objective Fuctio Values of the objective fuctio should be compared i a total order Objective fuctios which do ot have umeric values ca be cosidered Numerical values, however, is desirable as some algorithms require them to wor For eample, values of objective fuctios for partial problems may be summed up to obtai the value for the whole problem Liear Programmig (LP) Miimizatio of liear objective fuctio uder liear iequality costrait y Area of feasible solutios Optimum solutio Iteger Programmig (IP) Liear Programmig i Iteger Domai Oly itegral solutios are feasible y Optimum solutio Combiatorial Optimizatio Maimizatio/miimizatio of objective fuctio uder combiatorial costraits o a fiite domai A variety of costraits No restrictio o objective fuctios Objective fuctios may be o-liear No efficiet geeral algorithm is ow Usually, some smoothess is assumed: Similar argumets result i similar values 6

7 Kapsac Problem: a Typical Combiatorial Optimizatio A apsac of some give capacity Goods of various sizes ad values Fid the highest-valued set of goods that fits i the apsac Formulatio of the Kapsac Problem Variables : Boolea variables idicatig whether or ot the item is icluded i the set Costrait: Sum of the sizes of items s should ot eceed the apsac capacity c (liear iequality) 1 ( s ) c Objective fuctio: Sum of the values v (liear) f ( 1,, ) ( v ) ma 1 Ca be regarded as a -dimesioal iteger programmig problem with 0-1 domai Travellig Salesma Problem: aother typical problem A umber of cities ad distaces betwee them are give The shortest route to visit all the cities should be foud Formulatio of TSP Variables : The -th city visited The domai is the set of cities (fiite domai) Costrait: Should visit all of cities { 1,..., } X Cost fuctio: Total travel distace f ( 1,..., ) d (, 1) 1 1 mi 7

8 Algorithms for Combiatorial Optimizatio Strict algorithms The strictly best solutio is to be foud i.e., No other feasible solutio is better Ofte requires large computatioal cost Approimate algorithms Fid a solutio close to the best i.e., Not ecessarily the real best A variety of defiitios o how close the obtaied solutio should be Ofte decreases the computatioal cost Simple Strict Algorithm: Geerate ad Evaluate Geerate all feasible solutios ad evaluate them 1. Geerate all feasible solutios systematically Efficiet algorithms for combiatorial costrait satisfactio may be used 2. Compute the objective fuctio for each 3. The solutio givig the maimum/miimum value is the optimum Simple ad easy to uderstad, but iefficiet whe there are a huge umber of feasible solutios, which is frequetly the case Compleity of Geerate-ad-Evaluate Scheme Huge umber of feasible solutios may eist Whe each selectio has b choices ad the umber of selectios (= tree depth) is d, there are b d leaves (epoetial) For a apsac problem with 10 items, there are 1024 feasible solutios; with 20 items, about oe millio; with 30, oe billio TSP with 10 cities, there are 362,880 feasible solutios; with 100 cities, the umber has 156 digits! Brach ad Boud The tree of feasible solutios is epaded i a depth-first order Durig tree epasio, if a ode is ow ever to have leaves better tha a already ow solutio, o further epasio below the ode is required If the merit upper boud of possible further choices is ow, the followig coditio ca be used to prue the braches below the ode [sum of merits of already chose part] + [upper boud of sum of merits of further choices] [merit of already ow solutio] 8

9 Brach ad Boud merit upper boud > + merit upper boud? ow solutio B&B for Kapsac Problems Decide whether to put a item ito the apsac i the descedig order of value per size, v /s The total value of the items i first m choices is m 1 ( v ) For all remaiig m items, value per size ever eceeds v m+1 /s m+1 ad thus a upper boud of the sum ca be give as v m1 s m1 S m s 1 Relaatio of Costraits Relaig IP to LP Some problems ca be more efficietly solved if costraits are relaed e.g., Wideig the domai of iteger programmig from iteger to real maes it a LP problem, for which we have efficiet algorithms The optimum of the origial problem ca ever be better tha that of the relaed problem A relaed problem gives a upper boud y optimum of iteger programmig optimum of liear programmig 9

10 Solvig Relaed Problems to Estimate Upper Bouds Durig tree search, choices ot made yet ca be cosidered to be a subproblem The solutio of a relaed subproblem gives a upper boud of the origial subproblem This iformatio ca be used for search pruig Useful whe Subproblems ca be clealy etracted, Relaed subproblems have optimum ot too much differet from the origial, ad Efficiet algorithms are ow Relaatio of Kapsac Problem to LP Origial Kapsac 1 ( s ) S ( 0or1) f (,, ) ( v ) ma 1 1 Relaed Kapsac 1 ( s ) S (0 1) f (,, ) ( v ) ma 1 1 This is a LP problem ad ca be solved i polyomial time Priciple of Optimality The optimum solutio of a subproblem forms a part of the optimum solutio of the whole problem If there eists a better solutio to a subproblem, replacig the part with that solutio improves the solutio of the whole problem Note: This is applicable oly whe subproblems are idepedet E.g., i TSP, optimum subroute for a subset of cities might ot be a part of the optimum route for all the cities; Start ad ed poits have to agree to merge the subroutes ito oe Dyamic Programmig (DP) A algorithm derived from the priciple of optimality 1. Solve small subproblems 2. Gradually combie solutios of subproblems 3. Record optimum solutios of subproblems to avoid recomputatio: memoizatio Coditios to apply dyamic programmig Subproblems should be mutually idepedet The umber of subproblems is small eough so that their optimum solutios ca be recorded 10

11 Optimizatio o State Trasitios A set of state S = { s 0, s 1,, s } Cost of trasitio from s i to s j is c i,j At time t, the state is t Miimize the total cost of trasitios from the iitial state s 0 to the fial state s f ( 0 1 0,, ) c t 0 t, t1 s, 0 s Optimum Solutio Epressed as Recurrece Equatios The cost sum after t = u ca be defied as 1 fu ( u,, ) c tu t, t 1 The followig recurrece formula is observed f ( mi u u 0 u mi u1 f ( 0,, f ( u u f,...,,, u1 ) f ( ) u1 u1 ),, f ( 0 ( u1,...,,, ) c ), u ) c u1, u u1 Computig Edit Distace Editig a Sequece Edit a character strig to mae aother A give fiite set of primitive editig operatios Deletio: Isertio: cost 3 ABCD ABD cost 4 ABD AYBD Replacemet: cost 5 AYBD XYBD Fid a sequece of operatios with the least cost to obtai the target strig A problem that completely fits the DP framewor States: Strigs beig edited Costs of primitive operatios are idepedet isertio deletio Record i each ode the miimum cost to reach there X Y B D A B C D I practice, oly a sigle row of memory is required at a time O( 2 ) time ad O() space 11

12 Matri Chai Multiplicatio: A More Comple DP Fidig the best order of computig matri multiplicatio chai Needs the product of may matrices Associativity allows ay computatio order ( A1 A2 A3 ) A4 A1 (A2 A3 A4 ) The order affects the computatioal cost The umber of scalar multiplicatios should be miimized The umber of possible orderig icreases epoetially proportioal to the umber of matrices p i-1 Matri Chai Multiplicatio Size of the matri A i is p i-1 p i Multiplyig a p q matri by a q r matri requires p q r scalar multiplicatios By A i, j, we mea the product A i A j A i, j has the size p i-1 p j p i A i = p i p i+1 A i+1 p i+1 p i+2 Ai+2 p i-1 p i+2 A i, i+2 Recurrece Equatio o Computatioal Cost With m i, j beig the least umber of scalar multiplicatios to obtai A i, As A i, j is obtaied by A i, A, j for some, we have the followig recurrece equatio m 0 mimi i j m p i, j,, j i1 p p j ; ; i j i j Matri Chai Computatio as DP Subproblems : Computig A i, j (1 i < j ) These subproblems are idepedet The umber of subproblems is O( 2 ) where is the umber of matrices Small eough to record 1. Mae a cost table of m i, j With iitial values of ifiity (cost uow) 2. Compute m i,j startig with m i,i+1 ad gradually wideig the rage 3. Whe m 1, is reached, trace bac the way it is computed 12

13 Matri Chai Computatio m 1,2 m 2,3 m 3,4 m 4,5 m 1,3 m 2,4 m 3,5 m 1,4 m 2,5 Approimatig Algorithms Algorithms, usually for NP-hard problems, that give solutios ot ecessarily optimal but hopefully acceptable (suboptimal) Polyomial-time approimatio Algorithms that repeatedly search for better solutios m 1,5 Polyomial-Time Approimatio By permittig (1+ ε) times the cost of the optimum solutio, computatioal time might be made proportioal to a polyomial of the problem size Computatioal compleity ca be polyomial for fied ε but rapidly becomes larger for smaller ε, for eample, O( 1/ε ), or eve O( ep(1/ε) ) Thus, the algorithms are ot practical for problems requirig small ε Iterative Improvemet Methods 1. Fid a iitial feasible solutio somehow, which should satisfy the costrait but may be far from optimal 2. The solutio is modified a bit without violatig the costraits, maig the et feasible solutio (eighbor solutio) 3. Repeat the process util some termiatio coditio is reached Small modificatios are epected to lead to a little better feasible solutios 13

14 Simple Iterative Improvemet I the step 2 of the previous page, always choose the best amog the eighbor solutios Simple ad efficiet Several ames Local Search Greedy Search Hill Climbig Local Search 1. If there eists a better feasible solutio i the eighborhood of the curret solutio, mae that solutio curret 2. Repeat this util there is o better solutios i the eighborhood Neighborhood: A set of feasible solutios that ca be easily derived from the curret solutio Usually, some of the variables comprisig the solutio are modified Neighborhoods too wide may icur high cost Should be able to cover all the feasible solutios K-Opt method: A Neighborhood Costructio Scheme for TSP paths i the solutio are cut ad With cities, eighborhood size is O( ) Commoly used are 2 ad 3 Or-Opt method: Aother Neighborhood for TSP s cities i the route are removed ad iserted agai at a differet positio Neighborhood size is O( 2 ) Commoly used s are 1 through 3 2-opt Or-opt 3-opt 14

15 Covergece to Local Optima Local search may result i a locally optimal solutio which is far from the global optimum cost iitial solutio oe of local optima solutio space The global optimum Summary Costrait satisfactio ad costrait optimizatio problems Combiatorial optimizatio Strict algorithms Pruig, forward checig, lower boud Dyamic programmig Approimate algorithms Algorithms with ow approimatio errors Iterative Improvemet But, local search may lead to a local optimum Net Wee Iterative improvemet methods that ca escape from local optima Are there geeral methods that are ot specific to problem domais? Metaheuristics 15