Una descrizione della Teoria della Complessità, e più in generale delle classi NP-complete, possono essere trovate in:

Transcription

1 AA. 2014/2015

2 Bibliography Una descrizione della Teoria della Complessità, e più in generale delle classi NP-complete, possono essere trovate in: M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, W. H. Freeman, 1st ed. (1979) Cormen, Leiserson, Rivest and Stein, Introduzione agli Algoritmi e Strutture Dati, McGraw-Hill (capitolo 34) A compendium of NP optimization problems at the link:

3 NP- Completeness Polynomial- time algorithms on inputs of size n: O(n k ) All problems can be solved in polynomial time? NO Halting Turing Problem: Given a description of an arbitrary computer program and a ;ixed input (;inite), decide whether the program ;inishes running or continues to run forever The problems can be tractable or inctractable If you can establish a problem as NP- complete, you provide good evidence for its intractability spending the time developing an approximation algorithm, rather than searching for a fast algorithm

4 What is a Problem? We define a problem Q to be a binary relation on a set I of problem instances and a set S of problem solutions: Q: I S Shortest-path problem: finding a shortest path between two given vertices in an unweigthed, undirected graph G=(V, E) instance: vertices is a triple consisting of a graph and two solution: a sequence of vertices in the graph (path) In general a given problem instance may have more than one solution

5 What is a Decision Problem? The theory of NP-completeness restricts attention to decision problems: those having yes/no solution. Decision problem GCP: "Is a given graph G=(V,E) K- colorable? Decision Problems: Q: I {0, 1} Example: if i=<g,k> is an instance of the graph coloring problem then COL(i)=1 (yes) if there exist a k- coloring, and COL(i)=0 (no) otherwise. Optimization Problems: those where some value must be minimized or maximized.

6 Review: P and NP Summary so far: P = problems that can be solved in polynomial time NP = problems for which a solution can be verified in polynomial time Unknown whether P = NP (most suspect not) Hamiltonian-cycle problem is in NP: Cannot solve in polynomial time Easy to verify solution in polynomial time Mario Pavone, AA. 2014/2015 Computazione Naturale CdL Magistrale in Informatica DMI UniCt mpavone@dmi.unict.it

7 Hamiltonian Cycle Problem A hamiltonian cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V Does a graph G have a hamiltonian cycle? Algorithm: lists all permutations of the vertices and check each permutation to see if it is a hamiltonian path there are m! Possible permutations of the vertices afterwards CHECK if the provided cycle is hamiltonian: i.e. if it is a permutation of the vertices of V and each consecutive edges along the cycle exists in the graph This verification can be implemented to run in O(n 2 ) time If a hamiltonian cycle exists in a graph can be verified in polynomial time Mario Pavone, AA. 2014/2015 Computazione Naturale CdL Magistrale in Informatica DMI UniCt mpavone@dmi.unict.it

8 Hamiltonian Cycle Problem Hamiltonian Cycle (input: a graph G) Does G have a Hamiltonian cycle? Solution: 0, 1, 2, 11, 10, 9, 8, 7, 6, 5, 14, 15, 6, 17, 18, 19, 12, 13, 3, 4 Such solution can be evaluated in a polynomial time! Mario Pavone, AA. 2014/2015 Computazione Naturale CdL Magistrale in Informatica DMI UniCt mpavone@dmi.unict.it

9 Reduction The crux of NP-Completeness is reducibility Informally, a problem P can be reduced to another problem Q if any instance of P can be easily rephrased as an instance of Q, the solution to which provides a solution to the instance of P This rephrasing is called transformation Intuitively: If P reduces to Q, P is no harder to solve than Q If P is polynomial-time reducible to Q, we denote this P p Q Mario Pavone, AA. 2014/2015 Computazione Naturale CdL Magistrale in Informatica DMI UniCt mpavone@dmi.unict.it

10 Reducibility An example: P: Given a set of Booleans, is at least one TRUE? Q: Given a set of integers, is their sum positive? Transformation: (x 1, x 2,, x n ) = (y 1, y 2,, y n ) where y i = 1 if x i = TRUE, y i = 0 if x i = FALSE Another example: Solving linear equations is reducible to solving quadratic equations How can we easily use a quadratic-equation solver to solve linear equations? Mario Pavone, AA. 2014/2015 Computazione Naturale CdL Magistrale in Informatica DMI UniCt mpavone@dmi.unict.it

11 NP-Completeness If P p Q then P is not more than a polynomial factor harder than Q A problem P is NP-Complete if P NP, and P' p P, for every P' NP If all problems Q NP are reducible to P, then we say that P is NP-Hard A problem P is NP-Complete if it is in the NP class and is NP-Hard (P is NP-Hard and P NP) If P p Q and P is NP-Complete, Q is also NP-Complete

12 Example of some NP-complete problems Given one NP-Complete problem, we can prove many interesting problems NP-Complete Graph coloring (= register allocation) Hamiltonian cycle Hamiltonian path Knapsack problem Traveling salesman Job scheduling with penalities Many, many more

13 Why Prove NP-Completeness? Though nobody has proven that P!= NP, if you prove a problem NP-Complete, most people accept that it is probably intractable Therefore it can be important to prove that a problem is NP- Complete Don t need to come up with an efficient algorithm Can instead work on approximation algorithms

14 How I Prove the NP-Completeness? What steps do we have to take to prove a problem P is NP- Complete? Pick a known NP-Complete problem Q Reduce Q to P Describe a transformation that maps instances of Q to instances of P, s.t. yes for P = yes for Q Prove the transformation works Prove it runs in polynomial time Oh yeah, prove P NP

15

16 SAT (Satifability) 0 Steve Cook in 1971 proved that SAT is NPcomplete 0 Variables: u 1, u 2, u 3,... u k 0 A literal is a variable u i or the negation of a variable u i 0 If u is set to true then u is false and if u is set to false then u is true 0 A clause is a set of literals. A clause is true if at least one of the literals in the clause is true 0 The input to SAT is a collection of clauses.

17 SAT (Satifability) 0 The output is the answer to: Is there an assignment of true/false to the variables so that every clause is satisfied (satisfied means the clause is true)? 0 If the answer is yes, such an assignment of the variables is called a truth assignment. 0 SAT is in NP: Certificate is true/false value for each variable in satisfying assignment.

18 Hamiltonian Cycle TSP The well-known traveling salesman problem: Optimization variant: a salesman must travel to n cities, visiting each city exactly once and finishing where he begins. How to minimize travel time? Model as complete graph with cost c(i,j) to go from city i to city j How would we turn this into a decision problem? A: ask if a TSP with cost < k

19 Hamiltonian Cycle TSP The steps to prove TSP is NP-Complete: Prove that TSP NP (Argue this) Reduce the undirected hamiltonian cycle problem to the TSP So if we had a TSP-solver, we could use it to solve the hamilitonian cycle problem in polynomial time How can we transform an instance of the hamiltonian cycle problem to an instance of the TSP? Can we do this in polynomial time?

20 The TSP Random asides: TSPs (and variants) have enormous practical importance E.g., for shipping and freighting companies Lots of research into good approximation algorithms Recently made famous as a DNA computing problem

21 Combinatorial Landscapes The notion of landscape is among the rare existing concepts which help to understand the behaviour of search algorithms and heuristics and to characterize the difficulty of a combinatorial problem.

22 Search Space Given a combinatorial problem P, a search space associated to a mathematical formulation of P is defined by a couple (S,f ) where S is a finite set of configurations (or nodes or points) and f a cost function which associates a real number to each configurations of S. For this structure two most common measures are the minimum and the maximum costs. In this case we have the combinatorial optimization problems. Combinatorial optimization problems are often hard to solve since such problems may have huge and complex search landscape

23 Example: K-SAT An instance of the K-SAT problem consists of a set V of variables, a collection C of clauses over V such that each clause c C has c = K The problem is to find a satisfying truth assignment for C The search space for the 2-SAT with V =2 is (S,f ) where S={ (T,T), (T,F), (F,T), (F,F) } and the cost function for 2-SAT computes only the number of satisfied clauses f sat (s)= #SatisfiedClauses(F,s), s S

24 SEARCH SPACE of K=2-SAT Let we consider F = (A B) ( A B) A B f sat (F,s) T T 1 T F 2 F T 1 F F 2

25 Example and Relevance of Landscape The search Landscape for the K-SAT problem is a N dimensional hypercube with N = number of variables = V Combinatorial optimization problems are often hard to solve since such problems may have huge and complex search landscape

26 K- SAT: HYPERCUBES

27 Department of Com Will get stuck in local optima. Plateaus can cause the algorithm to wande

28 Complexity of the Landscape PSP E 0 E E *

29 Complexity of the Landscape PSP Energy level No. of Conformations Total

30 Fitness Landscape Components of Nitness landscape: Set S of admissible solutions Fitness function that assigns a real value to each solution Distance measure that denines distance between any two solutions in S Example distance measures: Hamming distance for binary strings (num. mismatched bits). Euclidean distance metric for continuous vectors Easy Problems, and Hard Problems: Easy: few peaks; smooth surfaces, no ridges/plateaus Hard: many peaks; jagged or discontinuous surfaces, plateaus

31 Fitness Distance Correlation (FDC): Intuition Loose definition I FDC measures correlation between fitness and distance to global optimum. Simple functions I Being closer to a global optimum leads to higher fitness. I Climbing directly to the optimum is rather easy. Di cult functions I Being closer to a global optimum may lead to lower fitness. I Climbing directly to the optimum is not that easy. Martin Pelikan NK Landscapes, Problem Di culty, and Hybrid EAs

32 Correlation Length: Intuition Loose definition I Correlation length measures ruggedness of the landscape. Simple functions I Solutions close to each other have similar fitness values. I Fitness di erences increase slowly with distance. Di cult functions I Solutions close to each other have di erent fitness values. I Fitness di erences increase fast with distance. Martin Pelikan NK Landscapes, Problem Di culty, and Hybrid EAs