DESIGN AND ANALYSIS OF ALGORITHMS Unit 6 Chapter 17 TRACTABLE AND NON-TRACTABLE PROBLEMS http://milanvachhani.blogspot.in
COMPLEXITY FOR THE IMPATIENT You are a senior software engineer in a large software house. A demanding customer has asked for a program to solve a new, seemingly difficult, problem. On realizing that other customers also may come up with similar requirements, your manager has entrusted you the job of finding a most efficient general algorithm for the problem. After spending considerable time and effort on the job, you find that you are not able to find an efficient algorithm. What are the possibilities and what could you do?
In the above list of possibilities, a significant departure in attitude come at points marked with letter N. How do we decide that point?
In the previous chapters we have encountered a variety of algorithms, having efficiencies ranging over a wide spectrum. Even with the same basic algorithm, we noticed that running times can be reduced significantly by some adjustment of the algorithm. We noted this in case of the bogosort.c and bogosorti.c programs given in Chapters 1 and 4, where a small modification in the algorithm reduced the expected run times from O(n!) to O(n 2 ). Our studies in the later chapters showed us that the best running time we can expect for known sorting algorithm is O(n log(n)). This raises a question in our mind- Is this the minimum time that any sorting algorithm can achieve?" Note that here the question is not about a particular algorithm, but rather any algorithm whatsoever for sorting a given array. In other words, the question is about the problem itself-"does the sorting problem allow an algorithm having time less than O(n log(n))?"
Thus our focus now turns from a particular algorithm to a class of problems, e.g. sort problems. The range of conceptual abstractness of investigations involved is shown in next Figure. For example, suppose we have not developed efficient sorting algorithms like Quick-sort or Merge-sort and did all our sorting by an algorithm like bogosorti.c. Now suppose some clever researcher proves a lowerbound of O(n log(n)) on the running time of the class of sorting problems (i.e. all the possible sorting problem instances), then a host of developers will get busy trying to develop more efficient sorting algorithms, because now it is known that such an algorithm is possible. The above observations also points to the importance of finding the Lower bound on the expected running times of a class of problems.
Our major concern arises out of some of the results that we have already seen in the previous Chapters 4 and 14. Referring to previous figure, where we have plotted running times of algorithms with different behaviours, which one is the "odd-man-out"? Obviously the algorithm with run-time of 2 n is the" oddman-out". Even if the polynomial terms like n 4 are multiplied by large constant factors, say 10 3, their runtimes will still be much less than those of the algorithm with 2 n time. We are concerned about such difficult or hard algorithms. If we come across problems for which the only known algorithms are of such hard type, we should study those problems more closely. That is why we study Complexity classes of problems, with a view to categorize problems according to the timing behaviour of the best algorithms to solve them. The tentative situation observed till now is shown in next figure.
Computer scientists found that some problems are well behaved enough for the algorithms which solved them had polynomial time characteristics. Such problems were said to be tractable and were considered to be in a set called P, standing for polynomial time solvable problems. Others were said to be non-tractable. Soon it was found that matter was not that simple and clear-cut and required deeper study. After that rather rough and ready exposure to tractability, we present here, quick and terse elucidation of the current understanding about problem complexity which is accepted by most of the researchers.
Polynomial Time and Decision Problems In view of the considerations for run-times put forward above and in Chapters 4 and 14, we use the Polynomial- time as the cut-off for efficiency of algorithms and complexity of problem classes. That part is clear enough, but what about problems which are beyond that cutoff? To make our discussion simple, we take up for discussion a limited, but important class of problems, called Decision Problems. Decision problems have as their output either 1 or 0 (or "yes" or "no"). Some examples are:
Does a given graph G have an Euler tour? Does a text S contain a pattern R? Does an instance of 0/1 Knapsack have a solution with profit at least K? Does a graph G have an MST with weight at most K? Does a given set of n cities, in a TSproblem, have a tour of length less than a limit m? Note that in such problems we are not asking for a particular solution, but whether a solution exists at all. The algorithm to get the yes/no answer may or may not find the actual solution.
Problems and Languages The algorithm which solves a particular decision problem will be given the problem instance in form of a question, as concisely as possible and possibly coded in some language. For example, the decision problem "Does a text S contain a pattern R?" may be given to the algorithm as "in('this is good.', 'is')?" Thus a problem is expressed in form of a sentence or a string in some language. The given string may not result in a "yes" answer. Our definition of a language is slightly narrower. A language L is a set of strings x defined over some alphabet L. Every decision algorithm A defines a language L. A language L is the set consisting of every string x such that A outputs "yes" on input x. We say "A accepts x" if that is so. Expressed in symbolic form: For example, if an algorithm A determines whether or not a given graph G has an Euler tour, then the language L for A is all graphs with Euler tours.
Complexity Classes and Class P A complexity class is a collection of languages satisfying certain criteria. P is the complexity class consisting of all languages that are accepted by polynomial-time algorithms. For each language L in P there is a polynomial time decision algorithm A for L. If n= x, then x Є L, A runs in p(n) time on input x, the function p(n) is some polynomial in n. What about problems which are not in P? Consider the decision form of the Traveling Salesman problem: "Given a set of n cities, does it have a tour of length less than a limit m?"
There is no known polynomial time algorithm for solving this problem. However, suppose we find for a particular instance of this problem, that the answer for that instance is "yes." We are skeptical and would like to ensure correctness of the result by checking that the tour, which was basis for that answer, has indeed the required properties. It would then be a simple matter of computing its length and comparing that quantity to the given bound m. This verification procedure will have a polynomial time complexity. This idea of polynomial time verifiability is being described by the class NP. Note that polynomial time verifiability does not imply polynomial time solvability. Note well that NP stands for "Non-deterministic Polynomial-time" and not for "Non-Polynomial-time".
Complexity Class NP-1st Definition We say that an algorithm is non-deterministic if it uses some operation like: choose(v): chooses a value or a symbol v. Such an operation may also be used to choose an entire string y, with Y choices. How this choice is made is not specified. For example, it may be done by a random select out of a finite set. As if by some magic, which is out of purview of the algorithm, we get the chosen value. We say that a non-deterministic algorithm A accepts a string x if there exists some sequence of choose() operations that causes A to output "yes" on input x. NP is the complexity class consisting of all languages accepted by polynomial-time non-deterministic algorithms.
Example: Problem: Decide if a TS has a tour of length at most K. Algorithm: 1) Non-deterministically choose a set T of n edges, 2) Test that T forms a valid tour 3) Test that T has length at most K Analysis: Testing takes O(n) time, so this algorithm runs in polynomial time. Complexity Class NP - 2nd Definition: We say that an algorithm B verifies the acceptance of a language L iff, for all x in L, there exists a certificate y such that B outputs "yes on input (x, y). We have already had a rather imprecise glimpse of what a certificate looks like in our bogosort. c program, in form of the sorted () function. NP is the complexity class consisting of all languages verified by polynomial-time algorithms. Note that verification of a candidate solution is easier than computing the solution itself.
Example: We take the following problem as an example for the alternate definition of the class NP. Problem: Decide if a TS problem has a tour of length K or less. Verification Algorithm: 1) Use as a certificate, y, a set T of n edges, 2) Test that T forms a valid tour 3) Test that T has length at most K Analysis: Generation and Verification takes O(n) time, so this algorithm runs in polynomial time. At this stage we can say that: We know that P NP. The major open question is P = NP? Most researchers believe that P NP Generally accepted relationship between class P and class NP. P corresponds to the Tractable class, but NP is not Prepared necessarily by same : Dr. Milan as non-tractable Vachhani
Equivalence of the Two Definitions of NP: We show the Equivalence of the two definitions of NP by demonstrating how to implement an algorithm of one type in terms of the one of the other type. Suppose A is a non-deterministic algorithm. Let y be a certificate consisting of all the outcomes of the choose() steps that A uses. We can create a verification algorithm B' that uses y instead of A s choose steps. If A accepts x, then there is a certificate y that allows us to verify this (namely, the choose steps A made). If A runs in polynomialtime, so does this verification algorithm B'. Please see Figure.
Suppose B is a verification algorithm. Non-deterministically choose a certificate y and run B on y. If B runs in polynomialtime, so does this non-deterministic algorithm A. Please see Figure.
Polynomial Transformations We have to find a method to compare relative difficulty of solving various classes of problems. This is done by transforming a problem into another, by an auxiliary Polynomial-time algorithm. This is also called reduction technique. The main technique used for demonstrating that two problems are related is that of "reducing" one to the other, by giving a transformation that maps any instance of the first problem into an equivalent instance of the second. Such a transformation provides the means for converting any algorithm that solves the second problem into a corresponding algorithm for solving the first problem. In other words, if we have an algorithm solving the second problem, we can also solve the first problem. More specifically, if we have a polynomial time reduction from one problem to another, this ensures that any polynomial time algorithm for the second problem can be converted into a corresponding polynomial time algorithm for the first problem.
Cook in his 1971 paper showed that one particular problem in NP, called the "satisfiability problem, (see CIRCUIT-SAT below), has the property that every other problem in NP can be polynomially reduced to it. If the CIRCUIT-SAT problem can be solved with a polynomial time algorithm, then so can every problem in NP, and if any problem in NP is intractable, then the CIRCUIT-SAT problem also must be intractable. Thus CIRCUIT-SAT problem is the hardest problem in NP. Many other problems in NP are found to be as hard as CIRCUIT-SAT, and a class of problems, NP- Complete, is defined which contain them. Formally,
NP Problems A problem (language) L is NP-Hard if every problem in NP can be reduced to L in polynomial time. That is, for each language L' in NP, we can take an input x from L', transform it in polynomial time to an input x' for L such that x is in L' iff x' is in L. In other words, the problem is at least as difficult as all other problems in NP. L is NP-Complete if it's in NP and is NP-Hard. Please see Figure.
CIRCUIT-SAT is in NP An Interesting problem, known as CIRCUIT-SAT (or, simply SAT), is studied as an example of a NP problem. A Boolean circuit is a circuit of AND, OR, and NOT gates. For example, a simple logic circuit is shown in Figure. The CIRCUIT-SAT problem is to determine if there is an assignment of 0's and 1's to a given Boolean circuit's inputs so that the circuit outputs 1. The problem can also be enunciated in terms of Boolean expressions, e.g. f = a v b ^ c.
Non-deterministically choose a set of inputs to the circuit and then test the circuit s output. As this can be done in polynomial time, CIRCUIT-SAT is in NP. Cook-Levin theorem says that CIRCUIT-SAT is in NP- Complete. Some NP-Complete Problems Some of the well studied NP-Complete problems are: CNF-SAT and 3SAT Vertex Cover Clique Hamiltonian Cycle Set Cover SubsetSum 0/1 Knapsack Traveling Salesman
Is P equal to NP? Most of the present day researchers believe that P is a proper subset of NP, i.e. P NP. Implication of this belief is that the NPComplete problems are the hardest in NP. This is because if we could solve a NP-Complete problem in polynomial time, we could solve every problem in NP in polynomial time. That is, if an NP-Complete problem is solvable in polynomial time, then P = NP. Significance of a problem being NP-complete: Since so many people have attempted, without success, to find polynomial-time solutions to NP-Complete problems, showing your problem is NP-Complete is equivalent to showing that a lot of smart people have worked on your problem and found no polynomial-time algorithm. If a polynomial-time algorithm is known for one NP-Complete problem, then there is a polynomial-time algorithm for every NP-Complete problem. An exponential lower bound for such a problem has never been shown. If difficult instances of an NP-Complete problem arise in practice, then approximation schemes with bounds on the quality of the solution are needed.
VIDEO REFERENCES P, NP, NPC, NPH https://www.youtube.com/watch?v=f7u7uk6iiu4 https://www.youtube.com/watch?v=htlypzgpohc https://www.youtube.com/watch?v=1qehnctsveu https://www.youtube.com/watch?v=gq4k5hhiise https://www.youtube.com/watch?v=vgckklmcgei
THANK YOU Dr. Milan Vachhani Assistant Professor, Sunshine Group of Institutions, Rajkot milan.vachhani@gmail.com http://milanvachhani.blogspot.com