Artificial Intelligence 3 Problem Complexity Prof. Dr. Jana Koehler Fall 2016
Agenda Computability and Turing Machines Tractable and Intractable Problems P vs. NP Decision Problems Optimization problems Multi-Criteria Optimization Pareto Optimality 2
Length of the minimal Tour = 27 3
4
Inherently Intractable problems 5
Exponential vs. Polynomial Growth 6
Can Faster Computers Help? 7
Can we Live with Suboptimal Solutions? Only if suboptimal does not mean "arbitrarily bad" rule-based assignment of passengers led to very low transportation capacity preventing market acceptance of the system an ordering algorithm often finds no solution many solutions turn out to perform very poor in practices control is manually developed restricting the systems to a mass market seqment 8
An Informal Notion of Computability We have an intuitive notion of an algorithm that can compute a result over some input we can encode the input in the natural numbers 1,, n our algorithm (Java programm) will compute a function f Given the input x 1, x 2,, x n the arguments of the function f, the algorithm should stop returning f(x 1, x 2,, x n ) after a finite amount of steps If f is a partial function (undefined for some of its arguments), the algorithm should loop infinitely (not stop) Can we formalize this notion? 9
Turing Machine Explained 41- https://www.youtube.com/watch?v=dnrdvlacg5q 10
Turing Machine 11
Example States Q = z 0, z e Blank Input symbols Σ= 0,1, Tape symbols Γ= 0,1, Inital state q 0 = z 0 Accepting final states F = {z e } transition function δ δ(z 0,0) = (z 0,0,R) δ(z 0,1) = (z 0,1,R) δ(z 0, ) = (z e,,l) 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 12
A Working Turing Machine 42 - https://www.youtube.com/watch?v=e3kelemwfhy 13
Deterministic and Non-Deterministic Turing Machines Deterministic Turing machine (DTM): the set of rules prescribes at most one action to be performed for any given situation Non-deterministic Turing machine (NTM): may have a set of rules that prescribes more than one action for a given situation δ(z 0,0) = (z 0,0,R) δ(z 0,1) = (z 0,1,R) δ(z 0, ) = (z e,,l) δ is function δ(z 0,0) = (z 0,0,R) δ(z 0,0) = (z 0,0,L) δ(z 0,1) = (z 0,1,R) δ is a relation δ(z 0,1) = (z 0,0,R) Every Non-Deterministic TM (NDM) can be simulated by a Deterministic TM (DTM)! 14
Church's Thesis The formal definition of Turing-Computability captures exactly the class if intuitively computable functions. 15
The Problems computed by DTM and NTM An NTM only needs to have(!) some sequence of choices that results in "yes" to accept an input The length of an accepting computation of the DTM is, in general, exponential in the length of the shortest accepting computation of the NTM DTM needs to simulate all possible sequences of choices n! for n transition rules general property of simulations of NTMs by DTMs 16
The complexity Classes P and NP P = set of problems, a DTM can solve in polynomial time NP = set of problems, a NTM can solve in polynomial time an NTM always picks the best nondeterministic transition and ends in an accepting end state in an amount of time bounded polynomially on the size of the input Our computers (algorithms) are DTMs, hence they need exponential time on problems in NP! they have to explore all possible sequences in the worst case 17
P vs NP Video 43 - https://www.youtube.com/watch?v=yx40hbahx3s&spfreload=10 18
Example: The Subset Sum problem is in NP Given a set of integers, does some nonempty subset of them sum to 0? s= { 2, 3, 15, 14, 7, 10} yes! x = { 2, 3, 15, 10} because 2 + 3 + 10 + 15 =0 It is easy (in polynomial time) to verify that x is a solution To find this solution, 2 n -1 tries (all non-empty) subsets have to be tried, enumerating this exponential set takes exponential time An NTM can nondeterministically pick the sequence of transitions that directly generate this set, a DTM cannot 19
Decision Problems I. Given a set of integers, does some nonempty subset of them sum to 0? II. Does the Turing machine M halt on input x (Halteproblem)? III. Does x evenly divide y? Decision Procedure: method for solving a decision problem, given in the form of an algorithm I. Try all nonempty sum subsets, if there is one that equals 0, answer yes, otherwise no II.???? if the machine stops, ok, but what if not???? III. If the division algorithm returns the remainder 0, the answer is yes, otherwise it is no 20
Complete Decision Problems Decision problems, which can be solved by an algorithm, are called decidable A decision problem x is said to be complete for a set of decision problems S if x is a member of S and every problem s in S can be reduced to x Reduce a problem s in S to x any instance of the problem s can be transformed to an equivalent instance of x any algorithm that solves x can be transformed to solve s Polynomial-time reductions construct polynomial-time algorithms from polynomial-time algorithms 21
NP-Completeness (Cook 1971) Demonstrated P-time reductions Defined the class of NP problems and phrased them as decision problems Showed that every problem in NP can be P-time reduced to the Satisfiability problem, which is also in NP any algorithm for SAT will solve any other NP problem SAT is NP-hard A decision problem is NP-complete if it is in NP and it is NP-hard or if another NP-complete problem reduces to it 22
A SAT Problem Instance Given a formula in propositional logic, is there an assignment of truth values which makes the formula true? SAT solving is a hot AI-related research area 23
NP-Completeness Complete decision problems are used in computational complexity to characterize complexity classes of decision problems membership: SAT is in NP hardness: SAT is NP-hard Unless P=NP, there is no poynomial time solution for SAT 24
The Theory of NP-Completeness gives us at least 25
Examples of NP-complete Problems Subgraph Isomorphism Given 2 graphs G and H, does G contain a subgraph isomorph to H? Traveling Salesman Given a number of cities, a distance for each pair of cities, does there exist a tour of all cities no longer than k? 26
Between P-complete and NP-complete Graph Isomorphism Are two graphs G and H isomorph to each other? Integer factorization or prime factorization Is there a decomposition of an integer into smaller nontrivial divisors, which when multiplied together equal the original integer? No polynomial-time algorithms known Could not be proven yet to be NP-complete 27
The Halting Problem 44 - https://www.youtube.com/watch?v=92whn-pafcs 28
Our Current Complexity Map EXPTIME co-np complement of NP PSPACE P NP 29 AI is usually concerned with NP-hard problems
The Class PSPACE Set of all decision problems solvable by a deterministic Turing machine using a polynomial amount of space in the length of the input length of the solution must be polynomially bounded Examples Given a regular expression R, does it generate every string over its alphabet? Word problem for deterministic context-sensitive grammars Games (when played on an n n board): hex, Reversi, Rush Hour, Mahjong, Atomix, Sokoban the game can only last for a polynomial number of moves 32
The Class EXPTIME Set of all decision problems solvable by a deterministic Turing machine in exponential time in the size of the input the solution length is NOT polynomially bounded Checkers, GO, Chess the game can last for an exponential number of moves Does a DTM halt in at most k steps? 33
Towers of Hanoi A simple EXPTIME Problem Rebuild the tower on poole C using poole B for intermediate positions of n disks. the length of the shortest solution is A B C 34
Hard and Easy Problem Instances Complexity theory gives us a worst-case result! does not say anything about the difficulty of specific instances of a problem 35
From Decision to Optimization Problems Traveling Salesman Given a number of cities, a distance for each pair of cities, does there exist a tour of all cities no longer than k? k is an input parameter on a yes/no question does not necessarily require to compute the tour Optimizing TSP Given a number of cities, a distance for each pair of cities, find the shortest tour of all cities need to find a tour and show that no other tour is shorter 36
TSP Is there a tour of length < 50? clearly not! Is there a tour of length < 2500? What is the length of the shortest tour? 37
How can I recognize an NP-hard Problem in Practice? The problem is difficult to describe There is confusion about what really constitutes the problem Experts have different opinions on what is a good solution to the problem There seem to be many possible solutions to a problem When implementing, the code seems to require to explore many alternatives 38
Summary There are inherently intractable problems! these problems cannot be optimally solved without investing an effort exponential in the size of the problem instance they occur more often in practice and are not so easy to recognize Exponential growth cannot be compensated by adding more computational power! Giving up completeness is an option, but we need to be aware that solution quality can become arbitrarily bad! Be careful when designing optimization criteria, they may not only have the desired effects! 44