Discrete Optimization 2010 Lecture 10 P, N P, and N PCompleteness

Discrete Optimization 2010 Lecture 10 P, N P, and N PCompleteness Marc Uetz University of Twente m.uetz@utwente.nl Lecture 9: sheet 1 / 31 Marc Uetz Discrete Optimization

Outline 1 N P and co-n P 2 N P-completeness 3 Clique is N P-complete Lecture 9: sheet 2 / 31 Marc Uetz Discrete Optimization

!"#$%&"'()"#*+'%,*#$%&-".'$#/%0%1'2"-*34#/ Some Problems with Poly-Time Algorithms!!"#"$%$&'()##"#*&+,--&.!'+/&0#&-1*-23-"*45-16&%#1",-75-1&*,)(4! 8,"$9 :;*0,"54$<& =.! > /!?,%9@);9 :;*0,"54$ =." ;0*&"/!!)A"$%$&B;03&.!B/&"#&)&1",-75-16&7)()7"5)5-1&*,)(4! '40,5-95&:%*$-#5"#*&8)54&:;*0,"54$<& =.! > "/! C"*4-95&;)D-;&8,-E;0328%94&:;*0,"54$< =.! > " FG> /! '40,5-95GH0#*-95&8)54&.'8/&0#&-1*-23-"*45-1&1",-75-1&*,)(4! )7I7;"7&1"*,)(4< =."/! #0##-*)5"J-&),7&3-"*459&.K"L@95,)/< =.! > /! ),D"5,),I&1"*,)(4M&);;&()",9&.B;0I12N),94);;/< =.! O /! '0,5"#*&! #%$D-,9! P%DD;-&'0,5< =.! > /!!-,*-&'0,5< =.! ;0*&!/! H"#-),&8,0*,)$$"#*&.H8/! Q;;"(90"1&!-5401&.?4)74"I)#R9 +4-0,-$/ =..!S"/ T! H.#$%$&/ > /! '"$(;-A&:;*0,"54$ #05 (0;I#0$");&5"$-U (in.v05"7-<&b0,&*,)(4&(,0d;-$96&! case of LP, A Q m n, L( "9&54-&#%$D-,&0E&#01-96&" ) = encoding length of 54-&#%$D-,&0E&),79M&E0,& A, b, c) H"#-),&8,0*,)$$"#*6&! )#1&" ),-&54-&$)5,"A&1"$-#9"0#9<&# "9&)# ""! $)5,"A/ Lecture 9: sheet 3 / 31 Marc Uetz Discrete Optimization

!"#$%&'"()$#*%+,-."/- &")0-,#$ 1)2"',-.#* Some Problems without Poly-Time Algorithms!!"#$%&'()*+#&%,-#(*."/0&%-*1!+.2! 3(4%)%"*5'(%#"*."/)"#--'()! 6#7'-8-*9&':8%*18(;'"%<4%;*)"#=>2! 6#7'-8-*3(;%=%(;%(4*+%4*18(;'"%<4%;*)"#=>2! 6#?%,=#( +<>%;8&'()! @%>'<&%*A/84'()*."/0&%-! 6#7'-8-*984*18(;'"%<4%;*)"#=>2*! @%"4%7*9/$%"B*+%4*9/$%"B*+%4*.#<?'()! C(#=,#<?*."/0&%-! D'(*.#<?'()*."/0&%-! +#4',E'#0'&'4F! G*G*G* Lecture 9: sheet 4 / 31 Marc Uetz Discrete Optimization

Outline 1 N P and co-n P 2 N P-completeness 3 Clique is N P-complete Lecture 9: sheet 5 / 31 Marc Uetz Discrete Optimization

Problem class N P: Nondeterministic Polynomial Time Given: Decision problem P, I encoding length of instance I P Definition Algorithm A is nondeterministic polynomial time algorithm for P if For all Yes instances I, poly-length certificate z(i ) (i.e., z(i ) p( I ) for some polyn. p), and A(I, z(i )) = Yes For all No instances I, and any poly-length input z ( fake certificate ), A(I, z) = No A(I, z) is a poly-time algorithm for any poly-length z Definition P N P : P has nondeterministic poly-time algorithm Yes instances of P have a polynomial certificate that can be verified in polynomial time Lecture 9: sheet 6 / 31 Marc Uetz Discrete Optimization

Example of N P-problem TSP: Given graph G = (V, E) with V = n, integer edge weights d e 0 and k N: exist Hamiltonian cycle of total length k? Theorem TSP N P Proof: Encoding length TSP: Ω(n + log U), with U = max e d e Certificate Yes instance: The set of edges of the tour, T E O( n ) Certificate checking: Check if the following exhausts all nodes: start with any e = {u, v} T, in T \ {e} look for edge e incident with v, and iterate with e O( n 2 ) Check if e T d e k O( n 2 log U ) Lecture 9: sheet 7 / 31 Marc Uetz Discrete Optimization

Problem class co-n P: Complement-N P Given: Decision problem P, I encoding length of instance I P Definition Algorithm A is nondeterministic polynomial time algorithm for complement problem P if For all No instances I, poly-length certificate z(i ) (i.e., z(i ) p( I ) for some polyn. p), and A(I, z(i )) = Yes For all Yes instances I, and any poly-length input z ( fake certificate ), A(I, z) = No A(I, z) is a poly-time algorithm for any poly-length z Definition P co-n P: No instances of P have a polynomial certificate that can be verified in polynomial time P N P Lecture 9: sheet 8 / 31 Marc Uetz Discrete Optimization

Example of co-n P Problem TUM: Given integer matrix A Z m n : is A totally unimodular? Theorem TUM con P Proof Encoding length A: Ω(n + m) Certificate No instance: Square submatrix B of A with det(b) {0, ±1} O( nm ) Certifcate checking: Compute det(b) this is non-trivial, because well-known Laplace formula det(b) = j b ij( 1) i+j det(b i, j ) requires Ω(n!) time possible via computing LU decomposition of B, where B = LU, with L = lower triangular, U = upper triangular O( n 3 ) Lecture 9: sheet 9 / 31 Marc Uetz Discrete Optimization

P, N P, and co-n P Theorem P N P and P co-n P TSP N P P co-n P TUM MST Open conjectures P N P, worth 1.000.000$ claymath.org/millennium/p vs NP/ P = N P co-n P, worth world-fame and recognition as scientist Lecture 9: sheet 10 / 31 Marc Uetz Discrete Optimization

Proof P N P,co-N P Given Problem Q P. To show: Q N P i.e., for all Yes instances I Q exists polynomial length certificate z(i ), and an algorithm A that verifies the certificate in poly-time, so A(I, z(i )) = Yes I = Yes. Proof: We know Q P, so Q has a poly-time algorithm A Q with A Q (I ) = Yes I = Yes Now use z(i ) =, and define A(I, ) := A Q (I ) To see P co-n P, replace all underlined Yes by No, and let A(I, ) := A Q (I ) Lecture 9: sheet 11 / 31 Marc Uetz Discrete Optimization

Outline 1 N P and co-n P 2 N P-completeness 3 Clique is N P-complete Lecture 9: sheet 12 / 31 Marc Uetz Discrete Optimization

Completeness Informal Definition Given a class of decision problems X, problem P X is complete for that class if it is among the most difficult problems of X Needed: Formalization of most difficult, or at least as difficult as polynomial transformations and reductions between problems Lecture 9: sheet 13 / 31 Marc Uetz Discrete Optimization

Reduction (Turing Reduction) Definition Given two problems P and Q, say P polynomially reduces to Q (P α Q), if the following is true: Assuming a poly-time algorithm for Q, there is a poly-time algorithm for P. Intuition: Q is at least as hard as P (as to poly-time solvability) Definition Problems P, Q are polynomially equivalent if P α Q and Q α P Example: Q = decision problem and P = optimization problem for matching are polynomially equivalent (same for TSP) Lecture 9: sheet 14 / 31 Marc Uetz Discrete Optimization

Transformation (Karp or Many-One Reduction)!"#$%&'"(#)*'$%+,#-%'.+/#"0123456)*'$%7 Definition Given decision problems P and Q, say P polynomially transforms to Q!"#$%$&$'%()*$+"%)&,')-".'/%$&$'%)0-'12"34)P if the following is true: There is a polynomial 5%6)Q7)&8"% time algorithm P 0'29%'3$5229 &-5%4#'-34)&')Q $#)&8")#'22',$%/)$4)&-:"();8"-")"<$4&4)5)0'29%'3$52)&$3") (the transformation) T : P Q, such that 52/'-$&83)=&8")&-5%4#'-35&$'%>)T() ()P! Q7)4:.8)&85&) instance $%4&5%.")I I P " = P $4)?@AB Yes @ABC # $%4&5%.");=)I instance T >)" (I ) Q $4)?@AB Q @ABC = Yes 1!"#!"# &'()*+,'-(&.,)//! $% $% 0 note: T need not be injective neither surjective Lecture 9: sheet 15 / 31 Marc Uetz Discrete Optimization

Example of a transformation P!"#$%&'( P = Clique )%*(+"),-./"G!0V,E12"! 3$%&'("C " V 4%5/""6C6"# k 7 GivenQ G!"8+9(.(+9(+5":(5 = (V, E), k, is there a clique C V of size C k? )%*(+"),-./"G!0V,E12"! %+9(.(+9(+5";(5"S " V 4%5/""6S6"# k 7 Q = Independent Set Given G = (V, E), k, is there a indep. set S V of size S k? <(=%+(">,-+;=?,@-5%?+" >,-+;=?,@-5%?+"TA" A"P $ Q BC" T A"G $ G c Define4/(,("""G T : P c Q!"0V,E by T c (G) 1"3?@.$(@(+5"?="GA""""e := G c, where G c = %(V E, c E& c ) e and! E0G1""" e E c e E! T is polynomial time transformation, time O( n 2 ) clique C k in G indep. set S k in T (G) = G c Lecture 9: sheet 16 / 31 Marc Uetz Discrete Optimization

Reductions and Transformations Theorem Let decision problems P, Q be given. If P polynomially transforms to Q P α Q Proof. To show: If Q is poly-time solvable, so is P Assume a poly-time algorithm for Q, A Q. We know, poly-time transformation T : P Q. Poly-time algorithm for P: given instance I P compute T (I ) Q return Yes if A Q (T (I )) = Yes, No otherwise Lecture 9: sheet 17 / 31 Marc Uetz Discrete Optimization

Problem Class N P-complete Definition Given decision problem Q, then Q is N P-complete if 1 Q N P 2 P N P : P polynomially transforms to Q (Q is N P-hard) In words: Q is at least as hard as any problem in N P (as to poly-time solvability) Theorem If N P-complete problem X has a poly-time algorithm, P=N P. Proof: P N P, poly-time transf. T P : P X, X is poly-time solvable by A X, so A X T P yields poly-time algorithm for P Lecture 9: sheet 18 / 31 Marc Uetz Discrete Optimization

How N P-completeness helps us Garey & Johnson (1979): Computers and Intractability Lecture 9: sheet 19 / 31 Marc Uetz Discrete Optimization

On N P-completeness Is definition practical? need existence of T : P Q, P N P! Remark Transformations are transitive: given poly-time T 1 : P Q, T 2 : Q R, yield poly-time transf. T : P R, T := T 2 T 1. Theorem (Cook 1971) There exists an N P-complete problem, namely Satisfiability (SAT). Since 1971, showing that problem Q is N P-complete is easy show Q NP show poly-time transformation T : SAT Q (Transitivity may use any N P-complete problem instead of SAT.) Lecture 9: sheet 20 / 31 Marc Uetz Discrete Optimization

Satisfiability (SAT)!"#$!%$"&$'$#()*!"#+ SAT: Is a formula F in conjunctive normal form satisfiable?!"##$%&'()&*+&,$%- x boolean variables x 1,.... /(0 /(x, x n, each n 1(((x j! 23&$-%/(4*5%6( x j {false, true}!7$&5-%- C clauses C 1,.. /(0 /(C., C m, m /(((%&89(C each C i is i 9&-(&(3#*:(&-(3#$$#;- disjunction of certain literals C i <(4*5%(+3(&4($%&-4(#'%( $+4%*&$(+-(4*5% 7$&5-% =+4%*&$ =+4%*&$(>?%@&4+#'(#3(x A 1(4*5%(+3(x A <(3&$-%B F = C 1 C m = true : all C i = true! " &(4*549(&--+@':%'4(3#*( 4*549(&--+@':%'4(3#*(x. /(0/( /(x -589(49&4(C<&$$(8$&5-%- n &*%(4*5%1 Satisfiability problem: x {false, true} n such that F = true?!de5+f&$%'4$g/ x j! 2H/.6( Cook s theorem: Any N P problem can be equivalently reformulated as a SAT problem Lecture 9: sheet 21 / 31 Marc Uetz Discrete Optimization

!"#$%&'()$*$!"+,-./0#)#$1'-20#.($3456$!'45(7-'.4)&-5(8 The First Six N P-complete problems SATISFIABILITY!""#$%&'()*+, %&'()*+,$%-%- 3 - SATISFIABILITY 3-DIMENSIONAL MATCHING VERTEX COVER PARTITION HAMILTONIAN CYCLE CLIQUE 3-SAT? SAT with exactly 3 literals per clause Lecture 9: sheet 22 / 31 Marc Uetz Discrete Optimization

Clique is N P-complete Clique Given G = (V, E), is there a clique C V of size C k? 1 Clique N P Certificate for Yes instances: clique C itself (note C n, verifiable in O( n 2 ) time) 2 Give a poly-time transformation: T : 3-SAT Clique, where m = #clauses, n = #variables in 3-SAT, so that F 3-SAT = Yes G = T (F ) Clique = Yes F satisfiable G has clique of size m (Our graph will have 7m nodes) Lecture 9: sheet 23 / 31 Marc Uetz Discrete Optimization

Lecture 9: sheet 24 / 31 Marc Uetz Discrete Optimization

Clique is N P-complete The transformation For each clause C i : 7 nodes that correspond to all 7 partial truth assignments that make C i true (partial truth assignment? only 3 variables appear in clause C i, all other variables remain undecided, marked d figure) O( mn ) time introduce an edge between any two partial truth assignments if they are compatibe (don t contradict each other on a truth value of any variable) figure O( m 2 n ) time The proof (main idea) Any clique in G can only have one node per clause C i Thus a clique of size m is equivalent with a (partial) truth assignment that makes all m clauses C i true Lecture 9: sheet 25 / 31 Marc Uetz Discrete Optimization

The Simplest N P-complete problem: Partition Given Natural numbers a i 0, i = 1,..., n, with n i=1 a i = 2B (that is to say, i a i is even). Question Does there exist a subset S {1,..., n} with Theorem Partition is N P-complete Proof N P-hardness: Literature a i = a i? i S i S Lecture 9: sheet 26 / 31 Marc Uetz Discrete Optimization

N P-hard Optimization Problems Definition We call an optimization problem Q N P-hard if the corresponding decision problem is N P-complete. (Note: a poly-time algorithm for solving Q would yield P=N P) Lecture 9: sheet 27 / 31 Marc Uetz Discrete Optimization

Knapsack Problem Given n items with weights w j 0 and value v j 0 a knapsack with total (weight) capacity W Problem find subset S of items with weight W and maximal value Theorem The Knapsack problem is N P-hard (Exercise). Lecture 9: sheet 28 / 31 Marc Uetz Discrete Optimization

Solving the Knapsack Problem weights w j : values v j : 1 2 3 v 1 = 2 v 2 = 3 v 3 = 4 capacity W = 4 Construct directed graph G = (V, A): nodes u jk, k = w j,..., W, edges (u ik, u jl ) with lengths v j i < j and l k = w j item 1 2 size v 1 1 2 3 4 = W u 11 u 12 u 13 u 14 v 2 v 2 s v 2 u 22 u 23 u 24 t 3 v 3 v 3 u 33 u 34 length 0 Lecture 9: sheet 29 / 31 Marc Uetz Discrete Optimization

Solving the Knapsack Problem!"#$%&'%()*%"'(+,%-.*/-%0(1#23-%-#2! Observations!"#$%&'(!"#$%&'%(%$)(*+,#-'. )*(&!"%(+#$! /+-(0123(12#43(5+(.67+(%$5"-$%$8(,60+(*%5+*. any (s, t)-path traverses some rows of the digraph,.9)'(-'%-(0123(".(%,(9&& thereby 9&& 569,0 collecting :6$(-'+(.'6$-+.-(&%-'(:$67(.(-6(2(12#4;.3< the corresponding items! ='+(,60+(*%5+*.(0123(%$+(.'6$-+.-(&%-'(*+,#-'.(":(%,0(6,*8(": by definition, all these items fit in the knapsack, w i W 01>3(! 01,3?) length of path = total value,%- :6$(%**(12@>3#A of items collected along the path longest (s, t)-path = optimal solution Knapsack! $ "! " How to compute! longest paths in digraphs? " Same * as shortest paths only with -inequality + 2 reversed: (&) d(s, w) ) d(s, $ v) + c) vw %&' (digraph is acyclic,! so we even have a linear time algorithm) $ # #! # " ) 2> > Lecture 9: sheet 30 / 31 Marc Uetz Discrete Optimization

Solving the Knapsack Problem Knapsack is N P-hard Longest Path is polynomially solvable, so P And we have a transformation T : Knapsack Longest Path Hence, can we conclude that P=N P and claim 1.000.000 $ from the Clay Mathematics Institute? Lecture 9: sheet 31 / 31 Marc Uetz Discrete Optimization