Discrete Optimization 2010 Lecture 8 Lagrangian Relaxation / P, N P and co-n P

Transcription

1 Discrete Optimization 2010 Lecture 8 Lagrangian Relaxation / P, N P and co-n P Marc Uetz University of Twente m.uetz@utwente.nl Lecture 8: sheet 1 / 32 Marc Uetz Discrete Optimization

2 Outline 1 Lagrangian Relaxation 2 TSP and the 1-Tree Relaxation 3 Decision Problems, P, N P, and co-n P Lecture 8: sheet 2 / 32 Marc Uetz Discrete Optimization

3 LP and Lagrangian Relaxation max c t x s.t. Ax b ( complicating constraints) Bx d (IP) x integer Lagrangian relaxation (λ 0) L(λ) = max x c t x λ t (Ax b) or L(λ) = λ t b + max x (c t λ t A)x s.t. Bx d x integer (LR(λ)) Idea: penalize violation of constraints instead of enforcing them Exercise: L(λ) is an upper bound on optimum IP solution. Lecture 8: sheet 3 / 32 Marc Uetz Discrete Optimization

4 LP and Lagrangian Relaxation Best possible Lagrangian upper bound is Langragian Dual: LD = min λ 0 L(λ) a a note: λ is unrestricted in sign if we relax equality constraints Ax = b LP upper bound: LP = max x {c t x Ax b, Bx d} denote by ILP the optimum solution value of the integer program Theorem (for maximization problems) ILP LD LP (i.e., best Lagrangian bound is never worse than the LP relaxation) Lecture 8: sheet 4 / 32 Marc Uetz Discrete Optimization

5 Proof (of 2nd inequality) Use (strong) linear programming duality twice: LD = min λ 0 {b t λ + max x {(c t λ t A)x Bx d, x integer}} = }{{} LP duality min λ 0 {b t λ + max x {(c t λ t A)x Bx d}} min λ 0 {b t λ + min y {d t y B t y = (c A t λ), y 0}} = min λ,y {b t λ + d t y A t λ + B t y = c, y 0, λ 0}} = }{{} LP duality max x {c t x Ax b, Bx d} = LP Note: = holds in 2nd step for example if B is totally unimodular Lecture 8: sheet 5 / 32 Marc Uetz Discrete Optimization

6 An Example: TSP TSP Given (w.l.o.g. complete) undirected graph G = (V, E), V = n, edge lengths c e 0 e E, find shortest Hamiltonian cycle T (= tour). Example a short but non-optimal tour for all 15,112 cities in D Lecture 8: sheet 6 / 32 Marc Uetz Discrete Optimization

7 IP Formulation for TSP I variables x e = 1 if e T, 0 otherwise min e E e δ(v) c e x e x e = 2 x e S 1 e S x {0, 1} n v V S V Subtour elimination constraints (SEC), need to forbid solution: Lecture 8: sheet 7 / 32 Marc Uetz Discrete Optimization

8 IP Formulation for TSP II variables x e = 1 if e T, 0 otherwise min e E c e x e s.t. x e = n e E x e 2 e δ(s) x {0, 1} n S V Lecture 8: sheet 8 / 32 Marc Uetz Discrete Optimization

9 IP Formulation for TSP III (why valid?) variables x e = 1 if e T, 0 otherwise min e E c e x e s.t. x e = n e E x e = 2, x e = 2 v V \ {v 1 } e δ(v 1 ) e δ(v) x e S 1 S V \ {v 1 } e S x {0, 1} n [Held & Karp 1970] Lagrange-relax all but one of the constraints e δ(v) x e = 2 (all but for node v 1 ) Lecture 8: sheet 9 / 32 Marc Uetz Discrete Optimization

10 TSP & One-Tree Relaxation Claim Lagrange-relaxing the degree constraints yields a spanning tree on G \ {v 1 } + two edges incident with v 1 Proof. e E x e = n, so we have n 2 edges on G \ {v 1 } (with n 1 nodes), and e δ(s) x e 1 for all S V \ {v 1 }, so the solution on G \ {v 1 } is a connected subgraph But we know how to compute a minimum spanning tree in polynomial time Lecture 8: sheet 10 / 32 Marc Uetz Discrete Optimization

11 The One-Tree Relaxation (given λ) min c e x e λ v ( x e 2) (let λ v1 = 0) e E v V e δ(v) = min 2 λ v + (c e λ v λ u )x e (let λ v1 = 0) v V e={u,v} E s.t. x e = n, x e = 2, e E e δ(v 1 ) x e S 1, S V \ {v 1 } e S x {0, 1} n Solution Pick the two cheapest edges leaving v 1, and compute a minimum spanning tree (edge weights c e λ u λ v ) on the graph G \ v 1. Lecture 8: sheet 11 / 32 Marc Uetz Discrete Optimization

12 Held & Karp Bound for TSP To get a good lower bound for the TSP need values for λ... Procedure for Computing λ Start with λ v = 0 v V in iteration 1 in iteration k, compute optimal one-tree x and update λ: if k := e δ(v) x e = 1, let λ v = λ v + k (2 k) [making edges incident with v cheaper, i.e., more attractive ] if k := e δ(v) x e 3, let λ v = λ v + k (2 k) [making edges incident with v less attractive ] for some step size k 0 iterate until improvement of bound becomes marginal By right choice of step size k ( k 0, k k = ) this converges to the optimal solution of the Lagrangian dual, LD. Lecture 8: sheet 12 / 32 Marc Uetz Discrete Optimization

13 Subgradient Optimization for LD For the TSP (minimization problem) we have LD = max λ L(λ) L(λ) is linear in λ for given one-tree x L(λ, x ) = e E c e x e v V λ v ( e δ(v) x e 2) hence L(λ) = min x L(λ, x) is a piecewise linear, concave function L(λ) regions where for varying λ, same x is optimal λ Lecture 8: sheet 13 / 32 Marc Uetz Discrete Optimization

14 Subgradient Optimization for LD (cont d) Now for any given λ and the corresponding optimal one-tree x, consider d v := dl(λ, x ( ) = 2 ) dλ x v e δ(v) e Then d v coincides with our update step for λ, as we defined λ v = λ v + k (2 ) x e δ(v) e = λ v + k d v Exercise: vector d = dl(λ,x ) dλ R n is a subgradient of L( ) at λ, i.e., L(λ ) L(λ) + d t (λ λ) λ R n Note: Previous intuitive update of λs is just a special case of what is known as subgradient optimization Lecture 8: sheet 14 / 32 Marc Uetz Discrete Optimization

15 More Generally Given λ, let x be the optimal solution to a Lagrangian L(λ) with L(λ) = min x c t x λ t (Ax b) then b Ax is a subgradient of L( ) at λ, that is, Proof: Exercise L(λ ) L(λ) + (b Ax ) t (λ λ) λ Remark: For any L : R n R concave, the set dl(λ) := {d R n L(λ ) L(λ) + d t (λ λ) λ } is called subdifferential of L at λ. If L is differentiable at λ, then dl(λ) = { L(λ)}. If 0 dl(λ), L attains its minimum in λ Lecture 8: sheet 15 / 32 Marc Uetz Discrete Optimization

16 Remarks on the Held & Karp bound Very strong lower bound in practical implementations Reasonably fast to compute, so feasible within B & B Gives same lower bound as the LP relaxation, LP = LD [fractional 1-tree relaxation always has integer optimal solution, follows from a theorem by Edmonds 1971] There are examples where LD = max λ L(λ)< IP [to really solve TSP to optimality, can use Lagrangian relaxation only as a lower bound, e.g. within B & B] Lecture 8: sheet 16 / 32 Marc Uetz Discrete Optimization

17 Example for Suboptimality of 1-Tree Bound Exercise: Optimal tour costs 4, but there is always a 1-tree with cost 3, no matter what the values of the λ v s are Lecture 8: sheet 17 / 32 Marc Uetz Discrete Optimization

18 One-Trees: Trees With a Cycle! Lecture 8: sheet 18 / 32 Marc Uetz Discrete Optimization

19 Outline 1 Lagrangian Relaxation 2 TSP and the 1-Tree Relaxation 3 Decision Problems, P, N P, and co-n P Lecture 8: sheet 19 / 32 Marc Uetz Discrete Optimization

20 Decision Problems Definition A decision (recognition) problem P is a set of instances I that can be partitioned into Yes and No instances I Y, I N, such that I = I Y I N I Y I N = Examples All graphs G = (V, E) with edge lengths c e, and k N. Graphs with TSP tour of length k, and those without. All graphs G = (V, E), and k N. Graphs with a matching M E of cardinality k, and those without. Lecture 8: sheet 20 / 32 Marc Uetz Discrete Optimization

21 Optimization and Decision Problems Optimization (OPT) Given an instance I = (S, f ) of discrete optimization problem P, find a solution s S minimizing / maximizing f (s). Decision (DEC) Given an instance I = (S, f ) of discrete optimization problem P, and integer k, decide if solution s with f (s) / k. Clearly, if we can solve OPT, we can also solve DEC. What about the other direction? Lecture 8: sheet 21 / 32 Marc Uetz Discrete Optimization

22 Decision Problems are often General Enough Proposition (For quite many discrete optimization problems) if we can solve DEC in poly-time, we can also solve OPT in poly-time. There may be cases where OPT is provably more difficult (an open research question), but in many cases it isn t. How to solve OPT using an algorithm for DEC? 1 First compute optimal solution value ξ for OPT, by binary search on possible optimal values 2 Given ξ, construct a corresponding optimal solution Lecture 8: sheet 22 / 32 Marc Uetz Discrete Optimization

23 Example: Maximum Matching Given G = (V, E), find a matching of maximal cardinality. Assume algorithm A(k) that correctly tells us: Is there a matching of size k in a graph G? 1 Binary Search for optimal value ξ (note, ξ [0, m], integer) Let [a, b] = [0, m] While (b > a) k = a + b a 2 If A(k) = Yes on G, then a = k Otherwise b = k 1 return ξ := a(= b) O( log m ) A(k) 2 Finding the solution For all edges (e E) if A(ξ) = Yes on graph (G e) then delete e from G return remaining edges E(G) O( m ) A(k) Lecture 8: sheet 23 / 32 Marc Uetz Discrete Optimization

24 Problem class P: Deterministic Polynomial Time Given: Decision problem P, I encoding length of instance I P Definition Algorithm A is a deterministic polynomial time algorithm for P if there exists a polynomial p such that, A(I ) = Yes I is Yes instance t A (I ) p( I ) instances I of P (A is a poly-time algorithm) Definition Problem P P : P has deterministic poly-time algorithm Examples: decision problems for MST, ShortestPath, MaxFlow, MinCostFlow, Matching P Lecture 8: sheet 24 / 32 Marc Uetz Discrete Optimization

25 Certificates!"#$%&'(")*(+,$-." Given: TSP instance, a complete undirected graph with integer edge lengths c e 0, e E, integer k: tour k? )*+,$-./%"0".%&"%10'%-$2&!&3& &"%0$/$,&:;& Can we give a short certificate, that is, a short proof that I is a Yes instance? "#$&'&1>.,0&?,..@&& 1>.,0&?,..@&&0>'0&"0&"1&'&AB7)C "%10'%-$D This is simple: The tour T E itself (n edges, T O( n )) Verifying the certificate!?,..@2 0.E,&(&"01$F@ check that T is indeed a tour check its length e T c e k?!?,..@&#$,"@"-'0".%2 Possible in polynomial time, O( n 2 ) (&"1&"%8$$8&'&0.E,&! 84(9&! :& Note: Finding the certificate maybe hard, but verifying its correctness is easy (in polynomial time) Lecture 8: sheet 25 / 32 Marc Uetz Discrete Optimization

26 Problem class N P: Nondeterministic Polynomial Time Given: Decision problem P, I encoding length of instance I P Definition Algorithm A is nondeterministic polynomial time algorithm for P if For all Yes instances I, poly-length certificate z(i ) (i.e., z(i ) p( I ) for some polyn. p), and A(I, z(i )) = Yes For all No instances I, and any poly-length input z ( fake certificate ), A(I, z) = No A(I, z) is a poly-time algorithm for any poly-length z Definition P N P : P has nondeterministic poly-time algorithm Yes instances of P have a polynomial certificate that can be verified in polynomial time Example: We have just argued that TSP N P Lecture 8: sheet 26 / 32 Marc Uetz Discrete Optimization

27 Why nondeterministic? Definition uses existence of a polynomial length certificate z(i ), but not how to obtain it! We can interpret this equivalently as: Consider all possible certificates z of polynomial length, If the instance I = Yes, at least one yields the algorithm to terminate with Yes Could we guess the right certificate z(i ) for any given Yes instance I, we could solve the problem guessing = nondeterminism Lecture 8: sheet 27 / 32 Marc Uetz Discrete Optimization

28 What about No Instances!"#$%&'&()*+,$-./%"0".%&"%10'%-$2&!&3& &"%0$/$,&:;& "%10'%-$E Given: TSP instance I, so a complete undirected graph with integer edge lengths c e 0, e E, integer k.? Can we give a short certificate that I is a No instance???? ).F$>.=6&'GG&?,..@1&1$$F&0.&,$HI",$&$J?.%$%0"'G&G$%/0> All proofs seem to require exponential $J?.%$%0"'G&G$%/0>K length... check all (n 1)! LGG&>'#$&G$%/0>&! :MN& 2 tours, see they all have length k + 1 Lecture 8: sheet 28 / 32 Marc Uetz Discrete Optimization

29 No Instances with Certificates Given: An integer matrix A Z m n : is A totally unimodular? Short certificate for No instances: square submatrix B of A encoding length B A, so certainly polynomial verifying the certificate: verify that det(b) {0, ±1} Is certificate verification possible in polynomial time? Laplace formula det(b) = k j=1 b ij( 1) i+j det(b i, j ) requires Ω(n!) time but O( n 3 ) is possible, computing LU decomposition of B (B = LU, with L = lower triangular, U = upper triangular) Lecture 8: sheet 29 / 32 Marc Uetz Discrete Optimization

30 Problem class co-n P: Complement-N P Given: Decision problem P, I encoding length of instance I P Definition Algorithm A is nondeterministic polynomial time algorithm for the complement of P if For all No instances I, poly-length certificate z(i ) (i.e., z(i ) p( I ) for some polyn. p), and A(I, z(i )) = Yes For all Yes instances I, and any poly-length input z ( fake certificate ), A(I, z) = No A(I, z) is a poly-time algorithm for any poly-length z Definition P co-n P: complement of P has nondet. poly-time algorithm No instances of P have a polynomial certificate that can be verified in polynomial time Lecture 8: sheet 30 / 32 Marc Uetz Discrete Optimization

31 P, N P, and co-n P Theorem P N P and P co-n P TSP N P P co-n P TUM MST Open conjectures P N P, worth $ claymath.org/millennium/p vs NP/ P = N P co-n P, worth world-fame and recognition as scientist Lecture 8: sheet 31 / 32 Marc Uetz Discrete Optimization

32 Proof P N P,co-N P Given Problem Q P. To show: Q N P i.e., for all Yes instances I Q exists polynomial length certificate z(i ), and an algorithm A that verifies the certificate in poly-time, so A(I, z(i )) = Yes I = Yes. Proof: We know Q P, so Q has a poly-time algorithm A Q with A Q (I ) = Yes I = Yes Now use z(i ) =, and define A(I, ) := A Q (I ) To see P co-n P, replace all underlined Yes by No, and let A(I, ) := A Q (I ) Lecture 8: sheet 32 / 32 Marc Uetz Discrete Optimization