Lecture 10. Primal-Dual Interior Point Method for LP

Transcription

1 IE Lecture 10. Primal-Dual Interior Point Method for LP

2 IE Consider a linear program (P ) minimize c T x subject to Ax = b x 0 and its dual (D) maximize b T y subject to A T y + s = c s 0. Let F P be the feasible set for (P ). Let F D be the feasible set for (D).

3 IE Suppose that both (P ) and (D) satisfy the Slater Condition (strong feasibility). Then the central path {(x(µ), y(µ), s(µ)) µ (0, ), x(µ) F P, (y(µ), s(µ)) F D, x(µ) s(µ) = µe} exists, where stands for the Hadamard product. The idea now is: Try to approach the optimal solutions by tracing the central path as µ tends to zero. x i (µ)s i (µ) = µ for 1 i n Suppose that our current interior primal-dual solution is (x, s). Let our next target solution be (x, s ). Assume that (x ) T s < x T s Moreover, let w := Xs and w := X s. Denote x := x x and s := s s.

4 IE We have A x = 0 A T y + s = 0 S x + X s = w w + X s This is a nonlinear equation and is difficult to solve. Linearization (Newton s method) is a good approximation: Let D := (XS 1 ) 1/2 and A x = 0 A T y + s = 0 S x + X s = w w p x := D 1 x and p s := D s

5 IE What does this mean? Well, this suggests that we scale the problems (P ) minimize c T x subject to Ax = b x 0 into and (D) maximize b T y subject to A T y + s = c s 0 (P ) minimize (Dc) T x subject to ADx = b x 0 and (D) maximize b T y subject to (AD) T y + s = Dc s 0.

6 IE The scaling transformation is given as x = D 1 x and y = y and s = Ds. Facts: (P ) and (D) remain a pair of primal-dual linear programs. (P ) and (D) are identical to the original problems (P ) and (D). The feasible solutions x and s for the original problems (P ) and (D) become w and w.

7 IE Let v := w (and v := w ). It follows that ADp x = 0 (AD) T y + p s = 0 p x + p s = p where p := V 1 (w w). The solution of the above equation is apparent: p x = P AD p = [I DA T (AD 2 A T ) 1 AD]p and p s = p p x = DA T (AD 2 A T ) 1 ADp. Because the Newton method is only an approximation, we may not exactly have x + x = x and s + s = s as we expected.

8 IE We introduce the step length t (> 0) and consider the iterates x(t) := x + t x and s(t) := s + t s Our idea is to make X(t)s(t) stay close to the central line and converge to zero. Introduce the neighborhoods of the central line { } N 2 (β) := v 2 1 µ v2 e β 2 and N (β) := { } v 2 1 µ v2 e β where 0 < β < 1 is a constant and µ = et v 2 n.

9 IE Primal-dual path following method The essence of this method is to select the next target point w as w := ρµe where 0 < ρ < 1. In this case we have p := v + ρµv 1 Assume that Xs = v 2 N (β). Therefore, v 2 i (1 β)µ for all i Clearly, iff x + t x > 0 and s + t s > 0 v + tp x > 0 and v + tp s > 0

10 IE Lemma 1 It holds that 1). (p x ) (p s ) p 2 2 ; 2). (p x ) (p s ) 1 4 p 2 2.

11 IE Proof. 1). We have (p x ) (p s ) 2 = n [(p x ) i (p s ) i ] 2 i=1 n (p x ) 2 n i (p s ) 2 i i=1 i=1 1 2 [ p x p s 2 2 ] = 1 2 p 2 2

12 IE To prove 2). we observe for every i (p x ) i (p s ) i 1 2 (p x) (p s ) n [ (px ) 2 i + (p s ) 2 ] i i=1 = 1 4 p 2 2 Lemma 2 It holds that ( 1 1 p 2 1 β µ v2 e + (1 ρ) µ n) 2

13 IE Proof. Using v 2 i (1 β)µ we have p 2 = V 1 ( µe v 2 + (ρ 1)µe ) 2 ( β µ v2 e + (1 ρ) µ n) 2

14 IE The Algorithm: (Assume x (0) s (0) N (β)). Step 0 Let k := 0; Step 1 Compute x (k) and s (k). Step 2 Compute maximum step length t > 0 such that (x (k) + t x (k) ) (s (k) + t s (k) ) N (β) Step 3 Let and x (k+1) := x (k) + t x (k) s (k+1) := s (k) + t s (k) Let k := k + 1 and go to Step 1.

15 IE We are interested in how fast the algorithm converges. Clearly, v(t) 2 := (x + t x) (s + t s) = (1 t)v 2 + tρµe + t 2 p x p s and so µ(t) := et v(t) 2 n = (1 t + tρ)µ

16 IE Therefore = 1 µ(t) v(t)2 e ( ) ( ) tρ t + tρ µ v2 e + t2 p x p s (1 t + tρ) µ ( ) tρ t 2 p β + 1 t + tρ 2(1 t + tρ)µ ( ( ) tρ t 2 v 2 µ e 2 + (1 ρ) n 1 β + 1 t + tρ 2(1 t + tρ)(1 β) ( v 2 t µ e 2 + (1 ρ) ) 2 n = β ρβ t. 1 t + tρ 2(1 β) ) 2 Moreover, x(t) T s(t) = (1 (1 ρ)t) x T s

17 IE Now it follows: Theorem 1 Let β = Ω(1) and 1 ρ = Ω(1). Either the N 2 (β) or the N (β) neighborhood is used. There exists a constant c 1 = Ω(1) such that the sequence produced by the primal-dual algorithm satisfies ( (x (k+1) ) T s (k+1) 1 c ) 1 (x (k) ) T s (k) n Theorem 2 Let β = Ω(1) and 1 ρ = Ω( 1 n ). Assume that the N 2 (β) neighborhood is used. Then, there exists a constant c 2 = Ω(1) such that the sequence produced by the primal-dual algorithm satisfies ( (x (k+1) ) T s (k+1) 1 c ) 2 (x (k) ) T s (k) n

18 IE In general, if 1 ρ = Ω(1) then the algorithm is called a long step path following algorithm, and if 1 ρ = Ω( 1 n ) it is called a short step path following algorithm. In particular, there are two choices of ρ s that are interesting, i.e. ρ := 0 (primal-dual affine scaling) and ρ := 1 (centering). An important variant of primal-dual interior point algorithm is called the predictor-corrector algorithm. Lemma 3 If x s N 2 (1/2), then by letting ρ := 1 and the step length t := 1 we have x(1) s(1) N 2 (1/4)

19 IE The Predictor-Corrector Algorithm: (Assume x (0) s (0) N 2 (1/2)). Step 1 If k even, let ρ := 1; otherwise go to Step 2. Compute x (k) and s (k). Let x (k+1) := x (k) + x (k), s (k+1) := s (k) + s (k) Let k := k + 1 and go to Step 1. Step 2 Let ρ := 0. Compute maximum step length t > 0 such that (x (k) + t x (k) ) (s (k) + t s (k) ) N 2 (1/2) Let x (k+1) := x (k) + t x (k) Let k := k + 1 and go to Step 1. s (k+1) := s (k) + t s (k)

20 IE Theorem 3 In the predictor-corrector algorithm, there exists a constant c 3 = Ω(1) such that the sequence of iterates satisfies ( (x (k+1) ) T s (k+1) 1 c ) 3 (x (k) ) T s (k) n for all even k. The general convergence results are included in the following three theorems: Theorem 4 Assume that x (0) s (0) N (β) and (x (0) ) T s (0) 1/ϵ then in at most k = O(n log 1 ϵ ) iterations the long step primal-dual interior point algorithm will find iterates satisfying (x (k) ) T s (k) < ϵ.

21 IE Theorem 5 Assume that x (0) s (0) N 2 (β), (x (0) ) T s (0) 1/ϵ and the N 2 (β) neighborhood is used, then in at most k = O( n log 1 ϵ ) iterations the short step primal-dual interior point algorithm will find iterates satisfying (x (k) ) T s (k) < ϵ. Theorem 6 Assume that x (0) s (0) N 2 ( 1 2 ) and (x(0) ) T s (0) 1/ϵ then in at most k = O( n log 1 ϵ ) iterations the predictor-corrector algorithm will find iterates satisfying (x (k) ) T s (k) < ϵ.

22 IE Key References: M. Kojima, S. Mizuno, and A. Yoshise, A Primal-Dual Interior Point Algorithm for Linear Programming, ed. N. Megiddo, Progress in Mathematical Programming: Interior point and related methods, 29 37, S.J. Wright, Primal-Dual Interior-Point Methods, SIAM, S. Mizuno, M.J. Todd, and Y. Ye, On Adaptive-Step Primal-Dual Interior-Point Algorithms for Linear Programming, Mathematics of Operations Research, 18, , 1993.