1 Outline Part I: Linear Programming (LP) Interior-Point Approach 1. Simplex Approach Comparison Part II: Semidenite Programming (SDP) Concludin

Transcription

1 Sensitivity Analysis in LP and SDP Using Interior-Point Methods E. Alper Yldrm School of Operations Research and Industrial Engineering Cornell University Ithaca, NY joint with Michael J. Todd INFORMS Philadelphia November 8, 1999

2 1 Outline Part I: Linear Programming (LP) Interior-Point Approach 1. Simplex Approach Comparison Part II: Semidenite Programming (SDP) Concluding Remarks

3 Part I: Linear Programming (LP) 2 LP in standard form: LP P (b; c) min x c T x s.t. Ax = b; x 0; c and x 2 IR n, b 2 IR m, and A 2 IR mn. Associated dual LP: LP D(b; c) max y;s b T y s.t. A T y + s = c; s 0; y 2 IR m and s 2 IR n. The coecient matrix A is xed.

4 = b; Ax T y + s = c; A 3 Assumptions: 1. Both LP P (b; c) and LP D(b; c) have strictly feasible points. 2. A has full row rank. Central Path Solution to: XSe = e; with x > 0 and s > 0 for some > 0. Assumptions 1 and 2, such a solution exists and is unique for each Under. positive

5 4 Let (x; y; s) be the current iterate. Seek an approximation to the point on the central path with parameter. Using Newton's method: Ax = r p A T y + s = r d Sx + Xs = r xs ; (1) where r p = b Ax, r d = c A T y s, and r xs = e XSe.

6 (x 0 ; y 0 ; s 0 ) of LP P (b 0 ; c 0 ) and LP D(b 0 ; c 0 ) that satises X 0 S 0 e = v is points triple (x; y; s) solving (1) for r p = b 0 Ax, r d = c 0 A T y s, the 5 Denition. A Newton step from (x; y; s) targeting the feasible pair of and r xs = v XSe. Remark 1. Such a pair of points (x 0 ; y 0 ; s 0 ) might not exist, but the Newton step is still well-dened.

7 6 Proposition 1. Assume: (x; y; s) is a strictly feasible pair of points for LP P (b; c) and LP D(b; c), b is replaced by b 0 = b + b, where b 2 IR m, and a full Newton step is taken from (x; y; s) targeting the feasible pair of points (x 0 ; y 0 ; s 0 ) of LP P (b 0 ; c) and LP D(b 0 ; c) that satises X 0 S 0 e = XSe. If, and only if, ks 1 A T (AS 1 XA T ) 1 bk 1 1; then resulting iterate will be feasible for the new problem. Moreover, duality gap will decrease.

8 x i 7 Proof: r p = b, r d = 0, and r xs = 0. The third equation dening the Newton step: Sx + Xs = 0: Rewriting it componentwise, s i x i + x i s i = 0 so x i s i = + 0; i = 1; : : : ; n: i s

9 1 1 1: s S 8 Proof (Cont'd): The next iterate will be feasible i x i + x i 0; and s i + s i 0; i = 1; : : : ; n: Combining two observations, it is necessary and sucient to have: The result follows from the solution of the Newton step.

10 9 Proposition 2. Assume: (x; y; s) is a strictly feasible pair of points for LP P (b; c) and LP D(b; c), c is replaced by c 0 = c + c, where c 2 IR n, and a full Newton step is taken from (x; y; s) targeting the feasible pair (x 0 ; y 0 ; s 0 ) of LP P (b; c 0 ) and LP D(b; c 0 ) that satises X 0 S 0 e = XSe. If, and only if, ks 1 (I A T (AS 1 XA T ) 1 AS 1 X)ck 1 1; then resulting iterate will be feasible for the new problem. Moreover, duality gap will decrease.

11 10 Simplex Approach Partition A as B and N corresponding to the basic and nonbasic columns, respectively. Let (x ; y ; s ) be a pair of optimal solutions for LP P (b; c) and D(b; c). LP Partition x as x B and x N ; s as s B and s N ; c as c B and c N accordingly.

12 11 1. Let b be replaced by b + b. The optimal basis remains optimal i B 1 (b + b) 0 or B 1 b B 1 b = x B : 2. Let c be replaced by c + c. The optimal basis remains optimal i c T N +ct N c T B B 1 N c T B B 1 N 0 or c N N T B T c B s N :

13 12 Assumption: The optimal solution is unique and non-degenerate. Symmetrized simplex bounds 1. Change in b B 1 b B 1 b = x B so (X B ) 1 B 1 b e: The largest L 1 -box around the origin: k(x B ) 1 B 1 bk 1 1:

14 13 Symmetrized simplex bounds (Cont'd): 2. Change in c c N N T B T c B s N so (S N ) 1 (c N N T B T c B ) e: The largest L 1 -box around the origin: k(s N ) 1 (c N N T B T c B )k 1 1:

15 duality gap n. The bounds arising from Propositions 1 and 2 small at these points are given by evaluated (X B ) 1 B 1 + O() 1 1; these bounds are asymptotically the same as the symmetrized Moreover, bounds. simplex 14 Comparison Let (x; y; s) be any pair of strictly feasible solutions with Theorem 1. b O() and cb 1 1: N c O() O() (S N ) 1 N T B T + O() (S N ) 1 + O()

16 15 Conclusion of Part I the assumption of unique, non-degenerate solution, the interiorpoint Under approach yields asymptotically exactly the same bounds as those from the simplex approach that keep the current basis optimal (after arising with respect to the origin). symmetrization

17 Part II: Semidenite Programming (SDP) 16 SDP in standard form: SDP (b; C) min X C X AX = b; X 0; where AX = (A i X) m i=1 ; A i 2 SIR nn, b 2 IR m, C 2 SIR nn are given, and X 2 SIR nn. SIR nn : n n symmetric matrices. X 0: X is symmetric positive semidenite. P Q = Trace (P T Q) = P ij P ijq ij

18 17 Associated dual problem: SDD(b; C) max y;s b T y A y + S = C; S 0; where y 2 IR m, S 2 SIR nn, and A y = mx i=1 y i A i : Assumptions: 1. Both SDP (b; C) and SDD(b; C) have strictly feasible solutions. 2. A is surjective, i.e. A i s are linearly independent.

19 i X = b i ; for i = 1; : : : ; m; P A m y ia i + S = C; i=1 18 Central Path Solution to: (2) XS = I; together with X 0 and S 0 for > 0. Remark. Newton's method cannot be applied directly to (2) since the system is not square. Remedy: Symmetrize the third equation so that the residual lies in SIR nn.

20 ~ X = b; A ~y + ~ S = C; A 19 Let (X; y; S) be the current iterate. The directions we will examine will be Newton steps for nonlinear systems the form of ( ~ X; ~ S) = (X 0 ; S 0 ); (X; S) is some symmetrization of XS and where X 0 and S 0 are where targeted points. the More than 20 dierent symmetrizations!

21 20 The Newton step (X; y; S) is the solution to AX = r p ; A y + S = R d ; EX + FS = R EF ; where r p = b AX is the primal residual, R d = C A y S is the dual residual, the operators E = E(X; S) and F = F(X; S) are the derivatives of with respect to X ~ and S ~ respectively, evaluated at (X; S), and R EF = R EF (X; S) = (X 0 ; S 0 ) (X; S).

22 21 The solution to the Newton step is unique if E is nonsingular, and m m Schur complement matrix AE 1 FA nonsingular. is In this case, Newton step is given by: (AE 1 FA )y = r p AE 1 (R EF FR d ); S = R d A y; X = E 1 (R EF FS):

23 22 We consider the following search directions: AHO (Alizadeh, Haeberly, Overton) H..K..M (Helmberg et. al., Kojima et. al., Monteiro) NT (Nesterov, Todd)

24 23 Notation: (P Q)K := 1 2 (P KQT + QKP T ); P; Q 2 IR nn and K 2 SIR nn, and we will regard it as an operator where SIR nn to itself. from

25 24 Observations: 1. For the three directions, E = S M; F = M X I; where M is a symmetric positive denite matrix. E is invertible. M = I for AHO M = S for H..K..M M = W 1 for NT where W SW = X. 2. For all (X; y; S) strictly feasible, E 1 F is positive denite for H..K..M and NT. This also holds for AHO under additional assumptions on (X; y; S).

26 25 Proposition 3. Assume (X; y; S) is a strictly feasible pair of points for SDP (b; C) and SDD(b; C), AE 1 FA is nonsingular, b is replaced by b 0 = b + b, a full Newton step is taken from (X; y; S) targeting the feasible pair (X 0 ; y 0 ; S 0 ) of SDP (b 0 ; C) and SDD(b 0 ; C) that satises (X 0 ; S 0 ) = (X; S). If X 1 2 S 1 2 E 1 FA (AE 1 FA ) 1 b X ; A (AE 1 FA ) 1 b S ; then the resulting iterate will be feasible for the new problem. Moreover, the duality gap of the new iterate will decrease if E 1 F is positive denite.

27 Therefore, all eigenvalues of X 1 2XX Proof: next iterate will be feasible for the new problem if and only if The + X 0 and S + S 0. But, X X + X 0 holds i I + X 1 2XX 1 2 0: should be 1. A sucient condition is kx 1 2XX 1 2 k 2 1:

28 27 Proof (Cont'd): Similarly, S + S 0 holds i I + S 1 2SS 1 2 0: A sucient condition is ks 1 2SS 1 2 k 2 1: The result follows from the solution to the Newton system.

29 (X; y; S) is a strictly feasible pair of points for SDP (b; C) and C), SDD(b; a full Newton step is taken from (X; y; S) targeting the feasible pair 0 ; y 0 ; S 0 ) of SDP (b; C 0 ) and SDD(b; C 0 ) that satises (X 0 ; S 0 ) = (X 28 Proposition 4. Assume: AE 1 FA is nonsingular, C is replaced by C 0 = C + C, where C 2 SIR nn, and (X; S).

30 FC E FA (AE 1 FA ) 1 AE 1 1 FC X 2 E 1; the resulting iterate will be feasible for the new problem. Moreover, then duality gap of the new iterate will decrease if E 1 F is positive denite. the 29 If 1 2 X 1 2 S (AE FA ) 1 AE 1 1 FC S 2 C 1: 1 A 2

31 Concluding Remarks 30 For LP, the bounds arising from the interior-point approach compares nicely with those from the simplex approach under the assumption of unique, non-degenerate solution. The comparison in the degenerate case remains unanswered. For SDP, our preliminary computational tests indicate that the three directions perform similarly if the iterates lie in a relatively wide neighborhood of the central path. NT direction yields similar results even when the iterates get too close to the boundary whereas the other two directions deteriorate signicantly.