1. Introduction and background. Consider the primal-dual linear programs (LPs)

Transcription

1 SIAM J. OPIM. Vol. 9, No. 1, pp c 1998 Society for Industrial and Applied Mathematics ON HE DIMENSION OF HE SE OF RIM PERURBAIONS FOR OPIMAL PARIION INVARIANCE HARVEY J. REENBER, ALLEN. HOLDER, KEES ROOS, AND AMÁS ERLAKY Abstract. wo new dimension results are presented. For linear programs, it is shown that the sum of the dimension of the optimal set and the dimension of the set of objective perturbations for which the optimal partition is invariant equals the number of variables. A decoupling principle shows that the primal and dual results are additive. he main result is then extended to convex quadratic programs, but the dimension relationships are no longer dependent only on problem size. Furthermore, although the decoupling principle does not extend completely, the dimensions are additive, as in the linear case. Key words. linear programming, optimal partition, polyhedron, polyhedral combinatorics, quadratic programming, computational economics AMS subject classification. 90C05 PII. S Introduction and background. Consider the primal-dual linear programs LPs min{cx : x 0, Ax= b}, max{yb : s 0, ya+ s = c}, where c is a row vector in R n, called objective coefficients; x is a column vector in R n, called levels; b is a column vector in R m, called right-hand sides; y is a row vector in R m called prices; and A is an m n matrix with rank m. Let P and D denote the primal and dual polyhedra, respectively, and let P and D denote their optimality regions, which we assume to be nonempty. Let x,y,s be a strictly complementary optimal solution, and let the optimal partition be denoted by B N, where B = σx {j : x j > 0} and N = σs {j : s j > 0}. For background, see [6]. his paper first presents a result concerning the dimension of P D in connection with the set of direction vectors in R n respectively, R m for which the optimal partition does not change when the objective coefficients respectively, right-hand sides are perturbed in that direction. After establishing fundamental relations for LPs, we consider extensions to convex quadratic programs. he technical terms used throughout this paper are defined in the Mathematical Programming lossary [2]. Received by the editors February 10, 1997; accepted for publication in revised form February 9, 1998; published electronically December 2, University of Colorado at Denver, Mathematics Department, P.O. Box , Denver, CO hgreenbe@carbon.cudenver.edu, hgreenbe; agholder@tiger.cudenver.edu, agholder. Faculty of echnical Mathematics and Informatics, Delft University of echnology, Delft, he Netherlands c.roos@twi.tudelft.nl; t.terlaky@twi.tudelft.nl, People/Staf/.erlaky.html. 207

2 208 H. J. REENBER, A.. HOLDER, K. ROOS, AND. ERLAKY 2. Linear programs. Following reenberg [3], let r =b, c denote the rim data, and let H denote the set of rim direction vectors, h =δb, δc, for which the optimal partition does not change on the interval [r, r + θh] for some θ>0, i.e., H = {δb, δc : there is x 0, y 0, θ>0 such that Ax = b + θ δb, x B > 0, x N =0; ya + s = c + θ δc, s B =0,s N > 0}. Here we follow the notation in [1, 6], where a subscript on a vector means it is the subvector restricted to the indexes in the subscript. For example, x B is the vector of positive levels. his notation extends to matrices: A partitions into [A B A N ]. Let H c denote the projection of H onto R n for changing only c: H c {δc :0,δc H}. Similarly, let H b denote the projection of H onto R m for changing only b: H b {δb :δb, 0 H}. reenberg [3] showed that H is a convex cone that satisfies a decoupling principle: H = H b H c. o help build intuition, notice first that if the dimension of the primal optimality region, dimp, is zero, this means it is an extreme point. In that case, every vector in R n can be used to change c without changing the optimal partition, so dimh c =n. At the other extreme, suppose dimp =n m, such as when c = 0, so every feasible solution is optimal in the primal LP. In that case, H c consists of change vectors that maintain equal net effects among the positive variables, so dimh c =m. his latter case can be illustrated with the following. Example. min{ j 0x j : j x j =1,x 0}. In this case, B = {1,...,n}. In order for this partition not to change for the LP: min{ j δc jx j : j x j =1,x 0}, it is necessary and sufficient that δc j = δc 1 for all j. hus, dimh c =1. In both cases, we see that dimp + dimh c =n. his is what we shall prove in general along with related results. heorem 2.1. he following equations hold for any LP whose primal and dual sets have nonempty strict interiors. 1. dimp + dimh c =n. 2. dimd + dimh b =m. 3. dimp D + dimh =n + m. Proof. From Lemma IV.44 in [6], we have dimp = B ranka B. he conditions for δc H c are ya B = c B + θδc B and ya N <c N + θδc N for some θ>0. hus, δc N can be arbitrary, so dimh c = N + dim{δc B : δy R m δya B = δc B } = N + ranka B. his implies dimp + dimh c = B + N = n.

3 RIM PERURBAIONS FOR OPIMAL PARIION INVARIANCE 209 he second statement has a similar argument. From Lemma IV.44 in [6], dimd = m ranka B. he conditions for δb H b are A B x B = b + θδband x B > 0. hus, dimh b = ranka B, so dimd + dimh b =m. he last statement follows from the decoupling principle, upon adding the first two equations H = H b H c dimh = dimh b + dimh c. We now consider some corollaries whose proofs follow directly from the theorem but whose meanings lend insight into how perturbation relates to the dimensions of the primal and dual optimality regions. he dimension of a set is sometimes called the degrees of freedom. If there are n variables and no constraints on their values, the set has the full degrees of freedom, which is n; i.e., each variable can vary independently. When the set is defined by a system of m independent equations, as in our case, we sometimes refer to m as the degrees of freedom lost. Because we assume that there exists a strict interior solution x >0, there are no implied equalities among the nonnegativity constraints, so dimp =n m. hus, the feasibility region has m degrees of freedom lost due to the equations that relate the variables. A meaningful special case is when there is an excess number of columns, say B = m + k, and there is enough linear independence retained in the columns so that ranka B =m recall that we assume ranka =m. hen, dimp =k, so dimh c =n k. Expressed in words, the degrees of freedom lost in varying objective coefficients equals the number of excess columns over those of a basic optimal solution. Furthermore, ranka B =mis equivalent to dimd = 0 i.e., unique dual solution, so we can say the following. Corollary 2.2. he following are equivalent. 1. he dual solution is unique. 2. dimh c =n + m B. 3. dimh b =m. Another special case arises when the LP is a conversion from the inequality constraints, A x b, where A is m n, and ranka =m. In that case, A =[A I], and n = n + m. Suppose x > 0, so B includes all of the structural variables and some of the surplus variables, say B = n + k. hen, dimp =k, and heorem 2.1 implies dimh c =n + m k. Since we do not allow the costs of the surplus variables to be nonzero, we can reduce this by m, giving dimh c =n k. Expressed in words, this says that the degrees of freedom lost in varying structural cost coefficients equals the number of positive surplus variables. A similar result follows for the primal. he next corollary says, in part, that dimp = 0 if and only if dimh c =n. Expressed in words, this says that the primal solution is unique if and only if every objective coefficient can be perturbed independently without changing the optimal partition. he last equivalence includes the special case of a nondegenerate basic solution, in which case B = m, so every right-hand side can be perturbed without changing the optimal partition. Corollary 2.3. he following are equivalent. 1. he primal solution is unique. 2. dimh c =n. 3. dimh b = B. hese corollaries combine into the following, which is the familiar case of a unique strictly complementary optimum which is basic. Corollary 2.4. he following are equivalent.

4 210 H. J. REENBER, A.. HOLDER, K. ROOS, AND. ERLAKY 1. he primal-dual solution is unique. 2. dimh c =n and dimh b =m. 3. dimh =m + n. he following corollary says that dimh c m, and it follows from the main theorem since the maximum dimension of P is n m. he analogous bound for dimh b is merely that it is nonnegative since the maximum dimension of D is m. Corollary 2.5. here are at least m degrees of freedom to vary the objective coefficients without changing the optimal partition. In the next section, we extend heorem 2.1 to convex quadratic programs, and note that care must be taken when specializing it to an LP. 3. Quadratic programs. We now extend heorem 2.1 to the convex quadratic program min{cx x Qx : Ax = b, x 0}, where Q is symmetric and positive semidefinite. We use the Wolfe dual [2] max{yb 1 2 u Qu : ya + s u Q = c, s 0}. Let QP and QD denote primal and dual feasibility regions, respectively. Let us introduce the following notation: QP = {x : x QP, and x is primal optimal}, QD = {y, s :y, s, u QD and y, s, u is dual optimal}, QD = {y, s, u :y, s, u QD and y, s, u is dual optimal}. Here QP and QD denote their optimality regions, except that we define QD exclusive of the u-variables, while QD denotes the full dual optimality region to distinguish it from QD. We shall explain this shortly. Following Jansen [4] and Berkelaar, Roos, and erlaky [1], an optimal partition is defined by three sets B N, where B = {j : x j > 0 for some x QP }, N = {j : s j > 0 for some y, s QD }, and = {1,...,n}\B N. We assume that the solution obtained is maximal [1]: x j > 0 j B and s j > 0 j N. üler and Ye [5] show that many interior point algorithms converge to a solution whose support sets comprise the maximal partition: B = σx,n = σs, and = {1,...,n}\B N. Unlike linear programming, there is no guarantee of a strictly complementary optimal solution, so need not be empty. For this and other reasons, there are some important differences see [1, 4] for details that affect our extension of heorem 2.1. In particular, the decoupling principle does not apply since a change in c affects both primal and dual optimality conditions.

5 RIM PERURBAIONS FOR OPIMAL PARIION INVARIANCE 211 We begin our extension with the following lemma. In the proof we use the following notation: col = column space of = {u : u = x for some x R n }, N = null space of = {x : x =0}. Lemma 3.1. Let F and be m n and g n matrices, respectively, and consider the set: S = {v : v = Fu for some u u =0}. hen, dims = rank F rank. Proof. Without loss in generality assume has full row rank, and let {u 1,...,u g } be a basis for col. Let {v 1,...,v s } be a basis for S where dims =s, and consider the following set in col : { w1 u 1... F wg u g v } vs, 0 where w i F [ ] 1 u i. Once we prove that this is a basis for col F, we have that g + s = rank F, which implies the desired result. First, we shall prove that these vectors are linearly independent. Suppose wi α i + vj β u j =0. i 0 i j Since {u 1,...,u g } is a basis, α = 0, which then implies β = 0, because {v 1,...,v s } are also linearly independent. Second, we shall prove that these vectors span col F. Let v u = F x for some x R n. Decompose x = y + z, where y col and z N. hen, x = y = λ, where y = λ, and Fx = Fy + Fz. Since Fz S, F x = F λ + j β jv j. We thus have u = x = λ, but since u col, λ = i α iu i. his implies λ = i α i[ ] 1 u i,so Fx = F i α i[ ] 1 u i + j β jv j = i α if [ ] 1 u i + j β jv j. By the definition of w, we have derived α, β such that v = u i α wi i + u j β vj j. i 0 o prove the main theorem, we use the following dimension results of Berkelaar, Roos, and erlaky [1]: 3.1 dimqp = B rank, 3.2 dimqd =m ranka B A +n rankq. he last portion, n rankq, accounts for the u-variables because the dual conditions can use x Q in place of u Q, leaving u to appear only in the equation Qu = Qx. For

6 212 H. J. REENBER, A.. HOLDER, K. ROOS, AND. ERLAKY our purposes it is not necessary or desirable to include this, so we define the dual optimality region exclusive of the u-variables: QD = {y, s :y, s, x QD for some x QP }. hen, 3.2 yields the dimension of the dual optimality region that we shall use: 3.3 dimqd =m ranka B A. As in the linear case, s N > 0 implies that each component of c N can vary independently, so dimh is the sum of N and the dimension of the set of other possible changes. Keeping x N = 0 and s B = 0, the partition does not change if and only if there exists δy, δu, δx B to satisfy the following primal-dual conditions: 3.4 Here we follow the notation in [1]: A B Q Q 0 δy δc B B δu = δc δb. δx 0 Q Q B B 0 Q I = rows of Q associated with index set I, Q J = columns of Q associated with index set J, Q IJ = submatrix of Q associated with index sets I and J. he quadratic extensions rely on the fact that the rank of the matrix in 3.4 is related to the rank of the matrices found in statements 3.1 and 3.3. hese relations are formalized in the following lemma. Lemma 3.2. he following relations hold for Q positive semidefinite: A B Q 3.5 ranka B A + rank = rank Q 0 B rankq 3.6 A = rank + ranka Q B. B Proof. o prove 3.5, performing elementary row and column operations on the large matrix first on the right produces the following matrix of the same rank: A B 0 0 Q B B. 0 Q 0 So, A B Q rank Q 0 B = rank A B Q B 0 A B + rankq.

7 RIM PERURBAIONS FOR OPIMAL PARIION INVARIANCE 213 he positive semidefiniteness of Q implies that Q B is linearly dependent on [1]. Hence, A B Q B 0 A B à B QBB 0 à à B à B à à B, QBB where is used to represent a series of row and column operations that preserve rank, and represents an arbitrary matrix of appropriate size. Hence, A B Q rank Q 0 B = rankq à + rank B ÃB + rank 0 à A = rankq + rank B + rank 0 QBB, which yields the result. he proof of 3.6 is similar, using the positive semidefiniteness property of Q in reducing the large matrix to row echelon form. We now have what we need to prove the following extension of heorem 2.1. heorem 3.3. he following equations hold for any convex quadratic program whose primal and dual sets are not empty. 1. dimqp + dimh c =n + ranka B A ranka B. 2. dimqd + dimh b =m ranka B A + ranka B. 3. dimqp QD + dimh =n + m. Proof. o prove 1, we set δb = 0 in 3.4, and apply Lemmas 3.1 and 3.2 to

8 214 H. J. REENBER, A.. HOLDER, K. ROOS, AND. ERLAKY produce the following: A B Q dimh c = N + rank A Q 0 0 Q B rank Q B 0 0 A B = N + ranka B A + rank + rankq rankq ranka B = N + ranka B A rank ranka B. Adding 3.1 to the last statement and substituting n = B + N gives the first result. Similarly, to prove 2, set δc B = 0 and δc = 0 in 3.4. hen, Lemma 3.1 implies the equation A B Q dimh b = rank A A B Q 0 B rank Q Q 0. Using row and column operations on the matrix in the last term together with Lemma 3.2 we obtain the dimension of H b : A dimh b = rank B + rank = ranka B, A rank B Q B where the last equation follows from 3.6. Adding this to 3.3 yields the second result. he third result does not follow from a decoupling principle, as in the linear case where H = H b H c. Rather, it needs a development similar to the first two parts just obtained. Using Lemmas 3.1 and 3.2 yields the following equations

9 RIM PERURBAIONS FOR OPIMAL PARIION INVARIANCE 215 A B Q dimh = N + rank Q 0 B rank A = N + rank B + rank he sum of the last statement with 3.1 and 3.3, plus substituting n = B + N, implies the third result. Notice that the statements in heorem 3.3 reduce to the corresponding statements in heorem 2.1 when = and Q = 0, which is the case for an LP. his reduction occurs because we eliminated the u-variables. In fact, the statements in the theorem imply each of the following. dimqp + dimh c n with equality if =. dimqd + dimh b m with equality if =. dimqp QD + dimh m + n with equality if =. he reduction of QD also enables us to have the following extension of Corollary 2.2. In fact, u is unique if and only if Q is positive definite because it can be any solution to Qu = Qx for any x QP. Corollary 3.4. he following are equivalent. 1. he dual solution is unique. 2. dimh c =n + m ranka B + ranka B. 3. dimh b =m + ranka B ranka B A. he above cases reduce to the corresponding LP cases in Corollary 2.2, where Q = 0 and =, as does the following extension of Corollary 2.3. Corollary 3.5. he following are equivalent. 1. he primal solution is unique. 2. dimh c =n + ranka B A ranka B. 3. dimh b = B ranka B + ranka B. Combining these, despite the absence of a decoupling principle, the dimensions are additive, so we also obtain the following extension of Corollary 2.4. Corollary 3.6. he following are equivalent. 1. he primal-dual solution is unique. 2. dimh c =n and dimh b =m. 3. dimh =m + n. Unlike the LP case, this shows that we lose degrees of freedom in varying the cost coefficients. For example, if δc j > 0 for j, the partition immediately changes since s j = δc j is optimal for the perturbed quadratic program. his loss appears in the last extension, which follows. Corollary 3.7. here are at least m + ranka B A ranka B degrees of freedom to vary the rim data without a change in the optimal partition. his lower bound on dimh c follows in the same way as in Corollary 2.5, and it is m when =. More generally, we see that the bound is at most m, which reflects the fact that we can lose some degrees of freedom by lacking strict complementarity..

10 216 H. J. REENBER, A.. HOLDER, K. ROOS, AND. ERLAKY 4. Concluding comments. For LPs, the dimension of the cone of rim direction vectors for which the optimal partition does not change has an Eulerian property with the dimension of the optimality region: they sum to the number of variables and equations. his decouples into Eulerian properties for varying the primal and dual right-hand sides separately: cost coefficients change with lost degrees of freedom equal to the dimension of primal space; right-hand sides change with lost degrees of freedom equal to the dimension of dual space. he comparable equation for quadratic programs is not Eulerian in that the sum of dimensions depends on the partition notably on the number of complementary coordinate pairs that are both zero. REFERENCES [1] A. Berkelaar, C. Roos, and. erlaky, he optimal set and optimal partition approach to linear and quadratic programming, in Advances in Sensitivity Analysis and Parametric Programming,. al and H. reenberg, eds., Kluwer Academic Publishers, Boston, MA, 1997, Chapter 6. [2] H. reenberg, Mathematical Programming lossary, hgreenbe/glossary/glossary.html, [3] H. reenberg, Rim Sensitivity Analysis from an Interior Solution, echnical report CCM 86, Center for Computational Mathematics, Mathematics Department, University of Colorado at Denver, Denver, CO, [4] B. Jansen, Interior Point echniques in Optimization: Complexity, Sensitivity, and Algorithms, Kluwer Academic Publishers, Boston, MA, [5] O. üler and Y. Ye, Convergence behavior of interior-point algorithms, Math. Programming, , pp [6] C. Roos,. erlaky, and J.-P. Vial, heory and Algorithms for Linear Optimization: An Interior Point Approach, John Wiley and Sons, Chichester, UK, 1997.