Optimization over a polyhedron

Transcription

1 32

2 Chapter 5 Optimization over a polyhedron A significant class of mathematical programming problems with convex feasible region is the case of polyhedron. Apply of course to this problem of the same optimality conditions described in paragraph4; however linearity allowws to get more specific conditions. Assume without loss of generality that S = {x R n : Ax b} where A is a m n real matrix and b R m. The problem under consideration is min f (x Ax b (P POL where f is continuously differentiable. 5.1 Feasible direction on a polyhedron Consider the following defintion: Definition 5.2 (Binding (or active constraints let S = {x R n : Ax b} and x S. If a T i x = b i we say that the i-th constraint is binding (or active in x. For any x S we denote by I( x the index set of all the active constraints in x, namely: I( x = {i {1,...,m} : a T i x = b i } Active constraint defitino can be given for polyhedron in a general form. Given S = {x R n : Ax = b}, we have that I( x = {1,...,m}. 33

3 34 CHAPTER 5. OPTIMIZATION OVER A POLYHEDRON Gievna polyhedorn in standard form S = {x R n : Ax = b, x 0}, we have I( x = {1,...,m} { j {1,...,n} : x j = 0}. Theorem 5.3 (Feasible directions Let S = {x R n : Ax b} and x S with I( x = {i {1,...,m} : a T i x = b i }. A vector d is a feasible direction x if and only if a T i d 0, for all i I( x. (5.1 Furthemore givena feasible direction d, the point x +t d is feasible for t satisfying 0 < t t max = min j / I( x: a T j d<0 a T j x b j d a T j (5.2 where in case { j / I( x : a T j d < 0} is empty, t max =. Proof. Let x feasible, i.e. a T i x b i, i = 1,...,m. Consider x = x +td with t > 0. We look for conditions on d which guarantee that a T i ( x +td = a T i x +ta T i d b i for t [0,t max. For any i I( x, we have a T i x = b i hence a T i x +ta T i d = b i +ta T i d. So that for all i I( x: a T i x +ta T i d b i if and only if a T i d 0. Consider j / I( x, so that a T j x > b i hence a T j x +tat j d > b j +ta T j d. We have two cases: either 1. a T j d 0 or 2. at j d < 0. If a T j d 0, then b j +ta T j d b j for all t > 0. if a T j d < 0, a sufficiently small stepsize t must be chosen such that a T j x t at j d b j for t > 0. In any case we do not have condition on d but only on t. In particular the value of t max can be found by considering only the non active constraints such that a T j d < 0 and solving the system in t We get a T j ( x +td = a T j x +ta T j d = a T j x t a T j d b j j I( x and such that a T j d < 0. Hence we get (5.2. t at j x b j a T i d j I( x. Let A I( x be the I( x n submatrix of A made up of the rows with index in I( x, A I( x = (a T i i I( x.

4 5.1. FEASIBLE DIRECTION ON A POLYHEDRON 35 Let x such that A x b, feasible directions in x are the solution of the linear system of inequalities A I( x d 0 where A I( x = (a T i i I( x and I( x = {i : a T i x = b i }. We can give a characterization also form polyhedron in a different form. Consider first the case of S = = {x R n : Ax = b} and let x S =. By definition a feasible direction d satisfies A( x +td = b for sufficiently small t. We get A x +tad = b +tad = b Ad = 0 for all t. We note that in this case both +d and d are feasible directions. Proposition 5.4 Given S = = {x R n : Ax = b} and x S =. A vector d R n is a feasible direction in x if and only if Ad = 0. (5.3 As an example consider x 1 + x 2 x 3 = 1 x 1 x 2 = 2 (5.4 Then Ad = 0 is ( d 1 d 2 d 3 = which has infinite solution of the type d = (t, t, 2t T = t(1, 1, 2 T with t R. Theorem 5.5 (Faesible direction over a standard polyhedron Given S = {x R n : Ax = b, x 0} and x S. let J( x = { j {1,...,n} : x j = 0}. A vector d R n is a feasible direction in x if and only if Ad = 0, (5.5 d j 0 per ogni j J( x.

5 36 CHAPTER 5. OPTIMIZATION OVER A POLYHEDRON 5.6 Optimality conditions over Ax = b Consideri a problem of the type min f (x Ax = b where A is a m n matrix with rows a T j, and b Rm with components b j. (P EQ Proposition 5.7 Let x S be a local minimizer of problem (P-EQ and assume that f is continuously differentiable overr n. Then it holds: f (x T d 0, for all d R n : Ad = 0. We can enter more into details of the condition above. Assume to this aim that the matrix A has rank equal to m. This assumption is purely simplifying. Indeed the result which is obtained is valid also in the general case, but the proof is more complex. Let A 1,A 2,...,A n be the columns of A, so that the constraints can be written as: Further we observe that A 1 x 1 + A 2 x A n x n b = 0. (5.6 (Ax b = A T ; We can extract from the matrix A, m linearly independent columns and aftere a reordering we assume that they are the first m, A 1,A 2,...,A m. We define B = [A 1,A 2,...,A m ], N = [A m+1,a m+2...,a n ], A = (B N x 1 x m+1 d 1 d m+1 x B = x 2..., x N = x m+2..., d B = d 2..., d N = d m+2..., x m x n d m d n where B is a square non singular m m matrix. B is called basis matrix, and N non basis matrix; the vector x B is called basic variable, whereas x N non basic variable. Accordingly the vector d is partitioned into d B basic direction and d N non basic direction. Constraint (5.6 becomes: Bx B + Nx N b = 0, (5.7 and (5.3: Since B is non singular, from (5.8 we get: Bd B + Nd N = 0. (5.8 d B = B 1 Nd N ; (5.9

6 5.6. OPTIMALITY CONDITIONS OVER AX = B 37 Hence with reference to a basis B, any feasibkle direction can be expressed as [ B d = 1 ] Nd N, (5.10 where d N R (n m can take any value. As an example consider system (5.4; we have B = ( ( 1, N = 0 d N ( x1, x B = x 2 ( d1, x N = x 3, d B = d 2, d N = d 3. Hence we get d B = ( d1 d 2 ( 1 1 = ( 1 0 ( 1/2 d 3 = 1/2 d 3. By partitioning also the gradient of f as follows: [ f (x B f (x = N f (x ], (5.11 where we denote by B f (x the vector of f x i with i B and N f (x the vector of f x i with i N. The necessary condition (5.7 can be written as: B f (x T d B + N f (x T d N 0, per ogni d B R m,e d N R (n m : Bd B + Nd N = 0. Using (5.9 we get Hence: which gives B f (x T B 1 Nd N + N f (x T d N 0,for all d N R n m. ( B f (x T B 1 N + N f (x T d N 0, for all d N R n m. (5.12 ( N f (x N T (B 1 T B f (x T d N 0, for all d N R n m. Reasoning as in the unconstrained case (see Theorem 3.18, the preceding inequality holds for all d N R n m when: N f (x N T (B 1 T B f (x = 0; (5.13 Indeed otherwise, taking d N = [ N f (x N T (B 1 T B f (x ], inequality would not be satisfied. equation (5.13 is the gradient of the fucntion in the only variables x N R n m. Indeed using x B = B 1 b B 1 Nx N,

7 38 CHAPTER 5. OPTIMIZATION OVER A POLYHEDRON problem (P-EQ can be written as min f x N R (B 1 b B 1 Nx N,x N n m which is an unconstrained problem in the variables x N of the composite function f (x B (x N,x N. Such condition can be expressed using the defintion of Lagrangian function for Problem (P- EQ which is m L(x, µ = f (x + µ j (a T j x b j, j=1 or, using vector notation, Finally we get the theorem. L(x, µ = f (x + µ T (Ax b. Proposition 5.8 (First order optimality condition over Ax = b Let x S be a local minimizer of problem (P-CONV and assume that f is continuously differentiable overr n. Then multipliers µ 1, µ 2,... µ m exist such that: or, using vector notation, x L(x, µ = f (x + m µ j a j = 0, (5.14 j=1 x L(x, µ = f (x + A T µ = 0. (5.15 proof. Condition (5.15 can be written using the partition B,N as ( B f (x ( B T N f (x + µ = 0. which is From the first equation we get µ N T B f (x + B T µ = 0, N f (x + N T µ = 0. and substituting back into the second equation we get (5.13. µ = (B 1 T B f (x, (5.16 Given a minimization problem with only equality constraints Ax = b, candaites to be local minimizer are stationary point of the Lagrangian function x L(x, µ = f (x + A T µ = 0. µ L(x, µ = Ax b = 0

8 5.9. OPTIMALITY CONDITION OVER A POLYHEDRON Optimality condition over a polyhedron Using the defintion of feasible direction, we can apply Theorem 3.7 to get Theorem 5.10 (First order necessary condition overa polyhedron If x is a local minimizer of problem (P-POLthen we have f (x T d 0 per ogni direzione d R n : a T i d 0, i I(x = {i : a T i x = b i }. This condition can be state as a non existence of a solution of a linear system. Indeed we can write If x is a local minimizer of problem (P-POLthen there exists NO solution d R n of the system A I(x d 0, f (x T d < 0. (5.17 Example 5.11 Consider the problem min (x (x x 1 2x x 1 x 2 12 x 1 0, x 2 0 And let x = (0, 12 T be a minimizer (see Figure 5.1. The gradient of the objective function is ( 2(x f (x = 2(x 2 12 so that f ( x = (20, 0 T. The system (5.17 is written as 2d 1 d 2 0 d d 1 < 0 (5.18 which does not have a solution. Under convexity assumption on the objective function we get the following (since the feasible region is convex too

9 40 CHAPTER 5. OPTIMIZATION OVER A POLYHEDRON x 2 x 1 Figure 5.1: Poliedro Esempio Theorem 5.12 (Necessary and sufficient conditon for convex problem Let f (x e a convex function overr n. A point x S is a global solution for problem (P-POL if and only if f (x T d 0 for all d R n : a T i d 0, i I(x = {i : a T i x = b i }. which can be written as Let f (x e a convex function overr n. A point x S is a globalsolution for problem (P-POL if and only if there exists NO solution d R n of the system A I d 0, f (x T d < 0. (5.19 Example 5.13 Consider again Example The objective functi is strictly convex Hence the point (0, 12 T which satisfies the condition is a global minimizer.

10 5.9. OPTIMALITY CONDITION OVER A POLYHEDRON 41 x 2 x x 1 Figure 5.2: Poliedro Esempio We can extend the conditions above to the case of polyhedron described by inequality and equality using characterization of feasible direction given in Section 5.1. Consider the problem min f (x a T i x b i a T j x = b j i = 1,...,m j = 1,..., p (5.20 Let x S be a local minimizer for problem (5.20 then there exists no solution d R n to the linear system a T i d 0 i I(x {1,...,m} a T j d = 0 j = 1,..., p f (x T d < 0. An important case is the problem min f (x Ax = b x 0 (P POL ST

11 42 CHAPTER 5. OPTIMIZATION OVER A POLYHEDRON In this case using Theorem 5.5, we get: Let x S be a local minimizer for problem (P-POL-ST then there exists no solution d R n to the linear system Ad = 0, with J(x = { j {1,...,n} : x j = 0}. d J(x 0 f (x T d < 0. (5.21 Of ocurse all the necessary conditions above become also sufficient if the objective fuction f is ocnvex. Consider now Linear Programming problems min c T x Ax b. (PL Using theorem 5.12, we get the following condition. A point x S be a global minimizer for problem (PL if and only if there exists no solution d R n to the linear system A I(x d 0, c T (5.22 d < 0. Analogously if we have problem of the type min c T x Ax = b x 0, (PL ST we get A point x S be a global minimizer for problem (PL-ST if and only if there exists no solution d R n to the linear system Ad = 0, con J(x = { j {1,...,n} : x j = 0}. d J(x 0 c T d < 0. (5.23 Non existence conditions of solution of linear system formulated above can be formulate as the existence conditions of an alternative system, as discussed in Chapter 6.