Quadratic Programming

Transcription

1 QuadProgram.nb 1 Revised: 5 April, 005 Quadratic Programming Quadratic Programs The general quadratic program proposes to minimie an obective function of the form: the linear constraints: A.x == b, x 0 Min: x.q.x/ + p.x subect to Note that we may assume that Q is a symmetric matrix (and that Q is the Hessian of the obective function.) This is a non-linear program problem, for the obective function is a quadratic function (if Q is non-ero.) ü Example 1 In[98]:= A = 83, 4<; b = 6; Q = 88, 0<, 80, 0<<; p = 8-5, 7<; X = Table@x i, 8i, 1, <D; q@x_d := X.Q.X ê + p.x In[91]:= Plot@H6-3 xl ê 4, 8x, 0, <D Out[91]= Ü Graphics Ü In[107]:= Expand@q@8x, H6-3 xl ê 4<DD Out[107]= 1 ÅÅÅÅÅÅÅ - ÅÅÅÅÅÅÅÅÅÅ 41 x + x 4

2 QuadProgram.nb In[108]:= xl ê 4, H6-3 xl ê 4<D<, 8x, 0, <D Out[108]= Ü Graphics Ü ü Example A = 81, 1<; b = ; Q = 88-, 0<, 80, -<<; p = 80, 0<; X = Table@x i, 8i, 1, <D; Expand@X.Q.X ê + p.xd -x 1 - x ü Numerical Example In[1]:= A = 81, 1<; b = 1; Q = 88, 1<, 81, <<; p = 80, -5<; X = Table@x i, 8i, 1, <D; q@x_d = X.Q.X ê + p.x Expand@q@XDD Out[6]= -5 x + 1 ÅÅÅÅ Hx 1 H x 1 + x L + x Hx 1 + x LL Out[7]= x 1-5 x + x 1 x + x

3 QuadProgram.nb 3 LP Algorithm for Quadratic Program: The method for reduction from the Quadratic to a Linear Program is due to Philip Wolfe: The Simplex Method for Quadratic Programming, Econometrica, vol. 7, (1959). See also the discussion of Example 5 in Chapter One: page 7-8. There is an interesting interview on the Web at: ü Development from Farkas Lemma Suppose X[0] is feasible: A.X[0]==b and X 0. Then X = X[0] + e Y is feasible whenever A.Y ==0 and such that ( Y 0 whenever X@0D = 0 ). If q[x[0]] is minimal then q[x[0]] q[x[0] + e Y] = q[x[0]] + e(p + Q.X[0]).Y + e Y.Q.Y/. Therefore, if X[0] is minimal, then (p + Q.X[0]).Y 0 But the Farkas alternative requires, for this condition to hold, that Y be a non-negative linear combinations of the rows of A, -A and the unit vectors e where X@0D = 0. That is p + Q.X[0] = Transpose[A].(-u) + v with x.v=0. ü Basic Algorithm Given the Quadratic Program as above, the associated Linear Program is: A.x == b Q.x + Transpose[A].u - v == -p x 0, v 0, x.v == 0 Note that the vector u is not necessarily positive. The only additional feature is the exclusion rule: x.v == 0, this requirement states that the i th components of x and v can not both be positive simultaneously. Finally, note that the existence of a solution of the associated LP is only necessary for the existence of a solution to the QP and is sufficient in case Q is positive semi-definite. In practice we solve the LP program: A.x==b Q.x + Transpose[A].u -v + D. == -p x 0l, u free, v 0, 0 Min: {1,...,1}. Initial Feasible Solution: x, u=0, v=0, where x is a basic feasible solution of A.x ==b, x 0, D is a diagonal matrix with entries ± 1 to correct the signs of and is a chosen such that Q.x + D. == - p, 0.

4 QuadProgram.nb 4 ü Numerical Example: Franklin, p. 184 In[109]:= A = 81, 1<; b = 1; Q = 88, 1<, 81, <<; p = 80, -5<; X = Table@x i, 8i, 1, <D; q@x_d := X.Q.X ê + p.x; Expand@q@XDD Out[115]= x 1-5 x + x 1 x + x There are two basic solutions for x, let's start with {1,0} In[118]:= x0 = 81, 0< Q.x0 + p Out[118]= 81, 0< Out[119]= 8, -4< In[14]:= Diag = DiagonalMatrix@8-1, 1<D; MatrixForm@DiagD = 8, 4< Q.x0 + Diag. + p Out[15]//MatrixForm= J N Out[16]= 8, 4< Out[17]= 80, 0< Note that Q.x + Diag. == -p Finally, we have the following linear program:

5 QuadProgram.nb 5 ü Equivalent LP In[18]:= LP = 881, 1, 0, 0, 0, 0, 0, 1<, 8, 1, 1, -1, 0, -1, 0, 0<, 81,, 1, 0, -1, 0, 1, 5<, 80, 0, 0, 0, 0, -1, -1, 0<<; MatrixForm@ LPD Out[19]//MatrixForm= k { The initial basic feasible solution is: x = {1,0}, ={,4} so we need to put them into the basic set: In[130]:= LP@@4DD = LP@@3DD + LP@@4DD; LP@@4DD = -LP@@DD + LP@@4DD; LP@@DD = - LP@@1DD + LP@@DD; LP@@3DD = - LP@@1DD + LP@@3DD; LP@@4DD = LP@@1DD + LP@@4DD; Out[135]//MatrixForm= k { At this point the cost is 6 and the basic feasible set is: x 1 =1, 1 =, = 4. We need to continue and "drive" the cost to ero by putting x into the basic set In[136]:= LP@@DD = LP@@1DD + LP@@DD; LP@@3DD = - LP@@1DD + LP@@3DD; LP@@4DD = - LP@@1DD + LP@@4DD; Out[139]//MatrixForm= k { Next we put v 1 into the basic set (this is O.K. by the exclusionary rule)

6 QuadProgram.nb 6 In[140]:= LP@@DD = -LP@@DD; LP@@4DD = -LP@@DD + LP@@4DD; Out[14]//MatrixForm= k { Now we put u 1 in and throw out In[143]:= LP@@DD = LP@@3DD + LP@@DD; LP@@4DD = -LP@@3DD + LP@@4DD; Out[145]//MatrixForm= k { The cost is now ero and the solution is x = 1 and {0,1} is the solution. The other variables are auxiliary. Exam : Linear & Quadratic Programs Date: Tuesday, 1 April Pedregal: Chapter Two covers Linear Programming as well with simple problems worked in detail. Franklin: Chapter One: Sections 1-5: Linear Programs, Canonical Forms and Dual Programs Section 8: Duality Principle: Know statement of Farkas Alternative (p. 56) and the application to the four cases Section 9: (page 68 & 69 only) Shadow costs from dual solution: If the i th requirement b i changes slightly by db i then the minimum cost changes by the product y i (db i ) Section 10: Simplex Method (with Extended Tableaux & Shadow Costs Omit Section 1: Lexicographic Order and avoidance of Cycling in degenerate LP problems Affine Scaling Method Franklin: Book Division Chapter 1 (p ) Wolfe's Method for Quadratic Programs: Minimie p.x + x.c.x/ subect to Ax==b, x 0. These programs allow degree two terms in the obective function. Exercises: You may want to first work thru the numerical example on page 185 and then try problems, 14, 17 and then 5. For a real test of understanding, work thru the sequence 6-10 on page 188.