Solution of Multivariable Optimization with Inequality Constraints by Lagrange Multipliers. where, x=[x 1 x 2. x n] T

Transcription

1 Solution of Multivariable Optiization with Inequality Constraints by Lagrange Multipliers Consider this proble: Miniize f where, =[. n] T subect to, g,, The g functions are labeled inequality constraints. They ean that only acceptable solutions are those satisfying these constraints. Another way to thin about an optiization proble with inequality constraint is we are trying to find a solution within a space bounded by these constraints. To start, we need to ae distinction between two possibilities for a iniu: Interior: Eterior: No inequality constraint is active. In this case, a iniu is associated with, f * One or ore inequality constraint is active. One possible way to thin about this proble is f * but this is point is the feasible iniu. 3

2 We can find a solution to the proble by adding non-negative slac slacness variables, y such that, g y,, Slac variables are not nown beforehand. The proble now is transfored into: Miniize f where, =[. n] T subect to, g y,, In this for, the Lagrange ultiplier ethod can be used to solve the above proble by creating this function, Miniize, L = f + λ g y where, λ is the Lagrange ultiplier. This proble can be solved necessary conditions. = f g + λ i = i =,,, n i i = = g y = =,,, λ = y = λ y = =,,, The total nuber of equations is n+, which can be solved siultaneously to obtain the optial point. The solution will indicate which constraint is active, if any, are associated with the solution. 4

3 It ay be useful to understand the solution a little better. The first set of equations state that the gradient is still zero for the case of an eterior iniu. The gradient now cobines the original function and the active constraints. The second set of equations ensure that The third set of equations indicate either y or is zero. o If =, it eans that this constraint is inactive. o If y =, it eans that this constraint is active. g i= g Typically, consider the case when p constraints are active, which eans that -p are inactive. The first equation becoes, f g = λ i i = i =,,, p Or, f = λ g = i =,,, p The figure below ay help understand constrained optiization. In this case, the global iniu is outside feasible range. Reeber that at iniu slope is zero. Unfeasible g g f Feasible Global Miniu g g 5

4 Eaple.7: Miniize Subect to, f 3 Place the proble in the standard canonical for: g g f Miniize Subect to, f Prepare the solution: f 3 g g 3 g g Condition for iniu: 3 + λ + λ = + y = + y = λ y = λ y = Here we have to eplore several possibilities: = = =, which is outside the feasible doain. = y = Equation will result in =. The first equation eans that =3 The function is = y = Equation 3 will result in =. The first equation eans that =- The function is 8 This case is trivial as both constraints cannot be active in the sae tie Hoewor:.6 using the Lagrange Multipliers ethod. Plot the contour plots and the constraints. Relate your solution to this plot. 6

5 Kuhn-Tucer Optiality Conditions Kuhn and Tucer etended the Lagrange s theory to include classical nonlinear prograing probles. Harold Kuhn 95-4, Wiipedia Albert Willia Tucer , Wiipedia Kuhn and Tucer focused on identifying the conditions that when satisfied are related to constrained iniu or, = f g + λ i = i =,,, n i i = > J The above equations are labeled Kuhn-Tucer conditions. These conditions are necessary but not necessary to ensure optiality. They are not however not sufficient. If we liit the discussion to conve prograing probles, the conditions becoe both necessary and sufficient. A proble where both the obective = f g function and the constraints are + λ i = i =,,, n i i conve. λ g = g = =,,, =,,, Note: The Hessian atri of a conve function is positive seidefinite. λ =,,, 7

6 8 Constraint Qualifications We can now that we can solve an optiization proble with equality and inequality constraints as: Find,, and vectors such that, h g f p g y feasibilit p h y feasibilit g,,,,,,,, To be ore specific, we need to state that, f, g, and h should be linearly independent.

7 Eaple.8: Miniize f g Subect to, g 6 h 6 Graphical inspection shows the iniu is at,5. However, we will solve as if we do not now this. Prepare the solution, f g g h Evaluate the function and constraints Hf = [ ] Positive seidefinite Hg = [ ] Linear: positive seidefinite Hg = [ ] Positive definite Conclusion: We can apply the Kuhn-Tucer conditions: , 9

8 Here we have to eplore several possibilities: = = Equation : = Equation : -= Solving these two equations together, =-.5 Equation 5: = , 6.5 violates Equation 4, STOP = Equation 6: = Equation 5: =+5 or -5,-5 violates Equation 5, STOP, 5 does not violate any constraint. f=-4 = Equation 7: + = 6 Equation 5: + = 6 Solutions:,5 or 5, Both solutions do not violate constraints. f=-4 or 4 Choose, 5 since we are looing for a iniu. Equation 6: = Equation 7: =+5 or -5,-5 violates Equation 5, STOP, 5 does not violate any constraint. f=-4 Note: The sae solution cae fro three out of the four cases since the two inequality constraints and the equality constraint intersect at the sae point, 5. Hoewor:.64,.69,.73 3