Water Resources Systems Plannin and Manaement: Introduction to Optimization: arane Multipliers MODUE - ECTURE NOTES 3 AGRANGE MUTIPIERS AND KUHN-TUCKER CONDITIONS INTRODUCTION In the previous lecture the optimization o unctions o multiple variables subected to equality constraints usin the method o constrained variation was dealt. Optimization o unctions o multiple variables subected to equality constraints usin arane multiplier and inequality constraints usin Kuhn-Tucker conditions will be discussed in the present lecture with eamples. CONSTRAINTED OPTIMIZATION PROBEM WITH EQUAITY CONSTRAINTS Solution by method o arane multipliers As discussed in the previous lecture, a unction o multiple variables, (), is to be optimized subect to one or more equality constraints o many variables. The problem statement is as ollows: Maimize (or minimize) (X), subect to (X) =, =,,, m where X () n with the condition that m n; or else i m > n then the problem becomes an over deined one and there will be no solution. et us consider a speciic case with n = and m=. Consider a quantity, called the arane multiplier as / / (, ) () Usin this in the constrained variation o [ iven in the previous lecture in eqn. 5 as d / / (, ) d And () written as (, ) (3) D Naesh Kumar, IISc, Banalore M3
Water Resources Systems Plannin and Manaement: Introduction to Optimization: arane Multipliers (, ) Also, the constraint equation has to be satisied at the etreme point (, ) (4) (, ) (5) Hence equations () to (5) represent the necessary conditions or the point [, ] to be an etreme point. Note that could be epressed in terms o / as well / has to be non-zero. Thus, these necessary conditions require that at least one o the partial derivatives o (, ) be non-zero at an etreme point. The conditions iven by equations () to (5) can also be enerated by constructin a unction, known as the aranian unction, as Alternatively, treatin as a unction o, and etremum are iven by (,, ) (, ) (, ) () (,, ) (, ) (, ) (,, ) (, ) (, ) (,, ) (, ), the necessary conditions or its The necessary and suicient conditions or a eneral problem are discussed net. Necessary conditions or a eneral problem For a eneral problem with n variables and m equality constraints the problem is deined as shown earlier Maimize (or minimize) (X), subect to (X) =, =,,, m (7) where X n In this case the arane unction,, will have one arane multiplier or each constraint (X) as (,,...,,,..., ) ( X) ( X) ( X)... ( X ) (8) n, m m m D Naesh Kumar, IISc, Banalore M3
Water Resources Systems Plannin and Manaement: Introduction to Optimization: arane Multipliers 3 is now a unction o n + m unknowns,,,..., n,,,..., m, and the necessary conditions or the problem deined above are iven by ( ) ( X), i,,..., n;,,..., m m X i i i ( X),,,..., m which represent n + m equations in terms o the n + m unknowns, i and. The solution to (9) this set o equations ives us X n and m () The vector X corresponds to the relative constrained minimum o (X) (subect to the veriication o suicient conditions). Suicient conditions or a eneral problem A suicient condition or (X) to have a relative minimum at X is that each root o the polynomial in, deined by the ollowin determinant equation be positive. n m n m n n nn n n mn n n m m mn () where i ( X, ), or i,,..., n;,,..., m i p pq ( X), where p,,..., m and q,,..., n q () D Naesh Kumar, IISc, Banalore M3
Water Resources Systems Plannin and Manaement: Introduction to Optimization: arane Multipliers 4 Similarly, a suicient condition or (X) to have a relative maimum at X is that each root o the polynomial in, deined by equation () be neative. I equation (), on solvin yields roots, some o which are positive and others neative, then the point X is neither a maimum nor a minimum. Eample Minimize ( X ) 3 5 7 5 Subect to 5 Solution ( X ) 5 (,,...,,,..., ) ( X) ( X) ( X)... ( X ) with n = and m = n, m m m = 3 5 7 5 ( 5) 7 (7 ) 5 (7 ) or 3 5 3 5 (5 ) 3( ) (5 ) and, D Naesh Kumar, IISc, Banalore M3
Water Resources Systems Plannin and Manaement: Introduction to Optimization: arane Multipliers 5 Hence X, ; λ 3 ( X,λ ) ( X,λ ) ( X,λ ) ( X,λ ) ( X,λ ) The determinant becomes or ( )[ ] ( )[ ] [ ] Since is neative, X, λ correspond to a maimum. KUHN-TUCKER CONDITIONS It was previously established that or both an unconstrained optimization problem and an optimization problem with an equality constraint the irst-order conditions are suicient or a lobal optimum when the obective and constraint unctions satisy appropriate concavity/conveity conditions. The same is true or an optimization problem with inequality constraints. The Kuhn-Tucker conditions are both necessary and suicient i the obective unction is concave and each constraint is linear or each constraint unction is concave, the problems belon to a class called the conve prorammin problems. D Naesh Kumar, IISc, Banalore M3
Water Resources Systems Plannin and Manaement: Introduction to Optimization: arane Multipliers Consider the ollowin optimization problem: Minimize (X) subect to (X) or =,,,p ; where X = [,,... n ] Then the Kuhn-Tucker conditions or X = [... n ] to be a local minimum are i m i i,,..., n,,..., m,,..., m,,..., m (3) In case o minimization problems, i the constraints are o the orm (X), then have to be nonpositive in (3). On the other hand, i the problem is one o maimization with the constraints in the orm (X), then have to be nonneative. It may be noted that sin convention has to be strictly ollowed or the Kuhn-Tucker conditions to be applicable. Eample Minimize 3 subect to the constraints 3 3 3 8 3 usin Kuhn-Tucker conditions. Solution: The Kuhn-Tucker conditions are iven by a) i i i i.e. (4) 4 (5) 3 () 3 D Naesh Kumar, IISc, Banalore M3
Water Resources Systems Plannin and Manaement: Introduction to Optimization: arane Multipliers 7 b) ( ) (7) 3 ( 3 8) (8) 3 c) (9) 3 3 8 () 3 d) () () From (7) either = or, 3 Case : = From (4), (5) and () we have = = /and 3 = /. Usin these in (8) we et 8 Thereore, or 8 From (),, thereore, =, X = [,, ], this solution set satisies all o (8) to () Case : 3 3 Usin (4), (5) and (), we have 4 3 or, 7 44. But conditions () and () ive us and simultaneously, which cannot be possible with 7 44. Hence the solution set or this optimization problem is X = [ ] Eample Minimize subect to the constraints D Naesh Kumar, IISc, Banalore M3
Water Resources Systems Plannin and Manaement: Introduction to Optimization: arane Multipliers 8 8 usin Kuhn-Tucker conditions. Solution The Kuhn-Tucker conditions are iven by 3 a) i.e. 3 i i i i b) (3) (4) i.e. ( 8) (5) ( ) () c) 8 (7) (8) d) (9) (3) From (5) either = or, ( 8) Case : = From (3) and (4) we have 3 and Usin these in () we et 5 ; or 5 Considerin, X = [ 3, ]. But this solution set violates (7) and (8) For 5, X = [ 45, 75]. But this solution set violates (7). D Naesh Kumar, IISc, Banalore M3
Water Resources Systems Plannin and Manaement: Introduction to Optimization: arane Multipliers 9 Case : ( 8) Usin 8 in (3) and (4), we have (3) Substitute (3) in (), we have 4. For this to be true, either or 4 For,. This solution set violates (7) and (8) For 4, 4 and 8. This solution set is satisyin all equations rom (7) to (3) and hence the desired. Thereore, the solution set or this optimization problem is X = [ 8, 4 ]. BIBIOGRAPHY / FURTHER READING:. Rao S.S., Enineerin Optimization Theory and Practice, Fourth Edition, John Wiley and Sons, 9.. Ravindran A., D.T. Phillips and J.J. Solber, Operations Research Principles and Practice, John Wiley & Sons, New York,. 3. Taha H.A., Operations Research An Introduction, 8 th edition, Pearson Education India, 8. 4. Vedula S., and P.P. Muumdar, Water Resources Systems: Modellin Techniques and Analysis, Tata McGraw Hill, New Delhi, 5. D Naesh Kumar, IISc, Banalore M3