St back, relax, and enjoy the rde of your lfe as we explore the condtons that enable us to clmb to the top of a concave functon or descend to the bottom of a convex functon whle constraned wthn a closed area. The General Nonlnear Constraned Optmzaton Problem The whole enchlada. Ths s t! At last the wat s over. The breathtakng adventure s about to begn. Dscover the key to successful optmzaton. Meet the explorers Kuhn and Tucker.
The Problem subj to : Max/Mn f( x, x,..., x ) g ( x, x,..., x ) 0, =,,..., m n h ( x, x,..., x ) = 0, k =,,..., K k n n
x 6 The Dffculty (,6) Mn f(x,x ) subj. to: x + x <= 8-3x + x <= 6 x, x >= 0 5 4 3. (,) f(x,x ) = (x 8) + (x 4).(8,4) nfeasble.(,6) f(x,x ) = (x ) + (x ) 0 3 4 5 6 7 8 x 3
More Dffcultes x objectve functon The feasble regon 0 5 0 5 global mnmum local mnmum pont x 4
Non-convex feasble regons x 3 + x ( ) ( ) 3 5
Convex feasble regons x 3 + x ( ) ( ) feasble regon 6
Non-convex feasble regons x 3 + x ( ) ( ) 3 7
Convex feasble regons x 3 + x ( ) ( ) feasble regon 8
Convex Feasble Regons The feasble regon s convex f the lne segment between every par of feasble ponts falls entrely wthn the feasble regon. The feasble regon defned by: g ( x) 0 ; =,,..., m = s convex f () each g (x) >= constrant s concave () each g (x) <= constrant s convex (3) each g (x) = constrant s lnear 9
g(x) Concave g(x) C g(x) < C g(x) > C g(x) < C x 0
g(x) Convex g(x) C g(x) > C g(x) < C g(x) > C x
Equalty Constrant g(x) C g(x) = C x
A Useful Result If g(x) s convex (concave), then g(x) s concave (convex) g(x) -g(x) concave convex x x 3
Sngle Constrant K-T Condtons Max f(x) s.t. g(x) <= 0 where x = (x,x,,x n ) convert constrant to an equalty: g(x) + y = 0 Form Lagrangan: L(x,y,u) = f(x) - u [ g(x) + y ] necessary condtons: L f g = u = 0 j =,,..., n x x x j j j L = gx + y = u L = yu = 0 y ( ) 0 4
The Kuhn-Tucker Condtons L f g = u = 0 j =,,..., n x x x j j j L = gx + y = u L = yu = 0 y ( ) 0 gx ( ) 0 ug( x) = 0 statonary pont feasblty complementary slackness 5
The Lagrangan Multpler L f g = u = x x x j j j 0 f x u j g = u x f x g x j = = j j f g Max f(x) s.t. g(x) <= 0 u >= 0 for max 6
K-T Necessary Condtons L f g I. = u = 0 j =,,..., n x x x j j j II. gx ( ) 0 III. ug( x) = 0 IV. u 0 (max) u 0(mn) A set of values for x, x,,x n and u that satsfes all of the above condtons s referred to as a Kuhn-Tucker pont. 7
Our Very Frst Kuhn-Tucker Problem Max x 3x + 0x + 6x + 5 st.. 3x + 5x 9 0 [ ] Lx (, x, u) = x 3x + 0x+ 6x+ 5 u3x+ 5x 9 It must be wonderful to fnd a Kuhn-Tucker pont! 8
K-T Condtons [ ] Lx (, x, u) = x 3x + 0x+ 6x+ 5 u3x+ 5x 9 L I. = 4x + 0 3u = 0 x L x = 6x + 6 5u = 0 II. 3x + 5x 9 0 III. u 3x + 5x 9 = 0 IV. u 0 ( ) 9
assume u = 0: L I. = 4x + 0 = 0 x =.5 x L = 6x + 6= 0 x = x II. 3x + 5x 9 0 III. u 3x + 5x 9 = 0 IV. u 0 ( ) I., III. and IV. are satsfed. nfeasble snce 3 (.5) + 5 () 9 =.5-9 >0 0
assume u > 0: 4x + 0 3u = 0 6x + 6 5u = 0 3x + 5x 9 = 0 ( ) x 5-0x + 50 + 8x - 8 =0 x (-3) -0x + 8x = -3 x (0/3) 0x + (00/3)x = 60 (-54/3) x = 8 x = 3 (8) /54 = 6/ x = (3 + 8 x )/0 = 3/ u = (-4x +0 ) /3 = 6/
L I. = 4x + 0 3u = 0 x L x = 6x + 6 5u = 0 II. 3x + 5x 9 0 x = 3/ x = 6/ u = 6/ III. u 3x + 5x 9 = 0 ( ) IV. u 0
Tell us about suffcency. The Kuhn-Tucker condtons are both necessary and suffcent f mnmzng a convex functon over a convex feasble regon or maxmzng a concave functon over a convex feasble regon. Wow! That s truly an amazng result! 3
Insect Infestatons Let x = the pounds of nsectcde allocated aganst the th acre of farmland; and d(x ) = k ln x s the number of nsects klled where k = relatve weght of the nsect nfestaton of acre d(x ) d(x ) = k ln x x Gven there are M pounds avalable and n acres. How many pounds should be allocated to each acre n order to maxmze the number of nsects klled? 4
The Formulaton n Max k ln x = n st.. x M = n n L( x, u) = k lnx u x M = = 5
The K-T Condtons L k I. = u = 0 x x II. n = x M III. n u x M = 0 = IV. u 0 n n L( x, u) = k lnx u x M = = 6
7 The K-T Condtons I. 0 II. III. 0 IV. 0 n n k L u x x x M u x M u = = = = = n n n k ux k u x um k u M = = = = = = = n km x k = =
f( x... x ) = k ln x What about Suffcency? n n = f( x... x ) k f( x... x ) k f( x... x ) = ; = ; = 0 n n n x x x x x x xj k 0... 0 x k 0 : 0 H( x... xn ) = x 0 0 : 0 kn 0 0... x n Clearly, the matrx s negatve defnte. 8
The General K-T Problem Max/Mn f(x) s.t. g (x) <= 0 ; =,,,m h k (x) = 0 ; k =,,,K f ( x) g ( x) h ( x) I. u v = 0 j =,,..., n x x x m K k k j = j k= j II. g ( x) 0 ; =,,..., m hk ( x) = 0 ; k =,,..., K III. ug ( x) = 0 ; =,,..., m IV. u 0 (max) u 0 (mn) =,,..., m v unrestrcted k 9
An Excellent Applcaton A closed cylndrcal tank s beng desgned to carry at least 0 cubc feet of chemcals. Metal for the top and sdes costs $ per square foot, but the heaver metal of the base costs $8 per square foot. The heght of the tank can be no more than twce ts dameter to keep t from beng top heavy. Fnd a mnmum cost desgn. let x = dameter of tank x = heght of tank x x mn πxx + π + 8 π 4 4 st.. x π x 4 x x 0 x 0, x 0 (cost) (volume) (heght-to-dameter rato) 30
An Excellent Example Contnued x x mn πxx + π + 8 π 4 4 st.. x π x 4 x x 0 x 0, x 0 x x π + 0 0 4 relax non-negatvty constrants and form the Lagrangan functon. x x Lx (, x, u, u) = πxx + π + 8 π 4 4 x + u x u x x 4 π 0 [ ] 3
x x Lx (, x, u, u) = πxx + π + 8 π 4 4 L x = πx + πx+ 4( πx) + u π x + u = 0 x L x + u x u x x 4 x = + = 0 πx u π u x 4 π 0 [ ] 3
x I. πx + 5( πx) + u π x + u = 0 x πx+ u π u = 0 4 x II. π x 0 4 x x x III. u π x 0 = 0 4 [ ] u x x = 0 IV. u 0 u 0 33
Case : Assume u = u = 0 ( π ) I. πx + 5 x = 0 π x = 0 then x = x = 0 nfeasble! Case : Assume u <0; u = 0 x I. πx + 5( πx) + u π x = 0 x πx+ u π = 0 4 x III. π x 0 = 0 4 soluton: x =.677 x = 5.493 snce x not x,, nfeasble 34
Case 3: Assume u = 0; u < 0 ( π ) I. πx + 5 x + u = 0 π x u = 0 III. x x = 0 then x = x ( π ) (4πx + 5 x + u = 0) + ( π x u = 0) 3π x = 0 or x = 0 and x = 0 35
Case 4: Assume u < 0; u < 0 x I. πx + 5( πx) + u π x + u = 0 x πx+ u π u = 0 4 x III. π x 0 = 0 4 [ x x ] = 0 soluton: x =.335 x = 4.670 u = -3.763 u = -.6 36
A Kuhn-Tucker Problem Ique S. Lowe, an MSC grad student, must cram for fnal exams n OR I and Statstcs. The amount of nformaton absorbed s a functon of Ique s study tme. However, dmnshng returns are observed n both subjects. That s g(x) = 00x x and h(x) = 500y y 3 where g,h = nformaton absorbed n 000 s of neural bts and x = hours studyng OR y = hours studyng statstcs Ique has only 30 hours avalable to study. How many hours should be spent on each subject? 37
Max f(x,y) = 00x x + 500y y 3 s.t. x + y <= 30 L(x,y,u) = 00x x + 500y y 3 u(x + y 30) I. L x = 00 4x u = 0 L y = 500 3y u =0 II. x + y <= 30 III. u (x + y 30)=0 IV. u>=0 38
I. L x = 00 4x u = 0 L y = 500 3y u =0 II. x + y <= 30 III. u (x + y 30)=0 IV. u>=0 Assume u = 0 from I: x = 5, y = (500/3).5 =.9 nfeasble Assume u >0, then 00 4x u = 0 500 3y u = 0 x + y 30 = 0 x = 30 y 400 + 4x 3y = 0 400 + 4(30 y) 3y = 0 3y + 4y 50 = 0 y =.5 x = 30.5 = 7.48 u = 00 4x = 00 4(7.48) > 0 39
nd Order Condtons Max f(x,y) = 00x x + 500y y 3 s.t. x + y <= 30 f xx = -4 < 0 f yy = -6y f xy = 0 f xx f yy (f xy ) = 4y > 0 for y > 0 f(,y) s concave and snce the constrant s lnear, the feasble regon s convex. Therefore a K-T pont s both necessary and suffcent to solve the problem! 40
optmze wth the best of them never agan wll you be satsfed wth non-optmalty experence the peak and valleys that only nonlnear optmzaton can brng a certanty to be the pnnacle of your career mssng ths opportunty s surely a nadr Engneerng Management and Systems proudly presents Nonlnear Optmzaton (MSC 53) Comng soon sgn up now seatng s lmted MSC 53: Nonlnear Optmzaton Ketterng Labs Room 3, TTH, :30 a.m. :00 p.m.* Textbook: Engneerng Optmzaton: Methods and Applcatons (nd ed) by Ravndran, Ragsdell, Reklats, Wley 006 3 credt hours *Class s also offered through dstance learnng. 4