Linear and Nonlinear Optimization

Transcription

1 Lner nd Nonlner Optmzton Ynyu Ye Deprtment of Mngement Scence nd Engneerng Stnford Unversty Stnford, CA 9430, U.S.A. Ynyu Ye, Stnford, MS&E Lecture Notes #

2 st Dy Questons 6 Homework (best fve), Mdterm, Fnl Regulr Students: 3%*H + %*M + 40%*F Project Students: 3%*H + %*M + 40%*F + Project 0% No dfference on tkng 3 or 4 unts No formul for cutoff between A/B etc. The more fun we ll hve, the more A s I wll gve out. Tetbook: Lner nd Nonlner Progrmmng (LY 4 th edton) Progrmmng Computer Scence. The softwre use wll help: Mtlb, Ecel Solver or publc free softwre. It s mostly PAPER AND PENCIL clss! Project: onlne LP model Form study group My CA tem Webste: web.stnford.edu/clss/msnde Frdy s problem sessons Ynyu Ye, Stnford, MS&E Lecture Notes #

3 Introducton to Optmzton Often consder the common gol of mngement scence & engneerng. Mmze or Mnmze f() for ll ϵ some set X Applctons n: Appled Scence, Engneerng, Economcs, Fnnce, Medcne, Sttstcs, Busness Generl Decson nd Polcy Mkng The fmous Eghteenth Century Swss mthemtcn nd physcst Leonhrd Euler ( ) proclmed tht nothng t ll tkes plce n the Unverse n whch some rule of mmum or mnmum does not pper. Ynyu Ye, Stnford, MS&E Lecture Notes # 3

4 The Prototypcl Optmzton Problem Mnmze: Subject to: f() h () = 0... h m () = 0 g () < 0... g r () < 0 The Functon could be: +, +y+y, ln()+e y, +m{,y}, etc Ynyu Ye, Stnford, MS&E Lecture Notes # 4

5 An Emple: Mmum Flow 6 3 A B 4 How much flow cn trvel from A to B, gven tht ech of the drected connectng routes hve flow lmts? 0 Ynyu Ye, Stnford, MS&E Lecture Notes #

6 Mmum Flow by Inspecton Ynyu Ye, Stnford, MS&E Lecture Notes # 6

7 An Emple: Mmum Flow 6 3 A B 4 0 Cut vlue from Source ste to Snk ste=7 Ynyu Ye, Stnford, MS&E Lecture Notes # 7

8 An Emple: Mmum Flow 6 3 A B 4 0 Cut vlue from Source ste to Snk ste=7 Ynyu Ye, Stnford, MS&E Lecture Notes # 8

9 An Emple: Mmum Flow 6 3 A B 4 0 Cut vlue from Source ste to Snk ste= Ynyu Ye, Stnford, MS&E Lecture Notes # 9

10 6 (k ) (k 3 ) (k 6 ) 6 m 6 outflow nflow s.t. k 0 k j k j j j,,3,4,,6, j,,3,4,,6 Ynyu Ye, Stnford, MS&E Lecture Notes # 0

11 3 Mn Ctegores n Optmzton Lner Optmzton (Progrmmng) Serch Algorthms Interor-Pont Algorthms Unconstrned Optmzton Constrned Optmzton Other Clssfctons: Qudrtc, Conve, Integer, Med-Integer, Bnry, etc. Ynyu Ye, Stnford, MS&E Lecture Notes #

12 Issues n Optmzton Problem Sze Smll by hnd Medum by softwre Lrge by decomposton Algorthm Complety Convergence Locl Convergence Tme Insght more thn just the soluton? Soluton structure propertes Senstvty nlyss Alternte formultons Ynyu Ye, Stnford, MS&E Lecture Notes #

13 Wht do you lern? Models the rt How we choose to represent rel problems Theory the scence Necessry nd Suffcent Condtons tht must be true for the optmlty of dfferent clsses of problems. Algorthms the tools How we pply the theory to robustly nd effcently solve problems nd gn nsght beyond the soluton. Ynyu Ye, Stnford, MS&E Lecture Notes # 3

14 Secton I: Lner Progrmmng Why do we study LP s Not becuse solvng non-lner problems re too dffcult But becuse rel-world problems re often formulted s lner equtons Ether becuse they ndeed re lner Or becuse t s uncler how to represent them nd lner s n ntutve compromse Ynyu Ye, Stnford, MS&E Lecture Notes # 4

15 LP, Nobel Prze, Ynyu Ye, Stnford, MS&E Lecture Notes #

16 nd Ntonl Medl of Scence Ynyu Ye, Stnford, MS&E Lecture Notes # 6

17 Art of Modelng & Vocbulry Decson Vrbles ϵ R n, yet to be decde Coeffcents, c ϵ R n, tht re gven nd fed Objectve nner product f =c T : R n R Constrnt Set X R n Fesble soluton ϵ X Optml soluton * ϵ X * Optml vlue z* = f ( * ) Ynyu Ye, Stnford, MS&E Lecture Notes # 7

18 LP Emple : Producton Mngement The Wyndor Glss Co. s producer of hgh-qulty glss products. It hs three plnts. Alumnum frmes nd hrdwre re mde n Plnt, wood frmes re mde n Plnt, nd Plnt 3 s used to produce glss nd ssemble the products. Wyndor produces two products whch requre the resources of the three plnts s follows: Plnt Alumnum Wood Resources Unt Proft $000 $000 m s.t.,,,., 0 Ynyu Ye, Stnford, MS&E Lecture Notes # 8

19 LP Emple : Trnsportton nd Assgnment Retler Retler Retler 3 Retler 4 SUPPLY Wrehouse (c ) (s ) Wrehouse (s ) Wrehouse (c 34 ) 800 (s 3 ) DEMAND 400 (d ) 900 (d ) 00 (d 3 ) 00 (d 4 ) 000 (s 4 ) mn 3 4 j c j j Abstrct Model s.t. 4 j 3 j j j 0, s d, j,,,3 j,,,3,4 j Ynyu Ye, Stnford, MS&E Lecture Notes # 9

20 LP Emple 3: Support Vector Mchne b j T 0 b T j 0 {y: y T + 0 = 0} s the norml drecton or slope vector nd 0 s the ntersect Ynyu Ye, Stnford, MS&E Lecture Notes # 0

21 LP Emple 3: Is Strct Seprton Possble Ynyu Ye, Stnford, MS&E Lecture Notes # j b T j T 0, 0, 0 0 Are there nd 0 such tht the followng (open) nequltes re ll stsfed Ths s specl LP, clled lner fesblty problem. j b T j T,, 0 0 j b T j T,, 0 0 Are there nd 0 such tht the followng nequltes re ll stsfed for rbtrrly smll ε. Dvde nd 0 by ε., the problem cn equvlently reformulted.

22 LP Emple 4: Electrc Vehcle Chrgng Schedule Perod Perod Perod 3 Perod 4 Perod Prce ($). (c ).3 (c ). (c 3 ).0 (c 4 ).0 (c ) Demnd (kw) 60 (d ) 0 (d ) 00 (d 3 ) 40 (d 4 ) 0 (d ) Chrgng (kw) 3 4 Inventory (I 0 ) I I I 3 I 4 I mn s.t. I I c 0, I d K, 0, I,.,,3,4,,,3,4, Ynyu Ye, Stnford, MS&E Lecture Notes #

23 LP Emple 4: When Dschrge s Allowed Perod Perod Perod 3 Perod 4 Perod Prce ($). (c ).3 (c ). (c 3 ).0 (c 4 ).0 (c ) Demnd (kw) 60 (d ) 0 (d ) 00 (d 3 ) 40 (d 4 ) 0 (d ) Chrgng (kw) 3 4 Inventory (I 0 ) I I I 3 I 4 I mn s.t. I I I c 0, d K,. I,,,3,4,,,3,4, Ynyu Ye, Stnford, MS&E Lecture Notes # 3

24 Lner Progrmmng Abstrcton Ynyu Ye, Stnford, MS&E Lecture Notes # s.t. m b b b c c, ,.,,, s.t. m 0. 0,.,, 0, 0 s.t. m

25 m (mn) s.t. Input j, : c Output,..., c : Abstrct Lner Progrmmng Model,..., m; j,..., n,constrnt left - hnd -sde tble or mtr coef.,..., m,objectve n c c m 0, coef.;, decson v rbles free, b,..., b m c n n n mn n n n n {,, {,, {,, } } } b b b, 0. n, constrnt rght - hnd -sde coef. m Ynyu Ye, Stnford, MS&E Lecture Notes #

26 LP n Compct Mtr Form A... m... m n n mn, b b b... b m, c c c... c n,... n Coeffcent mtr RHS vector Obj. vector decson vector m(mn) s.t. c T A {,, } b, {, }0 or free. Ynyu Ye, Stnford, MS&E Lecture Notes # 6

27 Some Fcts of Lner Progrmmng Add constnt to the objectve functon does not chnge the optmlty Scle the objectve coeffcents does not chnge the optmlty Scle the rght hnd sde coeffcents does not chnge the optmlty but the soluton scled ccordngly Reorder the decson vrbles (together wth ther correspondng objectve nd constrnt coeffcents) does not chnge the optmlty Reorder the constrnts (together wth ther rght hnd sde coeffcents) does not chnge the optmlty Multply both sdes of n equlty constrnt by constnt does not chnge the optmlty Pre multply both sdes of ll equlty constrnts by non sngulr mtr does not chnge the optmlty Ynyu Ye, Stnford, MS&E Lecture Notes # 7

28 Homework Before net clss red Chpters.,.,. nd. n the tet book Ynyu Ye, Stnford, MS&E Lecture Notes # 8