The General Nonlinear Constrained Optimization Problem
|
|
- Cathleen Pearson
- 5 years ago
- Views:
Transcription
1 St back, relax, and enjoy the rde of your lfe as we explore the condtons that enable us to clmb to the top of a concave functon or descend to the bottom of a convex functon whle constraned wthn a closed area. The General Nonlnear Constraned Optmzaton Problem The whole enchlada. Ths s t! At last the wat s over. The breathtakng adventure s about to begn. Dscover the key to successful optmzaton. Meet the explorers Kuhn and Tucker.
2 The Problem subj to : Max/Mn f( x, x,..., x ) g ( x, x,..., x ) 0, =,,..., m n h ( x, x,..., x ) = 0, k =,,..., K k n n
3 x 6 The Dffculty (,6) Mn f(x,x ) subj. to: x + x <= 8-3x + x <= 6 x, x >= (,) f(x,x ) = (x 8) + (x 4).(8,4) nfeasble.(,6) f(x,x ) = (x ) + (x ) x 3
4 More Dffcultes x objectve functon The feasble regon global mnmum local mnmum pont x 4
5 Non-convex feasble regons x 3 + x ( ) ( ) 3 5
6 Convex feasble regons x 3 + x ( ) ( ) feasble regon 6
7 Non-convex feasble regons x 3 + x ( ) ( ) 3 7
8 Convex feasble regons x 3 + x ( ) ( ) feasble regon 8
9 Convex Feasble Regons The feasble regon s convex f the lne segment between every par of feasble ponts falls entrely wthn the feasble regon. The feasble regon defned by: g ( x) 0 ; =,,..., m = s convex f () each g (x) >= constrant s concave () each g (x) <= constrant s convex (3) each g (x) = constrant s lnear 9
10 g(x) Concave g(x) C g(x) < C g(x) > C g(x) < C x 0
11 g(x) Convex g(x) C g(x) > C g(x) < C g(x) > C x
12 Equalty Constrant g(x) C g(x) = C x
13 A Useful Result If g(x) s convex (concave), then g(x) s concave (convex) g(x) -g(x) concave convex x x 3
14 Sngle Constrant K-T Condtons Max f(x) s.t. g(x) <= 0 where x = (x,x,,x n ) convert constrant to an equalty: g(x) + y = 0 Form Lagrangan: L(x,y,u) = f(x) - u [ g(x) + y ] necessary condtons: L f g = u = 0 j =,,..., n x x x j j j L = gx + y = u L = yu = 0 y ( ) 0 4
15 The Kuhn-Tucker Condtons L f g = u = 0 j =,,..., n x x x j j j L = gx + y = u L = yu = 0 y ( ) 0 gx ( ) 0 ug( x) = 0 statonary pont feasblty complementary slackness 5
16 The Lagrangan Multpler L f g = u = x x x j j j 0 f x u j g = u x f x g x j = = j j f g Max f(x) s.t. g(x) <= 0 u >= 0 for max 6
17 K-T Necessary Condtons L f g I. = u = 0 j =,,..., n x x x j j j II. gx ( ) 0 III. ug( x) = 0 IV. u 0 (max) u 0(mn) A set of values for x, x,,x n and u that satsfes all of the above condtons s referred to as a Kuhn-Tucker pont. 7
18 Our Very Frst Kuhn-Tucker Problem Max x 3x + 0x + 6x + 5 st.. 3x + 5x 9 0 [ ] Lx (, x, u) = x 3x + 0x+ 6x+ 5 u3x+ 5x 9 It must be wonderful to fnd a Kuhn-Tucker pont! 8
19 K-T Condtons [ ] Lx (, x, u) = x 3x + 0x+ 6x+ 5 u3x+ 5x 9 L I. = 4x + 0 3u = 0 x L x = 6x + 6 5u = 0 II. 3x + 5x 9 0 III. u 3x + 5x 9 = 0 IV. u 0 ( ) 9
20 assume u = 0: L I. = 4x + 0 = 0 x =.5 x L = 6x + 6= 0 x = x II. 3x + 5x 9 0 III. u 3x + 5x 9 = 0 IV. u 0 ( ) I., III. and IV. are satsfed. nfeasble snce 3 (.5) + 5 () 9 =.5-9 >0 0
21 assume u > 0: 4x + 0 3u = 0 6x + 6 5u = 0 3x + 5x 9 = 0 ( ) x 5-0x x - 8 =0 x (-3) -0x + 8x = -3 x (0/3) 0x + (00/3)x = 60 (-54/3) x = 8 x = 3 (8) /54 = 6/ x = (3 + 8 x )/0 = 3/ u = (-4x +0 ) /3 = 6/
22 L I. = 4x + 0 3u = 0 x L x = 6x + 6 5u = 0 II. 3x + 5x 9 0 x = 3/ x = 6/ u = 6/ III. u 3x + 5x 9 = 0 ( ) IV. u 0
23 Tell us about suffcency. The Kuhn-Tucker condtons are both necessary and suffcent f mnmzng a convex functon over a convex feasble regon or maxmzng a concave functon over a convex feasble regon. Wow! That s truly an amazng result! 3
24 Insect Infestatons Let x = the pounds of nsectcde allocated aganst the th acre of farmland; and d(x ) = k ln x s the number of nsects klled where k = relatve weght of the nsect nfestaton of acre d(x ) d(x ) = k ln x x Gven there are M pounds avalable and n acres. How many pounds should be allocated to each acre n order to maxmze the number of nsects klled? 4
25 The Formulaton n Max k ln x = n st.. x M = n n L( x, u) = k lnx u x M = = 5
26 The K-T Condtons L k I. = u = 0 x x II. n = x M III. n u x M = 0 = IV. u 0 n n L( x, u) = k lnx u x M = = 6
27 7 The K-T Condtons I. 0 II. III. 0 IV. 0 n n k L u x x x M u x M u = = = = = n n n k ux k u x um k u M = = = = = = = n km x k = =
28 f( x... x ) = k ln x What about Suffcency? n n = f( x... x ) k f( x... x ) k f( x... x ) = ; = ; = 0 n n n x x x x x x xj k x k 0 : 0 H( x... xn ) = x 0 0 : 0 kn x n Clearly, the matrx s negatve defnte. 8
29 The General K-T Problem Max/Mn f(x) s.t. g (x) <= 0 ; =,,,m h k (x) = 0 ; k =,,,K f ( x) g ( x) h ( x) I. u v = 0 j =,,..., n x x x m K k k j = j k= j II. g ( x) 0 ; =,,..., m hk ( x) = 0 ; k =,,..., K III. ug ( x) = 0 ; =,,..., m IV. u 0 (max) u 0 (mn) =,,..., m v unrestrcted k 9
30 An Excellent Applcaton A closed cylndrcal tank s beng desgned to carry at least 0 cubc feet of chemcals. Metal for the top and sdes costs $ per square foot, but the heaver metal of the base costs $8 per square foot. The heght of the tank can be no more than twce ts dameter to keep t from beng top heavy. Fnd a mnmum cost desgn. let x = dameter of tank x = heght of tank x x mn πxx + π + 8 π 4 4 st.. x π x 4 x x 0 x 0, x 0 (cost) (volume) (heght-to-dameter rato) 30
31 An Excellent Example Contnued x x mn πxx + π + 8 π 4 4 st.. x π x 4 x x 0 x 0, x 0 x x π relax non-negatvty constrants and form the Lagrangan functon. x x Lx (, x, u, u) = πxx + π + 8 π 4 4 x + u x u x x 4 π 0 [ ] 3
32 x x Lx (, x, u, u) = πxx + π + 8 π 4 4 L x = πx + πx+ 4( πx) + u π x + u = 0 x L x + u x u x x 4 x = + = 0 πx u π u x 4 π 0 [ ] 3
33 x I. πx + 5( πx) + u π x + u = 0 x πx+ u π u = 0 4 x II. π x 0 4 x x x III. u π x 0 = 0 4 [ ] u x x = 0 IV. u 0 u 0 33
34 Case : Assume u = u = 0 ( π ) I. πx + 5 x = 0 π x = 0 then x = x = 0 nfeasble! Case : Assume u <0; u = 0 x I. πx + 5( πx) + u π x = 0 x πx+ u π = 0 4 x III. π x 0 = 0 4 soluton: x =.677 x = snce x not x,, nfeasble 34
35 Case 3: Assume u = 0; u < 0 ( π ) I. πx + 5 x + u = 0 π x u = 0 III. x x = 0 then x = x ( π ) (4πx + 5 x + u = 0) + ( π x u = 0) 3π x = 0 or x = 0 and x = 0 35
36 Case 4: Assume u < 0; u < 0 x I. πx + 5( πx) + u π x + u = 0 x πx+ u π u = 0 4 x III. π x 0 = 0 4 [ x x ] = 0 soluton: x =.335 x = u = u =
37 A Kuhn-Tucker Problem Ique S. Lowe, an MSC grad student, must cram for fnal exams n OR I and Statstcs. The amount of nformaton absorbed s a functon of Ique s study tme. However, dmnshng returns are observed n both subjects. That s g(x) = 00x x and h(x) = 500y y 3 where g,h = nformaton absorbed n 000 s of neural bts and x = hours studyng OR y = hours studyng statstcs Ique has only 30 hours avalable to study. How many hours should be spent on each subject? 37
38 Max f(x,y) = 00x x + 500y y 3 s.t. x + y <= 30 L(x,y,u) = 00x x + 500y y 3 u(x + y 30) I. L x = 00 4x u = 0 L y = 500 3y u =0 II. x + y <= 30 III. u (x + y 30)=0 IV. u>=0 38
39 I. L x = 00 4x u = 0 L y = 500 3y u =0 II. x + y <= 30 III. u (x + y 30)=0 IV. u>=0 Assume u = 0 from I: x = 5, y = (500/3).5 =.9 nfeasble Assume u >0, then 00 4x u = y u = 0 x + y 30 = 0 x = 30 y x 3y = (30 y) 3y = 0 3y + 4y 50 = 0 y =.5 x = 30.5 = 7.48 u = 00 4x = 00 4(7.48) > 0 39
40 nd Order Condtons Max f(x,y) = 00x x + 500y y 3 s.t. x + y <= 30 f xx = -4 < 0 f yy = -6y f xy = 0 f xx f yy (f xy ) = 4y > 0 for y > 0 f(,y) s concave and snce the constrant s lnear, the feasble regon s convex. Therefore a K-T pont s both necessary and suffcent to solve the problem! 40
41 optmze wth the best of them never agan wll you be satsfed wth non-optmalty experence the peak and valleys that only nonlnear optmzaton can brng a certanty to be the pnnacle of your career mssng ths opportunty s surely a nadr Engneerng Management and Systems proudly presents Nonlnear Optmzaton (MSC 53) Comng soon sgn up now seatng s lmted MSC 53: Nonlnear Optmzaton Ketterng Labs Room 3, TTH, :30 a.m. :00 p.m.* Textbook: Engneerng Optmzaton: Methods and Applcatons (nd ed) by Ravndran, Ragsdell, Reklats, Wley credt hours *Class s also offered through dstance learnng. 4
Solutions to exam in SF1811 Optimization, Jan 14, 2015
Solutons to exam n SF8 Optmzaton, Jan 4, 25 3 3 O------O -4 \ / \ / The network: \/ where all lnks go from left to rght. /\ / \ / \ 6 O------O -5 2 4.(a) Let x = ( x 3, x 4, x 23, x 24 ) T, where the varable
More informationprinceton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg
prnceton unv. F 17 cos 521: Advanced Algorthm Desgn Lecture 7: LP Dualty Lecturer: Matt Wenberg Scrbe: LP Dualty s an extremely useful tool for analyzng structural propertes of lnear programs. Whle there
More informationConvex Optimization. Optimality conditions. (EE227BT: UC Berkeley) Lecture 9 (Optimality; Conic duality) 9/25/14. Laurent El Ghaoui.
Convex Optmzaton (EE227BT: UC Berkeley) Lecture 9 (Optmalty; Conc dualty) 9/25/14 Laurent El Ghaou Organsatonal Mdterm: 10/7/14 (1.5 hours, n class, double-sded cheat sheet allowed) Project: Intal proposal
More informationKernel Methods and SVMs Extension
Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationLagrange Multipliers Kernel Trick
Lagrange Multplers Kernel Trck Ncholas Ruozz Unversty of Texas at Dallas Based roughly on the sldes of Davd Sontag General Optmzaton A mathematcal detour, we ll come back to SVMs soon! subject to: f x
More informationMMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More informationChapter Newton s Method
Chapter 9. Newton s Method After readng ths chapter, you should be able to:. Understand how Newton s method s dfferent from the Golden Secton Search method. Understand how Newton s method works 3. Solve
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationWhich Separator? Spring 1
Whch Separator? 6.034 - Sprng 1 Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng 3 Margn of a pont " # y (w $ + b) proportonal
More informationThe Minimum Universal Cost Flow in an Infeasible Flow Network
Journal of Scences, Islamc Republc of Iran 17(2): 175-180 (2006) Unversty of Tehran, ISSN 1016-1104 http://jscencesutacr The Mnmum Unversal Cost Flow n an Infeasble Flow Network H Saleh Fathabad * M Bagheran
More informationEEL 6266 Power System Operation and Control. Chapter 3 Economic Dispatch Using Dynamic Programming
EEL 6266 Power System Operaton and Control Chapter 3 Economc Dspatch Usng Dynamc Programmng Pecewse Lnear Cost Functons Common practce many utltes prefer to represent ther generator cost functons as sngle-
More informationSupport Vector Machines. Vibhav Gogate The University of Texas at dallas
Support Vector Machnes Vbhav Gogate he Unversty of exas at dallas What We have Learned So Far? 1. Decson rees. Naïve Bayes 3. Lnear Regresson 4. Logstc Regresson 5. Perceptron 6. Neural networks 7. K-Nearest
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More informationCollege of Computer & Information Science Fall 2009 Northeastern University 20 October 2009
College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:
More information6) Derivatives, gradients and Hessian matrices
30C00300 Mathematcal Methods for Economsts (6 cr) 6) Dervatves, gradents and Hessan matrces Smon & Blume chapters: 14, 15 Sldes by: Tmo Kuosmanen 1 Outlne Defnton of dervatve functon Dervatve notatons
More informationSolutions HW #2. minimize. Ax = b. Give the dual problem, and make the implicit equality constraints explicit. Solution.
Solutons HW #2 Dual of general LP. Fnd the dual functon of the LP mnmze subject to c T x Gx h Ax = b. Gve the dual problem, and make the mplct equalty constrants explct. Soluton. 1. The Lagrangan s L(x,
More information14 Lagrange Multipliers
Lagrange Multplers 14 Lagrange Multplers The Method of Lagrange Multplers s a powerful technque for constraned optmzaton. Whle t has applcatons far beyond machne learnng t was orgnally developed to solve
More informationCHAPTER 6 CONSTRAINED OPTIMIZATION 1: K-T CONDITIONS
Chapter 6: Constraned Optzaton CHAPER 6 CONSRAINED OPIMIZAION : K- CONDIIONS Introducton We now begn our dscusson of gradent-based constraned optzaton. Recall that n Chapter 3 we looked at gradent-based
More information18-660: Numerical Methods for Engineering Design and Optimization
8-66: Numercal Methods for Engneerng Desgn and Optmzaton n L Department of EE arnege Mellon Unversty Pttsburgh, PA 53 Slde Overve lassfcaton Support vector machne Regularzaton Slde lassfcaton Predct categorcal
More informationFeature Selection: Part 1
CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?
More informationNatural Language Processing and Information Retrieval
Natural Language Processng and Informaton Retreval Support Vector Machnes Alessandro Moschtt Department of nformaton and communcaton technology Unversty of Trento Emal: moschtt@ds.untn.t Summary Support
More informationTHE ARIMOTO-BLAHUT ALGORITHM FOR COMPUTATION OF CHANNEL CAPACITY. William A. Pearlman. References: S. Arimoto - IEEE Trans. Inform. Thy., Jan.
THE ARIMOTO-BLAHUT ALGORITHM FOR COMPUTATION OF CHANNEL CAPACITY Wllam A. Pearlman 2002 References: S. Armoto - IEEE Trans. Inform. Thy., Jan. 1972 R. Blahut - IEEE Trans. Inform. Thy., July 1972 Recall
More informationAssortment Optimization under MNL
Assortment Optmzaton under MNL Haotan Song Aprl 30, 2017 1 Introducton The assortment optmzaton problem ams to fnd the revenue-maxmzng assortment of products to offer when the prces of products are fxed.
More informationCHAPTER 7 CONSTRAINED OPTIMIZATION 1: THE KARUSH-KUHN-TUCKER CONDITIONS
CHAPER 7 CONSRAINED OPIMIZAION : HE KARUSH-KUHN-UCKER CONDIIONS 7. Introducton We now begn our dscusson of gradent-based constraned optzaton. Recall that n Chapter 3 we looked at gradent-based unconstraned
More informationCHAPTER 7 CONSTRAINED OPTIMIZATION 2: SQP AND GRG
Chapter 7: Constraned Optmzaton CHAPER 7 CONSRAINED OPIMIZAION : SQP AND GRG Introducton In the prevous chapter we eamned the necessary and suffcent condtons for a constraned optmum. We dd not, however,
More informationPhysics 5153 Classical Mechanics. Principle of Virtual Work-1
P. Guterrez 1 Introducton Physcs 5153 Classcal Mechancs Prncple of Vrtual Work The frst varatonal prncple we encounter n mechancs s the prncple of vrtual work. It establshes the equlbrum condton of a mechancal
More informationFirst Year Examination Department of Statistics, University of Florida
Frst Year Examnaton Department of Statstcs, Unversty of Florda May 7, 010, 8:00 am - 1:00 noon Instructons: 1. You have four hours to answer questons n ths examnaton.. You must show your work to receve
More informationMath1110 (Spring 2009) Prelim 3 - Solutions
Math 1110 (Sprng 2009) Solutons to Prelm 3 (04/21/2009) 1 Queston 1. (16 ponts) Short answer. Math1110 (Sprng 2009) Prelm 3 - Solutons x a 1 (a) (4 ponts) Please evaluate lm, where a and b are postve numbers.
More information8/25/17. Data Modeling. Data Modeling. Data Modeling. Patrice Koehl Department of Biological Sciences National University of Singapore
8/5/17 Data Modelng Patrce Koehl Department of Bologcal Scences atonal Unversty of Sngapore http://www.cs.ucdavs.edu/~koehl/teachng/bl59 koehl@cs.ucdavs.edu Data Modelng Ø Data Modelng: least squares Ø
More informationPROBLEM SET 7 GENERAL EQUILIBRIUM
PROBLEM SET 7 GENERAL EQUILIBRIUM Queston a Defnton: An Arrow-Debreu Compettve Equlbrum s a vector of prces {p t } and allocatons {c t, c 2 t } whch satsfes ( Gven {p t }, c t maxmzes βt ln c t subject
More informationOn the Multicriteria Integer Network Flow Problem
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 5, No 2 Sofa 2005 On the Multcrtera Integer Network Flow Problem Vassl Vasslev, Marana Nkolova, Maryana Vassleva Insttute of
More informationLecture 10 Support Vector Machines. Oct
Lecture 10 Support Vector Machnes Oct - 20-2008 Lnear Separators Whch of the lnear separators s optmal? Concept of Margn Recall that n Perceptron, we learned that the convergence rate of the Perceptron
More informationLeast squares cubic splines without B-splines S.K. Lucas
Least squares cubc splnes wthout B-splnes S.K. Lucas School of Mathematcs and Statstcs, Unversty of South Australa, Mawson Lakes SA 595 e-mal: stephen.lucas@unsa.edu.au Submtted to the Gazette of the Australan
More information( ) 2 ( ) ( ) Problem Set 4 Suggested Solutions. Problem 1
Problem Set 4 Suggested Solutons Problem (A) The market demand functon s the soluton to the followng utlty-maxmzaton roblem (UMP): The Lagrangean: ( x, x, x ) = + max U x, x, x x x x st.. x + x + x y x,
More information8.1 Arc Length. What is the length of a curve? How can we approximate it? We could do it following the pattern we ve used before
.1 Arc Length hat s the length of a curve? How can we approxmate t? e could do t followng the pattern we ve used before Use a sequence of ncreasngly short segments to approxmate the curve: As the segments
More information1 Convex Optimization
Convex Optmzaton We wll consder convex optmzaton problems. Namely, mnmzaton problems where the objectve s convex (we assume no constrants for now). Such problems often arse n machne learnng. For example,
More information3.1 ML and Empirical Distribution
67577 Intro. to Machne Learnng Fall semester, 2008/9 Lecture 3: Maxmum Lkelhood/ Maxmum Entropy Dualty Lecturer: Amnon Shashua Scrbe: Amnon Shashua 1 In the prevous lecture we defned the prncple of Maxmum
More informationDesign and Analysis of Algorithms
Desgn and Analyss of Algorthms CSE 53 Lecture 4 Dynamc Programmng Junzhou Huang, Ph.D. Department of Computer Scence and Engneerng CSE53 Desgn and Analyss of Algorthms The General Dynamc Programmng Technque
More information15 Lagrange Multipliers
15 The Method of s a powerful technque for constraned optmzaton. Whle t has applcatons far beyond machne learnng t was orgnally developed to solve physcs equatons), t s used for several ey dervatons n
More informationSome modelling aspects for the Matlab implementation of MMA
Some modellng aspects for the Matlab mplementaton of MMA Krster Svanberg krlle@math.kth.se Optmzaton and Systems Theory Department of Mathematcs KTH, SE 10044 Stockholm September 2004 1. Consdered optmzaton
More informationWeek 5: Neural Networks
Week 5: Neural Networks Instructor: Sergey Levne Neural Networks Summary In the prevous lecture, we saw how we can construct neural networks by extendng logstc regresson. Neural networks consst of multple
More information2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification
E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationLecture 20: November 7
0-725/36-725: Convex Optmzaton Fall 205 Lecturer: Ryan Tbshran Lecture 20: November 7 Scrbes: Varsha Chnnaobreddy, Joon Sk Km, Lngyao Zhang Note: LaTeX template courtesy of UC Berkeley EECS dept. Dsclamer:
More informationHere is the rationale: If X and y have a strong positive relationship to one another, then ( x x) will tend to be positive when ( y y)
Secton 1.5 Correlaton In the prevous sectons, we looked at regresson and the value r was a measurement of how much of the varaton n y can be attrbuted to the lnear relatonshp between y and x. In ths secton,
More informationAmiri s Supply Chain Model. System Engineering b Department of Mathematics and Statistics c Odette School of Business
Amr s Supply Chan Model by S. Ashtab a,, R.J. Caron b E. Selvarajah c a Department of Industral Manufacturng System Engneerng b Department of Mathematcs Statstcs c Odette School of Busness Unversty of
More informationAn Interactive Optimisation Tool for Allocation Problems
An Interactve Optmsaton ool for Allocaton Problems Fredr Bonäs, Joam Westerlund and apo Westerlund Process Desgn Laboratory, Faculty of echnology, Åbo Aadem Unversty, uru 20500, Fnland hs paper presents
More informationarxiv: v1 [math.oc] 3 Aug 2010
arxv:1008.0549v1 math.oc] 3 Aug 2010 Test Problems n Optmzaton Xn-She Yang Department of Engneerng, Unversty of Cambrdge, Cambrdge CB2 1PZ, UK Abstract Test functons are mportant to valdate new optmzaton
More informationTHE SUMMATION NOTATION Ʃ
Sngle Subscrpt otaton THE SUMMATIO OTATIO Ʃ Most of the calculatons we perform n statstcs are repettve operatons on lsts of numbers. For example, we compute the sum of a set of numbers, or the sum of the
More informationMath 217 Fall 2013 Homework 2 Solutions
Math 17 Fall 013 Homework Solutons Due Thursday Sept. 6, 013 5pm Ths homework conssts of 6 problems of 5 ponts each. The total s 30. You need to fully justfy your answer prove that your functon ndeed has
More informationThe exam is closed book, closed notes except your one-page cheat sheet.
CS 89 Fall 206 Introducton to Machne Learnng Fnal Do not open the exam before you are nstructed to do so The exam s closed book, closed notes except your one-page cheat sheet Usage of electronc devces
More informationConic Programming in GAMS
Conc Programmng n GAMS Armn Pruessner, Mchael Busseck, Steven Drkse, Ale Meeraus GAMS Development Corporaton INFORMS 003, Atlanta October 19- Drecton What ths talk s about Overvew: the class of conc programs
More informationProblem Do any of the following determine homomorphisms from GL n (C) to GL n (C)?
Homework 8 solutons. Problem 16.1. Whch of the followng defne homomomorphsms from C\{0} to C\{0}? Answer. a) f 1 : z z Yes, f 1 s a homomorphsm. We have that z s the complex conjugate of z. If z 1,z 2
More informationEEE 241: Linear Systems
EEE : Lnear Systems Summary #: Backpropagaton BACKPROPAGATION The perceptron rule as well as the Wdrow Hoff learnng were desgned to tran sngle layer networks. They suffer from the same dsadvantage: they
More informationPHYS 705: Classical Mechanics. Calculus of Variations II
1 PHYS 705: Classcal Mechancs Calculus of Varatons II 2 Calculus of Varatons: Generalzaton (no constrant yet) Suppose now that F depends on several dependent varables : We need to fnd such that has a statonary
More informationP R. Lecture 4. Theory and Applications of Pattern Recognition. Dept. of Electrical and Computer Engineering /
Theory and Applcatons of Pattern Recognton 003, Rob Polkar, Rowan Unversty, Glassboro, NJ Lecture 4 Bayes Classfcaton Rule Dept. of Electrcal and Computer Engneerng 0909.40.0 / 0909.504.04 Theory & Applcatons
More informationVery Large Scale Continuous and Discrete Variable. Woptimization,
Very Large Scale Contnuous and Dscrete Varable Optmzaton Garret N. Vanderplaats * Vanderplaats Research & Development, Inc. 1767 S. 8 th Street Colorado Sprngs, CO 80906 An optmzaton algorthm s presented
More informationUNIVERSITY OF TORONTO Faculty of Arts and Science. December 2005 Examinations STA437H1F/STA1005HF. Duration - 3 hours
UNIVERSITY OF TORONTO Faculty of Arts and Scence December 005 Examnatons STA47HF/STA005HF Duraton - hours AIDS ALLOWED: (to be suppled by the student) Non-programmable calculator One handwrtten 8.5'' x
More informationECE559VV Project Report
ECE559VV Project Report (Supplementary Notes Loc Xuan Bu I. MAX SUM-RATE SCHEDULING: THE UPLINK CASE We have seen (n the presentaton that, for downlnk (broadcast channels, the strategy maxmzng the sum-rate
More informationEconomics 101. Lecture 4 - Equilibrium and Efficiency
Economcs 0 Lecture 4 - Equlbrum and Effcency Intro As dscussed n the prevous lecture, we wll now move from an envronment where we looed at consumers mang decsons n solaton to analyzng economes full of
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationTwo-Way and Multiple-Access Energy Harvesting Systems with Energy Cooperation
Two-Way and Multple-Access Energy Harvestng Systems wth Energy Cooperaton Berk Gurakan, Omur Ozel, Jng Yang 2, and Sennur Ulukus Department of Electrcal and Computer Engneerng, Unversty of Maryland, College
More informationCOS 521: Advanced Algorithms Game Theory and Linear Programming
COS 521: Advanced Algorthms Game Theory and Lnear Programmng Moses Charkar February 27, 2013 In these notes, we ntroduce some basc concepts n game theory and lnear programmng (LP). We show a connecton
More informationSupport Vector Machines
CS 2750: Machne Learnng Support Vector Machnes Prof. Adrana Kovashka Unversty of Pttsburgh February 17, 2016 Announcement Homework 2 deadlne s now 2/29 We ll have covered everythng you need today or at
More informationMaximal Margin Classifier
CS81B/Stat41B: Advanced Topcs n Learnng & Decson Makng Mamal Margn Classfer Lecturer: Mchael Jordan Scrbes: Jana van Greunen Corrected verson - /1/004 1 References/Recommended Readng 1.1 Webstes www.kernel-machnes.org
More informationEstimation: Part 2. Chapter GREG estimation
Chapter 9 Estmaton: Part 2 9. GREG estmaton In Chapter 8, we have seen that the regresson estmator s an effcent estmator when there s a lnear relatonshp between y and x. In ths chapter, we generalzed the
More informationModule 9. Lecture 6. Duality in Assignment Problems
Module 9 1 Lecture 6 Dualty n Assgnment Problems In ths lecture we attempt to answer few other mportant questons posed n earler lecture for (AP) and see how some of them can be explaned through the concept
More informationSupport Vector Machines. Jie Tang Knowledge Engineering Group Department of Computer Science and Technology Tsinghua University 2012
Support Vector Machnes Je Tang Knowledge Engneerng Group Department of Computer Scence and Technology Tsnghua Unversty 2012 1 Outlne What s a Support Vector Machne? Solvng SVMs Kernel Trcks 2 What s a
More informationGame Theory. Lecture Notes By Y. Narahari. Department of Computer Science and Automation Indian Institute of Science Bangalore, India February 2008
Game Theory Lecture Notes By Y. Narahar Department of Computer Scence and Automaton Indan Insttute of Scence Bangalore, Inda February 2008 Chapter 10: Two Person Zero Sum Games Note: Ths s a only a draft
More information6.854J / J Advanced Algorithms Fall 2008
MIT OpenCourseWare http://ocw.mt.edu 6.854J / 18.415J Advanced Algorthms Fall 2008 For nformaton about ctng these materals or our Terms of Use, vst: http://ocw.mt.edu/terms. 18.415/6.854 Advanced Algorthms
More informationChapter 6 Support vector machine. Séparateurs à vaste marge
Chapter 6 Support vector machne Séparateurs à vaste marge Méthode de classfcaton bnare par apprentssage Introdute par Vladmr Vapnk en 1995 Repose sur l exstence d un classfcateur lnéare Apprentssage supervsé
More informationVQ widely used in coding speech, image, and video
at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng
More informationGlobal Optimization of Truss. Structure Design INFORMS J. N. Hooker. Tallys Yunes. Slide 1
Slde 1 Global Optmzaton of Truss Structure Desgn J. N. Hooker Tallys Yunes INFORMS 2010 Truss Structure Desgn Select sze of each bar (possbly zero) to support the load whle mnmzng weght. Bar szes are dscrete.
More informationProseminar Optimierung II. Victor A. Kovtunenko SS 2012/2013: LV
Prosemnar Optmerung II Vctor A. Kovtunenko Insttute for Mathematcs and Scentfc Computng, Karl-Franzens Unversty of Graz, Henrchstr. 36, 8010 Graz, Austra; Lavrent ev Insttute of Hydrodynamcs, Sberan Dvson
More informationPlease review the following statement: I certify that I have not given unauthorized aid nor have I received aid in the completion of this exam.
Please revew the followng statement: I certfy that I have not gven unauthorzed ad nor have I receved ad n the completon of ths exam. Sgnature: Instructor s Name and Secton: (Crcle Your Secton) Sectons:
More informationFIRST AND SECOND ORDER NECESSARY OPTIMALITY CONDITIONS FOR DISCRETE OPTIMAL CONTROL PROBLEMS
Yugoslav Journal of Operatons Research 6 (6), umber, 53-6 FIRST D SECOD ORDER ECESSRY OPTIMLITY CODITIOS FOR DISCRETE OPTIML COTROL PROBLEMS Boban MRIKOVIĆ Faculty of Mnng and Geology, Unversty of Belgrade
More informationThe Geometry of Logit and Probit
The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.
More informationSTATIC OPTIMIZATION: BASICS
STATIC OPTIMIZATION: BASICS 7A- Lecture Overvew What s optmzaton? What applcatons? How can optmzaton be mplemented? How can optmzaton problems be solved? Why should optmzaton apply n human movement? How
More informationAppendix for Causal Interaction in Factorial Experiments: Application to Conjoint Analysis
A Appendx for Causal Interacton n Factoral Experments: Applcaton to Conjont Analyss Mathematcal Appendx: Proofs of Theorems A. Lemmas Below, we descrbe all the lemmas, whch are used to prove the man theorems
More informationStatistics II Final Exam 26/6/18
Statstcs II Fnal Exam 26/6/18 Academc Year 2017/18 Solutons Exam duraton: 2 h 30 mn 1. (3 ponts) A town hall s conductng a study to determne the amount of leftover food produced by the restaurants n the
More informationA 2D Bounded Linear Program (H,c) 2D Linear Programming
A 2D Bounded Lnear Program (H,c) h 3 v h 8 h 5 c h 4 h h 6 h 7 h 2 2D Lnear Programmng C s a polygonal regon, the ntersecton of n halfplanes. (H, c) s nfeasble, as C s empty. Feasble regon C s unbounded
More informationAnalytical Chemistry Calibration Curve Handout
I. Quck-and Drty Excel Tutoral Analytcal Chemstry Calbraton Curve Handout For those of you wth lttle experence wth Excel, I ve provded some key technques that should help you use the program both for problem
More informationSELECTED SOLUTIONS, SECTION (Weak duality) Prove that the primal and dual values p and d defined by equations (4.3.2) and (4.3.3) satisfy p d.
SELECTED SOLUTIONS, SECTION 4.3 1. Weak dualty Prove that the prmal and dual values p and d defned by equatons 4.3. and 4.3.3 satsfy p d. We consder an optmzaton problem of the form The Lagrangan for ths
More informationComputing Correlated Equilibria in Multi-Player Games
Computng Correlated Equlbra n Mult-Player Games Chrstos H. Papadmtrou Presented by Zhanxang Huang December 7th, 2005 1 The Author Dr. Chrstos H. Papadmtrou CS professor at UC Berkley (taught at Harvard,
More informationCME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 13
CME 30: NUMERICAL LINEAR ALGEBRA FALL 005/06 LECTURE 13 GENE H GOLUB 1 Iteratve Methods Very large problems (naturally sparse, from applcatons): teratve methods Structured matrces (even sometmes dense,
More informationCALCULUS CLASSROOM CAPSULES
CALCULUS CLASSROOM CAPSULES SESSION S86 Dr. Sham Alfred Rartan Valley Communty College salfred@rartanval.edu 38th AMATYC Annual Conference Jacksonvlle, Florda November 8-, 202 2 Calculus Classroom Capsules
More informationCathy Walker March 5, 2010
Cathy Walker March 5, 010 Part : Problem Set 1. What s the level of measurement for the followng varables? a) SAT scores b) Number of tests or quzzes n statstcal course c) Acres of land devoted to corn
More informationLecture 3: Dual problems and Kernels
Lecture 3: Dual problems and Kernels C4B Machne Learnng Hlary 211 A. Zsserman Prmal and dual forms Lnear separablty revsted Feature mappng Kernels for SVMs Kernel trck requrements radal bass functons SVM
More informationPlease review the following statement: I certify that I have not given unauthorized aid nor have I received aid in the completion of this exam.
ME 270 Fall 2013 Fnal Exam NME (Last, Frst): Please revew the followng statement: I certfy that I have not gven unauthorzed ad nor have I receved ad n the completon of ths exam. Sgnature: INSTRUCTIONS
More informationCS4495/6495 Introduction to Computer Vision. 3C-L3 Calibrating cameras
CS4495/6495 Introducton to Computer Vson 3C-L3 Calbratng cameras Fnally (last tme): Camera parameters Projecton equaton the cumulatve effect of all parameters: M (3x4) f s x ' 1 0 0 0 c R 0 I T 3 3 3 x1
More informationSupport Vector Machines
Support Vector Machnes Konstantn Tretyakov (kt@ut.ee) MTAT.03.227 Machne Learnng So far Supervsed machne learnng Lnear models Least squares regresson Fsher s dscrmnant, Perceptron, Logstc model Non-lnear
More informationDesign and Optimization of Fuzzy Controller for Inverse Pendulum System Using Genetic Algorithm
Desgn and Optmzaton of Fuzzy Controller for Inverse Pendulum System Usng Genetc Algorthm H. Mehraban A. Ashoor Unversty of Tehran Unversty of Tehran h.mehraban@ece.ut.ac.r a.ashoor@ece.ut.ac.r Abstract:
More informationAdvanced Topics in Optimization. Piecewise Linear Approximation of a Nonlinear Function
Advanced Tocs n Otmzaton Pecewse Lnear Aroxmaton of a Nonlnear Functon Otmzaton Methods: M8L Introducton and Objectves Introducton There exsts no general algorthm for nonlnear rogrammng due to ts rregular
More informationSupporting Information for: Two Monetary Models with Alternating Markets
Supportng Informaton for: Two Monetary Models wth Alternatng Markets Gabrele Camera Chapman Unversty & Unversty of Basel YL Chen St. Lous Fed November 2015 1 Optmal choces n the CIA model On date t, gven
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Exerments-I MODULE III LECTURE - 2 EXPERIMENTAL DESIGN MODELS Dr. Shalabh Deartment of Mathematcs and Statstcs Indan Insttute of Technology Kanur 2 We consder the models
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationYong Joon Ryang. 1. Introduction Consider the multicommodity transportation problem with convex quadratic cost function. 1 2 (x x0 ) T Q(x x 0 )
Kangweon-Kyungk Math. Jour. 4 1996), No. 1, pp. 7 16 AN ITERATIVE ROW-ACTION METHOD FOR MULTICOMMODITY TRANSPORTATION PROBLEMS Yong Joon Ryang Abstract. The optmzaton problems wth quadratc constrants often
More informationLecture 14: Bandits with Budget Constraints
IEOR 8100-001: Learnng and Optmzaton for Sequental Decson Makng 03/07/16 Lecture 14: andts wth udget Constrants Instructor: Shpra Agrawal Scrbed by: Zhpeng Lu 1 Problem defnton In the regular Mult-armed
More informationxp(x µ) = 0 p(x = 0 µ) + 1 p(x = 1 µ) = µ
CSE 455/555 Sprng 2013 Homework 7: Parametrc Technques Jason J. Corso Computer Scence and Engneerng SUY at Buffalo jcorso@buffalo.edu Solutons by Yngbo Zhou Ths assgnment does not need to be submtted and
More information