Convex Optimization. Optimality conditions. (EE227BT: UC Berkeley) Lecture 9 (Optimality; Conic duality) 9/25/14. Laurent El Ghaoui.
|
|
- Eustace Leonard
- 5 years ago
- Views:
Transcription
1 Convex Optmzaton (EE227BT: UC Berkeley) Lecture 9 (Optmalty; Conc dualty) 9/25/14 Laurent El Ghaou Organsatonal Mdterm: 10/7/14 (1.5 hours, n class, double-sded cheat sheet allowed) Project: Intal proposal by 10/28/14 Project mdpont revew: 11/4/14 Project fnal report: Fnals week Optmalty condtons Optmalty condtons mn f 0 (x) f (x) 0, = 1,..., m. Recall: f 0 (x ), x x 0 for all feasble x X Can we smplfy ths usng Lagrangan? g(λ) = nf x L(x, λ) := f 0 (x) + λ f (x) Assume strong dualty and that both p and d attaned! Thus, there exsts a par (x, λ ) such that p = f 0 (x ) = d = g(λ ) = mn L(x, λ ) L(x, λ ) f 0 (x ) = p x Thus, equaltes hold n above chan. x argmn x L(x, λ ).
2 Optmalty condtons x argmn x L(x, λ ). If f 0, f 1,..., f m are dfferentable, ths mples x L(x, λ ) x=x = f 0 (x ) + λ f (x ) = 0. Moreover, snce L(x, λ ) = f 0 (x ), we also have λ f (x ) = 0. But λ 0 and f (x ) 0, so complementary slackness λ f (x ) = 0, = 1,..., m. KKT Optmalty condtons Karush-Kuhn-Tucker Condtons (KKT) f (x ) 0, = 1,..., m (prmal feasblty) λ 0, = 1,..., m (dual feasblty) λ f (x ) = 0, = 1,..., m (compl. slackness) x L(x, λ ) x=x = 0 (Lagrangan statonarty) We showed: f strong dualty holds, and (x, λ ) exst, then KKT condtons are necessary for par (x, λ ) to be optmal If problem s convex, then KKT also suffcent Exercse: Prove the above suffcency of KKT. Hnt: Use that L(x, λ ) s convex, and conclude from KKT condtons that g(λ ) = f 0 (x ), so that (x, λ ) optmal prmal-dual par. Read Ch. 5 of BV Mnmax Example: Lasso-lke problem p := mn x Ax b 2 + λ x 1. x 1 = max x T v v 1 x 2 = max x T u u 2 1. p = mn max x u,v = max mn u,v x = max u,v = max u Saddle-pont formulaton u T (b Ax) + v T x u 2 1, u T (b Ax) + x T v u 2 1, u T b, A T u = v, u 2 1, v λ u T b, u 2 1, A T u λ. v λ v λ
3 Mnmax problems Mnmax theory treats problems nvolvng a combnaton of mnmzaton and maxmzaton Let X and Y be arbtrary nonempty sets Let φ : X Y R ± nf over y Y, followed by sup over x X sup nf φ(x, y) = sup ψ(y(x)) sup over x X, followed by nf over y Y nf sup φ(x, y) = nf ξ(x(y)) When are nf sup and sup nf equal? Weak mnmax Theorem Let φ : X Y R ± be any functon. Then, Proof: sup nf φ(x, y) nf x, y, x, y, x, sup φ(x, y) nf φ(x, y) φ(x, y) x X nf φ(x, y) sup φ(x, y ) x X y Y sup nf φ(x, y) sup φ(x, y ) x X y Y = sup nf φ(x, y) nf x X sup y Y φ(x, y ). Exercse: Show that weak dualty s follows from above mnmax nequalty. Hnt: Use φ = L (Lagrangan), and sutably choose y. Strong mnmax If nf sup equals sup nf, common value called saddle-value Value exsts f there s a saddle-pont,.e., par (x, y ) φ(x, y ) φ(x, y ) φ(x, y) for all x X, y Y. Exercse: Verfy above nequalty! Strong mnmax Classes of problems dual to each other can be generated by studyng classes of functons φ, More nterestng queston: Startng from the prmal problem over X, how to ntroduce a space Y and a useful functon φ on X Y so that we have a saddle-pont? Suffcent condtons for saddle-pont Functon φ s contnuous, and It s convex-concave (φ(, y) convex for every y Y, and φ(x, ) concave for every x X ), and Both X and Y are convex; one of them s compact.
4 Strong mnmax Def. Let φ be as before. A pont (x, y ) s a saddle-pont of φ (mn over X and max over Y) ff the nfmum n the expresson nf sup φ(x, y) s attaned at x, and the supremum n the expresson sup nf φ(x, y) s attaned at y, and these two extrema are equal. x argmn max φ(x, y) y argmax mn φ(x, y). x argmn Optmalty va mnmax max φ(x, y) y Pont (x, y ) s a saddle-pont f and only f argmax mn φ(x, y). 0 φ(x, y ) = x φ(x, y ) y φ(x, y ) When φ s of convex-concave form, yelds KKT condtons. Conc dualty Consder lnear program LP Dualty Correspondng dual s mn c T x Ax b. max b T λ A T λ + c = 0, λ 0. LP dualty facts: If ether p or d fnte, then p = d, and both prmal, dual problem have optmal solutons If p =, then d = (follows from weak-dualty) If d =, then p = (agan, weak-dualty) Proof: See lecture notes. If LP s feasble, strong dualty holds.
5 Consder SOCP SOCP Dualty mn f T x A x + b 2 c T x + d, = 1,..., m. Lagrangan (ordnary) L(x, λ) := f T x + λ ( A x + b 2 c T x + d ) Recall that x 2 = sup u T x u 2 1. λ A x + b 2 = max u (λ u ) T (A x + b ) u 2 1 = max v v T (A x + b ) v 2 λ Thus, wth v 1,..., v m also as dual varables we have p = nf x sup f T x + v T λ,v 1,...,v m (A x + b ) λ (c T x + d ) s.t. v 2 λ, = 1,..., m. The dual problem s SOCP Dualty d = sup nf λ,v 1,...,v m x f T x + v T (A x + b ) λ (c T x + d ) s.t. v 2 λ, = 1,..., m. Inner mnmzaton over x very easy (unconstraned) f + AT v λ c = 0 Dual problem becomes d = sup λ T d + v T λ,v 1,...,v m s.t. f + AT v λ c = 0, v 2 λ, = 1,..., m. Also an SOCP, lke the prmal Apply Slater to obtan a condton for strong dualty. b SDP prmal form SDP dualty p := mn Tr(CX ), s.t. Tr(A X ) = b, = 1,..., m, X 0. How to handle the matrx constrant X 0? Introduce conc Lagrangan L(X, ν, Y ) := Tr(CX ) + ν (Tr(A X ) b ) Tr(YX ) where we have a matrx dual varable Y 0. Note: Tr(YX ) 0; so p sup ν,y L(X, ν, Y ) for any feasble X As before, p d := sup ν,y 0 nf X L(X, ν, Y ) Smplfyng nf X L, we obtan dual functon b T ν f C g(ν, Y ) = ν A Y = 0, otherwse. SDP Dualty Dual problem max ν,y 0 max ν b T ν s.t. C ν A = Y 0 b T ν s.t. ν A C. Ths s the conc form we saw n Lecture 5! Weak-dualty: Tr(CX ) ν T b for any feasble par (X, ν) Strong-dualty: If prmal strctly feasble,x 0 such that Tr(A X ) = b, for = 1,..., m, we have strong dualty. Alternatvely, f dual strctly feasble, we have strong dualty. But, contrary to LPs, feasblty alone does not suffce!
6 Example: falure of strong dualty Prmal problem x p = mn x 2 0 x 1 x 2 0. X 0 x 2 0 [ ] x1 x Any prmal feasble requres 2 0; x x and x Thus, we have x 2 = 0, whereby p = 0. Prmal obj: Tr(CX ) wth c 23 = c 32 = 1/2 (rest zeros). Lagrangan: Tr([C X ] T Y ) Example: falure of strong dualty Tr([C X ] T Y ) = (x 2 + 1)y 11 x 1 y 22 + x 2 2x 2 y 23 = y 11 x 1 y 22 + x 2 x 2 y 11 2x 2 y 23. g(y ) = nf X 0 Tr([C X ]T Y ) = Dual functon y 11 y 22 = 0, 1 y 11 2y 23 = 0 Dual SDP otherwse. d = max Y 0 y 11, y 22 = 0, 1 y 11 2y 23 = 0. Any feasble Y satsfes, y 23 = 0 (snce y 22 = 0) Thus y 11 = 1, so d = 1. dualty gap: p d = 1 S. Boyd, EE364b Sldes References Bertsekas, Nonlnear Programmng
Solutions HW #2. minimize. Ax = b. Give the dual problem, and make the implicit equality constraints explicit. Solution.
Solutons HW #2 Dual of general LP. Fnd the dual functon of the LP mnmze subject to c T x Gx h Ax = b. Gve the dual problem, and make the mplct equalty constrants explct. Soluton. 1. The Lagrangan s L(x,
More informationLagrange Multipliers Kernel Trick
Lagrange Multplers Kernel Trck Ncholas Ruozz Unversty of Texas at Dallas Based roughly on the sldes of Davd Sontag General Optmzaton A mathematcal detour, we ll come back to SVMs soon! subject to: f x
More informationSELECTED SOLUTIONS, SECTION (Weak duality) Prove that the primal and dual values p and d defined by equations (4.3.2) and (4.3.3) satisfy p d.
SELECTED SOLUTIONS, SECTION 4.3 1. Weak dualty Prove that the prmal and dual values p and d defned by equatons 4.3. and 4.3.3 satsfy p d. We consder an optmzaton problem of the form The Lagrangan for ths
More informationprinceton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg
prnceton unv. F 17 cos 521: Advanced Algorthm Desgn Lecture 7: LP Dualty Lecturer: Matt Wenberg Scrbe: LP Dualty s an extremely useful tool for analyzng structural propertes of lnear programs. Whle there
More informationMMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More informationSolutions to exam in SF1811 Optimization, Jan 14, 2015
Solutons to exam n SF8 Optmzaton, Jan 4, 25 3 3 O------O -4 \ / \ / The network: \/ where all lnks go from left to rght. /\ / \ / \ 6 O------O -5 2 4.(a) Let x = ( x 3, x 4, x 23, x 24 ) T, where the varable
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More information10-801: Advanced Optimization and Randomized Methods Lecture 2: Convex functions (Jan 15, 2014)
0-80: Advanced Optmzaton and Randomzed Methods Lecture : Convex functons (Jan 5, 04) Lecturer: Suvrt Sra Addr: Carnege Mellon Unversty, Sprng 04 Scrbes: Avnava Dubey, Ahmed Hefny Dsclamer: These notes
More information6.854J / J Advanced Algorithms Fall 2008
MIT OpenCourseWare http://ocw.mt.edu 6.854J / 18.415J Advanced Algorthms Fall 2008 For nformaton about ctng these materals or our Terms of Use, vst: http://ocw.mt.edu/terms. 18.415/6.854 Advanced Algorthms
More informationModule 9. Lecture 6. Duality in Assignment Problems
Module 9 1 Lecture 6 Dualty n Assgnment Problems In ths lecture we attempt to answer few other mportant questons posed n earler lecture for (AP) and see how some of them can be explaned through the concept
More informationProseminar Optimierung II. Victor A. Kovtunenko SS 2012/2013: LV
Prosemnar Optmerung II Vctor A. Kovtunenko Insttute for Mathematcs and Scentfc Computng, Karl-Franzens Unversty of Graz, Henrchstr. 36, 8010 Graz, Austra; Lavrent ev Insttute of Hydrodynamcs, Sberan Dvson
More informationSome modelling aspects for the Matlab implementation of MMA
Some modellng aspects for the Matlab mplementaton of MMA Krster Svanberg krlle@math.kth.se Optmzaton and Systems Theory Department of Mathematcs KTH, SE 10044 Stockholm September 2004 1. Consdered optmzaton
More informationAssortment Optimization under MNL
Assortment Optmzaton under MNL Haotan Song Aprl 30, 2017 1 Introducton The assortment optmzaton problem ams to fnd the revenue-maxmzng assortment of products to offer when the prces of products are fxed.
More informationLecture 20: Lift and Project, SDP Duality. Today we will study the Lift and Project method. Then we will prove the SDP duality theorem.
prnceton u. sp 02 cos 598B: algorthms and complexty Lecture 20: Lft and Project, SDP Dualty Lecturer: Sanjeev Arora Scrbe:Yury Makarychev Today we wll study the Lft and Project method. Then we wll prove
More informationCollege of Computer & Information Science Fall 2009 Northeastern University 20 October 2009
College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationCOS 521: Advanced Algorithms Game Theory and Linear Programming
COS 521: Advanced Algorthms Game Theory and Lnear Programmng Moses Charkar February 27, 2013 In these notes, we ntroduce some basc concepts n game theory and lnear programmng (LP). We show a connecton
More informationThe General Nonlinear Constrained Optimization Problem
St back, relax, and enjoy the rde of your lfe as we explore the condtons that enable us to clmb to the top of a concave functon or descend to the bottom of a convex functon whle constraned wthn a closed
More informationSupport Vector Machines. Jie Tang Knowledge Engineering Group Department of Computer Science and Technology Tsinghua University 2012
Support Vector Machnes Je Tang Knowledge Engneerng Group Department of Computer Scence and Technology Tsnghua Unversty 2012 1 Outlne What s a Support Vector Machne? Solvng SVMs Kernel Trcks 2 What s a
More informationMaximal Margin Classifier
CS81B/Stat41B: Advanced Topcs n Learnng & Decson Makng Mamal Margn Classfer Lecturer: Mchael Jordan Scrbes: Jana van Greunen Corrected verson - /1/004 1 References/Recommended Readng 1.1 Webstes www.kernel-machnes.org
More informationSupplement: Proofs and Technical Details for The Solution Path of the Generalized Lasso
Supplement: Proofs and Techncal Detals for The Soluton Path of the Generalzed Lasso Ryan J. Tbshran Jonathan Taylor In ths document we gve supplementary detals to the paper The Soluton Path of the Generalzed
More informationThe Second Anti-Mathima on Game Theory
The Second Ant-Mathma on Game Theory Ath. Kehagas December 1 2006 1 Introducton In ths note we wll examne the noton of game equlbrum for three types of games 1. 2-player 2-acton zero-sum games 2. 2-player
More informationAdvanced Introduction to Machine Learning Homework 1 Solutions October 6, 2014
10-715 Advanced Introducton to Machne Learnng Homewor 1 Solutons October 6, 2014 1 Regresson Samy 1.1 Mult-Tas Regreson 1. The Cost functon can be decomposed as, J 0 Θ Y j,: Θ j,: X 2 j1 where Y j,:, Θ
More informationp(z) = 1 a e z/a 1(z 0) yi a i x (1/a) exp y i a i x a i=1 n i=1 (y i a i x) inf 1 (y Ax) inf Ax y (1 ν) y if A (1 ν) = 0 otherwise
Dustn Lennon Math 582 Convex Optmzaton Problems from Boy, Chapter 7 Problem 7.1 Solve the MLE problem when the nose s exponentally strbute wth ensty p(z = 1 a e z/a 1(z 0 The MLE s gven by the followng:
More informationPerfect Competition and the Nash Bargaining Solution
Perfect Competton and the Nash Barganng Soluton Renhard John Department of Economcs Unversty of Bonn Adenauerallee 24-42 53113 Bonn, Germany emal: rohn@un-bonn.de May 2005 Abstract For a lnear exchange
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationMixed Taxation and Production Efficiency
Floran Scheuer 2/23/2016 Mxed Taxaton and Producton Effcency 1 Overvew 1. Unform commodty taxaton under non-lnear ncome taxaton Atknson-Stgltz (JPubE 1976) Theorem Applcaton to captal taxaton 2. Unform
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationFIRST AND SECOND ORDER NECESSARY OPTIMALITY CONDITIONS FOR DISCRETE OPTIMAL CONTROL PROBLEMS
Yugoslav Journal of Operatons Research 6 (6), umber, 53-6 FIRST D SECOD ORDER ECESSRY OPTIMLITY CODITIOS FOR DISCRETE OPTIML COTROL PROBLEMS Boban MRIKOVIĆ Faculty of Mnng and Geology, Unversty of Belgrade
More informationNatural Language Processing and Information Retrieval
Natural Language Processng and Informaton Retreval Support Vector Machnes Alessandro Moschtt Department of nformaton and communcaton technology Unversty of Trento Emal: moschtt@ds.untn.t Summary Support
More informationLecture 20: November 7
0-725/36-725: Convex Optmzaton Fall 205 Lecturer: Ryan Tbshran Lecture 20: November 7 Scrbes: Varsha Chnnaobreddy, Joon Sk Km, Lngyao Zhang Note: LaTeX template courtesy of UC Berkeley EECS dept. Dsclamer:
More informationAppendix B. Criterion of Riemann-Stieltjes Integrability
Appendx B. Crteron of Remann-Steltes Integrablty Ths note s complementary to [R, Ch. 6] and [T, Sec. 3.5]. The man result of ths note s Theorem B.3, whch provdes the necessary and suffcent condtons for
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationOptimization for Machine Learning
Optimization for Machine Learning (Problems; Algorithms - A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html
More informationAPPROXIMATE PRICES OF BASKET AND ASIAN OPTIONS DUPONT OLIVIER. Premia 14
APPROXIMAE PRICES OF BASKE AND ASIAN OPIONS DUPON OLIVIER Prema 14 Contents Introducton 1 1. Framewor 1 1.1. Baset optons 1.. Asan optons. Computng the prce 3. Lower bound 3.1. Closed formula for the prce
More informationNP-Completeness : Proofs
NP-Completeness : Proofs Proof Methods A method to show a decson problem Π NP-complete s as follows. (1) Show Π NP. (2) Choose an NP-complete problem Π. (3) Show Π Π. A method to show an optmzaton problem
More informationLecture 17: Lee-Sidford Barrier
CSE 599: Interplay between Convex Optmzaton and Geometry Wnter 2018 Lecturer: Yn Tat Lee Lecture 17: Lee-Sdford Barrer Dsclamer: Please tell me any mstake you notced. In ths lecture, we talk about the
More informationOPTIMALITY CONDITIONS FOR NONLINEAR PROGRAMMING WITH MIXED CONSTRAINTS AND ρ -LOCALLY ARCWISE CONNECTED FUNCTIONS
HE PUBLSHNG HOUSE PROCEEDNGS OF HE ROMANAN ACADEMY, Seres A, OF HE ROMANAN ACADEMY Volume 7, Number /6, pp - OPMALY CONDONS FOR NONLNEAR PROGRAMMNG WH MXED CONSRANS AND ρ -LOCALLY ARCWSE CONNECED FUNCONS
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationLecture 21: Numerical methods for pricing American type derivatives
Lecture 21: Numercal methods for prcng Amercan type dervatves Xaoguang Wang STAT 598W Aprl 10th, 2014 (STAT 598W) Lecture 21 1 / 26 Outlne 1 Fnte Dfference Method Explct Method Penalty Method (STAT 598W)
More informationPROBLEM SET 7 GENERAL EQUILIBRIUM
PROBLEM SET 7 GENERAL EQUILIBRIUM Queston a Defnton: An Arrow-Debreu Compettve Equlbrum s a vector of prces {p t } and allocatons {c t, c 2 t } whch satsfes ( Gven {p t }, c t maxmzes βt ln c t subject
More informationEconomics 101. Lecture 4 - Equilibrium and Efficiency
Economcs 0 Lecture 4 - Equlbrum and Effcency Intro As dscussed n the prevous lecture, we wll now move from an envronment where we looed at consumers mang decsons n solaton to analyzng economes full of
More informationThe Minimum Universal Cost Flow in an Infeasible Flow Network
Journal of Scences, Islamc Republc of Iran 17(2): 175-180 (2006) Unversty of Tehran, ISSN 1016-1104 http://jscencesutacr The Mnmum Unversal Cost Flow n an Infeasble Flow Network H Saleh Fathabad * M Bagheran
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationGame Theory. Lecture Notes By Y. Narahari. Department of Computer Science and Automation Indian Institute of Science Bangalore, India February 2008
Game Theory Lecture Notes By Y. Narahar Department of Computer Scence and Automaton Indan Insttute of Scence Bangalore, Inda February 2008 Chapter 10: Two Person Zero Sum Games Note: Ths s a only a draft
More information1 Convex Optimization
Convex Optmzaton We wll consder convex optmzaton problems. Namely, mnmzaton problems where the objectve s convex (we assume no constrants for now). Such problems often arse n machne learnng. For example,
More informationWorst-Case Robust Multiuser Transmit Beamforming Using Semidefinite Relaxation: Duality and Implications
Worst-Case Robust Multuser Transmt Beamforg Usng Semdefnte Relaxaton: Dualty and mplcatons Tsung-Hu Chang, Wng-Kn Ma, and Chong-Yung Ch nsttute of Commun. Eng. & Dept. of Elect. Eng. Natonal Tsng Hua Unversty,
More informationPHYS 705: Classical Mechanics. Calculus of Variations II
1 PHYS 705: Classcal Mechancs Calculus of Varatons II 2 Calculus of Varatons: Generalzaton (no constrant yet) Suppose now that F depends on several dependent varables : We need to fnd such that has a statonary
More informationLecture 3: Dual problems and Kernels
Lecture 3: Dual problems and Kernels C4B Machne Learnng Hlary 211 A. Zsserman Prmal and dual forms Lnear separablty revsted Feature mappng Kernels for SVMs Kernel trck requrements radal bass functons SVM
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationComputing Correlated Equilibria in Multi-Player Games
Computng Correlated Equlbra n Mult-Player Games Chrstos H. Papadmtrou Presented by Zhanxang Huang December 7th, 2005 1 The Author Dr. Chrstos H. Papadmtrou CS professor at UC Berkley (taught at Harvard,
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationECE 534: Elements of Information Theory. Solutions to Midterm Exam (Spring 2006)
ECE 534: Elements of Informaton Theory Solutons to Mdterm Eam (Sprng 6) Problem [ pts.] A dscrete memoryless source has an alphabet of three letters,, =,, 3, wth probabltes.4,.4, and., respectvely. (a)
More informationAffine transformations and convexity
Affne transformatons and convexty The purpose of ths document s to prove some basc propertes of affne transformatons nvolvng convex sets. Here are a few onlne references for background nformaton: http://math.ucr.edu/
More informationOn the Global Linear Convergence of the ADMM with Multi-Block Variables
On the Global Lnear Convergence of the ADMM wth Mult-Block Varables Tany Ln Shqan Ma Shuzhong Zhang May 31, 01 Abstract The alternatng drecton method of multplers ADMM has been wdely used for solvng structured
More informationSupport Vector Machines
Support Vector Machnes Konstantn Tretyakov (kt@ut.ee) MTAT.03.227 Machne Learnng So far Supervsed machne learnng Lnear models Least squares regresson Fsher s dscrmnant, Perceptron, Logstc model Non-lnear
More informationTHE STRUCTURED DISTANCE TO ILL-POSEDNESS FOR CONIC SYSTEMS. A.S. Lewis. April 18, 2003
THE STRUCTURED DISTANCE TO ILL-POSEDNESS FOR CONIC SYSTEMS A.S. Lews Aprl 18, 2003 Key words: condton number, conc system, dstance to nfeasblty, structured sngular values, sublnear maps, surjectvty AMS
More informationDuality. Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities
Duality Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities Lagrangian Consider the optimization problem in standard form
More information( ) 2 ( ) ( ) Problem Set 4 Suggested Solutions. Problem 1
Problem Set 4 Suggested Solutons Problem (A) The market demand functon s the soluton to the followng utlty-maxmzaton roblem (UMP): The Lagrangean: ( x, x, x ) = + max U x, x, x x x x st.. x + x + x y x,
More information6) Derivatives, gradients and Hessian matrices
30C00300 Mathematcal Methods for Economsts (6 cr) 6) Dervatves, gradents and Hessan matrces Smon & Blume chapters: 14, 15 Sldes by: Tmo Kuosmanen 1 Outlne Defnton of dervatve functon Dervatve notatons
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationLinear, affine, and convex sets and hulls In the sequel, unless otherwise specified, X will denote a real vector space.
Lnear, affne, and convex sets and hulls In the sequel, unless otherwse specfed, X wll denote a real vector space. Lnes and segments. Gven two ponts x, y X, we defne xy = {x + t(y x) : t R} = {(1 t)x +
More informationLecture 6: Support Vector Machines
Lecture 6: Support Vector Machnes Marna Melă mmp@stat.washngton.edu Department of Statstcs Unversty of Washngton November, 2018 Lnear SVM s The margn and the expected classfcaton error Maxmum Margn Lnear
More informationON A DETERMINATION OF THE INITIAL FUNCTIONS FROM THE OBSERVED VALUES OF THE BOUNDARY FUNCTIONS FOR THE SECOND-ORDER HYPERBOLIC EQUATION
Advanced Mathematcal Models & Applcatons Vol.3, No.3, 2018, pp.215-222 ON A DETERMINATION OF THE INITIAL FUNCTIONS FROM THE OBSERVED VALUES OF THE BOUNDARY FUNCTIONS FOR THE SECOND-ORDER HYPERBOLIC EUATION
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationSupport Vector Machines
Support Vector Machnes Konstantn Tretyakov (kt@ut.ee) MTAT.03.227 Machne Learnng So far So far Supervsed machne learnng Lnear models Non-lnear models Unsupervsed machne learnng Generc scaffoldng So far
More informationGeneral viscosity iterative method for a sequence of quasi-nonexpansive mappings
Avalable onlne at www.tjnsa.com J. Nonlnear Sc. Appl. 9 (2016), 5672 5682 Research Artcle General vscosty teratve method for a sequence of quas-nonexpansve mappngs Cuje Zhang, Ynan Wang College of Scence,
More informationInexact Alternating Minimization Algorithm for Distributed Optimization with an Application to Distributed MPC
Inexact Alternatng Mnmzaton Algorthm for Dstrbuted Optmzaton wth an Applcaton to Dstrbuted MPC Ye Pu, Coln N. Jones and Melane N. Zelnger arxv:608.0043v [math.oc] Aug 206 Abstract In ths paper, we propose
More informationThe Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction
ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationcoordinates. Then, the position vectors are described by
Revewng, what we have dscussed so far: Generalzed coordnates Any number of varables (say, n) suffcent to specfy the confguraton of the system at each nstant to tme (need not be the mnmum number). In general,
More informationLagrangian Duality and Convex Optimization
Lagrangian Duality and Convex Optimization David Rosenberg New York University February 11, 2015 David Rosenberg (New York University) DS-GA 1003 February 11, 2015 1 / 24 Introduction Why Convex Optimization?
More informationSupport Vector Machines
CS 2750: Machne Learnng Support Vector Machnes Prof. Adrana Kovashka Unversty of Pttsburgh February 17, 2016 Announcement Homework 2 deadlne s now 2/29 We ll have covered everythng you need today or at
More informationYong Joon Ryang. 1. Introduction Consider the multicommodity transportation problem with convex quadratic cost function. 1 2 (x x0 ) T Q(x x 0 )
Kangweon-Kyungk Math. Jour. 4 1996), No. 1, pp. 7 16 AN ITERATIVE ROW-ACTION METHOD FOR MULTICOMMODITY TRANSPORTATION PROBLEMS Yong Joon Ryang Abstract. The optmzaton problems wth quadratc constrants often
More informationUniversity of California, Davis Date: June 22, 2009 Department of Agricultural and Resource Economics. PRELIMINARY EXAMINATION FOR THE Ph.D.
Unversty of Calforna, Davs Date: June 22, 29 Department of Agrcultural and Resource Economcs Department of Economcs Tme: 5 hours Mcroeconomcs Readng Tme: 2 mnutes PRELIMIARY EXAMIATIO FOR THE Ph.D. DEGREE
More informationMEM Chapter 4b. LMI Lab commands
1 MEM8-7 Chapter 4b LMI Lab commands setlms lmvar lmterm getlms lmedt lmnbr matnbr lmnfo feasp dec2mat evallm showlm setmvar mat2dec mncx dellm delmvar gevp 2 Intalzng the LMI System he descrpton of an
More information14 Lagrange Multipliers
Lagrange Multplers 14 Lagrange Multplers The Method of Lagrange Multplers s a powerful technque for constraned optmzaton. Whle t has applcatons far beyond machne learnng t was orgnally developed to solve
More informationCSCE 790S Background Results
CSCE 790S Background Results Stephen A. Fenner September 8, 011 Abstract These results are background to the course CSCE 790S/CSCE 790B, Quantum Computaton and Informaton (Sprng 007 and Fall 011). Each
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationSupport Vector Machines. Vibhav Gogate The University of Texas at dallas
Support Vector Machnes Vbhav Gogate he Unversty of exas at dallas What We have Learned So Far? 1. Decson rees. Naïve Bayes 3. Lnear Regresson 4. Logstc Regresson 5. Perceptron 6. Neural networks 7. K-Nearest
More informationLab session: numerical simulations of sponateous polarization
Lab sesson: numercal smulatons of sponateous polarzaton Emerc Boun & Vncent Calvez CNRS, ENS Lyon, France CIMPA, Hammamet, March 2012 Spontaneous cell polarzaton: the 1D case The Hawkns-Voturez model for
More informationFuzzy Approaches for Multiobjective Fuzzy Random Linear Programming Problems Through a Probability Maximization Model
Fuzzy Approaches for Multobjectve Fuzzy Random Lnear Programmng Problems Through a Probablty Maxmzaton Model Htosh Yano and Kota Matsu Abstract In ths paper, two knds of fuzzy approaches are proposed for
More informationTHE CHINESE REMAINDER THEOREM. We should thank the Chinese for their wonderful remainder theorem. Glenn Stevens
THE CHINESE REMAINDER THEOREM KEITH CONRAD We should thank the Chnese for ther wonderful remander theorem. Glenn Stevens 1. Introducton The Chnese remander theorem says we can unquely solve any par of
More informationDuality in linear programming
MPRA Munch Personal RePEc Archve Dualty n lnear programmng Mhaela Albc and Dela Teselos and Raluca Prundeanu and Ionela Popa Unversty Constantn Brancoveanu Ramncu Valcea 7 January 00 Onlne at http://mpraubun-muenchende/986/
More informationTHE ARIMOTO-BLAHUT ALGORITHM FOR COMPUTATION OF CHANNEL CAPACITY. William A. Pearlman. References: S. Arimoto - IEEE Trans. Inform. Thy., Jan.
THE ARIMOTO-BLAHUT ALGORITHM FOR COMPUTATION OF CHANNEL CAPACITY Wllam A. Pearlman 2002 References: S. Armoto - IEEE Trans. Inform. Thy., Jan. 1972 R. Blahut - IEEE Trans. Inform. Thy., July 1972 Recall
More informationVARIATION OF CONSTANT SUM CONSTRAINT FOR INTEGER MODEL WITH NON UNIFORM VARIABLES
VARIATION OF CONSTANT SUM CONSTRAINT FOR INTEGER MODEL WITH NON UNIFORM VARIABLES BÂRZĂ, Slvu Faculty of Mathematcs-Informatcs Spru Haret Unversty barza_slvu@yahoo.com Abstract Ths paper wants to contnue
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationREAL ANALYSIS I HOMEWORK 1
REAL ANALYSIS I HOMEWORK CİHAN BAHRAN The questons are from Tao s text. Exercse 0.0.. If (x α ) α A s a collecton of numbers x α [0, + ] such that x α
More informationExercise Solutions to Real Analysis
xercse Solutons to Real Analyss Note: References refer to H. L. Royden, Real Analyss xersze 1. Gven any set A any ɛ > 0, there s an open set O such that A O m O m A + ɛ. Soluton 1. If m A =, then there
More informationWelfare Properties of General Equilibrium. What can be said about optimality properties of resource allocation implied by general equilibrium?
APPLIED WELFARE ECONOMICS AND POLICY ANALYSIS Welfare Propertes of General Equlbrum What can be sad about optmalty propertes of resource allocaton mpled by general equlbrum? Any crteron used to compare
More informationMaximizing the number of nonnegative subsets
Maxmzng the number of nonnegatve subsets Noga Alon Hao Huang December 1, 213 Abstract Gven a set of n real numbers, f the sum of elements of every subset of sze larger than k s negatve, what s the maxmum
More informationMinimax Optimal Control of Stochastic Uncertain Systems with Relative Entropy Constraints
[IEEE Trans. Aut. Control, 45, 398-412, 2000] Mnmax Optmal Control of Stochastc Uncertan Systems wth Relatve Entropy Constrants Ian R. Petersen Matthew R. James Paul Dupus Abstract Ths paper consders a
More informationGlobal Optimization of Truss. Structure Design INFORMS J. N. Hooker. Tallys Yunes. Slide 1
Slde 1 Global Optmzaton of Truss Structure Desgn J. N. Hooker Tallys Yunes INFORMS 2010 Truss Structure Desgn Select sze of each bar (possbly zero) to support the load whle mnmzng weght. Bar szes are dscrete.
More informationCommunication Complexity 16:198: February Lecture 4. x ij y ij
Communcaton Complexty 16:198:671 09 February 2010 Lecture 4 Lecturer: Troy Lee Scrbe: Rajat Mttal 1 Homework problem : Trbes We wll solve the thrd queston n the homework. The goal s to show that the nondetermnstc
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationHidden Markov Models & The Multivariate Gaussian (10/26/04)
CS281A/Stat241A: Statstcal Learnng Theory Hdden Markov Models & The Multvarate Gaussan (10/26/04) Lecturer: Mchael I. Jordan Scrbes: Jonathan W. Hu 1 Hdden Markov Models As a bref revew, hdden Markov models
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationLinear precoding via conic optimization for fixed MIMO receivers
Lnear precodng va conc optmzaton for fxed MIMO recevers Am Wesel, Yonna C Eldar, and Shlomo Shama (Shtz) Department of Electrcal Engneerng Technon - Israel Insttute of Technology June 8, 004 Abstract We
More information