Lecture 13: Duality Uses and Correspondences
|
|
- Mervyn Roberts
- 6 years ago
- Views:
Transcription
1 10-725/36-725: Conve Optimization Fall 2016 Lectre 13: Dality Uses and Correspondences Lectrer: Ryan Tibshirani Scribes: Yichong X, Yany Liang, Yanning Li Note: LaTeX template cortesy of UC Berkeley EECS dept. Disclaimer: These notes have not been sbjected to the sal scrtiny reserved for formal pblications. They may be distribted otside this class only with the permission of the Instrctor Last Class KKT conditions For the problem the KKT conditions are f() sbject to h i () 0, i = 1,..., m l j () = 0, j = 1,..., r Stationary: 0 f() + m i h i () + r j=1 v j l j (), Complementary slackness: i h i () = 0, i, Primal feasibility: h i () 0, h j () = 0, i, j, Dal feasibility: i 0, i. The KKT conditions are always sfficient, and are necessary for optimality nder strong dality Uses of dality For a primal feasible and a dal feasible, v, f() g(, v) is called the dality gap between and, v. Since f() g(, v) we have f() f( ) f() g(, v). So a zero dality gap implies optimality. algorithms. Also the dality gap can be sed as a stopping criterion in Under strong dality, if we are given dal optimal, v, any primal soltion imizes L(,, v ) over all, becase of the stationary condition. This can be sed to characterize or compte primal soltions. Eplicitly, given a dal soltion, v, any primal soltion solves f() + m i h i () + r vj l j (). j=1 13-1
2 13-2 Lectre 13: Dality Uses and Correspondences Soltions of this nconstrained problem can often be epressed eplicitly, giving an eplicit characterization of primal soltions from dal soltions. Eample (B & V page 249). Consider the problem f i ( i ) sbject to a T = b where each f i : R R is smooth and strictly conve. The dal fnction is g(v) = = bv + = bv f i ( i ) + v(b a T ) f i ( i ) a i v i i fi (a i v), where f i is the conjgate of f i, which we will define later. The dal problem is ths or ma v v bv fi (a i v) fi (a i v) bv. This is a conve imization problam with a scalar variable - it is mch easier to solve than the primal problem. Given v, the primal soltion solves i f i ( i ) + a i v i. Since each f i is strictly conve, the problem i f i ( i )+a i v i has a niqe soltion, which can be compted by solving f i ( i ) = a i v for each i Dal norms Let be an arbitrary norm. For eample: l p norm: p = ( n i p ) 1/p, for p 1. Trace norm: X tr = r σ i(x). Define its dal norm as = ma z 1 zt. The definition gives s the ineqality z T z, similar to Cachy-Schwartz ineqality. eamples: For the
3 Lectre 13: Dality ses and correspondences 13-3 l p norm dal: ( p ) = q, where 1/p + 1/q = 1. Trace norm: ( X tr ) = X op = σ 1 (X). We can show that Theorem 13.1 The dal of dal norm is the original norm: i.e., =. Proof: Consider the problem whose optimal vale is. Its Lagrangian is y y sbject to y =, L(y, ) = y + T ( y) = y y T + T. From the definition of, if > 1, let z be the maimizer of ma z 1 z T, i.e., z T =. Note that z = 1. Let y = tz for t > 0. Ths y y T = t( z z T ) = t(1 ) which as t. So in sch case y y y T =. If 1, we have y y T y y 0. This can be realized by setting y = 0. So y y y T = 0. Therefore the Lagrange dal problem is ma T sbject to 1, whose optimal vale is the dal of, i.e.,. By strong dality we have = Conjgate fnction Definition Given a fnction f : R n R, define its conjgate f : R n R, f (y) = ma yt f() Since y T f() is conve in y for any fied, f is always conve as it is a pointwise maimm of conve (affine) fnctions in y (f need not be conve). f (y) is the maimm gap between linear fnction y T and f() and Figre 13.1 shows how it looks like when f is a scalar fnction. For differentiable f, conjgation is called the Legendre transform.
4 13-4 Lectre 13: Dality Uses and Correspondences Given y, the pper dashed line represents the fnction g() = y and solid line represents f(). f (y) is the biggest gap between g and f where g is above f. The lower dashed line is drawn to find sch biggest gap and the absolte vale of intercept corresponds to the vale of biggest gap Figre 13.1: From [1] pp Properties Fenchel s ineqality: for any, y, Proof: f (y) = ma z z T y f(z) T y f() f() + f (y) T y Hence conjgate of conjgate f satisfies f f. Proof: f () = ma z z T f (z) ma z f() = f() ( comes from Fenchel s ineqality) If f is closed and conve, then f = f Proof: f() shares the same vale with y f(y), sbject to y =. The dal is ma T f () = f (). Since strong dality holds, the eqality follows. If f is closed and conve, then for any, y, If f(, v) = f 1 () + f 2 (v), then f (y) y f() f() + f (y) = T y f (w, z) = f 1 (w) + f 2 (z) Proof: f (w, z) = ma,v ( T, v T )(w, z) T f(, v) = ma,v T w+v T z f 1 () f 2 (v) = ma { T w f 1 ()} + ma v {v T z f 2 (v)} = f 1 (w) + f 2 (z) Eamples Simple qadratic: let f() = 1 2 T Q, where Q 0. Then y T 1 2 T Q is strictly concave in y and is maimized at y = Q 1, so f (y) = 1 2 yt Q 1 y
5 Lectre 13: Dality ses and correspondences 13-5 Indicator fnction: if f() = I C (), then its conjgate is called the spport fnction of C. Norm: if f() =, then its conjgate is where is the dal norm of. f (y) = ma C yt := I C(y) f (y) = I {z: z 1}(y) Eample: lasso dal Given y R n, X R n p, recall the lasso problem: 1 β 2 y Xβ λ β 1 Its dal fnction is jst a constant (eqal to f ). Therefore we transform the primal to So dal fnction is now g() = β,z 1 β,z 2 y z λ β 1 sbject to z = Xβ 1 2 y z λ β 1 + T (z Xβ) = y T I {v: v 1}(X T /λ) Therefore the lasso dal problem is or eqivalently sbject to sbject to ma y T X T λ y 2 2 X T λ Check: Slater s condition holds, and hence so does strong dality. Bt note: the optimal vale of the last problem is not the optimal lasso objective vale. Frther, note the given the dal soltion, any lasso soltion β satisfies Xβ = y This is from KKT stationarity condition for z (i.e. z y + β = 0). So the lasso fit is jst the dal residal (see Figre 13.2).
6 13-6 Lectre 13: Dality Uses and Correspondences Figre 13.2: The lasso soltion and its dal soltion Conjgates and dal problems Conjgates appear freqently in derivation of dal problems, via in imization of the Lagrangian. E.g., consider f () = f() T f() + g() Eqivalently:,z f() + g(z) sbject to = z. Lagrange dal fnction is: g() =,z f() + g(z) + T (z ) =,z f() T + g(z) ( ) T z = {f() T } + {g(z) ( ) T z} z = ma{ T f()} ma z {( )T z g(z)} = f () g ( ) Eamples of this last calclation: Indicator fnction: the dal of is f() + I C () ma f () IC( ) where I C is the spport fnction of C. Norms: the dal of f() +
7 Lectre 13: Dality ses and correspondences 13-7 is or eqivalently where is the dal norm of. ma f () I {z: z 1}( ) ma f () sbject to Shifting linear transformations Dal formlations can help s by shifting a linear transformation between one part of the objective and another. Let s consider f() + g(a) Eqivalently:,z f() + g(z) sbject to A = z. Like before: Then dal is: g() =,z f() + g(z) + T (z A) = ma (AT ) T f() ma ( )T z g(z) = f (A T ) g ( ) ma f (A T ) g ( ) Eample: for a norm and its dal norm,,, the problems and are primal and dal paris. f() + A ma f (A T ) sbject to 1 z 13.4 Dal cones Definition Recall that set K R n is a cone if K, t 0, we have t K. The dal cone of K is defined as K = {y : y T 0 for all K} Important properties: K is closed and conve. K 1 K 2 K2 K1 K is the closre of the conve hll of K. (Hence if K is conve and closed, K = K)
8 13-8 Lectre 13: Dality Uses and Correspondences Left. The halfspace with inward normal y contains the cone K, so y K. Right. The halfspace with inward normal z does not contain K, so z / K. Figre 13.3: From B & V [1] pp Eamples Linear sbspace: the dal cone of a linear sbspace V is V, its orthogonal complement. E.g. (row(a)) = (A). Norm cone: the dal cone of the norm cone is the norm cone of its dal norm K = {(, t) R n+1 : t} K = {(y, s) R n+1 : y s} Positive semidefinite cone: the conve cone S n + is self-dal, i.e. (S n +) = S n +. Y 0 Tr(Y X) 0 for all X Dal cones and dal problems Consider the cone constrained problem its dal problem is f() sbject to A K ma f (A T ) IK( ) where I K (y) = ma z K z T y is the spport fnction of K. If K is a cone, we have IK ( ) = I K (), the this is eqivalent to where K is the dal cone of K. ma f (A T ) sbject to K It is sally easier to handle cone constraints like K than constraints that the linear transform of is in a cone, i.e. A K.
9 Lectre 13: Dality ses and correspondences Doble dal Consider general imization problem with linear constraints: sbject to f() A b, C = d The Lagrangian is and hence the dal problem is L(,, v) = f() + (A T + C T v) T b T d T v f ( A T C T v) b T d T v sbject to 0 Recall property: f = f if f is closed and conve. Hence in this case, we can show that the dal of the dal is the primal. Actally this also goes beyond linear constraints. Consider f() sbject to h i () 0, i = 1,..., m l j () = 0, j = 1,..., r If f and h 1,...h m are closed and conve, and l 1,...l r are affine, then the dal of the dal is the primal. This is proved by viewing the imization problem in terms of a bifnction. In this framework, the dal fnction corresponds to the conjgate of this bifnction. See Chapter 29 and 30 of Rockafellar. [2] 13.6 Dal sbtleties We often transform the dal into an eqivalent problem and still call this the dal. Under strong dality, we can se soltions of the (transformed) dal problem to characterize or compte the primal soltions. Warning: the optimal vale of this transformed dal problem is not necessarily the optimal primal vale. A common trick in deriving dals for nconstrained problems is to first transform the primal by adding a dmmy variable and an eqality constraint. e.g. The previos eample of the lasso dal. Usally there is ambigity in how to do this. Different choices can lead to different dal problems. References [1] Stephen Boyd and Lieven Vandenberghe. Conve optimization. Cambridge University Press, [2] R. Tyrrell Rockafellar. Conve analysis. Princeton University Press, 1970.
Duality Uses and Correspondences. Ryan Tibshirani Convex Optimization
Duality Uses and Correspondences Ryan Tibshirani Conve Optimization 10-725 Recall that for the problem Last time: KKT conditions subject to f() h i () 0, i = 1,... m l j () = 0, j = 1,... r the KKT conditions
More informationLecture 8: September 26
10-704: Information Processing and Learning Fall 2016 Lectrer: Aarti Singh Lectre 8: September 26 Note: These notes are based on scribed notes from Spring15 offering of this corse. LaTeX template cortesy
More information10-725/36-725: Convex Optimization Spring Lecture 21: April 6
10-725/36-725: Conve Optimization Spring 2015 Lecturer: Ryan Tibshirani Lecture 21: April 6 Scribes: Chiqun Zhang, Hanqi Cheng, Waleed Ammar Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationLecture 23: November 19
10-725/36-725: Conve Optimization Fall 2018 Lecturer: Ryan Tibshirani Lecture 23: November 19 Scribes: Charvi Rastogi, George Stoica, Shuo Li Charvi Rastogi: 23.1-23.4.2, George Stoica: 23.4.3-23.8, Shuo
More informationLecture 16: October 22
0-725/36-725: Conve Optimization Fall 208 Lecturer: Ryan Tibshirani Lecture 6: October 22 Scribes: Nic Dalmasso, Alan Mishler, Benja LeRoy Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationLecture 6: September 12
10-725: Optimization Fall 2013 Lecture 6: September 12 Lecturer: Ryan Tibshirani Scribes: Micol Marchetti-Bowick Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have not
More informationLecture 4: September 12
10-725/36-725: Conve Optimization Fall 2016 Lecture 4: September 12 Lecturer: Ryan Tibshirani Scribes: Jay Hennig, Yifeng Tao, Sriram Vasudevan Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationB-469 Simplified Copositive and Lagrangian Relaxations for Linearly Constrained Quadratic Optimization Problems in Continuous and Binary Variables
B-469 Simplified Copositive and Lagrangian Relaxations for Linearly Constrained Qadratic Optimization Problems in Continos and Binary Variables Naohiko Arima, Snyong Kim and Masakaz Kojima October 2012,
More informationKarush-Kuhn-Tucker Conditions. Lecturer: Ryan Tibshirani Convex Optimization /36-725
Karush-Kuhn-Tucker Conditions Lecturer: Ryan Tibshirani Convex Optimization 10-725/36-725 1 Given a minimization problem Last time: duality min x subject to f(x) h i (x) 0, i = 1,... m l j (x) = 0, j =
More informationLecture 23: Conditional Gradient Method
10-725/36-725: Conve Optimization Spring 2015 Lecture 23: Conditional Gradient Method Lecturer: Ryan Tibshirani Scribes: Shichao Yang,Diyi Yang,Zhanpeng Fang Note: LaTeX template courtesy of UC Berkeley
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationThe Linear Quadratic Regulator
10 The Linear Qadratic Reglator 10.1 Problem formlation This chapter concerns optimal control of dynamical systems. Most of this development concerns linear models with a particlarly simple notion of optimality.
More informationOptimization for Machine Learning
Optimization for Machine Learning (Problems; Algorithms - A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html
More informationLecture 14: Optimality Conditions for Conic Problems
EE 227A: Conve Optimization and Applications March 6, 2012 Lecture 14: Optimality Conditions for Conic Problems Lecturer: Laurent El Ghaoui Reading assignment: 5.5 of BV. 14.1 Optimality for Conic Problems
More informationConvex Optimization Overview (cnt d)
Conve Optimization Overview (cnt d) Chuong B. Do November 29, 2009 During last week s section, we began our study of conve optimization, the study of mathematical optimization problems of the form, minimize
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationIntroduction to Machine Learning Spring 2018 Note Duality. 1.1 Primal and Dual Problem
CS 189 Introduction to Machine Learning Spring 2018 Note 22 1 Duality As we have seen in our discussion of kernels, ridge regression can be viewed in two ways: (1) an optimization problem over the weights
More informationLecture 14: Newton s Method
10-725/36-725: Conve Optimization Fall 2016 Lecturer: Javier Pena Lecture 14: Newton s ethod Scribes: Varun Joshi, Xuan Li Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes
More informationLecture 26: April 22nd
10-725/36-725: Conve Optimization Spring 2015 Lecture 26: April 22nd Lecturer: Ryan Tibshirani Scribes: Eric Wong, Jerzy Wieczorek, Pengcheng Zhou Note: LaTeX template courtesy of UC Berkeley EECS dept.
More informationFormal Methods for Deriving Element Equations
Formal Methods for Deriving Element Eqations And the importance of Shape Fnctions Formal Methods In previos lectres we obtained a bar element s stiffness eqations sing the Direct Method to obtain eact
More informationLecture 23: November 21
10-725/36-725: Convex Optimization Fall 2016 Lecturer: Ryan Tibshirani Lecture 23: November 21 Scribes: Yifan Sun, Ananya Kumar, Xin Lu Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationOperations Research Letters
Operations Research Letters 37 (2009) 1 6 Contents lists available at ScienceDirect Operations Research Letters journal homepage: www.elsevier.com/locate/orl Duality in robust optimization: Primal worst
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More informationLagrangian Duality and Convex Optimization
Lagrangian Duality and Convex Optimization David Rosenberg New York University February 11, 2015 David Rosenberg (New York University) DS-GA 1003 February 11, 2015 1 / 24 Introduction Why Convex Optimization?
More informationLecture 5: September 15
10-725/36-725: Convex Optimization Fall 2015 Lecture 5: September 15 Lecturer: Lecturer: Ryan Tibshirani Scribes: Scribes: Di Jin, Mengdi Wang, Bin Deng Note: LaTeX template courtesy of UC Berkeley EECS
More informationLecture 6: September 17
10-725/36-725: Convex Optimization Fall 2015 Lecturer: Ryan Tibshirani Lecture 6: September 17 Scribes: Scribes: Wenjun Wang, Satwik Kottur, Zhiding Yu Note: LaTeX template courtesy of UC Berkeley EECS
More informationDual methods and ADMM. Barnabas Poczos & Ryan Tibshirani Convex Optimization /36-725
Dual methods and ADMM Barnabas Poczos & Ryan Tibshirani Convex Optimization 10-725/36-725 1 Given f : R n R, the function is called its conjugate Recall conjugate functions f (y) = max x R n yt x f(x)
More information4 Exact laminar boundary layer solutions
4 Eact laminar bondary layer soltions 4.1 Bondary layer on a flat plate (Blasis 1908 In Sec. 3, we derived the bondary layer eqations for 2D incompressible flow of constant viscosity past a weakly crved
More informationConvexity II: Optimization Basics
Conveity II: Optimization Basics Lecturer: Ryan Tibshirani Conve Optimization 10-725/36-725 See supplements for reviews of basic multivariate calculus basic linear algebra Last time: conve sets and functions
More informationLagrangian Duality for Dummies
Lagrangian Duality for Dummies David Knowles November 13, 2010 We want to solve the following optimisation problem: f 0 () (1) such that f i () 0 i 1,..., m (2) For now we do not need to assume conveity.
More informationA Computational Study with Finite Element Method and Finite Difference Method for 2D Elliptic Partial Differential Equations
Applied Mathematics, 05, 6, 04-4 Pblished Online November 05 in SciRes. http://www.scirp.org/jornal/am http://d.doi.org/0.46/am.05.685 A Comptational Stdy with Finite Element Method and Finite Difference
More informationMixed Type Second-order Duality for a Nondifferentiable Continuous Programming Problem
heoretical Mathematics & Applications, vol.3, no.1, 13, 13-144 SSN: 179-9687 (print), 179-979 (online) Scienpress Ltd, 13 Mied ype Second-order Dality for a Nondifferentiable Continos Programming Problem.
More informationNon-Lecture I: Linear Programming. Th extremes of glory and of shame, Like east and west, become the same.
The greatest flood has the soonest ebb; the sorest tempest the most sdden calm; the hottest love the coldest end; and from the deepest desire oftentimes enses the deadliest hate. Th extremes of glory and
More informationECON3120/4120 Mathematics 2, spring 2009
University of Oslo Department of Economics Arne Strøm ECON3/4 Mathematics, spring 9 Problem soltions for Seminar 4, 6 Febrary 9 (For practical reasons some of the soltions may inclde problem parts that
More informationExecution time certification for gradient-based optimization in model predictive control
Exection time certification for gradient-based optimization in model predictive control Giselsson, Ponts Pblished in: [Host pblication title missing] 2012 Link to pblication Citation for pblished version
More informationLecture 5: September 12
10-725/36-725: Convex Optimization Fall 2015 Lecture 5: September 12 Lecturer: Lecturer: Ryan Tibshirani Scribes: Scribes: Barun Patra and Tyler Vuong Note: LaTeX template courtesy of UC Berkeley EECS
More informationDual Methods. Lecturer: Ryan Tibshirani Convex Optimization /36-725
Dual Methods Lecturer: Ryan Tibshirani Conve Optimization 10-725/36-725 1 Last time: proimal Newton method Consider the problem min g() + h() where g, h are conve, g is twice differentiable, and h is simple.
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationOn Multiobjective Duality For Variational Problems
The Open Operational Research Jornal, 202, 6, -8 On Mltiobjective Dality For Variational Problems. Hsain *,, Bilal Ahmad 2 and Z. Jabeen 3 Open Access Department of Mathematics, Jaypee University of Engineering
More informationUses of duality. Geoff Gordon & Ryan Tibshirani Optimization /
Uses of duality Geoff Gordon & Ryan Tibshirani Optimization 10-725 / 36-725 1 Remember conjugate functions Given f : R n R, the function is called its conjugate f (y) = max x R n yt x f(x) Conjugates appear
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationVectors in Rn un. This definition of norm is an extension of the Pythagorean Theorem. Consider the vector u = (5, 8) in R 2
MATH 307 Vectors in Rn Dr. Neal, WKU Matrices of dimension 1 n can be thoght of as coordinates, or ectors, in n- dimensional space R n. We can perform special calclations on these ectors. In particlar,
More informationCDS 110b: Lecture 1-2 Introduction to Optimal Control
CDS 110b: Lectre 1-2 Introdction to Optimal Control Richard M. Mrray 4 Janary 2006 Goals: Introdce the problem of optimal control as method of trajectory generation State the maimm principle and give eamples
More informationDual Ascent. Ryan Tibshirani Convex Optimization
Dual Ascent Ryan Tibshirani Conve Optimization 10-725 Last time: coordinate descent Consider the problem min f() where f() = g() + n i=1 h i( i ), with g conve and differentiable and each h i conve. Coordinate
More informationLecture 7: Weak Duality
EE 227A: Conve Optimization and Applications February 7, 2012 Lecture 7: Weak Duality Lecturer: Laurent El Ghaoui 7.1 Lagrange Dual problem 7.1.1 Primal problem In this section, we consider a possibly
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationDuality revisited. Javier Peña Convex Optimization /36-725
Duality revisited Javier Peña Conve Optimization 10-725/36-725 1 Last time: barrier method Main idea: approimate the problem f() + I C () with the barrier problem f() + 1 t φ() tf() + φ() where t > 0 and
More informationLecture 25: November 27
10-725: Optimization Fall 2012 Lecture 25: November 27 Lecturer: Ryan Tibshirani Scribes: Matt Wytock, Supreeth Achar Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have
More informationA Characterization of the Domain of Beta-Divergence and Its Connection to Bregman Variational Model
entropy Article A Characterization of the Domain of Beta-Divergence and Its Connection to Bregman Variational Model Hyenkyn Woo School of Liberal Arts, Korea University of Technology and Edcation, Cheonan
More informationThe Dual of the Maximum Likelihood Method
Department of Agricltral and Resorce Economics University of California, Davis The Dal of the Maximm Likelihood Method by Qirino Paris Working Paper No. 12-002 2012 Copyright @ 2012 by Qirino Paris All
More informationi=1 y i 1fd i = dg= P N i=1 1fd i = dg.
ECOOMETRICS II (ECO 240S) University of Toronto. Department of Economics. Winter 208 Instrctor: Victor Agirregabiria SOLUTIO TO FIAL EXAM Tesday, April 0, 208. From 9:00am-2:00pm (3 hors) ISTRUCTIOS: -
More information10701 Recitation 5 Duality and SVM. Ahmed Hefny
10701 Recitation 5 Duality and SVM Ahmed Hefny Outline Langrangian and Duality The Lagrangian Duality Eamples Support Vector Machines Primal Formulation Dual Formulation Soft Margin and Hinge Loss Lagrangian
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationDuality. Geoff Gordon & Ryan Tibshirani Optimization /
Duality Geoff Gordon & Ryan Tibshirani Optimization 10-725 / 36-725 1 Duality in linear programs Suppose we want to find lower bound on the optimal value in our convex problem, B min x C f(x) E.g., consider
More informationA fundamental inverse problem in geosciences
A fndamental inverse problem in geosciences Predict the vales of a spatial random field (SRF) sing a set of observed vales of the same and/or other SRFs. y i L i ( ) + v, i,..., n i ( P)? L i () : linear
More informationLecture 24: August 28
10-725: Optimization Fall 2012 Lecture 24: August 28 Lecturer: Geoff Gordon/Ryan Tibshirani Scribes: Jiaji Zhou,Tinghui Zhou,Kawa Cheung Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationLecture 9: September 28
0-725/36-725: Convex Optimization Fall 206 Lecturer: Ryan Tibshirani Lecture 9: September 28 Scribes: Yiming Wu, Ye Yuan, Zhihao Li Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationEssentials of optimal control theory in ECON 4140
Essentials of optimal control theory in ECON 4140 Things yo need to know (and a detail yo need not care abot). A few words abot dynamic optimization in general. Dynamic optimization can be thoght of as
More informationCRITERIA FOR TOEPLITZ OPERATORS ON THE SPHERE. Jingbo Xia
CRITERIA FOR TOEPLITZ OPERATORS ON THE SPHERE Jingbo Xia Abstract. Let H 2 (S) be the Hardy space on the nit sphere S in C n. We show that a set of inner fnctions Λ is sfficient for the prpose of determining
More informationLecture 15: October 15
10-725: Optimization Fall 2012 Lecturer: Barnabas Poczos Lecture 15: October 15 Scribes: Christian Kroer, Fanyi Xiao Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have
More informationFEA Solution Procedure
EA Soltion Procedre (demonstrated with a -D bar element problem) EA Procedre for Static Analysis. Prepare the E model a. discretize (mesh) the strctre b. prescribe loads c. prescribe spports. Perform calclations
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationConvex Optimization Problems. Prof. Daniel P. Palomar
Conve Optimization Problems Prof. Daniel P. Palomar The Hong Kong University of Science and Technology (HKUST) MAFS6010R- Portfolio Optimization with R MSc in Financial Mathematics Fall 2018-19, HKUST,
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationConvex Optimization. Dani Yogatama. School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA. February 12, 2014
Convex Optimization Dani Yogatama School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA February 12, 2014 Dani Yogatama (Carnegie Mellon University) Convex Optimization February 12,
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationLecture 1: January 12
10-725/36-725: Convex Optimization Fall 2015 Lecturer: Ryan Tibshirani Lecture 1: January 12 Scribes: Seo-Jin Bang, Prabhat KC, Josue Orellana 1.1 Review We begin by going through some examples and key
More informationLecture 4: January 26
10-725/36-725: Conve Optimization Spring 2015 Lecturer: Javier Pena Lecture 4: January 26 Scribes: Vipul Singh, Shinjini Kundu, Chia-Yin Tsai Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationSecond-Order Wave Equation
Second-Order Wave Eqation A. Salih Department of Aerospace Engineering Indian Institte of Space Science and Technology, Thirvananthapram 3 December 016 1 Introdction The classical wave eqation is a second-order
More informationUC Berkeley Department of Electrical Engineering and Computer Science. EECS 227A Nonlinear and Convex Optimization. Solutions 5 Fall 2009
UC Berkeley Department of Electrical Engineering and Computer Science EECS 227A Nonlinear and Convex Optimization Solutions 5 Fall 2009 Reading: Boyd and Vandenberghe, Chapter 5 Solution 5.1 Note that
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.-Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual
More informationLecture 10: Duality in Linear Programs
10-725/36-725: Convex Optimization Spring 2015 Lecture 10: Duality in Linear Programs Lecturer: Ryan Tibshirani Scribes: Jingkun Gao and Ying Zhang Disclaimer: These notes have not been subjected to the
More informationMAT389 Fall 2016, Problem Set 6
MAT389 Fall 016, Problem Set 6 Trigonometric and hperbolic fnctions 6.1 Show that e iz = cos z + i sin z for eer comple nmber z. Hint: start from the right-hand side and work or wa towards the left-hand
More informationVIBRATION MEASUREMENT UNCERTAINTY AND RELIABILITY DIAGNOSTICS RESULTS IN ROTATING SYSTEMS
VIBRATIO MEASUREMET UCERTAITY AD RELIABILITY DIAGOSTICS RESULTS I ROTATIG SYSTEMS. Introdction M. Eidkevicite, V. Volkovas anas University of Technology, Lithania The rotating machinery technical state
More informationLecture 17: Primal-dual interior-point methods part II
10-725/36-725: Convex Optimization Spring 2015 Lecture 17: Primal-dual interior-point methods part II Lecturer: Javier Pena Scribes: Pinchao Zhang, Wei Ma Note: LaTeX template courtesy of UC Berkeley EECS
More informationOptimization in Predictive Control Algorithm
Latest rends in Circits, Systems, Sinal Processin and Atomatic Control Optimization in Predictive Control Alorithm JAN ANOŠ, MAREK KUBALČÍK omas Bata University in Zlín, Faclty of Applied Informatics Nám..
More information3 2D Elastostatic Problems in Cartesian Coordinates
D lastostatic Problems in Cartesian Coordinates Two dimensional elastostatic problems are discssed in this Chapter, that is, static problems of either plane stress or plane strain. Cartesian coordinates
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationON OPTIMALITY CONDITIONS FOR ABSTRACT CONVEX VECTOR OPTIMIZATION PROBLEMS
J. Korean Math. Soc. 44 2007), No. 4, pp. 971 985 ON OPTIMALITY CONDITIONS FOR ABSTRACT CONVEX VECTOR OPTIMIZATION PROBLEMS Ge Myng Lee and Kwang Baik Lee Reprinted from the Jornal of the Korean Mathematical
More informationCHARACTERIZATIONS OF EXPONENTIAL DISTRIBUTION VIA CONDITIONAL EXPECTATIONS OF RECORD VALUES. George P. Yanev
Pliska Std. Math. Blgar. 2 (211), 233 242 STUDIA MATHEMATICA BULGARICA CHARACTERIZATIONS OF EXPONENTIAL DISTRIBUTION VIA CONDITIONAL EXPECTATIONS OF RECORD VALUES George P. Yanev We prove that the exponential
More informationLecture 14: October 17
1-725/36-725: Convex Optimization Fall 218 Lecture 14: October 17 Lecturer: Lecturer: Ryan Tibshirani Scribes: Pengsheng Guo, Xian Zhou Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationRelaxations Applicable to Mixed Integer Predictive Control Comparisons and Efficient Computations
echnical report from Atomatic Control at Linköpings niversitet Relaxations Applicable to Mixed Integer Predictive Control Comparisons and Efficient Comptations Daniel Axehill, Anders Hansson, Lieven Vandenberghe
More informationOn the Method of Lagrange Multipliers
On the Method of Lagrange Multipliers Reza Nasiri Mahalati November 6, 2016 Most of what is in this note is taken from the Convex Optimization book by Stephen Boyd and Lieven Vandenberghe. This should
More informationPrimal/Dual Decomposition Methods
Primal/Dual Decomposition Methods Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2018-19, HKUST, Hong Kong Outline of Lecture Subgradients
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More informationICS-E4030 Kernel Methods in Machine Learning
ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This
More informationComplex Variables. For ECON 397 Macroeconometrics Steve Cunningham
Comple Variables For ECON 397 Macroeconometrics Steve Cnningham Open Disks or Neighborhoods Deinition. The set o all points which satis the ineqalit
More informationSubgradient Method. Ryan Tibshirani Convex Optimization
Subgradient Method Ryan Tibshirani Convex Optimization 10-725 Consider the problem Last last time: gradient descent min x f(x) for f convex and differentiable, dom(f) = R n. Gradient descent: choose initial
More informationTechnical Note. ODiSI-B Sensor Strain Gage Factor Uncertainty
Technical Note EN-FY160 Revision November 30, 016 ODiSI-B Sensor Strain Gage Factor Uncertainty Abstract Lna has pdated or strain sensor calibration tool to spport NIST-traceable measrements, to compte
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More information3.1 The Basic Two-Level Model - The Formulas
CHAPTER 3 3 THE BASIC MULTILEVEL MODEL AND EXTENSIONS In the previos Chapter we introdced a nmber of models and we cleared ot the advantages of Mltilevel Models in the analysis of hierarchically nested
More informationSTABILIZATIO ON OF LONGITUDINAL AIRCRAFT MOTION USING MODEL PREDICTIVE CONTROL AND EXACT LINEARIZATION
8 TH INTERNATIONAL CONGRESS OF THE AERONAUTICAL SCIENCES STABILIZATIO ON OF LONGITUDINAL AIRCRAFT MOTION USING MODEL PREDICTIVE CONTROL AND EXACT LINEARIZATION Čeliovsý S.*, Hospodář P.** *CTU Prage, Faclty
More informationLagrange duality. The Lagrangian. We consider an optimization program of the form
Lagrange duality Another way to arrive at the KKT conditions, and one which gives us some insight on solving constrained optimization problems, is through the Lagrange dual. The dual is a maximization
More informationSTATIC LECTURE 4: CONSTRAINED OPTIMIZATION II - KUHN TUCKER THEORY
STATIC LECTURE 4: CONSTRAINED OPTIMIZATION II - KUHN TUCKER THEORY UNIVERSITY OF MARYLAND: ECON 600 1. Some Eamples 1 A general problem that arises countless times in economics takes the form: (Verbally):
More informationECE Optimization for wireless networks Final. minimize f o (x) s.t. Ax = b,
ECE 788 - Optimization for wireless networks Final Please provide clear and complete answers. PART I: Questions - Q.. Discuss an iterative algorithm that converges to the solution of the problem minimize
More informationThe Lagrangian L : R d R m R r R is an (easier to optimize) lower bound on the original problem:
HT05: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford Convex Optimization and slides based on Arthur Gretton s Advanced Topics in Machine Learning course
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course otes for EE7C (Spring 018): Conve Optimization and Approimation Instructor: Moritz Hardt Email: hardt+ee7c@berkeley.edu Graduate Instructor: Ma Simchowitz Email: msimchow+ee7c@berkeley.edu October
More information