A Minimax Theorem with Applications to Machine Learning, Signal Processing, and Finance
|
|
- Bathsheba Lawrence
- 6 years ago
- Views:
Transcription
1 A Minimax Theorem with Applications to Machine Learning, Signal Processing, and Finance Seung Jean Kim Stephen Boyd Alessandro Magnani MIT ORC Sear 12/8/05
2 Outline A imax theorem Robust Fisher discriant analysis Robust matched filtering An extension Worst-case Sharpe ratio maximization MIT ORC Sear 12/8/05 1
3 A fractional function f(a, B, x) = at x xt Bx, a, x Rn, B S n ++ f is homogeneous, quasiconcave in x provided a T x 0: for γ 0, { {x f(a, B, x) γ} = x } γ xt Bx a T x is convex (a second-order cone) f is quasiconvex in (a, B) provided a T x 0: for γ 0, {(a, B) f(a, B, x) γ} = { (a, B) } γ xt Bx a T x is convex (since x T Bx is concave in B) f maximized over x by x = B 1 a; optimal value a T B 1 a MIT ORC Sear 12/8/05 2
4 A Rayleigh quotient f(a, B, x) 2 is Rayleigh quotient of (aa T, B) evaluated at x: f(a, B, x) 2 = (xt a) 2 x T Bx convex in (a, B) not quasiconcave in x MIT ORC Sear 12/8/05 3
5 Interpretation via Gaussian suppose z N (a, B), and x R n z T x N (a T x, x T Bx), so Prob(z T x 0) = Φ ( a T ) x xt Bx x = B 1 a gives hyperplane through origin that maximizes probability of z being on one side ( ) maximum probability is Φ at B 1 a a T B 1 a measures how well a linear function can discriate z from zero MIT ORC Sear 12/8/05 4
6 A picture expand confidence ellipsoid until it touches origin tangent is plane that maximizes Prob(z T x 0) PSfrag replacements x 0 MIT ORC Sear 12/8/05 5
7 Interpretation via Chebyshev bound suppose E z = a, E(z a) T (z a) = B (otherwise arbitrary) E z T x = a T x, E(z T x a T x) 2 = x T Bx, so by Chebyshev bound Prob(z T x 0) Ψ ψ is increasing (bound is sharp) ( a T ) x, Ψ(u) = u2 + xt Bx 1 + u 2 + x = B 1 a gives hyperplane through origin that maximizes Chebyshev lower bound for Prob(z T x 0) maximum value of Chebyshev lower bound is a T B 1 a 1 + a T B 1 a MIT ORC Sear 12/8/05 6
8 Worst-case discriation probability analysis uncertain statistics: (a, B) U R n \{0} S n ++ (convex and compact) for fixed x, find worst-case statistics: where z N (a, B) imize Prob(z T x > 0) subject to (a, B) U MIT ORC Sear 12/8/05 7
9 Worst-case discriation probability analysis worst-case discriation probability analysis is equivalent to imize subject to a T x xt Bx (a, B) U if optimal value is positive, i.e., a T x > 0 for all (a, B) U, equivalent to convex problem (a T x) 2 imize x T Bx subject to (a, B) U if optimal value is negative, can solve via bisection, convex optimization MIT ORC Sear 12/8/05 8
10 Worst-case discriation probability maximization find x that maximizes worst-case discriation probability (a,b) U Prob(zT x 0) equivalent to finding x that maximizes (a,b) U a T x xt Bx not concave in x; not clear how to solve studied in context of robust signal processing in 1980s (Poor, Verdú,... ) for some specific Us MIT ORC Sear 12/8/05 9
11 Weak imax property max x 0 (a,b) U x T a xt Bx max (a,b) U x 0 x T a xt Bx LHS: worst-case discriation probability maximization problem RHS can be evaluated via convex optimization: max (a,b) U x 0 x T a xt Bx = (a,b) U at B 1 a = [ (a,b) U at B 1 a ] 1/2 finds statistics hardest to discriate from zero MIT ORC Sear 12/8/05 10
12 Strong imax property in fact, we have equality: max x 0 (a,b) U x T a xt Bx = max (a,b) U x 0 x T a xt Bx (and common value is 0) equivalently, with z N (a, B), max x 0 (a,b) U Prob(zT x 0) = max (a,b) U x 0 Prob(zT x 0) (and common value is 1/2) MIT ORC Sear 12/8/05 11
13 Computing saddle point via convex optimization find least favorable statistics (a, B ), i.e., imize a T B 1 a over (a, B) U (x, a, B ) with x = B 1 a is saddle point: x T a xt B x x T a x T B x x T a x 0, (a, B) U x T Bx, x solves worst-case discriation probability maximization problem (a, B ) is worst-case statistics for x MIT ORC Sear 12/8/05 12
14 Interpretation of least favorable statistics find statistics with imum Mahalanobis distance between mean and zero: a T B 1 a = (a,b) U at B 1 a PSfrag replacements 0 MIT ORC Sear 12/8/05 13
15 Proof via convex analysis let (a, B ) be least favorable: (a,b) U at B 1 a = a T B 1 a by imax inequality max x 0 (a,b) U x T a xt Bx max (a,b) U x 0 x T a xt Bx = a T B 1 a x = B 1 a gives lower bound on a T B 1 a : (a,b) U x T a x T Bx max x 0 (a,b) U x T a xt Bx a T B 1 a if (a,b) U x T a x T Bx = x T a x T B x = a T B 1 a, then imax equality must hold MIT ORC Sear 12/8/05 14
16 (a, B ) is optimal for (a,b) U a T B 1 a, so 2x T (a a ) x T (B B )x 0, (a, B) U with x = B 1 a (since z imizes convex differentiable fct. f over convex set C iff f(z ) T (z z ) 0 z C) we conclude (a, B ) is optimal for (a,b) U (x T a) 2 /x T Bx, since its optimality condition is also 2x T (a a ) x T (B B )x 0, (a, B) U (a, B ) must be optimal for (a,b) U x T a/(x T Bx ) 1/2, since (a,b) U x T a/(x T Bx ) 1/2 > 0 MIT ORC Sear 12/8/05 15
17 Fisher linear discriant analysis x N (µ x, Σ x ), y N (µ y, Σ y ) find w that maximizes Prob(w T x > w T y) x y N (µ x µ y, Σ x + Σ y ), so ( ) Prob(w T x > w T w T (µ x µ y ) y) = Φ wt (Σ x + Σ y )w equivalent to maximizing w T (µ x µ y ) wt (Σ x + Σ y )w = f(µ x µ y, Σ x + Σ y, w) proposed by Fisher in 1930s MIT ORC Sear 12/8/05 16
18 Worst-case Fisher linear discriation uncertain statistics: (µ x, µ y, Σ x, Σ y ) V R n R n S n ++ S n ++ convex and compact; µ x µ y for each (µ x, µ y, Σ x, Σ y ) V worst-case discriation probability (for fixed w): (µ x,µ y,σ x,σ y ) V Prob(wT x > w T y) can compute via convex optimization MIT ORC Sear 12/8/05 17
19 Robust Fisher linear discriant analysis worst-case discriation probability maximization: find w that maximizes worst-case discriation probability equivalent to maximizing (over w) (µ x,µ y,σ x,σ y ) V Prob(wT x > w T y) (µ x,µ y,σ x,σ y ) V w T (µ x µ y ) wt (Σ x + Σ y )w our max- problem, with a = µ x µ y, B = Σ x + Σ y U = {(µ x µ y, Σ x + Σ y ) (µ x, µ y, Σ x, Σ y ) V} MIT ORC Sear 12/8/05 18
20 Example x, y R 2 ; only µ y is uncertain, µ y E PSfrag replacements E µ x µ y 0 robust optimal noal optimal MIT ORC Sear 12/8/05 19
21 Results P nom P wc noal optimal robust optimal P nom : noal discriation probability P wc : worst-case discriation probability MIT ORC Sear 12/8/05 20
22 Matched filtering y(t) = s(t)a + v(t) R n s(t) R is desired signal; y(t) R n is received signal v(t) N (0, Σ) is noise filtered output with weight vector w R n : z(t) = w T y(t) = s(t)w T a + w T v(t) standard matched filtering: choose w to maximize (amplitude) signal to noise ratio (SNR) w T a wt Σw MIT ORC Sear 12/8/05 21
23 Robust matched filtering uncertain data: (a, Σ) U R n \{0} S n ++ (convex and compact) worst-case SNR for fixed w: (a,σ) U w T a wt Σw robust matched filtering: find w that maximizes worst-case SNR can be solved via convex optimization & imax theorem for x T a/ x T Bx MIT ORC Sear 12/8/05 22
24 Example a = (2, 3, 2, 2) is fixed (no uncertainty) uncertain Σ has form 1 + 1? + 1? 1 + means Σ ij [0, 1], means Σ ij [ 1, 0]? means Σ ij [ 1, 1] (and of course Σ 0) we take noal noise covariance as Σ = MIT ORC Sear 12/8/05 23
25 Results noal SNR worst-case SNR noal optimal robust optimal least favorable covariance: MIT ORC Sear 12/8/05 24
26 Extension of imax theorem U convex, compact; X convex cone max x X (a,b) U a T x xt Bx = max (a,b) U w X a T x xt Bx provided there exists x X s.t. a T x > 0 for all a with (a, B) U, i.e., LHS > 0 MIT ORC Sear 12/8/05 25
27 Computing saddle point via convex optimization convexity of -max problem: max (a,b) U x X X is dual cone a T x xt Bx = [ (a + λ) T B 1 (a + λ) ] 1/2 (a,b) U λ X [ = (a,b) U, λ X (a + λ)t B 1 (a + λ) if (a, B, λ ) is optimal for -max problem, (x, a, B ) with x = B 1 (a + λ ) is a saddle point: x T a xt B x x T a x T B x ] 1/2 x T a x X, (a, B) U x T Bx, MIT ORC Sear 12/8/05 26
28 A key lemma (a T x) 2 max x X x T Bx = λ X (a + λ)t B 1 (a + λ) using homogeneity of (a T x) 2 /x T Bx, write LHS as by convex duality max x X x T Bx = x X, a T x=1 (a T x) 2 x T Bx = [ x T Bx x X, a T x=1 ] 1 [ max (1/4)( λ + µa) T B 1 ( λ + µa) + µ ] λ X, µ R defining λ = λ/µ and optimizing over µ, RHS becomes x T 1 Bx = max x X, a T x=1 λ X (a + λ) T B 1 (a + λ) MIT ORC Sear 12/8/05 27
29 Asset allocation n risky assets with (single period) returns a N (µ, Σ) portfolio w W (convex); 1 T w = 1 for all w W w T a N (w T µ, w T Σw), so probability of beating risk-free asset with return µ rf is ( µ Prob(a T T ) w µ rf w > µ rf ) = Φ wt Σw maximized by w W that maximizes Sharpe ratio µt w µ rf wt Σw (called tangency portfolio w tp ) we re only interested in case when maximum Sharpe ratio 0 MIT ORC Sear 12/8/05 28
30 Interpretation via Chebyshev bound suppose E a = µ, E (a µ) T (a µ) = Σ (otherwise arbitrary) E a T w = µ T x, E (a T x µ T x) 2 = w T Σw, so by Chebyshev bound, Prob(a T w µ rf ) Ψ increasing (bound is tight) maximized by tangency portfolio ( µ T w µ rf wt Σw ), Ψ(u) = u u 2 + MIT ORC Sear 12/8/05 29
31 PSfrag replacements CML EF tangency portfolio µ T w maximum Sharpe ratio µ rf wt Σw EF: efficient frontier; CML: capital market line MIT ORC Sear 12/8/05 30
32 Asset allocation with a risk-free asset affine combination of risk-free asset and risky portfolio w W: [ ] (1 θ)w x = θ θ is amount of risk-free asset Tobin s two-fund separation theorem: risk s and return r of any x = ((1 θ)w, θ) with w W cannot lie above capital market line (CML) r = µ rf + S s, S = max w W µ T w µ rf wt Σw CML is formed by affine combinations of two portfolios: tangency portfolio (w tp, 0) and risk-free portfolio (0, 1) MIT ORC Sear 12/8/05 31
33 Worst-case Sharpe ratio uncertain statistics: (µ, Σ) V R n S n ++ (convex and compact) for fixed w W, find worst-case statistics by computing worst-case SR (µ,σ) V µ T w µ rf wt Σw = (µ,σ) V (µ µ rf 1) T w wt Σw... can be computed via convex optimization related to worst-case probability of beating risk-free asset ( (µ,σ) V Prob(aT w > µ rf ) = Φ (µ,σ) V µ T w µ rf wt Σw ) MIT ORC Sear 12/8/05 32
34 Worst-case Sharpe ratio maximization find portfolio that maximizes worst-case SR: maximize subject to (µ,σ) V w W µ T w µ rf wt Σw this portfolio (called robust tangency portfolio w rtp ) maximizes worst-case probability of beating risk-free asset (µ,σ) V Prob(wT a > µ rf ), a N (µ, Σ) worst-case Chebyshev lower bound for Prob(a T w µ rf ) (µ,σ) V inf{prob(at w µ rf ) E a = µ, E (a µ) T (a µ) = Σ} MIT ORC Sear 12/8/05 33
35 Solution via imax 1 T w = 1 for all w W so µ T w µ rf wt Σw = (µ µ rf1) T w wt Σw, w W can use imax theorem for a T x/ x T Bx, with a = µ µ rf 1, B = Σ, X = RW U = {(µ µ rf 1, Σ) (µ, Σ) V} MIT ORC Sear 12/8/05 34
36 Minimax result for Sharpe ratio suppose there exists portfolio w s.t. µ T w > µ rf for all (µ, Σ) V strong imax property: 0 < max w W (µ,σ) V µ T w µ rf wt Σw = (µ,σ) V max w W µ T w µ rf wt Σw saddle point (w, µ, Σ ) can be computed via convex optimization µ T w µ rf wt Σ w µ T w µ rf w T Σ w µt w µ rf, w W, (µ, Σ) V w T Σw MIT ORC Sear 12/8/05 35
37 Saddle-point property PSfrag replacements EF for (µ, Σ ) P(w ) ` w T Σ w, µ T w µ T w µ rf P(w ) = wt Σw {(µ T w, w T Σw ) (µ, Σ) V } is risk-return set of w MIT ORC Sear 12/8/05 36
38 PSfrag replacements Example 7 risky assets with noal returns µ i, variances σ i 2, correlation matrix Ω 10% µi µ rf = Ω = σ i 30% MIT ORC Sear 12/8/05 37
39 Example total short position is limited to 30% of total long position mean uncertainty model: 1 T µ 1 T µ T µ, µ i µ i 0.2 µ i, i = 1,..., 6 covariance uncertainty model: Σ Σ F 0.1 Σ F, Σ ij Σ ij 0.2 Σ ij, i, j = 1,..., 6 Σ is noal covariance MIT ORC Sear 12/8/05 38
40 Results noal SR worst-case SR noal MP robust MP P nom P wc noal MP robust MP P nom : probability of beating risk-free asset with noal statistics ( µ, Σ) P wc : probability of beating risk-free asset with worst-case statistics MIT ORC Sear 12/8/05 39
41 Two-fund separation under model uncertainty robust two-fund separation theorem: risk-return set of any x = ((1 θ)w, θ) with w W cannot lie above robust capital market line (RCML) r = µ rf + S s µ T w µ rf wt Σw = with slope S = max max w W (µ,σ) V (µ,σ) V w W RCML is formed by affine combinations of two portfolios: µ T w µ rf wt Σw robust tangency portfolio (w rtp, 0), risk-free portfolio (0, 1) for V = {(µ, Σ)}, reduces to Tobin s two-fund separation theorem MIT ORC Sear 12/8/05 40
42 Two-fund separation under model uncertainty P(1.5w, 0.5) P(w, 0) PSfrag replacements return P(0, 1) = {(0, µ rf )} P(0.5w, 0.5) S P((1 θ)w, θ) risk P((1 θ)w, θ) is risk-return set of ((1 θ)w, θ) MIT ORC Sear 12/8/05 41
43 Summary & comments the function f(a, B, x) = at x xt Bx comes up alot what was known (??): simple imax result, no constraints on x, product form for U what is new (??): general imax theorem; convex optimization method for computing saddle point imax theorem for TrAX TrBX imax theorem for xt Ax x T Bx holds (with conditions) generally false MIT ORC Sear 12/8/05 42
44 References Kim and Boyd, Two-fund separation under model uncertainty, in preparation Kim, Magnani, and Boyd, Robust Fisher discriant analysis, Verdú and Poor, On Minimax Robustness: A General Approach and Applications, IEEE Transactions on Information Theory, 30(2): , 1984 Sion, On general imax theorems, Pacific Journal of Mathematics, 8(1): , 1958 MIT ORC Sear 12/8/05 43
A Minimax Theorem with Applications to Machine Learning, Signal Processing, and Finance
A Minimax Theorem with Applications to Machine Learning, Signal Processing, and Finance Seung-Jean Kim Stephen Boyd Information Systems Laboratory Electrical Engineering Department, Stanford University
More informationRobust Fisher Discriminant Analysis
Robust Fisher Discriminant Analysis Seung-Jean Kim Alessandro Magnani Stephen P. Boyd Information Systems Laboratory Electrical Engineering Department, Stanford University Stanford, CA 94305-9510 sjkim@stanford.edu
More informationRobust Efficient Frontier Analysis with a Separable Uncertainty Model
Robust Efficient Frontier Analysis with a Separable Uncertainty Model Seung-Jean Kim Stephen Boyd October 2007 Abstract Mean-variance (MV) analysis is often sensitive to model mis-specification or uncertainty,
More informationChance constrained optimization
Chance constrained optimization chance constraints and percentile optimization chance constraints for log-concave distributions convex approximation of chance constraints sources: Rockafellar & Uryasev,
More informationApplications of Robust Optimization in Signal Processing: Beamforming and Power Control Fall 2012
Applications of Robust Optimization in Signal Processing: Beamforg and Power Control Fall 2012 Instructor: Farid Alizadeh Scribe: Shunqiao Sun 12/09/2012 1 Overview In this presentation, we study the applications
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationConvex Optimization in Classification Problems
New Trends in Optimization and Computational Algorithms December 9 13, 2001 Convex Optimization in Classification Problems Laurent El Ghaoui Department of EECS, UC Berkeley elghaoui@eecs.berkeley.edu 1
More informationCS295: Convex Optimization. Xiaohui Xie Department of Computer Science University of California, Irvine
CS295: Convex Optimization Xiaohui Xie Department of Computer Science University of California, Irvine Course information Prerequisites: multivariate calculus and linear algebra Textbook: Convex Optimization
More informationConvex optimization problems. Optimization problem in standard form
Convex optimization problems optimization problem in standard form convex optimization problems linear optimization quadratic optimization geometric programming quasiconvex optimization generalized inequality
More informationPortfolios that Contain Risky Assets 17: Fortune s Formulas
Portfolios that Contain Risky Assets 7: Fortune s Formulas C. David Levermore University of Maryland, College Park, MD Math 420: Mathematical Modeling April 2, 208 version c 208 Charles David Levermore
More informationIntroduction to Convex Optimization
Introduction to Convex Optimization Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2018-19, HKUST, Hong Kong Outline of Lecture Optimization
More informationOptimal Investment Strategies: A Constrained Optimization Approach
Optimal Investment Strategies: A Constrained Optimization Approach Janet L Waldrop Mississippi State University jlc3@ramsstateedu Faculty Advisor: Michael Pearson Pearson@mathmsstateedu Contents Introduction
More informationRobust Rayleigh quotient minimization and nonlinear eigenvalue problems
Robust Rayleigh quotient imization and nonlinear eigenvalue problems Zhaojun Bai, Ding Lu, and Bart Vandereycken August 1, 2018 Abstract We study the robust Rayleigh quotient optimization problem where
More informationMarkowitz Efficient Portfolio Frontier as Least-Norm Analytic Solution to Underdetermined Equations
Markowitz Efficient Portfolio Frontier as Least-Norm Analytic Solution to Underdetermined Equations Sahand Rabbani Introduction Modern portfolio theory deals in part with the efficient allocation of investments
More informationRobust portfolio selection under norm uncertainty
Wang and Cheng Journal of Inequalities and Applications (2016) 2016:164 DOI 10.1186/s13660-016-1102-4 R E S E A R C H Open Access Robust portfolio selection under norm uncertainty Lei Wang 1 and Xi Cheng
More informationCSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization
CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization April 6, 2018 1 / 34 This material is covered in the textbook, Chapters 9 and 10. Some of the materials are taken from it. Some of
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More informationEE 227A: Convex Optimization and Applications April 24, 2008
EE 227A: Convex Optimization and Applications April 24, 2008 Lecture 24: Robust Optimization: Chance Constraints Lecturer: Laurent El Ghaoui Reading assignment: Chapter 2 of the book on Robust Optimization
More informationDuality Methods in Portfolio Allocation with Transaction Constraints and Uncertainty
Duality Methods in Portfolio Allocation with Transaction Constraints and Uncertainty Christopher W. Miller Department of Mathematics University of California, Berkeley January 9, 2014 Project Overview
More informationModern Portfolio Theory with Homogeneous Risk Measures
Modern Portfolio Theory with Homogeneous Risk Measures Dirk Tasche Zentrum Mathematik Technische Universität München http://www.ma.tum.de/stat/ Rotterdam February 8, 2001 Abstract The Modern Portfolio
More informationA Brief Review on Convex Optimization
A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 Minimum volume ellipsoid around a set Löwner-John ellipsoid
More informationGaussian Estimation under Attack Uncertainty
Gaussian Estimation under Attack Uncertainty Tara Javidi Yonatan Kaspi Himanshu Tyagi Abstract We consider the estimation of a standard Gaussian random variable under an observation attack where an adversary
More informationCSCI : Optimization and Control of Networks. Review on Convex Optimization
CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Stanford University (Joint work with Persi Diaconis, Arpita Ghosh, Seung-Jean Kim, Sanjay Lall, Pablo Parrilo, Amin Saberi, Jun Sun, Lin
More informationRobust Optimization: Applications in Portfolio Selection Problems
Robust Optimization: Applications in Portfolio Selection Problems Vris Cheung and Henry Wolkowicz WatRISQ University of Waterloo Vris Cheung (University of Waterloo) Robust optimization 2009 1 / 19 Outline
More informationBayesian Optimization
Practitioner Course: Portfolio October 15, 2008 No introduction to portfolio optimization would be complete without acknowledging the significant contribution of the Markowitz mean-variance efficient frontier
More informationDistributionally robust optimization techniques in batch bayesian optimisation
Distributionally robust optimization techniques in batch bayesian optimisation Nikitas Rontsis June 13, 2016 1 Introduction This report is concerned with performing batch bayesian optimization of an unknown
More informationMFM Practitioner Module: Risk & Asset Allocation. John Dodson. January 25, 2012
MFM Practitioner Module: Risk & Asset Allocation January 25, 2012 Optimizing Allocations Once we have 1. chosen the markets and an investment horizon 2. modeled the markets 3. agreed on an objective with
More informationMFM Practitioner Module: Risk & Asset Allocation. John Dodson. February 18, 2015
MFM Practitioner Module: Risk & Asset Allocation February 18, 2015 No introduction to portfolio optimization would be complete without acknowledging the significant contribution of the Markowitz mean-variance
More informationThe Lagrangian L : R d R m R r R is an (easier to optimize) lower bound on the original problem:
HT05: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford Convex Optimization and slides based on Arthur Gretton s Advanced Topics in Machine Learning course
More informationMinimax Problems. Daniel P. Palomar. Hong Kong University of Science and Technolgy (HKUST)
Mini Problems Daniel P. Palomar Hong Kong University of Science and Technolgy (HKUST) ELEC547 - Convex Optimization Fall 2009-10, HKUST, Hong Kong Outline of Lecture Introduction Matrix games Bilinear
More informationA Second order Cone Programming Formulation for Classifying Missing Data
A Second order Cone Programming Formulation for Classifying Missing Data Chiranjib Bhattacharyya Department of Computer Science and Automation Indian Institute of Science Bangalore, 560 012, India chiru@csa.iisc.ernet.in
More informationMathematics for Economics and Finance
Mathematics for Economics and Finance Michael Harrison and Patrick Waldron B 375482 Routledge Taylor & Francis Croup LONDON AND NEW YORK Contents List of figures ix List of tables xi Foreword xiii Preface
More informationFE670 Algorithmic Trading Strategies. Stevens Institute of Technology
FE670 Algorithmic Trading Strategies Lecture 8. Robust Portfolio Optimization Steve Yang Stevens Institute of Technology 10/17/2013 Outline 1 Robust Mean-Variance Formulations 2 Uncertain in Expected Return
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Abstract. We consider the problem of choosing the edge weights of an undirected graph so as to maximize or minimize some function of the
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 1 Minimum volume ellipsoid around a set Löwner-John ellipsoid
More informationNonlinear Programming Models
Nonlinear Programming Models Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Nonlinear Programming Models p. Introduction Nonlinear Programming Models p. NLP problems minf(x) x S R n Standard form:
More informationminimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1,
4 Duality 4.1 Numerical perturbation analysis example. Consider the quadratic program with variables x 1, x 2, and parameters u 1, u 2. minimize x 2 1 +2x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x
More informationLecture: Examples of LP, SOCP and SDP
1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationDisciplined Convex Programming
Disciplined Convex Programming Stephen Boyd Michael Grant Electrical Engineering Department, Stanford University University of Pennsylvania, 3/30/07 Outline convex optimization checking convexity via convex
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationMinimum Error Rate Classification
Minimum Error Rate Classification Dr. K.Vijayarekha Associate Dean School of Electrical and Electronics Engineering SASTRA University, Thanjavur-613 401 Table of Contents 1.Minimum Error Rate Classification...
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationSubgradients. subgradients and quasigradients. subgradient calculus. optimality conditions via subgradients. directional derivatives
Subgradients subgradients and quasigradients subgradient calculus optimality conditions via subgradients directional derivatives Prof. S. Boyd, EE392o, Stanford University Basic inequality recall basic
More informationMaximum likelihood estimation
Maximum likelihood estimation Guillaume Obozinski Ecole des Ponts - ParisTech Master MVA Maximum likelihood estimation 1/26 Outline 1 Statistical concepts 2 A short review of convex analysis and optimization
More informationRobustness and bootstrap techniques in portfolio efficiency tests
Robustness and bootstrap techniques in portfolio efficiency tests Dept. of Probability and Mathematical Statistics, Charles University, Prague, Czech Republic July 8, 2013 Motivation Portfolio selection
More informationLecture 2: Convex functions
Lecture 2: Convex functions f : R n R is convex if dom f is convex and for all x, y dom f, θ [0, 1] f is concave if f is convex f(θx + (1 θ)y) θf(x) + (1 θ)f(y) x x convex concave neither x examples (on
More informationConvex Optimization M2
Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial
More informationThomas Knispel Leibniz Universität Hannover
Optimal long term investment under model ambiguity Optimal long term investment under model ambiguity homas Knispel Leibniz Universität Hannover knispel@stochastik.uni-hannover.de AnStAp0 Vienna, July
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More information7. Statistical estimation
7. Statistical estimation Convex Optimization Boyd & Vandenberghe maximum likelihood estimation optimal detector design experiment design 7 1 Parametric distribution estimation distribution estimation
More informationConvex Functions. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Convex Functions Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Definition convex function Examples
More informationSecond-order cone programming formulation for two player zero-sum game with chance constraints
Second-order cone programming formulation for two player zero-sum game with chance constraints Vikas Vikram Singh a, Abdel Lisser a a Laboratoire de Recherche en Informatique Université Paris Sud, 91405
More informationA General Projection Property for Distribution Families
A General Projection Property for Distribution Families Yao-Liang Yu Yuxi Li Dale Schuurmans Csaba Szepesvári Department of Computing Science University of Alberta Edmonton, AB, T6G E8 Canada {yaoliang,yuxi,dale,szepesva}@cs.ualberta.ca
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization
Semidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization Instructor: Farid Alizadeh Author: Ai Kagawa 12/12/2012
More informationMachine Learning. Support Vector Machines. Manfred Huber
Machine Learning Support Vector Machines Manfred Huber 2015 1 Support Vector Machines Both logistic regression and linear discriminant analysis learn a linear discriminant function to separate the data
More informationConvex optimization examples
Convex optimization examples multi-period processor speed scheduling minimum time optimal control grasp force optimization optimal broadcast transmitter power allocation phased-array antenna beamforming
More informationROBUST LINEAR DISCRIMINANT ANALYSIS WITH A LAPLACIAN ASSUMPTION ON PROJECTION DISTRIBUTION
ROBUST LINEAR DISCRIMINANT ANALYSIS WITH A LAPLACIAN ASSUMPTION ON PROJECTION DISTRIBUTION Shujian Yu, Zheng Cao, Xiubao Jiang Department of Electrical and Computer Engineering, University of Florida 0
More informationWORST-CASE VALUE-AT-RISK AND ROBUST PORTFOLIO OPTIMIZATION: A CONIC PROGRAMMING APPROACH
WORST-CASE VALUE-AT-RISK AND ROBUST PORTFOLIO OPTIMIZATION: A CONIC PROGRAMMING APPROACH LAURENT EL GHAOUI Department of Electrical Engineering and Computer Sciences, University of California, Berkeley,
More informationStochastic Subgradient Methods
Stochastic Subgradient Methods Stephen Boyd and Almir Mutapcic Notes for EE364b, Stanford University, Winter 26-7 April 13, 28 1 Noisy unbiased subgradient Suppose f : R n R is a convex function. We say
More informationNew Rank-One Matrix Decomposition Techniques and Applications to Signal Processing
New Rank-One Matrix Decomposition Techniques and Applications to Signal Processing Yongwei Huang Hong Kong Baptist University SPOC 2012 Hefei China July 1, 2012 Outline Trust-region subproblems in nonlinear
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history
More informationMathematical Optimization Models and Applications
Mathematical Optimization Models and Applications Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 1, 2.1-2,
More informationLECTURE 9 LECTURE OUTLINE. Min Common/Max Crossing for Min-Max
Min-Max Problems Saddle Points LECTURE 9 LECTURE OUTLINE Min Common/Max Crossing for Min-Max Given φ : X Z R, where X R n, Z R m consider minimize sup φ(x, z) subject to x X and maximize subject to z Z.
More informationCharacterizing Robust Solution Sets of Convex Programs under Data Uncertainty
Characterizing Robust Solution Sets of Convex Programs under Data Uncertainty V. Jeyakumar, G. M. Lee and G. Li Communicated by Sándor Zoltán Németh Abstract This paper deals with convex optimization problems
More informationThe Asymptotic Theory of Transaction Costs
The Asymptotic Theory of Transaction Costs Lecture Notes by Walter Schachermayer Nachdiplom-Vorlesung, ETH Zürich, WS 15/16 1 Models on Finite Probability Spaces In this section we consider a stock price
More informationWE consider an array of sensors. We suppose that a narrowband
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 56, NO. 4, APRIL 2008 1539 Robust Beamforming via Worst-Case SINR Maximization Seung-Jean Kim, Member, IEEE, Alessandro Magnani, Almir Mutapcic, Student Member,
More informationMathematical Foundations -1- Convexity and quasi-convexity. Convex set Convex function Concave function Quasi-concave function Supporting hyperplane
Mathematical Foundations -1- Convexity and quasi-convexity Convex set Convex function Concave function Quasi-concave function Supporting hyperplane Mathematical Foundations -2- Convexity and quasi-convexity
More informationOn deterministic reformulations of distributionally robust joint chance constrained optimization problems
On deterministic reformulations of distributionally robust joint chance constrained optimization problems Weijun Xie and Shabbir Ahmed School of Industrial & Systems Engineering Georgia Institute of Technology,
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationRobust Portfolio Selection Based on a Joint Ellipsoidal Uncertainty Set
Robust Portfolio Selection Based on a Joint Ellipsoidal Uncertainty Set Zhaosong Lu March 15, 2009 Revised: September 29, 2009 Abstract The separable uncertainty sets have been widely used in robust portfolio
More informationRobust Predictions in Games with Incomplete Information
Robust Predictions in Games with Incomplete Information joint with Stephen Morris (Princeton University) November 2010 Payoff Environment in games with incomplete information, the agents are uncertain
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationOptimization Problems with Probabilistic Constraints
Optimization Problems with Probabilistic Constraints R. Henrion Weierstrass Institute Berlin 10 th International Conference on Stochastic Programming University of Arizona, Tucson Recommended Reading A.
More informationDistributionally Robust Convex Optimization
Submitted to Operations Research manuscript OPRE-2013-02-060 Authors are encouraged to submit new papers to INFORMS journals by means of a style file template, which includes the journal title. However,
More informationPortfolio Selection under Model Uncertainty:
manuscript No. (will be inserted by the editor) Portfolio Selection under Model Uncertainty: A Penalized Moment-Based Optimization Approach Jonathan Y. Li Roy H. Kwon Received: date / Accepted: date Abstract
More informationGaussian discriminant analysis Naive Bayes
DM825 Introduction to Machine Learning Lecture 7 Gaussian discriminant analysis Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. is 2. Multi-variate
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationExploring Granger Causality for Time series via Wald Test on Estimated Models with Guaranteed Stability
Exploring Granger Causality for Time series via Wald Test on Estimated Models with Guaranteed Stability Nuntanut Raksasri Jitkomut Songsiri Department of Electrical Engineering, Faculty of Engineering,
More informationIE 521 Convex Optimization
Lecture 14: and Applications 11th March 2019 Outline LP SOCP SDP LP SOCP SDP 1 / 21 Conic LP SOCP SDP Primal Conic Program: min c T x s.t. Ax K b (CP) : b T y s.t. A T y = c (CD) y K 0 Theorem. (Strong
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 0-0 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More information44 CHAPTER 2. BAYESIAN DECISION THEORY
44 CHAPTER 2. BAYESIAN DECISION THEORY Problems Section 2.1 1. In the two-category case, under the Bayes decision rule the conditional error is given by Eq. 7. Even if the posterior densities are continuous,
More informationChapter 1. Introduction. 1.1 Problem statement and examples
Chapter 1 Introduction 1.1 Problem statement and examples In this course we will deal with optimization problems. Such problems appear in many practical settings in almost all aspects of science and engineering.
More informationRobust Markowitz portfolio selection. ambiguous covariance matrix
under ambiguous covariance matrix University Paris Diderot, LPMA Sorbonne Paris Cité Based on joint work with A. Ismail, Natixis MFO March 2, 2017 Outline Introduction 1 Introduction 2 3 and Sharpe ratio
More informationMachine Learning. Support Vector Machines. Fabio Vandin November 20, 2017
Machine Learning Support Vector Machines Fabio Vandin November 20, 2017 1 Classification and Margin Consider a classification problem with two classes: instance set X = R d label set Y = { 1, 1}. Training
More informationMathematical Economics: Lecture 16
Mathematical Economics: Lecture 16 Yu Ren WISE, Xiamen University November 26, 2012 Outline 1 Chapter 21: Concave and Quasiconcave Functions New Section Chapter 21: Concave and Quasiconcave Functions Concave
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization
More informationConvex Optimization: Applications
Convex Optimization: Applications Lecturer: Pradeep Ravikumar Co-instructor: Aarti Singh Convex Optimization 1-75/36-75 Based on material from Boyd, Vandenberghe Norm Approximation minimize Ax b (A R m
More informationRisk Management of Portfolios by CVaR Optimization
Risk Management of Portfolios by CVaR Optimization Thomas F. Coleman a Dept of Combinatorics & Optimization University of Waterloo a Ophelia Lazaridis University Research Chair 1 Joint work with Yuying
More informationRobust portfolio selection based on a joint ellipsoidal uncertainty set
Optimization Methods & Software Vol. 00, No. 0, Month 2009, 1 16 Robust portfolio selection based on a joint ellipsoidal uncertainty set Zhaosong Lu* Department of Mathematics, Simon Fraser University,
More informationAmbiguity in portfolio optimization
May/June 2006 Introduction: Risk and Ambiguity Frank Knight Risk, Uncertainty and Profit (1920) Risk: the decision-maker can assign mathematical probabilities to random phenomena Uncertainty: randomness
More information6.079/6.975 S. Boyd & P. Parrilo December 10 11, Final exam
6.079/6.975 S. Boyd & P. Parrilo December 10 11, 2009. Final exam This is a 24 hour take-home final exam. Please turn it in to Professor Stephen Boyd, (Stata Center), on Friday December 11, at 5PM (or
More informationRobust Novelty Detection with Single Class MPM
Robust Novelty Detection with Single Class MPM Gert R.G. Lanckriet EECS, U.C. Berkeley gert@eecs.berkeley.edu Laurent El Ghaoui EECS, U.C. Berkeley elghaoui@eecs.berkeley.edu Michael I. Jordan Computer
More informationQuadratic Two-Stage Stochastic Optimization with Coherent Measures of Risk
Noname manuscript No. (will be inserted by the editor) Quadratic Two-Stage Stochastic Optimization with Coherent Measures of Risk Jie Sun Li-Zhi Liao Brian Rodrigues Received: date / Accepted: date Abstract
More informationNotes on Recursive Utility. Consider the setting of consumption in infinite time under uncertainty as in
Notes on Recursive Utility Consider the setting of consumption in infinite time under uncertainty as in Section 1 (or Chapter 29, LeRoy & Werner, 2nd Ed.) Let u st be the continuation utility at s t. That
More informationRobust Portfolio Selection Based on a Multi-stage Scenario Tree
Robust Portfolio Selection Based on a Multi-stage Scenario Tree Ruijun Shen Shuzhong Zhang February 2006; revised November 2006 Abstract The aim of this paper is to apply the concept of robust optimization
More information