Approximate Optimal Designs for Multivariate Polynomial Regression
|
|
- Hester O’Neal’
- 5 years ago
- Views:
Transcription
1 Approximate Optimal Designs for Multivariate Polynomial Regression Fabrice Gamboa Collaboration with: Yohan de Castro, Didier Henrion, Roxana Hess, Jean-Bernard Lasserre Universität Potsdam 16th of February 2018
2 Short bibliography Convex design Polynomial regression Moment spaces Moment formulation SDP relaxation Examples Agenda 1 Short bibliography 2 Convex design 3 Polynomial regression 4 Moment spaces 5 Moment formulation 6 SDP relaxation 7 Examples F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
3 Overview 1 Short bibliography 2 Convex design 3 Polynomial regression 4 Moment spaces 5 Moment formulation 6 SDP relaxation 7 Examples F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
4 Short bibliography : Optimal design H. Dette and W. J. Studden. The theory of canonical moments with applications in statistics, probability, and analysis, volume 338. John Wiley & Sons, J. Kiefer. General equivalence theory for optimum designs (approximate theory). The annals of Statistics, pages , I. Molchanov and S. Zuyev. Optimisation in space of measures and optimal design. ESAIM : Probability and Statistics, 8 :12 24, F. Pukelsheim. Optimal design of experiments. SIAM, G. Sagnol and R. Harman. Computing exact D-optimal designs by mixed integer second-order cone programming. The Annals of Statistics, 43(5) : , F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
5 Short bibliography : Optimization J. B. Lasserre. Moments, positive polynomials and their applications, volume 1 of Imperial College Press Optimization Series. Imperial College Press, London, J. B. Lasserre and T. Netzer. SOS approximations of nonnegative polynomials via simple high degree perturbations. Mathematische Zeitschrift, 256(1) :99 112, A. S. Lewis. Convex analysis on the Hermitian matrices. SIAM Journal on Optimization, 6(1) : , L. Vandenberghe, S. Boyd, and S.-P. Wu. Determinant maximization with linear matrix inequality constraints. SIAM journal on matrix analysis and applications, 19(2) : , F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
6 Overview 1 Short bibliography 2 Convex design 3 Polynomial regression 4 Moment spaces 5 Moment formulation 6 SDP relaxation 7 Examples F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
7 Classical linear model What are you dealing with : Regression model F : X R n R p, continuous. Noisy linear model observed at t i X, i = 1,..., N z i = θ, F(t i ) + ε i. θ R p unknown, ε second order homoscedastic centred white noise Best choice for t i X, i = 1,..., N? F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
8 Information matrix Normalized inverse covariance matrix of the optimal linear unbiased estimate of θ (Gauss Markov) M(ξ) = 1 N N F(t i )F T (t i ) = i=1 l w i F(x i )F T (x i ). t i, i = 1,, N are picked Nw i times within x i, i = 1,, l, (l < N) ( ) x1 x ξ = 2 x l w 1 w 2 w l i=1 w j = n j N, Simplification of the frame (if not discrete optimisation) 0 < w j < 1, w j = 1 Best choice for the design ξ? F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
9 Concave matricial criteria Wish to maximize the information matrix with respect to ξ M(ξ) = l w i F(x i )F T (x i ) = i=1 ( ) x1 x ξ = 2 x l w 1 w 2 w l X σ ξ (dx) := F(x)F T (x)dσ ξ. l w j δ xj (dx). Concave criteria for symmetric p p non negative matrix : φ q (q [, 1]) ( 1 p trace(mq )) 1/q if q, 0 M > 0, φ q (M) := det(m) 1/p if q = 0 λ min (M) if q = { ( 1 det M = 0, φ q (M) := p trace(mq )) 1/q if q (0, 1] 0 if q [, 0]. j=1 F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
10 Optimal design Wish to maximize φ q (M(ξ)) ( with respect to ξ) ( ) x1 x ξ = 2 x l w 1 w 2 w l l σ ξ (dx) := w j δ xj (dx). j=1 φ q (M) concave with respect to M and positively homogenous, φ q (M) is isotonic with respect to Loewner ordering Main idea : extend the optimization problem to all probability measures M(P) = F(x)F T (x)dp(x), P P(X). Within the solutions build one with finite support X F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
11 Overview 1 Short bibliography 2 Convex design 3 Polynomial regression 4 Moment spaces 5 Moment formulation 6 SDP relaxation 7 Examples F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
12 Our frame : polynomial regression R[x] real polynomials in the variables x = (x 1,..., x n ) d N R[x] d := {p R[x] : deg p d} deg p := total degree of p Assumption F = (f 1,..., f p ) (R[x] d ) p. X R n is a given closed basic semi-algebraic set X := {x R m : g j (x) 0, j = 1,..., m} (1) g j R[x], deg g j = d j, j = 1,..., m, and X compact e.g : g 1 (x) := R 2 x 2 F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
13 Overview 1 Short bibliography 2 Convex design 3 Polynomial regression 4 Moment spaces 5 Moment formulation 6 SDP relaxation 7 Examples F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
14 Some facts and notations x α 1 1 xα n n, α = (α 1,..., α n ) N n basis of R[x] (x α := x α 1 1 xα n R[x] d has dimension s(d) := ( ) n+d n. Basis (x α ) α d, α := α α n. n ) v d (x) := ( 1, x }{{} 1,..., x n, x 2 1 }{{}, x 1x 2,..., x 1 x n, x 2 2,..., x2 n,...,..., x d 1 }{{},..., xd n ) }{{} degree 0 degree 1 degree 2 degree d There exists a unique matrix A of size p ( ) n+d n such that x X, F(x) = A v d (x). (2) M + (X) is the cone of nonnegative Borel measures supported on X (the dual of cone of nonnegative elements of C(X)) F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
15 Moments, the moment cone and the moment matrix µ M + (X)α N n, y α = y α (µ) = X x α dµ y = y α (µ) = (y α ) α N n moment sequence of µ M d (X) moment cone=convex cone of truncated sequences M d (X) := {y R (n+d n ) : µ M+ (X) s.t. (3) } y α = x α dµ, α N n, α d. P d (X) convex cone of nonnegative polynomials of degree d (may be also viewed as a vector of coefficients) M d (X) = P d (X) and P d (X) = M d (X) If X = [a, b], M d (X) is representable using positive semidefinitness of Hankel matrices. No more true in general F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31 X
16 Generalized Hankel matrix I Sequence y = (y α ) α N n is given, build linear form L y : R[x] R mapping a polynomials f = α N n f αx α to L y (f) = α N n f αy α. Theorem A sequence y = (y α ) α N n has a representing measure µ supported on X if and only if L y (f) 0 for all polynomials f R[x] nonnegative on X. The generalized Hankel matrix associated to a truncated sequence y = (y α ) α 2d M d (y)(α, β) = L y (x α x β ) = y α+β, ( α, β d). (M d (y)(α, β)) is symmetric (M d (y)(α, β)) is linear in y If y has a representing measure, then M d (y) 0 F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
17 Generalized Hankel matrix II Generalized Hankel matrix associated to polynomial f = α r f αx α R[x] r and a sequence y = (y α ) α 2d+r M d (fy)(α, β) = L y (f(x) x α x β ) = γ r f γ y γ+α+β, ( α, β d) If y has a representing measure µ, and Suppµ {x R n : f(x) 0} then M d (fy) 0 The converse statement holds in the infinite case y = (y α ) α N n has a representing measure µ M + (X) d N, M d (y) 0 and M d (g j y) 0, j = 1,..., m F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
18 Approximations of the moment cone Set v j := d j /2, j = 1,..., m, (half the degree of the g j ). For δ N, M 2d (X) can be approximate by { M SDP 2(d+δ) (X) := y d,δ R (n+2d n ) : yδ R (n+2(d+δ) n ) such that (4) Good approximation y d,δ = (y δ,α ) α 2d and } M d+δ (y δ ) 0, M d+δ vj (g j y δ ) 0, j = 1,..., m. M 2d (X) M SDP 2(d+2) (X) MSDP 2(d+1) (X) MSDP 2d (X). This hierarchy converges M 2d (X) = δ=0 MSDP 2(d+δ) (X) F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
19 Overview 1 Short bibliography 2 Convex design 3 Polynomial regression 4 Moment spaces 5 Moment formulation 6 SDP relaxation 7 Examples F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
20 Optimal design moment formulation For l ( ) n+2d n and W the simplex Carathéodory s theorem gives { } y M 2d (X) : y 0 = 1 = {y R (n+2d n ) : yα = x α dµ α 2d, X l } µ = w i δ xi, x i X, w W, i=1 Our design problem becomes : Step 1 find y solution ρ = max φ q (M) (5) s.t. M = A γ y γ 0, γ 2d y M 2d (X), y 0 = 1, Step 2 find an atomic measure µ representing y F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
21 Equivalence Theorem Theorem (Equivalence theorem) Let q (, 1), Step 1 is is a convex optimization problem with a unique optimal solution y M 2d (X). Denote by p d the polynomial x p d (x) := v d(x) M d (y ) q 1 v d (x) = M d (y ) q 1 2 v d (x) 2 2. (6) y is the vector of moments of a discrete measure µ supported on at least ( ) ( n+d n and at most n+2d ) n points in the set (Step 2) } Ω := {x X : trace(m d (y ) q ) p d (x) = 0. y { M 2d (X) is the unique solution } to Step 1 iff y y M 2d (X) : y 0 = 1 and p :=trace(m d (y ) q ) p d 0 on X. F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
22 Overview 1 Short bibliography 2 Convex design 3 Polynomial regression 4 Moment spaces 5 Moment formulation 6 SDP relaxation 7 Examples F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
23 The SDP relaxation scheme Ideal moment Problem on M 2d (X) is not SDP representable Use the outer approximations of Step 1 ρ δ = max y s.t. φ q (M d (y)) y M SDP 2(d+δ) (X), y 0 = 1. ( δ > 0, ρ δ ρ.) Theorem (Equivalence theorem for SDP relaxation) Let q (, 1), 1 SDP relaxed Problem has a unique optimal solution y R (n+2d n ). 2 M d (y ) is positive definite. p := trace(m d (y ) q ) p d is non-negative on X and L y (p ) = 0. If y is coming from a measure µ then ρ δ = ρ and yd,δ is the solution of unrelaxed Step 1 F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
24 Asymptotics on δ Theorem Let q (, 1), y d,δ optimal solution of relaxed, p d,δ R[x] 2d dual polynomial associated. Then, 1 ρ δ ρ as δ, 2 For every α, α 2d, lim δ y d,δ,α = y α, 3 p d,δ p d as δ, 4 If the dual polynomial p := trace(m d (y ) q ) p d to the unrelaxed Step 1 belongs to P SOS 2(d+δ) (X) for some δ, then finite convergence takes place. (X) is the set of sum of squares of polynomial of degrees less than d + δ P SOS 2(d+δ) F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
25 Overview 1 Short bibliography 2 Convex design 3 Polynomial regression 4 Moment spaces 5 Moment formulation 6 SDP relaxation 7 Examples F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
26 Example I : interval X = [ 1, 1], polynomial regression model d j=0 θ jx j D-optimal design : for d = 5 and δ = 0 we obtain the sequence y (1, 0, 0.56, 0, 0.45, 0, 0.40, 0, 0.37, 0, 0.36). Recover the corresponding atomic measure from the sequence y : supported by -1, , , 0.285, and 1 (for d = 5, δ=0). The points match with the known analytic solution to the problem (critical points of the Legendre polynomial) y x FIGURE: Polynomial p D-optimality, n = 1 F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
27 Example II : Wynn s polygon Polygon given by the vertices ( 1, 1), ( 1, 1), (1, 1) and (2, 2), scaled to fit the unit circle, i.e., we consider the design space { X = x R 2 : x 1, x , x1 1 3 (x 2 + 2), x (x 1 + } 2), x x D-optimal design x x 1 x x 1 x x 1 FIGURE: The polynomial p d ( ) 2+d 2 where for d = 1, d = 2, d = 3. The red points correspond to the good level set F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
28 Example III : Ring of ellipses An ellipse with a hole in the form of a smaller ellipse X = {x R 2 : 9x x , 5x x 2 2 2}. D-optimal design x 2 0 x 2 0 x x 1 x 1 x 1 FIGURE: The boundary in bold black. The support of the optimal design measure (red points). Size of the points corresponds to the respective weights for d = 1 (left), d = 2 (middle), d = 3 (right) and δ = 3. F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
29 Example IV : Folium Zero set of f(x) = x 1 (x 2 1 2x2 2 )(x2 1 + x2 2 )2 is a curve with a triple singular point at the origin called a folium, X = {x R 2 : f(x) 0, x x 2 2 1}. D-optimal design x 2 0 x 2 0 x x 1 x 1 x 1 FIGURE: Boundary in bold black The support of the optimal design measure (red points). The good level set in thin blue d = 1 (left), d = 2 (middle), d = 3 (right), δ = 3 F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
30 The 3-dimensional unit sphere Polynomial regression α d θ αx α on the unit sphere X = {x R 3 : x x x 2 3 = 1}. D-optimal design FIGURE: Optimal design in red d = 1 (left), d = 2 (middle), d = 3 (right) and δ = 0 F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
31 The 3-dimensional unit sphere+constraint on moments Fix y 020 := 2ω, y 002 := ω, y 110 := 0.01ω and y 101 := 0.95ω. ω chosen such that the problem is feasible D-optimal design x x 1 FIGURE: Support points d = 1 without constraint in red and constrained in blue F. Gamboa (IMT Toulouse) Polynomial design 16th of February / 31
APPROXIMATE OPTIMAL DESIGNS FOR MULTIVARIATE POLYNOMIAL REGRESSION
The Annals of Statistics 2019, Vol. 47, No. 1, 127 155 https://doi.org/10.1214/18-aos1683 Institute of Mathematical Statistics, 2019 APPROXIMATE OPTIMAL DESIGNS FOR MULTIVARIATE POLYNOMIAL REGRESSION BY
More informationA new look at nonnegativity on closed sets
A new look at nonnegativity on closed sets LAAS-CNRS and Institute of Mathematics, Toulouse, France IPAM, UCLA September 2010 Positivstellensatze for semi-algebraic sets K R n from the knowledge of defining
More informationConvergence rates of moment-sum-of-squares hierarchies for volume approximation of semialgebraic sets
Convergence rates of moment-sum-of-squares hierarchies for volume approximation of semialgebraic sets Milan Korda 1, Didier Henrion,3,4 Draft of December 1, 016 Abstract Moment-sum-of-squares hierarchies
More informationStrong duality in Lasserre s hierarchy for polynomial optimization
Strong duality in Lasserre s hierarchy for polynomial optimization arxiv:1405.7334v1 [math.oc] 28 May 2014 Cédric Josz 1,2, Didier Henrion 3,4,5 Draft of January 24, 2018 Abstract A polynomial optimization
More informationGlobal Optimization with Polynomials
Global Optimization with Polynomials Geoffrey Schiebinger, Stephen Kemmerling Math 301, 2010/2011 March 16, 2011 Geoffrey Schiebinger, Stephen Kemmerling (Math Global 301, 2010/2011) Optimization with
More informationConvex Optimization & Parsimony of L p-balls representation
Convex Optimization & Parsimony of L p -balls representation LAAS-CNRS and Institute of Mathematics, Toulouse, France IMA, January 2016 Motivation Unit balls associated with nonnegative homogeneous polynomials
More informationMoments and Positive Polynomials for Optimization II: LP- VERSUS SDP-relaxations
Moments and Positive Polynomials for Optimization II: LP- VERSUS SDP-relaxations LAAS-CNRS and Institute of Mathematics, Toulouse, France Tutorial, IMS, Singapore 2012 LP-relaxations LP- VERSUS SDP-relaxations
More informationOptimization over Polynomials with Sums of Squares and Moment Matrices
Optimization over Polynomials with Sums of Squares and Moment Matrices Monique Laurent Centrum Wiskunde & Informatica (CWI), Amsterdam and University of Tilburg Positivity, Valuations and Quadratic Forms
More informationThe moment-lp and moment-sos approaches
The moment-lp and moment-sos approaches LAAS-CNRS and Institute of Mathematics, Toulouse, France CIRM, November 2013 Semidefinite Programming Why polynomial optimization? LP- and SDP- CERTIFICATES of POSITIVITY
More informationMoments and Positive Polynomials for Optimization II: LP- VERSUS SDP-relaxations
Moments and Positive Polynomials for Optimization II: LP- VERSUS SDP-relaxations LAAS-CNRS and Institute of Mathematics, Toulouse, France EECI Course: February 2016 LP-relaxations LP- VERSUS SDP-relaxations
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationLinear conic optimization for nonlinear optimal control
Linear conic optimization for nonlinear optimal control Didier Henrion 1,2,3, Edouard Pauwels 1,2 Draft of July 15, 2014 Abstract Infinite-dimensional linear conic formulations are described for nonlinear
More informationOptimal experimental design, an introduction, Jesús López Fidalgo
Optimal experimental design, an introduction Jesus.LopezFidalgo@uclm.es University of Castilla-La Mancha Department of Mathematics Institute of Applied Mathematics to Science and Engineering Books (just
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationarxiv: v1 [math.oc] 31 Jan 2017
CONVEX CONSTRAINED SEMIALGEBRAIC VOLUME OPTIMIZATION: APPLICATION IN SYSTEMS AND CONTROL 1 Ashkan Jasour, Constantino Lagoa School of Electrical Engineering and Computer Science, Pennsylvania State University
More information460 HOLGER DETTE AND WILLIAM J STUDDEN order to examine how a given design behaves in the model g` with respect to the D-optimality criterion one uses
Statistica Sinica 5(1995), 459-473 OPTIMAL DESIGNS FOR POLYNOMIAL REGRESSION WHEN THE DEGREE IS NOT KNOWN Holger Dette and William J Studden Technische Universitat Dresden and Purdue University Abstract:
More informationAdvanced SDPs Lecture 6: March 16, 2017
Advanced SDPs Lecture 6: March 16, 2017 Lecturers: Nikhil Bansal and Daniel Dadush Scribe: Daniel Dadush 6.1 Notation Let N = {0, 1,... } denote the set of non-negative integers. For α N n, define the
More informationHilbert s 17th Problem to Semidefinite Programming & Convex Algebraic Geometry
Hilbert s 17th Problem to Semidefinite Programming & Convex Algebraic Geometry Rekha R. Thomas University of Washington, Seattle References Monique Laurent, Sums of squares, moment matrices and optimization
More informationNon-commutative polynomial optimization
Non-commutative polynomial optimization S. Pironio 1, M. Navascuès 2, A. Acín 3 1 Laboratoire d Information Quantique (Brussels) 2 Department of Mathematics (Bristol) 3 ICFO-The Institute of Photonic Sciences
More informationStochastic Design Criteria in Linear Models
AUSTRIAN JOURNAL OF STATISTICS Volume 34 (2005), Number 2, 211 223 Stochastic Design Criteria in Linear Models Alexander Zaigraev N. Copernicus University, Toruń, Poland Abstract: Within the framework
More informationSemialgebraic Relaxations using Moment-SOS Hierarchies
Semialgebraic Relaxations using Moment-SOS Hierarchies Victor Magron, Postdoc LAAS-CNRS 17 September 2014 SIERRA Seminar Laboratoire d Informatique de l Ecole Normale Superieure y b sin( par + b) b 1 1
More informationOn Polynomial Optimization over Non-compact Semi-algebraic Sets
On Polynomial Optimization over Non-compact Semi-algebraic Sets V. Jeyakumar, J.B. Lasserre and G. Li Revised Version: April 3, 2014 Communicated by Lionel Thibault Abstract The optimal value of a polynomial
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationFormal Proofs, Program Analysis and Moment-SOS Relaxations
Formal Proofs, Program Analysis and Moment-SOS Relaxations Victor Magron, Postdoc LAAS-CNRS 15 July 2014 Imperial College Department of Electrical and Electronic Eng. y b sin( par + b) b 1 1 b1 b2 par
More informationSolving Global Optimization Problems with Sparse Polynomials and Unbounded Semialgebraic Feasible Sets
Solving Global Optimization Problems with Sparse Polynomials and Unbounded Semialgebraic Feasible Sets V. Jeyakumar, S. Kim, G. M. Lee and G. Li June 6, 2014 Abstract We propose a hierarchy of semidefinite
More informationConvex computation of the region of attraction for polynomial control systems
Convex computation of the region of attraction for polynomial control systems Didier Henrion LAAS-CNRS Toulouse & CTU Prague Milan Korda EPFL Lausanne Region of Attraction (ROA) ẋ = f (x,u), x(t) X, u(t)
More informationCONVEXITY IN SEMI-ALGEBRAIC GEOMETRY AND POLYNOMIAL OPTIMIZATION
CONVEXITY IN SEMI-ALGEBRAIC GEOMETRY AND POLYNOMIAL OPTIMIZATION JEAN B. LASSERRE Abstract. We review several (and provide new) results on the theory of moments, sums of squares and basic semi-algebraic
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationOn construction of constrained optimum designs
On construction of constrained optimum designs Institute of Control and Computation Engineering University of Zielona Góra, Poland DEMA2008, Cambridge, 15 August 2008 Numerical algorithms to construct
More informationLecture 5. 1 Goermans-Williamson Algorithm for the maxcut problem
Math 280 Geometric and Algebraic Ideas in Optimization April 26, 2010 Lecture 5 Lecturer: Jesús A De Loera Scribe: Huy-Dung Han, Fabio Lapiccirella 1 Goermans-Williamson Algorithm for the maxcut problem
More informationarxiv: v2 [math.oc] 30 Sep 2015
Symmetry, Integrability and Geometry: Methods and Applications Moments and Legendre Fourier Series for Measures Supported on Curves Jean B. LASSERRE SIGMA 11 (215), 77, 1 pages arxiv:158.6884v2 [math.oc]
More informationThe moment-lp and moment-sos approaches in optimization
The moment-lp and moment-sos approaches in optimization LAAS-CNRS and Institute of Mathematics, Toulouse, France Workshop Linear Matrix Inequalities, Semidefinite Programming and Quantum Information Theory
More information6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC
6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC 2003 2003.09.02.10 6. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets Tarski-Seidenberg and quantifier elimination Feasibility
More informationAppendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS
Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution
More informationHow to generate weakly infeasible semidefinite programs via Lasserre s relaxations for polynomial optimization
CS-11-01 How to generate weakly infeasible semidefinite programs via Lasserre s relaxations for polynomial optimization Hayato Waki Department of Computer Science, The University of Electro-Communications
More informationRegion of attraction approximations for polynomial dynamical systems
Region of attraction approximations for polynomial dynamical systems Milan Korda EPFL Lausanne Didier Henrion LAAS-CNRS Toulouse & CTU Prague Colin N. Jones EPFL Lausanne Region of Attraction (ROA) ẋ(t)
More informationConvex computation of the region of attraction for polynomial control systems
Convex computation of the region of attraction for polynomial control systems Didier Henrion LAAS-CNRS Toulouse & CTU Prague Milan Korda EPFL Lausanne Region of Attraction (ROA) ẋ = f (x,u), x(t) X, u(t)
More informationTHEOREMS, ETC., FOR MATH 516
THEOREMS, ETC., FOR MATH 516 Results labeled Theorem Ea.b.c (or Proposition Ea.b.c, etc.) refer to Theorem c from section a.b of Evans book (Partial Differential Equations). Proposition 1 (=Proposition
More informationApproximating Pareto Curves using Semidefinite Relaxations
Approximating Pareto Curves using Semidefinite Relaxations Victor Magron, Didier Henrion,,3 Jean-Bernard Lasserre, arxiv:44.477v [math.oc] 6 Jun 4 June 7, 4 Abstract We consider the problem of constructing
More informationLift-and-Project Techniques and SDP Hierarchies
MFO seminar on Semidefinite Programming May 30, 2010 Typical combinatorial optimization problem: max c T x s.t. Ax b, x {0, 1} n P := {x R n Ax b} P I := conv(k {0, 1} n ) LP relaxation Integral polytope
More informationMinimum Ellipsoid Bounds for Solutions of Polynomial Systems via Sum of Squares
Journal of Global Optimization (2005) 33: 511 525 Springer 2005 DOI 10.1007/s10898-005-2099-2 Minimum Ellipsoid Bounds for Solutions of Polynomial Systems via Sum of Squares JIAWANG NIE 1 and JAMES W.
More informationPOLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS
POLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS Sercan Yıldız syildiz@samsi.info in collaboration with Dávid Papp (NCSU) OPT Transition Workshop May 02, 2017 OUTLINE Polynomial optimization and
More informationCSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming
More informationStability of optimization problems with stochastic dominance constraints
Stability of optimization problems with stochastic dominance constraints D. Dentcheva and W. Römisch Stevens Institute of Technology, Hoboken Humboldt-University Berlin www.math.hu-berlin.de/~romisch SIAM
More informationA GENERALIZATION OF THE LÖWNER-JOHN S ELLIPSOID THEOREM
A GENERALIZATION OF THE LÖWNER-JOHN S ELLIPSOID THEOREM JEAN B. LASSERRE Abstract. We address the following generalization P of the Löwner- John ellipsoid problem. Given a (non necessarily convex) compact
More informationWhat can be expressed via Conic Quadratic and Semidefinite Programming?
What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Abstract Tremendous recent
More informationUnbounded Convex Semialgebraic Sets as Spectrahedral Shadows
Unbounded Convex Semialgebraic Sets as Spectrahedral Shadows Shaowei Lin 9 Dec 2010 Abstract Recently, Helton and Nie [3] showed that a compact convex semialgebraic set S is a spectrahedral shadow if the
More informationExact SDP Relaxations for Classes of Nonlinear Semidefinite Programming Problems
Exact SDP Relaxations for Classes of Nonlinear Semidefinite Programming Problems V. Jeyakumar and G. Li Revised Version:August 31, 2012 Abstract An exact semidefinite linear programming (SDP) relaxation
More informationA JOINT+MARGINAL APPROACH TO PARAMETRIC POLYNOMIAL OPTIMIZATION
A JOINT+MARGINAL APPROACH TO PARAMETRIC POLNOMIAL OPTIMIZATION JEAN B. LASSERRE Abstract. Given a compact parameter set R p, we consider polynomial optimization problems (P y) on R n whose description
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More informationLecture 3: Semidefinite Programming
Lecture 3: Semidefinite Programming Lecture Outline Part I: Semidefinite programming, examples, canonical form, and duality Part II: Strong Duality Failure Examples Part III: Conditions for strong duality
More informationApproximation algorithms for nonnegative polynomial optimization problems over unit spheres
Front. Math. China 2017, 12(6): 1409 1426 https://doi.org/10.1007/s11464-017-0644-1 Approximation algorithms for nonnegative polynomial optimization problems over unit spheres Xinzhen ZHANG 1, Guanglu
More informationNonlinear Programming Models
Nonlinear Programming Models Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Nonlinear Programming Models p. Introduction Nonlinear Programming Models p. NLP problems minf(x) x S R n Standard form:
More informationFakultät für Mathematik und Informatik
Fakultät für Mathematik und Informatik Preprint 2017-03 S. Dempe, F. Mefo Kue Discrete bilevel and semidefinite programg problems ISSN 1433-9307 S. Dempe, F. Mefo Kue Discrete bilevel and semidefinite
More informationModule 04 Optimization Problems KKT Conditions & Solvers
Module 04 Optimization Problems KKT Conditions & Solvers Ahmad F. Taha EE 5243: Introduction to Cyber-Physical Systems Email: ahmad.taha@utsa.edu Webpage: http://engineering.utsa.edu/ taha/index.html September
More informationSEMIDEFINITE PROGRAMMING VS. LP RELAXATIONS FOR POLYNOMIAL PROGRAMMING
MATHEMATICS OF OPERATIONS RESEARCH Vol. 27, No. 2, May 2002, pp. 347 360 Printed in U.S.A. SEMIDEFINITE PROGRAMMING VS. LP RELAXATIONS FOR POLYNOMIAL PROGRAMMING JEAN B. LASSERRE We consider the global
More informationNotes on the decomposition result of Karlin et al. [2] for the hierarchy of Lasserre by M. Laurent, December 13, 2012
Notes on the decomposition result of Karlin et al. [2] for the hierarchy of Lasserre by M. Laurent, December 13, 2012 We present the decomposition result of Karlin et al. [2] for the hierarchy of Lasserre
More informationSPECTRAHEDRA. Bernd Sturmfels UC Berkeley
SPECTRAHEDRA Bernd Sturmfels UC Berkeley Mathematics Colloquium, North Carolina State University February 5, 2010 Positive Semidefinite Matrices For a real symmetric n n-matrix A the following are equivalent:
More informationThe Geometry of Semidefinite Programming. Bernd Sturmfels UC Berkeley
The Geometry of Semidefinite Programming Bernd Sturmfels UC Berkeley Positive Semidefinite Matrices For a real symmetric n n-matrix A the following are equivalent: All n eigenvalues of A are positive real
More informationCONVEX OPTIMIZATION OVER POSITIVE POLYNOMIALS AND FILTER DESIGN. Y. Genin, Y. Hachez, Yu. Nesterov, P. Van Dooren
CONVEX OPTIMIZATION OVER POSITIVE POLYNOMIALS AND FILTER DESIGN Y. Genin, Y. Hachez, Yu. Nesterov, P. Van Dooren CESAME, Université catholique de Louvain Bâtiment Euler, Avenue G. Lemaître 4-6 B-1348 Louvain-la-Neuve,
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More informationCopositive Programming and Combinatorial Optimization
Copositive Programming and Combinatorial Optimization Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria joint work with I.M. Bomze (Wien) and F. Jarre (Düsseldorf) IMA
More informationComputing uniform convex approximations for convex envelopes and convex hulls
Computing uniform convex approximations for convex envelopes and convex hulls Rida Laraki, Jean-Bernard Lasserre To cite this version: Rida Laraki, Jean-Bernard Lasserre. Computing uniform convex approximations
More informationReal Symmetric Matrices and Semidefinite Programming
Real Symmetric Matrices and Semidefinite Programming Tatsiana Maskalevich Abstract Symmetric real matrices attain an important property stating that all their eigenvalues are real. This gives rise to many
More informationLinear Regression and Its Applications
Linear Regression and Its Applications Predrag Radivojac October 13, 2014 Given a data set D = {(x i, y i )} n the objective is to learn the relationship between features and the target. We usually start
More informationRanks of Real Symmetric Tensors
Ranks of Real Symmetric Tensors Greg Blekherman SIAM AG 2013 Algebraic Geometry of Tensor Decompositions Real Symmetric Tensor Decompositions Let f be a form of degree d in R[x 1,..., x n ]. We would like
More informationGlobal Optimization with Polynomials
Global Optimization with Polynomials Deren Han Singapore-MIT Alliance National University of Singapore Singapore 119260 Email: smahdr@nus.edu.sg Abstract The class of POP (Polynomial Optimization Problems)
More informationSPECTRAHEDRA. Bernd Sturmfels UC Berkeley
SPECTRAHEDRA Bernd Sturmfels UC Berkeley GAeL Lecture I on Convex Algebraic Geometry Coimbra, Portugal, Monday, June 7, 2010 Positive Semidefinite Matrices For a real symmetric n n-matrix A the following
More informationInterior Point Methods: Second-Order Cone Programming and Semidefinite Programming
School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods: Second-Order Cone Programming and Semidefinite Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio
More informationCommon-Knowledge / Cheat Sheet
CSE 521: Design and Analysis of Algorithms I Fall 2018 Common-Knowledge / Cheat Sheet 1 Randomized Algorithm Expectation: For a random variable X with domain, the discrete set S, E [X] = s S P [X = s]
More informationA new approximation hierarchy for polynomial conic optimization
A new approximation hierarchy for polynomial conic optimization Peter J.C. Dickinson Janez Povh July 11, 2018 Abstract In this paper we consider polynomial conic optimization problems, where the feasible
More informationTowards Solving Bilevel Optimization Problems in Quantum Information Theory
Towards Solving Bilevel Optimization Problems in Quantum Information Theory ICFO-The Institute of Photonic Sciences and University of Borås 22 January 2016 Workshop on Linear Matrix Inequalities, Semidefinite
More informationAliprantis, Border: Infinite-dimensional Analysis A Hitchhiker s Guide
aliprantis.tex May 10, 2011 Aliprantis, Border: Infinite-dimensional Analysis A Hitchhiker s Guide Notes from [AB2]. 1 Odds and Ends 2 Topology 2.1 Topological spaces Example. (2.2) A semimetric = triangle
More informationOn self-concordant barriers for generalized power cones
On self-concordant barriers for generalized power cones Scott Roy Lin Xiao January 30, 2018 Abstract In the study of interior-point methods for nonsymmetric conic optimization and their applications, Nesterov
More informationProblem 1A. Calculus. Problem 3A. Real analysis. f(x) = 0 x = 0.
Problem A. Calculus Find the length of the spiral given in polar coordinates by r = e θ, < θ 0. Solution: The length is 0 θ= ds where by Pythagoras ds = dr 2 + (rdθ) 2 = dθ 2e θ, so the length is 0 2e
More informationMean squared error minimization for inverse moment problems
Mean squared error minimization for inverse moment problems Didier Henrion 1,2,3, Jean B. Lasserre 1,2,4, Martin Mevissen 5 June 19, 2013 Abstract We consider the problem of approximating the unknown density
More informationConvex Optimization in Classification Problems
New Trends in Optimization and Computational Algorithms December 9 13, 2001 Convex Optimization in Classification Problems Laurent El Ghaoui Department of EECS, UC Berkeley elghaoui@eecs.berkeley.edu 1
More informationarxiv: v1 [math.oc] 18 Jul 2011
arxiv:1107.3493v1 [math.oc] 18 Jul 011 Tchebycheff systems and extremal problems for generalized moments: a brief survey Iosif Pinelis Department of Mathematical Sciences Michigan Technological University
More informationLecture: Examples of LP, SOCP and SDP
1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationComparison of Lasserre s measure based bounds for polynomial optimization to bounds obtained by simulated annealing
Comparison of Lasserre s measure based bounds for polynomial optimization to bounds obtained by simulated annealing Etienne de lerk Monique Laurent March 2, 2017 Abstract We consider the problem of minimizing
More information1. If 1, ω, ω 2, -----, ω 9 are the 10 th roots of unity, then (1 + ω) (1 + ω 2 ) (1 + ω 9 ) is A) 1 B) 1 C) 10 D) 0
4 INUTES. If, ω, ω, -----, ω 9 are the th roots of unity, then ( + ω) ( + ω ) ----- ( + ω 9 ) is B) D) 5. i If - i = a + ib, then a =, b = B) a =, b = a =, b = D) a =, b= 3. Find the integral values for
More informationCopositive Programming and Combinatorial Optimization
Copositive Programming and Combinatorial Optimization Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria joint work with M. Bomze (Wien) and F. Jarre (Düsseldorf) and
More informationLOWER BOUNDS FOR POLYNOMIALS USING GEOMETRIC PROGRAMMING
LOWER BOUNDS FOR POLYNOMIALS USING GEOMETRIC PROGRAMMING MEHDI GHASEMI AND MURRAY MARSHALL Abstract. We make use of a result of Hurwitz and Reznick [8] [19], and a consequence of this result due to Fidalgo
More informationTheorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension. n=1
Chapter 2 Probability measures 1. Existence Theorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension to the generated σ-field Proof of Theorem 2.1. Let F 0 be
More informationLargest dual ellipsoids inscribed in dual cones
Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that
More informationFRTN10 Multivariable Control, Lecture 13. Course outline. The Q-parametrization (Youla) Example: Spring-mass System
FRTN Multivariable Control, Lecture 3 Anders Robertsson Automatic Control LTH, Lund University Course outline The Q-parametrization (Youla) L-L5 Purpose, models and loop-shaping by hand L6-L8 Limitations
More informationTensor decomposition and moment matrices
Tensor decomposition and moment matrices J. Brachat (Ph.D.) & B. Mourrain GALAAD, INRIA Méditerranée, Sophia Antipolis Bernard.Mourrain@inria.fr SIAM AAG, Raleigh, NC, Oct. -9, 211 Collaboration with A.
More informationMIT Algebraic techniques and semidefinite optimization May 9, Lecture 21. Lecturer: Pablo A. Parrilo Scribe:???
MIT 6.972 Algebraic techniques and semidefinite optimization May 9, 2006 Lecture 2 Lecturer: Pablo A. Parrilo Scribe:??? In this lecture we study techniques to exploit the symmetry that can be present
More informationAgenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Second-order cone programming (SOCP)
Agenda 1 Cone programming 2 Convex cones 3 Generalized inequalities 4 Linear programming (LP) 5 Second-order cone programming (SOCP) 6 Semidefinite programming (SDP) 7 Examples Optimization problem in
More informationSTUDY PLAN MASTER IN (MATHEMATICS) (Thesis Track)
STUDY PLAN MASTER IN (MATHEMATICS) (Thesis Track) I. GENERAL RULES AND CONDITIONS: 1- This plan conforms to the regulations of the general frame of the Master programs. 2- Areas of specialty of admission
More informationRobust Integer Programming
Robust Integer Programming Technion Israel Institute of Technology Outline. Overview our theory of Graver bases for integer programming (with Berstein, De Loera, Hemmecke, Lee, Romanchuk, Rothblum, Weismantel)
More informationapproximation algorithms I
SUM-OF-SQUARES method and approximation algorithms I David Steurer Cornell Cargese Workshop, 201 meta-task encoded as low-degree polynomial in R x example: f(x) = i,j n w ij x i x j 2 given: functions
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Abstract. We consider the problem of choosing the edge weights of an undirected graph so as to maximize or minimize some function of the
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Stanford University (Joint work with Persi Diaconis, Arpita Ghosh, Seung-Jean Kim, Sanjay Lall, Pablo Parrilo, Amin Saberi, Jun Sun, Lin
More informationKernel Method: Data Analysis with Positive Definite Kernels
Kernel Method: Data Analysis with Positive Definite Kernels 2. Positive Definite Kernel and Reproducing Kernel Hilbert Space Kenji Fukumizu The Institute of Statistical Mathematics. Graduate University
More informationContraction Principles some applications
Contraction Principles some applications Fabrice Gamboa (Institut de Mathématiques de Toulouse) 7th of June 2017 Christian and Patrick 59th Birthday Overview Appetizer : Christian and Patrick secret lives
More informationOptimal Design in Regression. and Spline Smoothing
Optimal Design in Regression and Spline Smoothing by Jaerin Cho A thesis submitted to the Department of Mathematics and Statistics in conformity with the requirements for the degree of Doctor of Philosophy
More informationFundamental Theorems of Optimization
Fundamental Theorems of Optimization 1 Fundamental Theorems of Math Prog. Maximizing a concave function over a convex set. Maximizing a convex function over a closed bounded convex set. 2 Maximizing Concave
More informationCourse Outline. FRTN10 Multivariable Control, Lecture 13. General idea for Lectures Lecture 13 Outline. Example 1 (Doyle Stein, 1979)
Course Outline FRTN Multivariable Control, Lecture Automatic Control LTH, 6 L-L Specifications, models and loop-shaping by hand L6-L8 Limitations on achievable performance L9-L Controller optimization:
More information