# Lecture 1 Introduction

Size: px
Start display at page:

## Transcription

1 L. Vandenberghe EE236A (Fall ) Lecture 1 Introduction course overview linear optimization examples history approximate syllabus basic definitions linear optimization in vector and matrix notation halfspaces and polyhedra geometrical interpretation 1 1

2 Linear optimization minimize subject to n j=1 n j=1 n j=1 c j x j a ij x j b i, i = 1,...,m d ij x j = f i, i = 1,...,p n optimization variables: x 1,..., x n (real scalars) problem data (parameters): the coefficients c j, a ij, b i, d ij, f i j c jx j is the cost function or objective function j a ijx j b i and j d ijx j = f i are inequality and equality constraints called a linear optimization problem or linear program (LP) Introduction 1 2

3 Importance low complexity problems with several thousand variables, constraints routinely solved much larger problems (millions of variables) if problem data are sparse widely available software theoretical worst-case complexity is polynomial wide applicability originally developed for applications in economics and management today, used in all areas of engineering, data analysis, finance,... a key tool in combinatorial optimization extensive theory no simple formula for solution but extensive, useful (duality) theory Introduction 1 3

4 Example: open-loop control problem single-input/single-output system (input u(t), output y(t) at time t) y(t) = h 0 u(t)+h 1 u(t 1)+h 2 u(t 2)+h 3 u(t 3)+ output tracking problem: minimize deviation from desired output y des (t) max y(t) y des(t) t=0,...,n subject to input amplitude and slew rate constraints: u(t) U, u(t+1) u(t) S variables: u(0),..., u(m) (with u(t) = 0 for t < 0, t > M) solution: can be formulated as an LP, hence easily solved (more later) Introduction 1 4

5 example step response (s(t) = h t + +h 0 ) and desired output: step response y des (t) amplitude and slew rate constraint on u: u(t) 1.1, u(t) u(t 1) 0.25 Introduction 1 5

6 optimal solution (computed via linear optimization) 1.1 input u(t) 0.25 u(t) u(t 1) output and desired output Introduction 1 6

7 Example: assignment problem match N people to N tasks each person is assigned to one task; each task assigned to one person cost of assigning person i to task j is a ij combinatorial formulation minimize subject to N i,j=1 N i=1 N j=1 a ij x ij x ij = 1, j = 1,...,N x ij = 1, i = 1,...,N x ij {0,1}, i,j = 1,...,N variable x ij = 1 if person i is assigned to task j; x ij = 0 otherwise N! possible assignments, i.e., too many to enumerate Introduction 1 7

8 linear optimization formulation minimize subject to N i,j=1 N i=1 N j=1 a ij x ij x ij = 1, j = 1,...,N x ij = 1, i = 1,...,N 0 x ij 1, i,j = 1,...,N we have relaxed the constraints x ij {0,1} it can be shown that at the optimum x ij {0,1} (see later) hence, can solve (this particular) combinatorial problem efficiently (via linear optimization or specialized methods) Introduction 1 8

9 Brief history 1940s (Dantzig, Kantorovich, Koopmans, von Neumann,... ) foundations, motivated by economics and logistics problems 1947 (Dantzig): simplex algorithm 1950s 60s: applications in other disciplines 1979 (Khachiyan): ellipsoid algorithm: more efficient (polynomial-time) than simplex in worst case, much slower in practice 1984 (Karmarkar): projective (interior-point) algorithm: polynomial-time worst-case complexity, and efficient in practice since 1984: variations of interior-point methods (improved complexity or efficiency in practice), software for large-scale problems Introduction 1 9

10 Tentative syllabus linear and piecewise-linear optimization polyhedral geometry duality applications algorithms: simplex algorithm, interior-point algorithms, decomposition applications in network and combinatorial optimization extensions: linear-fractional programming introduction to integer linear programming Introduction 1 10

11 Vectors vector of length n (or n-vector) x = x 1 x 2. x n we also use the notation x = (x 1,x 2,...,x n ) x i is ith component or element (real unless specified otherwise) set of real n-vectors is denoted R n special vectors (with n determined from context) x = 0 (zero vector): x i = 0, i = 1,...,n x = 1 (vector of all ones): x i = 1, i = 1,...,n x = e i (ith basis or unit vector): x i = 1, x k = 0 for k i Introduction 1 11

12 Matrices matrix of size m n A = A 11 A 12 A 1n A 21. A 22. A 2n. A m1 A m2 A mn A ij (or a ij ) is the i,j element (or entry, coefficient) set of real m n-matrices is denoted R m n vectors can be viewed as matrices with one column special matrices (with size determined from context) X = 0 (zero matrix): X ij = 0 for i = 1,...,m, j = 1,...,n X = I (identity matrix): m = n with X ii = 1, X ij = 0 for i j Introduction 1 12

13 Operations matrix transpose A T scalar multiplication αa addition A+B and subtraction A B of matrices of the same size product y = Ax of a matrix with a vector of compatible length product C = AB of matrices of compatible size inner product of n-vectors: x T y = x 1 y 1 + +x n y n Introduction 1 13

14 LP in inner-product notation minimize subject to n j=1 n j=1 n j=1 c j x j a ij x j b i, i = 1,...,m d ij x j = f i, i = 1,...,p inner-product notation c, a i, d i are n-vectors: minimize c T x subject to a T i x b i, i = 1,...,m d T i x = f i, i = 1,...,p c = (c 1,...,c n ), a i = (a i1,...,a in ), d i = (d i1,...,d in ) Introduction 1 14

15 LP in matrix notation matrix notation minimize subject to n j=1 n j=1 n j=1 c j x j a ij x j b i, i = 1,...,m d ij x j = f i, i = 1,...,p minimize c T x subject to Ax b Dx = f A is m n-matrix with elements a ij, rows a T i D is p n-matrix with elements d ij, rows d T i inequality is component-wise vector inequality Introduction 1 15

16 Terminology minimize c T x subject to Ax b Dx = f x is feasible if it satisfies the constraints Ax b and Dx = f feasible set is set of all feasible points x is optimal if it is feasible and c T x c T x for all feasible x the optimal value of the LP is p = c T x unbounded problem: c T x unbounded below on feasible set (p = ) infeasible probem: feasible set is empty (p = + ) Introduction 1 16

17 Euclidean norm x = Vector norms x 2 1 +x x2 n = x T x l 1 -norm and l -norm x 1 = x 1 + x x n x = max{ x 1, x 2,..., x n } properties (satisfied by any norm f(x)) f(αx) = α f(x) (homogeneity) f(x+y) f(x)+f(y) (triangle inequality) f(x) 0 (nonnegativity); f(x) = 0 if only if x = 0 (definiteness) Introduction 1 17

18 Cauchy-Schwarz inequality x y x T y x y holds for all vectors x, y of the same size x T y = x y iff x and y are aligned (nonnegative multiples) x T y = x y iff x and y are opposed (nonpositive multiples) implies many useful inequalities as special cases, for example, n x n x i n x i=1 Introduction 1 18

19 Angle between vectors the angle θ = (x,y) between nonzero vectors x and y is defined as θ = arccos x T y x y (i.e., x T y = x y cosθ) we normalize θ so that 0 θ π relation between sign of inner product and angle x T y > 0 θ < π 2 x T y = 0 θ = π 2 x T y < 0 θ > π 2 (vectors make an acute angle) (orthogonal vectors) (vectors make an obtuse angle) Introduction 1 19

20 Projection projection of x on the line defined by nonzero y: the vector ˆty with ˆt = argmin t x ty expression for ˆt: ˆt = xt y y 2 = x cosθ y ( ) ˆty = x T y y y T y y x 0 Introduction 1 20

21 Hyperplanes and halfspaces hyperplane solution set of one linear equation with nonzero coefficient vector a a T x = b halfspace solution set of one linear inequality with nonzero coefficient vector a a T x b a is the normal vector Introduction 1 21

22 Geometrical interpretation G = {x a T x = b} H = {x a T x b} a a u = (b/ a 2 )a x G x u 0 x u x u H the vector u = (b/ a 2 )a satisfies a T u = b x is in hyperplane G if a T (x u) = 0 (x u is orthogonal to a) x is in halfspace H if a T (x u) 0 (angle (x u,a) π/2) Introduction 1 22

23 Example a T x = 0 x 2 a T x = 10 x 2 a = (2,1) a = (2,1) x 1 x 1 a T x = 5 a T x = 5 a T x 3 Introduction 1 23

24 Polyhedron solution set of a finite number of linear inequalities a T 1x b 1, a T 2x b 2,..., a T mx b m a 1 a 2 a 5 a 3 a 4 intersection of a finite number of halfspaces in matrix notation: Ax b if A is a matrix with rows a T i can include equalities: Fx = g is equivalent to Fx g, Fx g Introduction 1 24

25 Example x 1 +x 2 1, 2x 1 +x 2 2, x 1 0, x 2 0 x 2 2x 1 +x 2 = 2 x 1 +x 2 = 1 x 1 Introduction 1 25

26 Example 0 x 1 2, 0 x 2 2, 0 x 3 2, x 1 +x 2 +x 3 5 x 3 (0,0,2) (0,2,2) (1,2,2) (2,0,2) (2,1,2) (2,2,1) (0,2,0) x 2 (2,0,0) (2,2,0) x 1 Introduction 1 26

27 Geometrical interpretation of LP minimize c T x subject to Ax b c Ax b optimal solution dashed lines (hyperplanes) are level sets c T x = α for different α Introduction 1 27

28 Example minimize x 1 x 2 subject to 2x 1 +x 2 3 x 1 +4x 2 5 x 1 0, x 2 0 x 1 x 2 = 2 x 2 x 1 x 2 = 1 x 1 x 2 = 5 x 1 x 2 = 0 x 1 x 2 = 4 x 1 optimal solution is (1, 1) x 1 x 2 = 3 Introduction 1 28

### Lecture 1 Introduction and overview

ESE504 (Fall 2013) Lecture 1 Introduction and overview linear programming example course topics software integer linear programming 1 1 Linear program (LP) minimize subject to n c j x j j=1 n a ij x j

### Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS

Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution

### Motivating examples Introduction to algorithms Simplex algorithm. On a particular example General algorithm. Duality An application to game theory

Instructor: Shengyu Zhang 1 LP Motivating examples Introduction to algorithms Simplex algorithm On a particular example General algorithm Duality An application to game theory 2 Example 1: profit maximization

### In English, this means that if we travel on a straight line between any two points in C, then we never leave C.

Convex sets In this section, we will be introduced to some of the mathematical fundamentals of convex sets. In order to motivate some of the definitions, we will look at the closest point problem from

### Lecture 1: Review of linear algebra

Lecture 1: Review of linear algebra Linear functions and linearization Inverse matrix, least-squares and least-norm solutions Subspaces, basis, and dimension Change of basis and similarity transformations

### MS-E2140. Lecture 1. (course book chapters )

Linear Programming MS-E2140 Motivations and background Lecture 1 (course book chapters 1.1-1.4) Linear programming problems and examples Problem manipulations and standard form Graphical representation

### LMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009

LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix

### MS-E2140. Lecture 1. (course book chapters )

Linear Programming MS-E2140 Motivations and background Lecture 1 (course book chapters 1.1-1.4) Linear programming problems and examples Problem manipulations and standard form problems Graphical representation

### Math 5593 Linear Programming Week 1

University of Colorado Denver, Fall 2013, Prof. Engau 1 Problem-Solving in Operations Research 2 Brief History of Linear Programming 3 Review of Basic Linear Algebra Linear Programming - The Story About

### 8. Geometric problems

8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 Minimum volume ellipsoid around a set Löwner-John ellipsoid

### 15-780: LinearProgramming

15-780: LinearProgramming J. Zico Kolter February 1-3, 2016 1 Outline Introduction Some linear algebra review Linear programming Simplex algorithm Duality and dual simplex 2 Outline Introduction Some linear

### CS711008Z Algorithm Design and Analysis

CS711008Z Algorithm Design and Analysis Lecture 8 Linear programming: interior point method Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 / 31 Outline Brief

### Lectures 6, 7 and part of 8

Lectures 6, 7 and part of 8 Uriel Feige April 26, May 3, May 10, 2015 1 Linear programming duality 1.1 The diet problem revisited Recall the diet problem from Lecture 1. There are n foods, m nutrients,

### Lecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set.

Lecture # 3 Orthogonal Matrices and Matrix Norms We repeat the definition an orthogonal set and orthornormal set. Definition A set of k vectors {u, u 2,..., u k }, where each u i R n, is said to be an

### HW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.

HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard

### 15.083J/6.859J Integer Optimization. Lecture 10: Solving Relaxations

15.083J/6.859J Integer Optimization Lecture 10: Solving Relaxations 1 Outline The key geometric result behind the ellipsoid method Slide 1 The ellipsoid method for the feasibility problem The ellipsoid

### Linear and Integer Optimization (V3C1/F4C1)

Linear and Integer Optimization (V3C1/F4C1) Lecture notes Ulrich Brenner Research Institute for Discrete Mathematics, University of Bonn Winter term 2016/2017 March 8, 2017 12:02 1 Preface Continuous updates

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology 18.433: Combinatorial Optimization Michel X. Goemans February 28th, 2013 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory

### 8. Geometric problems

8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 1 Minimum volume ellipsoid around a set Löwner-John ellipsoid

### Integer Linear Programs

Lecture 2: Review, Linear Programming Relaxations Today we will talk about expressing combinatorial problems as mathematical programs, specifically Integer Linear Programs (ILPs). We then see what happens

### Optimization WS 13/14:, by Y. Goldstein/K. Reinert, 9. Dezember 2013, 16: Linear programming. Optimization Problems

Optimization WS 13/14:, by Y. Goldstein/K. Reinert, 9. Dezember 2013, 16:38 2001 Linear programming Optimization Problems General optimization problem max{z(x) f j (x) 0,x D} or min{z(x) f j (x) 0,x D}

### Algorithms and Theory of Computation. Lecture 13: Linear Programming (2)

Algorithms and Theory of Computation Lecture 13: Linear Programming (2) Xiaohui Bei MAS 714 September 25, 2018 Nanyang Technological University MAS 714 September 25, 2018 1 / 15 LP Duality Primal problem

### Introduction and Math Preliminaries

Introduction and Math Preliminaries Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Appendices A, B, and C, Chapter

### Chapter 1. Preliminaries

Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between

### A Brief Review on Convex Optimization

A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review

### 9. Geometric problems

9. Geometric problems EE/AA 578, Univ of Washington, Fall 2016 projection on a set extremal volume ellipsoids centering classification 9 1 Projection on convex set projection of point x on set C defined

### Complexity of linear programming: outline

Complexity of linear programming: outline I Assessing computational e ciency of algorithms I Computational e ciency of the Simplex method I Ellipsoid algorithm for LP and its computational e ciency IOE

### Inner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:

Inner products Definition: An inner product on a real vector space V is an operation (function) that assigns to each pair of vectors ( u, v) in V a scalar u, v satisfying the following axioms: 1. u, v

### EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17

EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko

### ECS130 Scientific Computing. Lecture 1: Introduction. Monday, January 7, 10:00 10:50 am

ECS130 Scientific Computing Lecture 1: Introduction Monday, January 7, 10:00 10:50 am About Course: ECS130 Scientific Computing Professor: Zhaojun Bai Webpage: http://web.cs.ucdavis.edu/~bai/ecs130/ Today

### MTAEA Vectors in Euclidean Spaces

School of Economics, Australian National University January 25, 2010 Vectors. Economists usually work in the vector space R n. A point in this space is called a vector, and is typically defined by its

### Lecture 15: October 15

10-725: Optimization Fall 2012 Lecturer: Barnabas Poczos Lecture 15: October 15 Scribes: Christian Kroer, Fanyi Xiao Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have

### COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion

COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F

### 3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions

A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence

### LP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra

LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality

### AM 121: Intro to Optimization Models and Methods

AM 121: Intro to Optimization Models and Methods Fall 2017 Lecture 2: Intro to LP, Linear algebra review. Yiling Chen SEAS Lecture 2: Lesson Plan What is an LP? Graphical and algebraic correspondence Problems

### Prof. M. Saha Professor of Mathematics The University of Burdwan West Bengal, India

CHAPTER 9 BY Prof. M. Saha Professor of Mathematics The University of Burdwan West Bengal, India E-mail : mantusaha.bu@gmail.com Introduction and Objectives In the preceding chapters, we discussed normed

### Today: Linear Programming

Today: Linear Programming COSC 581, Algorithms March 27, 2014 Many of these slides are adapted from several online sources Today s class: Chapter 29.1 Reading Assignments Reading assignment for next Thursday

### Optimization (168) Lecture 7-8-9

Optimization (168) Lecture 7-8-9 Jesús De Loera UC Davis, Mathematics Wednesday, April 2, 2012 1 DEGENERACY IN THE SIMPLEX METHOD 2 DEGENERACY z =2x 1 x 2 + 8x 3 x 4 =1 2x 3 x 5 =3 2x 1 + 4x 2 6x 3 x 6

### CS295: Convex Optimization. Xiaohui Xie Department of Computer Science University of California, Irvine

CS295: Convex Optimization Xiaohui Xie Department of Computer Science University of California, Irvine Course information Prerequisites: multivariate calculus and linear algebra Textbook: Convex Optimization

### DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology 18.453: Combinatorial Optimization Michel X. Goemans April 5, 2017 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory

### Lecture 1 Introduction and overview

ESE504 (Fall 2010) Lecture 1 Introduction and overview linear programming example course topics software integer linear programming 1 1 Linear program (LP) minimize subject to n c j x j j=1 n a ij x j

### COT 6936: Topics in Algorithms! Giri Narasimhan. ECS 254A / EC 2443; Phone: x3748

COT 6936: Topics in Algorithms! Giri Narasimhan ECS 254A / EC 2443; Phone: x3748 giri@cs.fiu.edu https://moodle.cis.fiu.edu/v2.1/course/view.php?id=612 Gaussian Elimination! Solving a system of simultaneous

### Geometric problems. Chapter Projection on a set. The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as

Chapter 8 Geometric problems 8.1 Projection on a set The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as dist(x 0,C) = inf{ x 0 x x C}. The infimum here is always achieved.

### CSCI : Optimization and Control of Networks. Review on Convex Optimization

CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one

### INNER PRODUCT SPACE. Definition 1

INNER PRODUCT SPACE Definition 1 Suppose u, v and w are all vectors in vector space V and c is any scalar. An inner product space on the vectors space V is a function that associates with each pair of

### CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming

CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cash-flow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150

### 2. Norm, distance, angle

L. Vandenberghe EE133A (Spring 2017) 2. Norm, distance, angle norm distance k-means algorithm angle hyperplanes complex vectors 2-1 Euclidean norm (Euclidean) norm of vector a R n : a = a 2 1 + a2 2 +

### Linear Algebra. Session 12

Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)

### 5. Orthogonal matrices

L Vandenberghe EE133A (Spring 2017) 5 Orthogonal matrices matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns 5-1 Orthonormal

### Linear and Integer Programming - ideas

Linear and Integer Programming - ideas Paweł Zieliński Institute of Mathematics and Computer Science, Wrocław University of Technology, Poland http://www.im.pwr.wroc.pl/ pziel/ Toulouse, France 2012 Literature

### Linear Programming Duality

Summer 2011 Optimization I Lecture 8 1 Duality recap Linear Programming Duality We motivated the dual of a linear program by thinking about the best possible lower bound on the optimal value we can achieve

### j=1 [We will show that the triangle inequality holds for each p-norm in Chapter 3 Section 6.] The 1-norm is A F = tr(a H A).

Math 344 Lecture #19 3.5 Normed Linear Spaces Definition 3.5.1. A seminorm on a vector space V over F is a map : V R that for all x, y V and for all α F satisfies (i) x 0 (positivity), (ii) αx = α x (scale

### CSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming

CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming

### Nonlinear Programming Models

Nonlinear Programming Models Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Nonlinear Programming Models p. Introduction Nonlinear Programming Models p. NLP problems minf(x) x S R n Standard form:

### EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 18

EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 18 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 31, 2012 Andre Tkacenko

### Topics in Theoretical Computer Science April 08, Lecture 8

Topics in Theoretical Computer Science April 08, 204 Lecture 8 Lecturer: Ola Svensson Scribes: David Leydier and Samuel Grütter Introduction In this lecture we will introduce Linear Programming. It was

### 3 Development of the Simplex Method Constructing Basic Solution Optimality Conditions The Simplex Method...

Contents Introduction to Linear Programming Problem. 2. General Linear Programming problems.............. 2.2 Formulation of LP problems.................... 8.3 Compact form and Standard form of a general

### Introduction to optimization

Introduction to optimization Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 24 The plan 1. The basic concepts 2. Some useful tools (linear programming = linear optimization)

### CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.

1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function

### AN INTRODUCTION TO CONVEXITY

AN INTRODUCTION TO CONVEXITY GEIR DAHL NOVEMBER 2010 University of Oslo, Centre of Mathematics for Applications, P.O.Box 1053, Blindern, 0316 Oslo, Norway (geird@math.uio.no) Contents 1 The basic concepts

### Resource Constrained Project Scheduling Linear and Integer Programming (1)

DM204, 2010 SCHEDULING, TIMETABLING AND ROUTING Lecture 3 Resource Constrained Project Linear and Integer Programming (1) Marco Chiarandini Department of Mathematics & Computer Science University of Southern

### 6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC

6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC 2003 2003.09.02.10 6. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets Tarski-Seidenberg and quantifier elimination Feasibility

### Lecture 6: Conic Optimization September 8

IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions

### Inner products and Norms. Inner product of 2 vectors. Inner product of 2 vectors x and y in R n : x 1 y 1 + x 2 y x n y n in R n

Inner products and Norms Inner product of 2 vectors Inner product of 2 vectors x and y in R n : x 1 y 1 + x 2 y 2 + + x n y n in R n Notation: (x, y) or y T x For complex vectors (x, y) = x 1 ȳ 1 + x 2

### Linear Algebra Massoud Malek

CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

### Introduction to Matrix Algebra

Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p

### Lecture 6 Simplex method for linear programming

Lecture 6 Simplex method for linear programming Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University,

### Linear Algebra Review. Vectors

Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors

### Convex Optimization & Machine Learning. Introduction to Optimization

Convex Optimization & Machine Learning Introduction to Optimization mcuturi@i.kyoto-u.ac.jp CO&ML 1 Why do we need optimization in machine learning We want to find the best possible decision w.r.t. a problem

### Quantum Computing Lecture 2. Review of Linear Algebra

Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces

### Lecture Note 5: Semidefinite Programming for Stability Analysis

ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State

### Duality. Peter Bro Mitersen (University of Aarhus) Optimization, Lecture 9 February 28, / 49

Duality Maximize c T x for x F = {x (R + ) n Ax b} If we guess x F, we can say that c T x is a lower bound for the optimal value without executing the simplex algorithm. Can we make similar easy guesses

### A Strongly Polynomial Simplex Method for Totally Unimodular LP

A Strongly Polynomial Simplex Method for Totally Unimodular LP Shinji Mizuno July 19, 2014 Abstract Kitahara and Mizuno get new bounds for the number of distinct solutions generated by the simplex method

### 15. Conic optimization

L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone

### The Transpose of a Vector

8 CHAPTER Vectors The Transpose of a Vector We now consider the transpose of a vector in R n, which is a row vector. For a vector u 1 u. u n the transpose is denoted by u T = [ u 1 u u n ] EXAMPLE -5 Find

### CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization

CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization April 6, 2018 1 / 34 This material is covered in the textbook, Chapters 9 and 10. Some of the materials are taken from it. Some of

### Lecture 7. Econ August 18

Lecture 7 Econ 2001 2015 August 18 Lecture 7 Outline First, the theorem of the maximum, an amazing result about continuity in optimization problems. Then, we start linear algebra, mostly looking at familiar

### Lecture 20: 6.1 Inner Products

Lecture 0: 6.1 Inner Products Wei-Ta Chu 011/1/5 Definition An inner product on a real vector space V is a function that associates a real number u, v with each pair of vectors u and v in V in such a way

### EE 546, Univ of Washington, Spring Proximal mapping. introduction. review of conjugate functions. proximal mapping. Proximal mapping 6 1

EE 546, Univ of Washington, Spring 2012 6. Proximal mapping introduction review of conjugate functions proximal mapping Proximal mapping 6 1 Proximal mapping the proximal mapping (prox-operator) of a convex

### A Review of Linear Programming

A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex

### Theory and Internet Protocols

Game Lecture 2: Linear Programming and Zero Sum Nash Equilibrium Xiaotie Deng AIMS Lab Department of Computer Science Shanghai Jiaotong University September 26, 2016 1 2 3 4 Standard Form (P) Outline

### which has a check digit of 9. This is consistent with the first nine digits of the ISBN, since

vector Then the check digit c is computed using the following procedure: 1. Form the dot product. 2. Divide by 11, thereby producing a remainder c that is an integer between 0 and 10, inclusive. The check

### III. Linear Programming

III. Linear Programming Thomas Sauerwald Easter 2017 Outline Introduction Standard and Slack Forms Formulating Problems as Linear Programs Simplex Algorithm Finding an Initial Solution III. Linear Programming

### 6.1 Matrices. Definition: A Matrix A is a rectangular array of the form. A 11 A 12 A 1n A 21. A 2n. A m1 A m2 A mn A 22.

61 Matrices Definition: A Matrix A is a rectangular array of the form A 11 A 12 A 1n A 21 A 22 A 2n A m1 A m2 A mn The size of A is m n, where m is the number of rows and n is the number of columns The

### Linear Programming. Linear Programming I. Lecture 1. Linear Programming. Linear Programming

Linear Programming Linear Programming Lecture Linear programming. Optimize a linear function subject to linear inequalities. (P) max " c j x j n j= n s. t. " a ij x j = b i # i # m j= x j 0 # j # n (P)

### Linear Programming. Chapter Introduction

Chapter 3 Linear Programming Linear programs (LP) play an important role in the theory and practice of optimization problems. Many COPs can directly be formulated as LPs. Furthermore, LPs are invaluable

### Algorithmic Game Theory and Applications. Lecture 7: The LP Duality Theorem

Algorithmic Game Theory and Applications Lecture 7: The LP Duality Theorem Kousha Etessami recall LP s in Primal Form 1 Maximize c 1 x 1 + c 2 x 2 +... + c n x n a 1,1 x 1 + a 1,2 x 2 +... + a 1,n x n

### Chapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.

Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space

### Basic Concepts in Matrix Algebra

Basic Concepts in Matrix Algebra An column array of p elements is called a vector of dimension p and is written as x p 1 = x 1 x 2. x p. The transpose of the column vector x p 1 is row vector x = [x 1

### Introduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs

Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following

### web: HOMEWORK 1

MAT 207 LINEAR ALGEBRA I 2009207 Dokuz Eylül University, Faculty of Science, Department of Mathematics Instructor: Engin Mermut web: http://kisideuedutr/enginmermut/ HOMEWORK VECTORS IN THE n-dimensional

### 2. Review of Linear Algebra

2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear

### A TOUR OF LINEAR ALGEBRA FOR JDEP 384H

A TOUR OF LINEAR ALGEBRA FOR JDEP 384H Contents Solving Systems 1 Matrix Arithmetic 3 The Basic Rules of Matrix Arithmetic 4 Norms and Dot Products 5 Norms 5 Dot Products 6 Linear Programming 7 Eigenvectors

### An introductory example

CS1 Lecture 9 An introductory example Suppose that a company that produces three products wishes to decide the level of production of each so as to maximize profits. Let x 1 be the amount of Product 1

### Lecture 23: 6.1 Inner Products

Lecture 23: 6.1 Inner Products Wei-Ta Chu 2008/12/17 Definition An inner product on a real vector space V is a function that associates a real number u, vwith each pair of vectors u and v in V in such

### Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization

Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The