Functions of Several Variables
|
|
- Kory Blake
- 5 years ago
- Views:
Transcription
1 Jim Lambers MAT 419/519 Summer Session Lecture 2 Notes These notes correspond to Section 1.2 in the text. Functions of Several Variables We now generalize the results from the previous section, pertaining to optimization of functions of one variable, to functions of several variables. However, we first need some notation and definitions. Definition An n-vector in R n is an ordered n-tuple x = (x 1, x 2,..., x n ) of real numbers x i, called the components of x. Vectors belong to vector spaces, which support two essential operations. We define addition of two vectors x = (x 1, x 2,..., x n ) and y = (y 1, y 2,..., y n ) in R n by and multiplication of x and a real number λ by x + y = (x 1 + y1, x 2 + y 2,..., x n + y n ), λx = (λx 1, λx 2,..., λx n ). Multiplication of numbers needs to be generalized to a sort of multiplication operation involving two vectors. Definition If x = (x 1, x 2,..., x n ) and y = (y 1, y 2,..., y n ) are vectors in R n, their dot product or inner product x y is defined by x y = x 1 y 1 + x 2 y x n y n = Two vectors x and y are orthogonal if x y = 0. n x k y k. We also need to generalize the notion of absolute value, or a number s magnitude, to the magnitude of a vector. Definition The norm or length x of a vector x = (x 1, x 2,..., x n ) in R n is defined by k=1 x = (x x x 2 n) 1/2 = (x x) 1/2. The norm is a real-valued function on R n with the following properties: 1
2 1. x 0 for all vectors x R n. 2. x = 0 if and only if x = αx = α x for all vectors x R n and all real numbers α. 4. x + y x + y, the Triangle Inequality 5. x y x y, the Cauchy-Schwarz Inequality Using the norm, the dot product can also be defined as x y = x y cos θ where θ is the angle between x and y. Just as the distance between two numbers x and y is given by x y, the distance between two points in n-dimensional space can be defined similarly. Definition If x, y R n, the distance d(x, y) between x and y is defined by ( n ) 1/2 d(x, y) = x y = (x i y i ) 2. The ball B(x, r) centered at x of radius r is the set of all vectors y R n such that d(x, y) < r. A point x in a set D R n is an interior point of D if there exists an r > 0 such that B(x, r) D. The interior D 0 is the set of all interior points of D. A set G R n is open if G 0 = G. A set F R n is closed if its complement G = F c in R n is open. Now, we are prepared to define minimizers and maximizers of functions of n variables. Definition Suppose that f : D R n R. A point x D is a 1. global minimizer of f on D if f(x ) f for all x D; 2. strict global minimizer of f on D if f(x ) < f for all x D; 3. local minimizer of f(x if there exists a δ > 0 such that f(x ) f whenever x B(x, δ); 4. strict local minimizer of f if there exists a δ > 0 such that f(x ) < f whenever x B(x, δ) and x x ; 5. critical point of f if the first partial derivatives of f exist at x and i=1 x i (x ) = 0, i = 1, 2,..., n. 2
3 Using this definition of a critical point, we can now characterize the location of maximizers and minimizers, as in Fermat s theorem in the single-variable case. Theorem Suppose that f is a real-valued function for which all first partial derivatives of f exist on a subset D of R n. If x is an interior point of D that is a local minimizer of f, then x is a critical point of f. This theorem can be proved by reduction to the single-variable case, in which all variables except one are fixed. We need to generalize Taylor s Formula to the multi-variable case. Given a function f : R n R whose first and second partial derivatives are continuous on an open set containing the line segment joining x and x. By defining the function [x, x] = {w R n w = x + t(x x ), 0 t 1} ϕ(t) = f(x + t(x x )) and applying Taylor s Formula in conjunction with the multi-variable Chain Rule, we obtain the following result. Theorem Suppose that x, x R n and that f : D R n R with continuous first and second partial derivatives on some open set containing the line segment [x, x]. Then there exists a z [x, x] such that where is the gradient of f, and is the Hessian of f. f = f(x ) + f(x ) (x x ) (x x ) Hf(z)(x x ) f = Hf = [ x 1 x 2 1 x 2 x 1. x n x 1 x 2 x n x 1 x 2 x 1 x n 2 f x 2 x 2 2 x n..... x n x 2 2 f x 2 n Now we can characterize local or global maximizers or minimizers based on the second partial derivatives, in the same way as in the single-variable case. Theorem Suppose that x is a critical point of f with continuous first and second partial derivatives on R n. Then: 3 ]
4 1. x is a global minimizer of f if (x x ) Hf(z)(x x ) 0 for all x R n and all z [x, x]; 2. x is a strict global minimizer of f if (x x ) Hf(z)(x x ) > 0 for all x R n and all z [x, x]; 3. x is a global maximizer of f if (x x ) Hf(z)(x x ) 0 for all x R n and all z [x, x]; 4. x is a strict global maximizer of f if (x x ) Hf(z)(x x ) < 0 for all x R n and all z [x, x]. This theorem can be proved using the multi-variable generalization of Taylor s Theorem, in conjunction with the continuity of the second partial derivatives. Unfortunately, the sign of (x x ) Hf(z)(x x ) is not so easily determined, in comparison to the single-variable counterpart f (z)(x x ) 2. To that end, we turn to concepts from linear algebra. Definition Let A be an n n symmetric matrix. The quadratic form associated with A is a function Q A : R n R defined by Q A (y) = y Ay = n a ij y i y j, y R n. i,j=1 Example Let f(x, y, z) = x 2 y 2 + 4z 2 2xy + 4yz. Then we have f(x, y, z) = (2x 2y, 2y 2x + 4z, 8z + 4y) and It follows that Hf(x, y, z) = Q Hf (x, y, z) = (x, y, z) Hf(x, y, z)(x, y, z) = (x, y, z) (2x 2y, 2x 2y + 4z, 4y + 8z) = 2x 2 2y 2 + 8z 2 4xy + 8yz = 2f(x, y, z). The following terms will enable us to more easily describe the conditions for local or global minimizers or maximizers. 4
5 Definition Suppose that A is an n n symmetric matrix and that Q A (y) = y Ay is the quadratic form associated with A. Then A and Q A are called: 1. positive semidefinite if Q A (y) 0 for all y R n ; 2. positive definite if Q A (y) > 0 for all y R n, y 0; 3. negative semidefinite if Q A (y) 0 for all y R n ; 4. negative definite if Q A (y) < 0 for all y R n, y 0; 5. indefinite if Q A (y) > 0 for some y R n and Q A (y) < 0 for other y R n. With these terms, the preceding theorem can be restated more concisely as follows: Theorem Suppose that x is a critical point of a function f with continuous first and second partial derivatives on R n and that Hf is the Hessian of f. Then x is a 1. global minimizer of f if Hf is positive semidefinite on R n ; 2. strict global minimizer of f if Hf is positive definite on R n ; 3. global maximizer of f if Hf is negative semidefinite on R n ; 4. strict global maximizer of f if Hf is negative definite on R n. It remains to determine when a given matrix is positive (or negative) definite (or semidefinite). This will be taken up in subsequent lectures. Exercises 1. Chapter 1, Exercise 3 2. Chapter 1, Exercise 4 3. Chapter 1, Exercise 5 5
MAT 419 Lecture Notes Transcribed by Eowyn Cenek 6/1/2012
(Homework 1: Chapter 1: Exercises 1-7, 9, 11, 19, due Monday June 11th See also the course website for lectures, assignments, etc) Note: today s lecture is primarily about definitions Lots of definitions
More informationMATH529 Fundamentals of Optimization Unconstrained Optimization II
MATH529 Fundamentals of Optimization Unconstrained Optimization II Marco A. Montes de Oca Mathematical Sciences, University of Delaware, USA 1 / 31 Recap 2 / 31 Example Find the local and global minimizers
More informationJim Lambers MAT 419/519 Summer Session Lecture 13 Notes
Jim Lambers MAT 419/519 Summer Session 2011-12 Lecture 13 Notes These notes correspond to Section 4.1 in the text. Least Squares Fit One of the most fundamental problems in science and engineering is data
More informationReal Analysis III. (MAT312β) Department of Mathematics University of Ruhuna. A.W.L. Pubudu Thilan
Real Analysis III (MAT312β) Department of Mathematics University of Ruhuna A.W.L. Pubudu Thilan Department of Mathematics University of Ruhuna Real Analysis III(MAT312β) 1/87 About course unit Course unit:
More informationIntroduction to Proofs
Real Analysis Preview May 2014 Properties of R n Recall Oftentimes in multivariable calculus, we looked at properties of vectors in R n. If we were given vectors x =< x 1, x 2,, x n > and y =< y1, y 2,,
More informationThe general programming problem is the nonlinear programming problem where a given function is maximized subject to a set of inequality constraints.
1 Optimization Mathematical programming refers to the basic mathematical problem of finding a maximum to a function, f, subject to some constraints. 1 In other words, the objective is to find a point,
More informationPreliminary draft only: please check for final version
ARE211, Fall2012 CALCULUS4: THU, OCT 11, 2012 PRINTED: AUGUST 22, 2012 (LEC# 15) Contents 3. Univariate and Multivariate Differentiation (cont) 1 3.6. Taylor s Theorem (cont) 2 3.7. Applying Taylor theory:
More information1 Directional Derivatives and Differentiability
Wednesday, January 18, 2012 1 Directional Derivatives and Differentiability Let E R N, let f : E R and let x 0 E. Given a direction v R N, let L be the line through x 0 in the direction v, that is, L :=
More informationAnalysis-3 lecture schemes
Analysis-3 lecture schemes (with Homeworks) 1 Csörgő István November, 2015 1 A jegyzet az ELTE Informatikai Kar 2015. évi Jegyzetpályázatának támogatásával készült Contents 1. Lesson 1 4 1.1. The Space
More informationFunctional Analysis MATH and MATH M6202
Functional Analysis MATH 36202 and MATH M6202 1 Inner Product Spaces and Normed Spaces Inner Product Spaces Functional analysis involves studying vector spaces where we additionally have the notion of
More informationVectors in Function Spaces
Jim Lambers MAT 66 Spring Semester 15-16 Lecture 18 Notes These notes correspond to Section 6.3 in the text. Vectors in Function Spaces We begin with some necessary terminology. A vector space V, also
More informationChapter 2: Unconstrained Extrema
Chapter 2: Unconstrained Extrema Math 368 c Copyright 2012, 2013 R Clark Robinson May 22, 2013 Chapter 2: Unconstrained Extrema 1 Types of Sets Definition For p R n and r > 0, the open ball about p of
More informationMath (P)refresher Lecture 8: Unconstrained Optimization
Math (P)refresher Lecture 8: Unconstrained Optimization September 2006 Today s Topics : Quadratic Forms Definiteness of Quadratic Forms Maxima and Minima in R n First Order Conditions Second Order Conditions
More informationJim Lambers MAT 610 Summer Session Lecture 2 Notes
Jim Lambers MAT 610 Summer Session 2009-10 Lecture 2 Notes These notes correspond to Sections 2.2-2.4 in the text. Vector Norms Given vectors x and y of length one, which are simply scalars x and y, the
More informationLecture Notes on Metric Spaces
Lecture Notes on Metric Spaces Math 117: Summer 2007 John Douglas Moore Our goal of these notes is to explain a few facts regarding metric spaces not included in the first few chapters of the text [1],
More informationB553 Lecture 3: Multivariate Calculus and Linear Algebra Review
B553 Lecture 3: Multivariate Calculus and Linear Algebra Review Kris Hauser December 30, 2011 We now move from the univariate setting to the multivariate setting, where we will spend the rest of the class.
More informationChapter 7. Extremal Problems. 7.1 Extrema and Local Extrema
Chapter 7 Extremal Problems No matter in theoretical context or in applications many problems can be formulated as problems of finding the maximum or minimum of a function. Whenever this is the case, advanced
More informationRiemannian geometry of surfaces
Riemannian geometry of surfaces In this note, we will learn how to make sense of the concepts of differential geometry on a surface M, which is not necessarily situated in R 3. This intrinsic approach
More informationORTHOGONALITY AND LEAST-SQUARES [CHAP. 6]
ORTHOGONALITY AND LEAST-SQUARES [CHAP. 6] Inner products and Norms Inner product or dot product of 2 vectors u and v in R n : u.v = u 1 v 1 + u 2 v 2 + + u n v n Calculate u.v when u = 1 2 2 0 v = 1 0
More informationMAT 473 Intermediate Real Analysis II
MAT 473 Intermediate Real Analysis II John Quigg Spring 2009 revised February 5, 2009 Derivatives Here our purpose is to give a rigorous foundation of the principles of differentiation in R n. Much of
More informationJim Lambers MAT 169 Fall Semester Lecture 6 Notes. a n. n=1. S = lim s k = lim. n=1. n=1
Jim Lambers MAT 69 Fall Semester 2009-0 Lecture 6 Notes These notes correspond to Section 8.3 in the text. The Integral Test Previously, we have defined the sum of a convergent infinite series to be the
More informationProf. M. Saha Professor of Mathematics The University of Burdwan West Bengal, India
CHAPTER 9 BY Prof. M. Saha Professor of Mathematics The University of Burdwan West Bengal, India E-mail : mantusaha.bu@gmail.com Introduction and Objectives In the preceding chapters, we discussed normed
More informationCHAPTER 4: HIGHER ORDER DERIVATIVES. Likewise, we may define the higher order derivatives. f(x, y, z) = xy 2 + e zx. y = 2xy.
April 15, 2009 CHAPTER 4: HIGHER ORDER DERIVATIVES In this chapter D denotes an open subset of R n. 1. Introduction Definition 1.1. Given a function f : D R we define the second partial derivatives as
More informationLinear Algebra. Session 12
Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)
More informationMATHEMATICAL ECONOMICS: OPTIMIZATION. Contents
MATHEMATICAL ECONOMICS: OPTIMIZATION JOÃO LOPES DIAS Contents 1. Introduction 2 1.1. Preliminaries 2 1.2. Optimal points and values 2 1.3. The optimization problems 3 1.4. Existence of optimal points 4
More informationMath General Topology Fall 2012 Homework 1 Solutions
Math 535 - General Topology Fall 2012 Homework 1 Solutions Definition. Let V be a (real or complex) vector space. A norm on V is a function : V R satisfying: 1. Positivity: x 0 for all x V and moreover
More informationLinear Analysis Lecture 5
Linear Analysis Lecture 5 Inner Products and V Let dim V < with inner product,. Choose a basis B and let v, w V have coordinates in F n given by x 1. x n and y 1. y n, respectively. Let A F n n be the
More informationDefinitions and Properties of R N
Definitions and Properties of R N R N as a set As a set R n is simply the set of all ordered n-tuples (x 1,, x N ), called vectors. We usually denote the vector (x 1,, x N ), (y 1,, y N ), by x, y, or
More informationMATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products.
MATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products. Orthogonal projection Theorem 1 Let V be a subspace of R n. Then any vector x R n is uniquely represented
More informationTangent Planes, Linear Approximations and Differentiability
Jim Lambers MAT 80 Spring Semester 009-10 Lecture 5 Notes These notes correspond to Section 114 in Stewart and Section 3 in Marsden and Tromba Tangent Planes, Linear Approximations and Differentiability
More informationMathematical Economics: Lecture 16
Mathematical Economics: Lecture 16 Yu Ren WISE, Xiamen University November 26, 2012 Outline 1 Chapter 21: Concave and Quasiconcave Functions New Section Chapter 21: Concave and Quasiconcave Functions Concave
More information1. General Vector Spaces
1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule
More informationLecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set.
Lecture # 3 Orthogonal Matrices and Matrix Norms We repeat the definition an orthogonal set and orthornormal set. Definition A set of k vectors {u, u 2,..., u k }, where each u i R n, is said to be an
More informationUnconstrained Geometric Programming
Jim Lambers MAT 49/59 Summer Session 20-2 Lecture 8 Notes These notes correspond to Section 2.5 in the text. Unconstrained Geometric Programming Previously, we learned how to use the A-G Inequality to
More informationConvex Analysis and Economic Theory Winter 2018
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Winter 2018 Supplement A: Mathematical background A.1 Extended real numbers The extended real number
More informationDifferentiable Functions
Differentiable Functions Let S R n be open and let f : R n R. We recall that, for x o = (x o 1, x o,, x o n S the partial derivative of f at the point x o with respect to the component x j is defined as
More informationECON 5111 Mathematical Economics
Test 1 October 1, 2010 1. Construct a truth table for the following statement: [p (p q)] q. 2. A prime number is a natural number that is divisible by 1 and itself only. Let P be the set of all prime numbers
More informationConvex Functions and Optimization
Chapter 5 Convex Functions and Optimization 5.1 Convex Functions Our next topic is that of convex functions. Again, we will concentrate on the context of a map f : R n R although the situation can be generalized
More informationx +3y 2t = 1 2x +y +z +t = 2 3x y +z t = 7 2x +6y +z +t = a
UCM Final Exam, 05/8/014 Solutions 1 Given the parameter a R, consider the following linear system x +y t = 1 x +y +z +t = x y +z t = 7 x +6y +z +t = a (a (6 points Discuss the system depending on the
More informationChapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.
Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space
More informationx 1. x n i + x 2 j (x 1, x 2, x 3 ) = x 1 j + x 3
Version: 4/1/06. Note: These notes are mostly from my 5B course, with the addition of the part on components and projections. Look them over to make sure that we are on the same page as regards inner-products,
More informationg(t) = f(x 1 (t),..., x n (t)).
Reading: [Simon] p. 313-333, 833-836. 0.1 The Chain Rule Partial derivatives describe how a function changes in directions parallel to the coordinate axes. Now we shall demonstrate how the partial derivatives
More informationl(y j ) = 0 for all y j (1)
Problem 1. The closed linear span of a subset {y j } of a normed vector space is defined as the intersection of all closed subspaces containing all y j and thus the smallest such subspace. 1 Show that
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More informationLecture 3. Econ August 12
Lecture 3 Econ 2001 2015 August 12 Lecture 3 Outline 1 Metric and Metric Spaces 2 Norm and Normed Spaces 3 Sequences and Subsequences 4 Convergence 5 Monotone and Bounded Sequences Announcements: - Friday
More informationj=1 [We will show that the triangle inequality holds for each p-norm in Chapter 3 Section 6.] The 1-norm is A F = tr(a H A).
Math 344 Lecture #19 3.5 Normed Linear Spaces Definition 3.5.1. A seminorm on a vector space V over F is a map : V R that for all x, y V and for all α F satisfies (i) x 0 (positivity), (ii) αx = α x (scale
More informationChapter 2: Preliminaries and elements of convex analysis
Chapter 2: Preliminaries and elements of convex analysis Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-14-15.shtml Academic year 2014-15
More informationLinear Algebra. Alvin Lin. August December 2017
Linear Algebra Alvin Lin August 207 - December 207 Linear Algebra The study of linear algebra is about two basic things. We study vector spaces and structure preserving maps between vector spaces. A vector
More informationMA 102 (Multivariable Calculus)
MA 102 (Multivariable Calculus) Rupam Barman and Shreemayee Bora Department of Mathematics IIT Guwahati Outline of the Course Two Topics: Multivariable Calculus Will be taught as the first part of the
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Introduction and a quick repetition of analysis/linear algebra First lecture, 12.04.2010 Jun.-Prof. Matthias Hein Organization of the lecture Advanced course, 2+2 hours,
More informationEC /11. Math for Microeconomics September Course, Part II Lecture Notes. Course Outline
LONDON SCHOOL OF ECONOMICS Professor Leonardo Felli Department of Economics S.478; x7525 EC400 20010/11 Math for Microeconomics September Course, Part II Lecture Notes Course Outline Lecture 1: Tools for
More informationMath 341: Convex Geometry. Xi Chen
Math 341: Convex Geometry Xi Chen 479 Central Academic Building, University of Alberta, Edmonton, Alberta T6G 2G1, CANADA E-mail address: xichen@math.ualberta.ca CHAPTER 1 Basics 1. Euclidean Geometry
More informationElementary linear algebra
Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The
More informationMathematics Department Stanford University Math 61CM/DM Inner products
Mathematics Department Stanford University Math 61CM/DM Inner products Recall the definition of an inner product space; see Appendix A.8 of the textbook. Definition 1 An inner product space V is a vector
More informationMath 212-Lecture Interior critical points of functions of two variables
Math 212-Lecture 24 13.10. Interior critical points of functions of two variables Previously, we have concluded that if f has derivatives, all interior local min or local max should be critical points.
More informationCHAPTER VIII HILBERT SPACES
CHAPTER VIII HILBERT SPACES DEFINITION Let X and Y be two complex vector spaces. A map T : X Y is called a conjugate-linear transformation if it is a reallinear transformation from X into Y, and if T (λx)
More information1 Lagrange Multiplier Method
1 Lagrange Multiplier Method Near a maximum the decrements on both sides are in the beginning only imperceptible. J. Kepler When a quantity is greatest or least, at that moment its flow neither increases
More informationEuclidean Space. This is a brief review of some basic concepts that I hope will already be familiar to you.
Euclidean Space This is a brief review of some basic concepts that I hope will already be familiar to you. There are three sets of numbers that will be especially important to us: The set of all real numbers,
More informationMA102: Multivariable Calculus
MA102: Multivariable Calculus Rupam Barman and Shreemayee Bora Department of Mathematics IIT Guwahati Differentiability of f : U R n R m Definition: Let U R n be open. Then f : U R n R m is differentiable
More informationTable of mathematical symbols - Wikipedia, the free encyclopedia
Página 1 de 13 Table of mathematical symbols From Wikipedia, the free encyclopedia For the HTML codes of mathematical symbols see mathematical HTML. Note: This article contains special characters. The
More informationNOTES ON MULTIVARIABLE CALCULUS: DIFFERENTIAL CALCULUS
NOTES ON MULTIVARIABLE CALCULUS: DIFFERENTIAL CALCULUS SAMEER CHAVAN Abstract. This is the first part of Notes on Multivariable Calculus based on the classical texts [6] and [5]. We present here the geometric
More informationThe Transpose of a Vector
8 CHAPTER Vectors The Transpose of a Vector We now consider the transpose of a vector in R n, which is a row vector. For a vector u 1 u. u n the transpose is denoted by u T = [ u 1 u u n ] EXAMPLE -5 Find
More information1. Subspaces A subset M of Hilbert space H is a subspace of it is closed under the operation of forming linear combinations;i.e.,
Abstract Hilbert Space Results We have learned a little about the Hilbert spaces L U and and we have at least defined H 1 U and the scale of Hilbert spaces H p U. Now we are going to develop additional
More informationVector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis.
Vector spaces DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Vector space Consists of: A set V A scalar
More informationOptimization Theory. A Concise Introduction. Jiongmin Yong
October 11, 017 16:5 ws-book9x6 Book Title Optimization Theory 017-08-Lecture Notes page 1 1 Optimization Theory A Concise Introduction Jiongmin Yong Optimization Theory 017-08-Lecture Notes page Optimization
More informationMATH 5720: Unconstrained Optimization Hung Phan, UMass Lowell September 13, 2018
MATH 57: Unconstrained Optimization Hung Phan, UMass Lowell September 13, 18 1 Global and Local Optima Let a function f : S R be defined on a set S R n Definition 1 (minimizers and maximizers) (i) x S
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More informationMath 117: Topology of the Real Numbers
Math 117: Topology of the Real Numbers John Douglas Moore November 10, 2008 The goal of these notes is to highlight the most important topics presented in Chapter 3 of the text [1] and to provide a few
More informationElements of Convex Optimization Theory
Elements of Convex Optimization Theory Costis Skiadas August 2015 This is a revised and extended version of Appendix A of Skiadas (2009), providing a self-contained overview of elements of convex optimization
More informationThe Derivative. Appendix B. B.1 The Derivative of f. Mappings from IR to IR
Appendix B The Derivative B.1 The Derivative of f In this chapter, we give a short summary of the derivative. Specifically, we want to compare/contrast how the derivative appears for functions whose domain
More informationFunctional Analysis. Franck Sueur Metric spaces Definitions Completeness Compactness Separability...
Functional Analysis Franck Sueur 2018-2019 Contents 1 Metric spaces 1 1.1 Definitions........................................ 1 1.2 Completeness...................................... 3 1.3 Compactness......................................
More informationYour first day at work MATH 806 (Fall 2015)
Your first day at work MATH 806 (Fall 2015) 1. Let X be a set (with no particular algebraic structure). A function d : X X R is called a metric on X (and then X is called a metric space) when d satisfies
More informationARE211, Fall2015. Contents. 2. Univariate and Multivariate Differentiation (cont) Taylor s Theorem (cont) 2
ARE211, Fall2015 CALCULUS4: THU, SEP 17, 2015 PRINTED: SEPTEMBER 22, 2015 (LEC# 7) Contents 2. Univariate and Multivariate Differentiation (cont) 1 2.4. Taylor s Theorem (cont) 2 2.5. Applying Taylor theory:
More informationInner Product and Orthogonality
Inner Product and Orthogonality P. Sam Johnson October 3, 2014 P. Sam Johnson (NITK) Inner Product and Orthogonality October 3, 2014 1 / 37 Overview In the Euclidean space R 2 and R 3 there are two concepts,
More informationLecture Notes 1: Vector spaces
Optimization-based data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector
More informationTutorials in Optimization. Richard Socher
Tutorials in Optimization Richard Socher July 20, 2008 CONTENTS 1 Contents 1 Linear Algebra: Bilinear Form - A Simple Optimization Problem 2 1.1 Definitions........................................ 2 1.2
More informationDS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More information(x, y) = d(x, y) = x y.
1 Euclidean geometry 1.1 Euclidean space Our story begins with a geometry which will be familiar to all readers, namely the geometry of Euclidean space. In this first chapter we study the Euclidean distance
More informationNATIONAL UNIVERSITY OF SINGAPORE Department of Mathematics MA4247 Complex Analysis II Lecture Notes Part II
NATIONAL UNIVERSITY OF SINGAPORE Department of Mathematics MA4247 Complex Analysis II Lecture Notes Part II Chapter 2 Further properties of analytic functions 21 Local/Global behavior of analytic functions;
More informationAnother consequence of the Cauchy Schwarz inequality is the continuity of the inner product.
. Inner product spaces 1 Theorem.1 (Cauchy Schwarz inequality). If X is an inner product space then x,y x y. (.) Proof. First note that 0 u v v u = u v u v Re u,v. (.3) Therefore, Re u,v u v (.) for all
More informationSolutions to Homework 7
Solutions to Homework 7 Exercise #3 in section 5.2: A rectangular box is inscribed in a hemisphere of radius r. Find the dimensions of the box of maximum volume. Solution: The base of the rectangular box
More informationCourse Summary Math 211
Course Summary Math 211 table of contents I. Functions of several variables. II. R n. III. Derivatives. IV. Taylor s Theorem. V. Differential Geometry. VI. Applications. 1. Best affine approximations.
More informationFunctional Analysis Exercise Class
Functional Analysis Exercise Class Week 9 November 13 November Deadline to hand in the homeworks: your exercise class on week 16 November 20 November Exercises (1) Show that if T B(X, Y ) and S B(Y, Z)
More information1.2 LECTURE 2. Scalar Product
6 CHAPTER 1. VECTOR ALGEBRA Pythagean theem. cos 2 α 1 + cos 2 α 2 + cos 2 α 3 = 1 There is a one-to-one crespondence between the components of the vect on the one side and its magnitude and the direction
More informationInner products and Norms. Inner product of 2 vectors. Inner product of 2 vectors x and y in R n : x 1 y 1 + x 2 y x n y n in R n
Inner products and Norms Inner product of 2 vectors Inner product of 2 vectors x and y in R n : x 1 y 1 + x 2 y 2 + + x n y n in R n Notation: (x, y) or y T x For complex vectors (x, y) = x 1 ȳ 1 + x 2
More informationa. Define a function called an inner product on pairs of points x = (x 1, x 2,..., x n ) and y = (y 1, y 2,..., y n ) in R n by
Real Analysis Homework 1 Solutions 1. Show that R n with the usual euclidean distance is a metric space. Items a-c will guide you through the proof. a. Define a function called an inner product on pairs
More informationOn the interior of the simplex, we have the Hessian of d(x), Hd(x) is diagonal with ith. µd(w) + w T c. minimize. subject to w T 1 = 1,
Math 30 Winter 05 Solution to Homework 3. Recognizing the convexity of g(x) := x log x, from Jensen s inequality we get d(x) n x + + x n n log x + + x n n where the equality is attained only at x = (/n,...,
More informationMath 10C - Fall Final Exam
Math 1C - Fall 217 - Final Exam Problem 1. Consider the function f(x, y) = 1 x 2 (y 1) 2. (i) Draw the level curve through the point P (1, 2). Find the gradient of f at the point P and draw the gradient
More informationIntroduction to Real Analysis Alternative Chapter 1
Christopher Heil Introduction to Real Analysis Alternative Chapter 1 A Primer on Norms and Banach Spaces Last Updated: March 10, 2018 c 2018 by Christopher Heil Chapter 1 A Primer on Norms and Banach Spaces
More informationSymmetric Matrices and Eigendecomposition
Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions
More informationPrague, II.2. Integrability (existence of the Riemann integral) sufficient conditions... 37
Mathematics II Prague, 1998 ontents Introduction.................................................................... 3 I. Functions of Several Real Variables (Stanislav Kračmar) II. I.1. Euclidean space
More informationMathematical Analysis Outline. William G. Faris
Mathematical Analysis Outline William G. Faris January 8, 2007 2 Chapter 1 Metric spaces and continuous maps 1.1 Metric spaces A metric space is a set X together with a real distance function (x, x ) d(x,
More informationMath 3191 Applied Linear Algebra
Math 191 Applied Linear Algebra Lecture 1: Inner Products, Length, Orthogonality Stephen Billups University of Colorado at Denver Math 191Applied Linear Algebra p.1/ Motivation Not all linear systems have
More informationMATH 304 Linear Algebra Lecture 18: Orthogonal projection (continued). Least squares problems. Normed vector spaces.
MATH 304 Linear Algebra Lecture 18: Orthogonal projection (continued). Least squares problems. Normed vector spaces. Orthogonality Definition 1. Vectors x,y R n are said to be orthogonal (denoted x y)
More informationCourse 212: Academic Year Section 1: Metric Spaces
Course 212: Academic Year 1991-2 Section 1: Metric Spaces D. R. Wilkins Contents 1 Metric Spaces 3 1.1 Distance Functions and Metric Spaces............. 3 1.2 Convergence and Continuity in Metric Spaces.........
More informationTopological properties
CHAPTER 4 Topological properties 1. Connectedness Definitions and examples Basic properties Connected components Connected versus path connected, again 2. Compactness Definition and first examples Topological
More informationConvex Optimization Theory. Chapter 5 Exercises and Solutions: Extended Version
Convex Optimization Theory Chapter 5 Exercises and Solutions: Extended Version Dimitri P. Bertsekas Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationFunctional Analysis Exercise Class
Functional Analysis Exercise Class Week: December 4 8 Deadline to hand in the homework: your exercise class on week January 5. Exercises with solutions ) Let H, K be Hilbert spaces, and A : H K be a linear
More informationInner Product Spaces 6.1 Length and Dot Product in R n
Inner Product Spaces 6.1 Length and Dot Product in R n Summer 2017 Goals We imitate the concept of length and angle between two vectors in R 2, R 3 to define the same in the n space R n. Main topics are:
More informationProjection Theorem 1
Projection Theorem 1 Cauchy-Schwarz Inequality Lemma. (Cauchy-Schwarz Inequality) For all x, y in an inner product space, [ xy, ] x y. Equality holds if and only if x y or y θ. Proof. If y θ, the inequality
More information