MATH529 Fundamentals of Optimization Unconstrained Optimization II
|
|
- Valerie Doyle
- 5 years ago
- Views:
Transcription
1 MATH529 Fundamentals of Optimization Unconstrained Optimization II Marco A. Montes de Oca Mathematical Sciences, University of Delaware, USA 1 / 31
2 Recap 2 / 31
3 Example Find the local and global minimizers and maximizers on R of f (x) = 3x 4 4x / 31
4 Graph of f (x) = 3x 4 4x / 31
5 Two theorems summarize the basic facts about global optimization of one variable functions. Theorem (1st order condition (Necessary, but not sufficient)) Suppose that f (x) is a differentiable function on R (or the function s domain I ). If x is a global minimizer of f (x), then f (x ) = 0. 5 / 31
6 Two theorems summarize the basic facts about global optimization of one variable functions. Theorem (1st order condition (Necessary, but not sufficient)) Suppose that f (x) is a differentiable function on R (or the function s domain I ). If x is a global minimizer of f (x), then f (x ) = 0. Theorem (2nd order condition (Sufficient, but not necessary)) Suppose that f (x), f (x), and f (x) are all continuous on R (or I ) and that x is a critical point of f (x). a) If f (x) 0 for all x R (or I ), then x is a global minimizer of f (x) on R (or I ). b) If f (x) > 0 for all x I such that x x, then x is a strict global minimizer of f (x) on R (or I ). 6 / 31
7 Local optimization is easier to verify. Theorem (1st order condition (Necessary, but not sufficient)) Suppose that f (x) is a differentiable function on R (or I ). If x is a local minimizer of f (x), then f (x ) = 0. 7 / 31
8 Local optimization is easier to verify. Theorem (1st order condition (Necessary, but not sufficient)) Suppose that f (x) is a differentiable function on R (or I ). If x is a local minimizer of f (x), then f (x ) = 0. Theorem (2nd order condition (Sufficient, but not necessary)) Suppose that f (x), f (x), and f (x) are all continuous on R (or I ) and that x is a critical point of f (x). If f (x ) > 0, then x is a strict local minimizer of f (x). 8 / 31
9 Exercise Find the local and global minimizers and maximizers on I = ( 1, 1) of f (x) = ln(1 x 2 ). 9 / 31
10 What about functions of many variables? 10 / 31
11 What about functions of many variables? Extend theorems that allow us to identify and classifly local minimizers of one variable functions to multivariable cases. 11 / 31
12 Notation: A vector in R n is an ordered n-tuple x = x 1 x 2 x 3. x n of real numbers called components of x. If x and y are vectors in R n, then their dot product or inner product is defined by y 2 x y = x T y = (x 1, x 2, x 3,..., x n ) y 3 =. y n y 1 n x i y i i=1 where x T is the transpose of x. 12 / 31
13 Notation: If f (x) is a function of n variables with continuous first and second partial derivatives on R n, then the gradient of f (x) is the vector f x 1 f f x n x 2 f f = x / 31
14 Notation: The Hessian of f (x), denoted by 2 f or Hf, is the symmetric n n matrix 2 f = Hf = x 2 1 x 2 x 1 x 3 x 1. x n x 1 x 1 x 2 x 2 2 x 3 x 2. x n x 2 x 1 x f x 1 x n x 2 x 3... x 2 3. x 2 x n... x 3 x n.... x n x f xn 2 14 / 31
15 Definition Suppose f (x) is a real-valued function defined on a subset D of R n. A point x in D is: A global minimizer for f (x) on D if f (x ) f (x) for all x D; 15 / 31
16 Definition Suppose f (x) is a real-valued function defined on a subset D of R n. A point x in D is: A global minimizer for f (x) on D if f (x ) f (x) for all x D; A strict global minimizer for f (x) on D if f (x ) < f (x) for all x D such that x x ; 16 / 31
17 Definition Suppose f (x) is a real-valued function defined on a subset D of R n. A point x in D is: A global minimizer for f (x) on D if f (x ) f (x) for all x D; A strict global minimizer for f (x) on D if f (x ) < f (x) for all x D such that x x ; A local minimizer for f (x) if there is a positive number δ such that f (x ) f (x) for all x D for which x in B(x, δ); 17 / 31
18 Definition Suppose f (x) is a real-valued function defined on a subset D of R n. A point x in D is: A global minimizer for f (x) on D if f (x ) f (x) for all x D; A strict global minimizer for f (x) on D if f (x ) < f (x) for all x D such that x x ; A local minimizer for f (x) if there is a positive number δ such that f (x ) f (x) for all x D for which x in B(x, δ); A strict local minimizer for f (x) if there is a positive number δ such that f (x ) < f (x) for all x D for which x in B(x, δ) and x x ; 18 / 31
19 Definition Suppose f (x) is a real-valued function defined on a subset D of R n. A point x in D is: A global minimizer for f (x) on D if f (x ) f (x) for all x D; A strict global minimizer for f (x) on D if f (x ) < f (x) for all x D such that x x ; A local minimizer for f (x) if there is a positive number δ such that f (x ) f (x) for all x D for which x in B(x, δ); A strict local minimizer for f (x) if there is a positive number δ such that f (x ) < f (x) for all x D for which x in B(x, δ) and x x ; A critical point (also called a stationary point) of f (x) if the first partial derivatives of f (x) exist at x and f x i = 0, for i = 1, 2, 3..., n. 19 / 31
20 Theorem (Multivariable Taylor s formula) Suppose that x, x are points in R n and that f (x) is a real-valued function of n variables with continuous first and second partial derivatives on some open set containing the line segment [x, x] = {w R n : w = x + t(x x ), 0 t 1} joining x and x. Then, there exists a z [x, x] such that f (x) = f (x ) + f (x ) T (x x ) (x x ) T Hf (z)(x x ) 20 / 31
21 Theorem (Local minimizer identification) Suppose that f (x) is a real-valued function for which all first partial derivatives of f (x) exist on a subset D R n. If x is an interior point of D that is a local minimizer of f (x), then f (x ) = / 31
22 Theorem (Classification of minimizers (maximizers)) Suppose that x is a critical point of a function f (x) with continuous first and second partial derivatives on R n. Then: x is a global minimizer of f (x) if (x x ) T Hf (z)(x x ) 0 for all x R n and all z [x, x]; x is a strict global minimizer of f (x) if (x x ) T Hf (z)(x x ) > 0 for all x R n such that x x and for all z [x, x]; x is a global maximizer of f (x) if (x x ) T Hf (z)(x x ) 0 for all x R n and all z [x, x]; x is a strict global maximizer of f (x) if (x x ) T Hf (z)(x x ) < 0 for all x R n such that x x and for all z [x, x]; 22 / 31
23 Practical ways to use the previous theorem: Conditions that involve the form (x x ) T Hf (z)(x x ), or in general v T Av, where A is a symmetric square matrix, call for methods to identify whether A (in our case the Hessian of the objective function) is positive or negative (semi)definite. 23 / 31
24 Quadratic forms: Let a 11 a a 1n a 21 a a 2n A = a n1 a n2... a nn The quadratic form Q A (x) = x T Ax = a 11 x a 12x 1 x 2 + a 13 x 1 x a ij x i x j a ii x 2 i a nn x 2 n. 24 / 31
25 Example Write the quadratic form associated with the following matrix: A = 0 2 1/ / / 31
26 Determining whether a quadratic form Q A (x) > 0 for all x R n. Example in class / 31
Functions of Several Variables
Jim Lambers MAT 419/519 Summer Session 2011-12 Lecture 2 Notes These notes correspond to Section 1.2 in the text. Functions of Several Variables We now generalize the results from the previous section,
More informationMAT 419 Lecture Notes Transcribed by Eowyn Cenek 6/1/2012
(Homework 1: Chapter 1: Exercises 1-7, 9, 11, 19, due Monday June 11th See also the course website for lectures, assignments, etc) Note: today s lecture is primarily about definitions Lots of definitions
More informationMath (P)refresher Lecture 8: Unconstrained Optimization
Math (P)refresher Lecture 8: Unconstrained Optimization September 2006 Today s Topics : Quadratic Forms Definiteness of Quadratic Forms Maxima and Minima in R n First Order Conditions Second Order Conditions
More informationChapter 2: Unconstrained Extrema
Chapter 2: Unconstrained Extrema Math 368 c Copyright 2012, 2013 R Clark Robinson May 22, 2013 Chapter 2: Unconstrained Extrema 1 Types of Sets Definition For p R n and r > 0, the open ball about p of
More informationMATH 5720: Unconstrained Optimization Hung Phan, UMass Lowell September 13, 2018
MATH 57: Unconstrained Optimization Hung Phan, UMass Lowell September 13, 18 1 Global and Local Optima Let a function f : S R be defined on a set S R n Definition 1 (minimizers and maximizers) (i) x S
More informationMATH 4211/6211 Optimization Basics of Optimization Problems
MATH 4211/6211 Optimization Basics of Optimization Problems Xiaojing Ye Department of Mathematics & Statistics Georgia State University Xiaojing Ye, Math & Stat, Georgia State University 0 A standard minimization
More informationHere each term has degree 2 (the sum of exponents is 2 for all summands). A quadratic form of three variables looks as
Reading [SB], Ch. 16.1-16.3, p. 375-393 1 Quadratic Forms A quadratic function f : R R has the form f(x) = a x. Generalization of this notion to two variables is the quadratic form Q(x 1, x ) = a 11 x
More informationChapter 13. Convex and Concave. Josef Leydold Mathematical Methods WS 2018/19 13 Convex and Concave 1 / 44
Chapter 13 Convex and Concave Josef Leydold Mathematical Methods WS 2018/19 13 Convex and Concave 1 / 44 Monotone Function Function f is called monotonically increasing, if x 1 x 2 f (x 1 ) f (x 2 ) It
More informationThe general programming problem is the nonlinear programming problem where a given function is maximized subject to a set of inequality constraints.
1 Optimization Mathematical programming refers to the basic mathematical problem of finding a maximum to a function, f, subject to some constraints. 1 In other words, the objective is to find a point,
More informationMathematical Economics: Lecture 16
Mathematical Economics: Lecture 16 Yu Ren WISE, Xiamen University November 26, 2012 Outline 1 Chapter 21: Concave and Quasiconcave Functions New Section Chapter 21: Concave and Quasiconcave Functions Concave
More informationSIMPLE MULTIVARIATE OPTIMIZATION
SIMPLE MULTIVARIATE OPTIMIZATION 1. DEFINITION OF LOCAL MAXIMA AND LOCAL MINIMA 1.1. Functions of variables. Let f(, x ) be defined on a region D in R containing the point (a, b). Then a: f(a, b) is a
More informationThe Derivative. Appendix B. B.1 The Derivative of f. Mappings from IR to IR
Appendix B The Derivative B.1 The Derivative of f In this chapter, we give a short summary of the derivative. Specifically, we want to compare/contrast how the derivative appears for functions whose domain
More informationMATHEMATICAL ECONOMICS: OPTIMIZATION. Contents
MATHEMATICAL ECONOMICS: OPTIMIZATION JOÃO LOPES DIAS Contents 1. Introduction 2 1.1. Preliminaries 2 1.2. Optimal points and values 2 1.3. The optimization problems 3 1.4. Existence of optimal points 4
More informationJUST THE MATHS UNIT NUMBER 1.6. ALGEBRA 6 (Formulae and algebraic equations) A.J.Hobson
JUST THE MATHS UNIT NUMBER 1.6 ALGEBRA 6 (Formulae and algebraic equations) by A.J.Hobson 1.6.1 Transposition of formulae 1.6. of linear equations 1.6.3 of quadratic equations 1.6. Exercises 1.6.5 Answers
More informationLecture Unconstrained optimization. In this lecture we will study the unconstrained problem. minimize f(x), (2.1)
Lecture 2 In this lecture we will study the unconstrained problem minimize f(x), (2.1) where x R n. Optimality conditions aim to identify properties that potential minimizers need to satisfy in relation
More information1 Overview. 2 A Characterization of Convex Functions. 2.1 First-order Taylor approximation. AM 221: Advanced Optimization Spring 2016
AM 221: Advanced Optimization Spring 2016 Prof. Yaron Singer Lecture 8 February 22nd 1 Overview In the previous lecture we saw characterizations of optimality in linear optimization, and we reviewed the
More informationMath 273a: Optimization Basic concepts
Math 273a: Optimization Basic concepts Instructor: Wotao Yin Department of Mathematics, UCLA Spring 2015 slides based on Chong-Zak, 4th Ed. Goals of this lecture The general form of optimization: minimize
More informationOR MSc Maths Revision Course
OR MSc Maths Revision Course Tom Byrne School of Mathematics University of Edinburgh t.m.byrne@sms.ed.ac.uk 15 September 2017 General Information Today JCMB Lecture Theatre A, 09:30-12:30 Mathematics revision
More informationFunctions of Several Variables
Functions of Several Variables The Unconstrained Minimization Problem where In n dimensions the unconstrained problem is stated as f() x variables. minimize f()x x, is a scalar objective function of vector
More informationChapter 11. Taylor Series. Josef Leydold Mathematical Methods WS 2018/19 11 Taylor Series 1 / 27
Chapter 11 Taylor Series Josef Leydold Mathematical Methods WS 2018/19 11 Taylor Series 1 / 27 First-Order Approximation We want to approximate function f by some simple function. Best possible approximation
More informationMATH529 Fundamentals of Optimization Constrained Optimization I
MATH529 Fundamentals of Optimization Constrained Optimization I Marco A. Montes de Oca Mathematical Sciences, University of Delaware, USA 1 / 26 Motivating Example 2 / 26 Motivating Example min cost(b)
More informationCHAPTER 4: HIGHER ORDER DERIVATIVES. Likewise, we may define the higher order derivatives. f(x, y, z) = xy 2 + e zx. y = 2xy.
April 15, 2009 CHAPTER 4: HIGHER ORDER DERIVATIVES In this chapter D denotes an open subset of R n. 1. Introduction Definition 1.1. Given a function f : D R we define the second partial derivatives as
More informationIntroduction to Unconstrained Optimization: Part 2
Introduction to Unconstrained Optimization: Part 2 James Allison ME 555 January 29, 2007 Overview Recap Recap selected concepts from last time (with examples) Use of quadratic functions Tests for positive
More informationMonotone Function. Function f is called monotonically increasing, if. x 1 x 2 f (x 1 ) f (x 2 ) x 1 < x 2 f (x 1 ) < f (x 2 ) x 1 x 2
Monotone Function Function f is called monotonically increasing, if Chapter 3 x x 2 f (x ) f (x 2 ) It is called strictly monotonically increasing, if f (x 2) f (x ) Convex and Concave x < x 2 f (x )
More informationWe have seen that for a function the partial derivatives whenever they exist, play an important role. This motivates the following definition.
\ Module 12 : Total differential, Tangent planes and normals Lecture 34 : Gradient of a scaler field [Section 34.1] Objectives In this section you will learn the following : The notions gradient vector
More informationComputational Optimization. Mathematical Programming Fundamentals 1/25 (revised)
Computational Optimization Mathematical Programming Fundamentals 1/5 (revised) If you don t know where you are going, you probably won t get there. -from some book I read in eight grade If you do get there,
More information1 Convexity, concavity and quasi-concavity. (SB )
UNIVERSITY OF MARYLAND ECON 600 Summer 2010 Lecture Two: Unconstrained Optimization 1 Convexity, concavity and quasi-concavity. (SB 21.1-21.3.) For any two points, x, y R n, we can trace out the line of
More informationPaul Schrimpf. October 18, UBC Economics 526. Unconstrained optimization. Paul Schrimpf. Notation and definitions. First order conditions
Unconstrained UBC Economics 526 October 18, 2013 .1.2.3.4.5 Section 1 Unconstrained problem x U R n F : U R. max F (x) x U Definition F = max x U F (x) is the maximum of F on U if F (x) F for all x U and
More informationg(t) = f(x 1 (t),..., x n (t)).
Reading: [Simon] p. 313-333, 833-836. 0.1 The Chain Rule Partial derivatives describe how a function changes in directions parallel to the coordinate axes. Now we shall demonstrate how the partial derivatives
More informationCourse Summary Math 211
Course Summary Math 211 table of contents I. Functions of several variables. II. R n. III. Derivatives. IV. Taylor s Theorem. V. Differential Geometry. VI. Applications. 1. Best affine approximations.
More informationPreliminary draft only: please check for final version
ARE211, Fall2012 CALCULUS4: THU, OCT 11, 2012 PRINTED: AUGUST 22, 2012 (LEC# 15) Contents 3. Univariate and Multivariate Differentiation (cont) 1 3.6. Taylor s Theorem (cont) 2 3.7. Applying Taylor theory:
More informationOptimization. Escuela de Ingeniería Informática de Oviedo. (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30
Optimization Escuela de Ingeniería Informática de Oviedo (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30 Unconstrained optimization Outline 1 Unconstrained optimization 2 Constrained
More informationChapter 7. Extremal Problems. 7.1 Extrema and Local Extrema
Chapter 7 Extremal Problems No matter in theoretical context or in applications many problems can be formulated as problems of finding the maximum or minimum of a function. Whenever this is the case, advanced
More informationSolution Methods. Richard Lusby. Department of Management Engineering Technical University of Denmark
Solution Methods Richard Lusby Department of Management Engineering Technical University of Denmark Lecture Overview (jg Unconstrained Several Variables Quadratic Programming Separable Programming SUMT
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More information3 Applications of partial differentiation
Advanced Calculus Chapter 3 Applications of partial differentiation 37 3 Applications of partial differentiation 3.1 Stationary points Higher derivatives Let U R 2 and f : U R. The partial derivatives
More information1 Introduction to Optimization
Unconstrained Convex Optimization 2 1 Introduction to Optimization Given a general optimization problem of te form min x f(x) (1.1) were f : R n R. Sometimes te problem as constraints (we are only interested
More informationEC /11. Math for Microeconomics September Course, Part II Problem Set 1 with Solutions. a11 a 12. x 2
LONDON SCHOOL OF ECONOMICS Professor Leonardo Felli Department of Economics S.478; x7525 EC400 2010/11 Math for Microeconomics September Course, Part II Problem Set 1 with Solutions 1. Show that the general
More informationLec3p1, ORF363/COS323
Lec3 Page 1 Lec3p1, ORF363/COS323 This lecture: Optimization problems - basic notation and terminology Unconstrained optimization The Fermat-Weber problem Least squares First and second order necessary
More informationREVIEW OF DIFFERENTIAL CALCULUS
REVIEW OF DIFFERENTIAL CALCULUS DONU ARAPURA 1. Limits and continuity To simplify the statements, we will often stick to two variables, but everything holds with any number of variables. Let f(x, y) be
More informationLecture 3: Basics of set-constrained and unconstrained optimization
Lecture 3: Basics of set-constrained and unconstrained optimization (Chap 6 from textbook) Xiaoqun Zhang Shanghai Jiao Tong University Last updated: October 9, 2018 Optimization basics Outline Optimization
More information8.7 Taylor s Inequality Math 2300 Section 005 Calculus II. f(x) = ln(1 + x) f(0) = 0
8.7 Taylor s Inequality Math 00 Section 005 Calculus II Name: ANSWER KEY Taylor s Inequality: If f (n+) is continuous and f (n+) < M between the center a and some point x, then f(x) T n (x) M x a n+ (n
More informationWeek 4: Calculus and Optimization (Jehle and Reny, Chapter A2)
Week 4: Calculus and Optimization (Jehle and Reny, Chapter A2) Tsun-Feng Chiang *School of Economics, Henan University, Kaifeng, China September 27, 2015 Microeconomic Theory Week 4: Calculus and Optimization
More informationMA102: Multivariable Calculus
MA102: Multivariable Calculus Rupam Barman and Shreemayee Bora Department of Mathematics IIT Guwahati Differentiability of f : U R n R m Definition: Let U R n be open. Then f : U R n R m is differentiable
More informationISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints
ISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints Instructor: Prof. Kevin Ross Scribe: Nitish John October 18, 2011 1 The Basic Goal The main idea is to transform a given constrained
More information12. Cholesky factorization
L. Vandenberghe ECE133A (Winter 2018) 12. Cholesky factorization positive definite matrices examples Cholesky factorization complex positive definite matrices kernel methods 12-1 Definitions a symmetric
More information1 Directional Derivatives and Differentiability
Wednesday, January 18, 2012 1 Directional Derivatives and Differentiability Let E R N, let f : E R and let x 0 E. Given a direction v R N, let L be the line through x 0 in the direction v, that is, L :=
More informationUnconstrained Optimization
1 / 36 Unconstrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University February 2, 2015 2 / 36 3 / 36 4 / 36 5 / 36 1. preliminaries 1.1 local approximation
More informationNumerical Optimization
Unconstrained Optimization Computer Science and Automation Indian Institute of Science Bangalore 560 01, India. NPTEL Course on Unconstrained Minimization Let f : R n R. Consider the optimization problem:
More informationOPER 627: Nonlinear Optimization Lecture 2: Math Background and Optimality Conditions
OPER 627: Nonlinear Optimization Lecture 2: Math Background and Optimality Conditions Department of Statistical Sciences and Operations Research Virginia Commonwealth University Aug 28, 2013 (Lecture 2)
More informationAnalysis/Calculus Review Day 3
Analysis/Calculus Review Day 3 Arvind Saibaba arvindks@stanford.edu Institute of Computational and Mathematical Engineering Stanford University September 15, 2010 Big- Oh and Little- Oh Notation We write
More informationLinear Models Review
Linear Models Review Vectors in IR n will be written as ordered n-tuples which are understood to be column vectors, or n 1 matrices. A vector variable will be indicted with bold face, and the prime sign
More informationOptimization and Calculus
Optimization and Calculus To begin, there is a close relationship between finding the roots to a function and optimizing a function. In the former case, we solve for x. In the latter, we solve: g(x) =
More informationLines, parabolas, distances and inequalities an enrichment class
Lines, parabolas, distances and inequalities an enrichment class Finbarr Holland 1. Lines in the plane A line is a particular kind of subset of the plane R 2 = R R, and can be described as the set of ordered
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationNONLINEAR. (Hillier & Lieberman Introduction to Operations Research, 8 th edition)
NONLINEAR PROGRAMMING (Hillier & Lieberman Introduction to Operations Research, 8 th edition) Nonlinear Programming g Linear programming has a fundamental role in OR. In linear programming all its functions
More informationLecture 2 - Unconstrained Optimization Definition[Global Minimum and Maximum]Let f : S R be defined on a set S R n. Then
Lecture 2 - Unconstrained Optimization Definition[Global Minimum and Maximum]Let f : S R be defined on a set S R n. Then 1. x S is a global minimum point of f over S if f (x) f (x ) for any x S. 2. x S
More informationReal Symmetric Matrices and Semidefinite Programming
Real Symmetric Matrices and Semidefinite Programming Tatsiana Maskalevich Abstract Symmetric real matrices attain an important property stating that all their eigenvalues are real. This gives rise to many
More informationThe coordinates of the vertex of the corresponding parabola are p, q. If a > 0, the parabola opens upward. If a < 0, the parabola opens downward.
Mathematics 10 Page 1 of 8 Quadratic Relations in Vertex Form The expression y ax p q defines a quadratic relation in form. The coordinates of the of the corresponding parabola are p, q. If a > 0, the
More informationSupport Vector Machine (SVM) & Kernel CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2012
Support Vector Machine (SVM) & Kernel CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Linear classifier Which classifier? x 2 x 1 2 Linear classifier Margin concept x 2
More informationScientific Computing: Optimization
Scientific Computing: Optimization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 March 8th, 2011 A. Donev (Courant Institute) Lecture
More informationNote: Every graph is a level set (why?). But not every level set is a graph. Graphs must pass the vertical line test. (Level sets may or may not.
Curves in R : Graphs vs Level Sets Graphs (y = f(x)): The graph of f : R R is {(x, y) R y = f(x)} Example: When we say the curve y = x, we really mean: The graph of the function f(x) = x That is, we mean
More informationTutorials in Optimization. Richard Socher
Tutorials in Optimization Richard Socher July 20, 2008 CONTENTS 1 Contents 1 Linear Algebra: Bilinear Form - A Simple Optimization Problem 2 1.1 Definitions........................................ 2 1.2
More informationPerformance Surfaces and Optimum Points
CSC 302 1.5 Neural Networks Performance Surfaces and Optimum Points 1 Entrance Performance learning is another important class of learning law. Network parameters are adjusted to optimize the performance
More informationAppendix A Taylor Approximations and Definite Matrices
Appendix A Taylor Approximations and Definite Matrices Taylor approximations provide an easy way to approximate a function as a polynomial, using the derivatives of the function. We know, from elementary
More informationPreliminaries Lectures. Dr. Abdulla Eid. Department of Mathematics MATHS 101: Calculus I
Preliminaries 2 1 2 Lectures Department of Mathematics http://www.abdullaeid.net/maths101 MATHS 101: Calculus I (University of Bahrain) Prelim 1 / 35 Pre Calculus MATHS 101: Calculus MATHS 101 is all about
More information(1) Recap of Differential Calculus and Integral Calculus (2) Preview of Calculus in three dimensional space (3) Tools for Calculus 3
Math 127 Introduction and Review (1) Recap of Differential Calculus and Integral Calculus (2) Preview of Calculus in three dimensional space (3) Tools for Calculus 3 MATH 127 Introduction to Calculus III
More informationCS100: DISCRETE STRUCTURES. Lecture 3 Matrices Ch 3 Pages:
CS100: DISCRETE STRUCTURES Lecture 3 Matrices Ch 3 Pages: 246-262 Matrices 2 Introduction DEFINITION 1: A matrix is a rectangular array of numbers. A matrix with m rows and n columns is called an m x n
More informationMath 10C - Fall Final Exam
Math 1C - Fall 217 - Final Exam Problem 1. Consider the function f(x, y) = 1 x 2 (y 1) 2. (i) Draw the level curve through the point P (1, 2). Find the gradient of f at the point P and draw the gradient
More informationSolutionbank Edexcel AS and A Level Modular Mathematics
Page of Exercise A, Question The curve C, with equation y = x ln x, x > 0, has a stationary point P. Find, in terms of e, the coordinates of P. (7) y = x ln x, x > 0 Differentiate as a product: = x + x
More informationSECTION A. f(x) = ln(x). Sketch the graph of y = f(x), indicating the coordinates of any points where the graph crosses the axes.
SECTION A 1. State the maximal domain and range of the function f(x) = ln(x). Sketch the graph of y = f(x), indicating the coordinates of any points where the graph crosses the axes. 2. By evaluating f(0),
More informationLinear algebra and applications to graphs Part 1
Linear algebra and applications to graphs Part 1 Written up by Mikhail Belkin and Moon Duchin Instructor: Laszlo Babai June 17, 2001 1 Basic Linear Algebra Exercise 1.1 Let V and W be linear subspaces
More informationMATH Max-min Theory Fall 2016
MATH 20550 Max-min Theory Fall 2016 1. Definitions and main theorems Max-min theory starts with a function f of a vector variable x and a subset D of the domain of f. So far when we have worked with functions
More informationAM 205: lecture 18. Last time: optimization methods Today: conditions for optimality
AM 205: lecture 18 Last time: optimization methods Today: conditions for optimality Existence of Global Minimum For example: f (x, y) = x 2 + y 2 is coercive on R 2 (global min. at (0, 0)) f (x) = x 3
More informationEC /11. Math for Microeconomics September Course, Part II Lecture Notes. Course Outline
LONDON SCHOOL OF ECONOMICS Professor Leonardo Felli Department of Economics S.478; x7525 EC400 20010/11 Math for Microeconomics September Course, Part II Lecture Notes Course Outline Lecture 1: Tools for
More informationQuadratic Programming
Quadratic Programming Outline Linearly constrained minimization Linear equality constraints Linear inequality constraints Quadratic objective function 2 SideBar: Matrix Spaces Four fundamental subspaces
More informationIn view of (31), the second of these is equal to the identity I on E m, while this, in view of (30), implies that the first can be written
11.8 Inequality Constraints 341 Because by assumption x is a regular point and L x is positive definite on M, it follows that this matrix is nonsingular (see Exercise 11). Thus, by the Implicit Function
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More information9.5. Polynomial and Rational Inequalities. Objectives. Solve quadratic inequalities. Solve polynomial inequalities of degree 3 or greater.
Chapter 9 Section 5 9.5 Polynomial and Rational Inequalities Objectives 1 3 Solve quadratic inequalities. Solve polynomial inequalities of degree 3 or greater. Solve rational inequalities. Objective 1
More informationRecitation 1. Gradients and Directional Derivatives. Brett Bernstein. CDS at NYU. January 21, 2018
Gradients and Directional Derivatives Brett Bernstein CDS at NYU January 21, 2018 Brett Bernstein (CDS at NYU) Recitation 1 January 21, 2018 1 / 23 Initial Question Intro Question Question We are given
More informationCalculus 2502A - Advanced Calculus I Fall : Local minima and maxima
Calculus 50A - Advanced Calculus I Fall 014 14.7: Local minima and maxima Martin Frankland November 17, 014 In these notes, we discuss the problem of finding the local minima and maxima of a function.
More informationOptimization Tutorial 1. Basic Gradient Descent
E0 270 Machine Learning Jan 16, 2015 Optimization Tutorial 1 Basic Gradient Descent Lecture by Harikrishna Narasimhan Note: This tutorial shall assume background in elementary calculus and linear algebra.
More informationCalculus and optimization
Calculus an optimization These notes essentially correspon to mathematical appenix 2 in the text. 1 Functions of a single variable Now that we have e ne functions we turn our attention to calculus. A function
More informationE 600 Chapter 3: Multivariate Calculus
E 600 Chapter 3: Multivariate Calculus Simona Helmsmueller August 21, 2017 Goals of this lecture: Know when an inverse to a function exists, be able to graphically and analytically determine whether a
More informationTable of mathematical symbols - Wikipedia, the free encyclopedia
Página 1 de 13 Table of mathematical symbols From Wikipedia, the free encyclopedia For the HTML codes of mathematical symbols see mathematical HTML. Note: This article contains special characters. The
More informationMAT 473 Intermediate Real Analysis II
MAT 473 Intermediate Real Analysis II John Quigg Spring 2009 revised February 5, 2009 Derivatives Here our purpose is to give a rigorous foundation of the principles of differentiation in R n. Much of
More informationMath 443 Differential Geometry Spring Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook.
Math 443 Differential Geometry Spring 2013 Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook. Endomorphisms of a Vector Space This handout discusses
More informationARE211, Fall2015. Contents. 2. Univariate and Multivariate Differentiation (cont) Taylor s Theorem (cont) 2
ARE211, Fall2015 CALCULUS4: THU, SEP 17, 2015 PRINTED: SEPTEMBER 22, 2015 (LEC# 7) Contents 2. Univariate and Multivariate Differentiation (cont) 1 2.4. Taylor s Theorem (cont) 2 2.5. Applying Taylor theory:
More informationBoston College. Math Review Session (2nd part) Lecture Notes August,2007. Nadezhda Karamcheva www2.bc.
Boston College Math Review Session (2nd part) Lecture Notes August,2007 Nadezhda Karamcheva karamche@bc.edu www2.bc.edu/ karamche 1 Contents 1 Quadratic Forms and Definite Matrices 3 1.1 Quadratic Forms.........................
More information6 Optimization. The interior of a set S R n is the set. int X = {x 2 S : 9 an open box B such that x 2 B S}
6 Optimization The interior of a set S R n is the set int X = {x 2 S : 9 an open box B such that x 2 B S} Similarly, the boundary of S, denoted@s, istheset @S := {x 2 R n :everyopenboxb s.t. x 2 B contains
More informationQuadratic Formula: - another method for solving quadratic equations (ax 2 + bx + c = 0)
In the previous lesson we showed how to solve quadratic equations that were not factorable and were not perfect squares by making perfect square trinomials using a process called completing the square.
More informationDifferentiable Functions
Differentiable Functions Let S R n be open and let f : R n R. We recall that, for x o = (x o 1, x o,, x o n S the partial derivative of f at the point x o with respect to the component x j is defined as
More informationStructural and Multidisciplinary Optimization. P. Duysinx and P. Tossings
Structural and Multidisciplinary Optimization P. Duysinx and P. Tossings 2018-2019 CONTACTS Pierre Duysinx Institut de Mécanique et du Génie Civil (B52/3) Phone number: 04/366.91.94 Email: P.Duysinx@uliege.be
More informationA A x i x j i j (i, j) (j, i) Let. Compute the value of for and
7.2 - Quadratic Forms quadratic form on is a function defined on whose value at a vector in can be computed by an expression of the form, where is an symmetric matrix. The matrix R n Q R n x R n Q(x) =
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
More informationChapter 8 ~ Quadratic Functions and Equations In this chapter you will study... You can use these skills...
Chapter 8 ~ Quadratic Functions and Equations In this chapter you will study... identifying and graphing quadratic functions transforming quadratic equations solving quadratic equations using factoring
More informationNonlinear Optimization
Nonlinear Optimization (Com S 477/577 Notes) Yan-Bin Jia Nov 7, 2017 1 Introduction Given a single function f that depends on one or more independent variable, we want to find the values of those variables
More informationECON 5111 Mathematical Economics
Test 1 October 1, 2010 1. Construct a truth table for the following statement: [p (p q)] q. 2. A prime number is a natural number that is divisible by 1 and itself only. Let P be the set of all prime numbers
More informationAPPENDIX : PARTIAL FRACTIONS
APPENDIX : PARTIAL FRACTIONS Appendix : Partial Fractions Given the expression x 2 and asked to find its integral, x + you can use work from Section. to give x 2 =ln( x 2) ln( x + )+c x + = ln k x 2 x+
More informationExtrema of Functions of Several Variables
Extrema of Functions of Several Variables MATH 311, Calculus III J. Robert Buchanan Department of Mathematics Fall 2011 Background (1 of 3) In single-variable calculus there are three important results
More information