446 CHAP. 8 NUMERICAL OPTIMIZATION. Newton's Search for a Minimum of f(x,y) Newton s Method
|
|
- Doreen Spencer
- 5 years ago
- Views:
Transcription
1 446 CHAP. 8 NUMERICAL OPTIMIZATION Newton's Search for a Minimum of f(xy) Newton s Method The quadratic approximation method of Section 8.1 generated a sequence of seconddegree Lagrange polynomials. It was implicitly assumed that near the minimum the shape of the quadratics approximated the shape of the objective function y = f (x). The resulting sequence of minimums of the quadratics produced a sequence converging
2 SEC. 8.3 GRADIENT AND NEWTON S METHODS 447 Table 8.7 Gradient Method for f (x y) = (x y)/(x 2 + y 2 + 2) k x k y k f (x k y k ) y P 1 P 2 P 3 P( 11) P x Figure 8.13 The countour graph of f (x y) = (x y)/(x 2 + y 2 + 2) and the gradient method. to the minimum of the objective function f. Newton s method extends this process to functions of N independent variables: z = f (x 1 x 2...x N ). Starting at an initial point P 0 a sequence of second-degree polynomials in N variables will be constructed recursively. If the objective function is well-behaved and the initial point is near the actual minimum then the sequence of minimums of the quadratics will converge to the minimum of the objective function. The process will use both the first- and second-order partial derivatives of the objective function. Recall that the gradient method used only the first partial derivatives. It is to be expected that Newton s method will be more efficient than the gradient method. Definition 8.5. Let z = f (X) be a function of X such that 2 f (X) x i x j exists for i j =
3 448 CHAP. 8 NUMERICAL OPTIMIZATION N. The Hessian matrix for f at X denoted by H f (X) is the N N matrix 2 f (X) (2) H f (X) = x i x j where i j = N. N N It is appropriate to think of the Hessian matrix of a function f as representing the second derivative of the function (precisely the case when N = 1). It is not difficult to show that the Hessian matrix of a function f equals the Jacobian matrix (see Section 3.7) of the gradient of f ; (3) H f (X) = J f (X). Example Find the Hessian matrix at the point ( 3 2) of the function f (x y) = (x y)/(x 2 + y 2 + 2). From Example 8.8 f x (x y) = x2 + 2xy + y (x 2 + y 2 + 2) 2 and f y (x y) = x2 2xy + y 2 2 (x 2 + y 2 + 2) 2. The second partials are f xx (x y) = 2(x3 3x 2 y 3x(y 2 + 2) + y(y 2 + 2)) (x 2 + y 2 + 2) 3 Evaluating the Hessian matrix at (x y) = ( 3 2) yields f xy (x y) = 2(x3 + 3x 2 y + x(2 3y 2 ) y(y 2 + 2)) (x 2 + y 2 + 2) 3 f yx (x y) = 2(x3 + 3x 2 y + x(2 3y 2 ) y(y 2 + 2)) (x 2 + y 2 + 2) 3 f yy (x y) = 2(2x + x3 6y 3x 2 y 3xy 2 + y 3 ) (x 2 + y 2 + 2) 3. H f (x y) = H f ( 3 2) = [ fxx (x y) f xy (x y) f yx (x y) f yy (x y) [ ] ]. Definition 8.6. The Taylor polynomial of degree two for f (X) centered at A is (4) Q(X) = f (A) + f (A) (X A) (X A)H f (A)(X A). Mathematical descriptions of Taylor polynomials of degree m can be found in most vector or advanced calculus textbooks.
4 SEC. 8.3 GRADIENT AND NEWTON S METHODS 449 Example Calculate the second-degree Taylor polynomial of f (x y) = (x y)/(x 2 + y 2 + 2) centered at the point A = ( 3 2). Treat the gradient of f as a 1 2 matrix. From Examples 8.8 and 8.10 respectively. Thus f ( 3 2) =[f x ( 3 2) f y ( 3 2)] = H f ( 3 2) = [ ] 225 Q(x y) = [ 9 19 ] [ x + 3 y + 2 ] ( ) [ x + 3 y + 2 ] [ x + 3 y + 2 ] = 69x2 61y x 763y 78xy Without ambiguity the matrix notation is dropped from the resultant 1 1 matrix. Assume that the first and second partial derivatives of z = f (x 1 x 2...x N ) exist and are continuous in a region containing the point P 0 and that there is a minimum at the point P. Substituting P 0 for A in formula (4) yields (5) Q(X) = f (P 0 ) + f (P 0 ) (X P 0 ) (X P 0)H f (P 0 )(X P 0 ) a second-degree polynomial in N variables; where X =[x 1 x 2 x N ]. A minimum of Q(X) occurs where (6) Q(X) = 0 or (7) f (P 0 ) + (X P 0 )(H f (P 0 )) = 0. If P 0 is close to the point P (where a minimum of f occurs) then H f (P 0 ) is invertible and equation (7) can be solved for X: (8) X = P 0 f (P 0 )((H f (P 0 )) 1 ). Substituting P 1 for X in formula (8) yields (9) P 1 = P 0 f (P 0 )((H f (P 0 )) 1 ).
5 450 CHAP. 8 NUMERICAL OPTIMIZATION When P k 1 is used in place of P 0 in formula (9) the following general rule is established: (10) P k = P k 1 f (P k 1 )((H f (P k 1 )) 1 ). In equation (7) the inverse of the Hessian matrix was used to solve for X. It would be better to solve the system of linear equations represented by equation (7) with one of the methods from Chapter 3. In general the methods in Chapter 3 are more reliable and efficient. The reader should realize that the inverse is primarily a theoretical tool and the computation and use of inverses is inherently inefficient. Example Use formula (10) to find P 1 and P 2 for the function f (x y) = (x y)/(x 2 + y 2 + 2). Use the initial point P 0 =[ ]. If P 0 =[ ] then f (P 0 ) = [ ] H f (P 0 ) = (H f (P 0 )) = Substituting P 0 f (P 0 ) and (H f (P 0 )) 1 into formula (10) yields P 1 = [ ] [ ] = [ ]. If P 1 = [ ] then f (P 1 ) = [ ] H f (P 1 ) = (H f (P 1 )) = Substituting P 1 f (P 1 ) and (H f (P 1 )) 1 into formula (10) yields P 2 = [ ] [ ] [ = [ ]. The process appears to be converging to the point P =[ 11] where the minimum occurs for the function f.atthefifth interation P 5 =[ 11]. ]
6 SEC. 8.3 GRADIENT AND NEWTON S METHODS 451 It should be noted that formula (9) is equivalent (take the transpose of both sides) to formula (30) in the optional Section 3.7. Formula (10) is also equivalent to step (iv) in the outline of Newton s method in Section 3.7. Thus Program 3.7 (Newton-Raphson method) can be used to produce the sequence {P k } k=0 (without using inverse matrices) that converges to P. Newton s method requires a good initial point if there is to be convergence. This is similar to the situation for the Newton-Raphson method for approximating a root of f (x) = 0. Unlike earlier examples the initial point in Example 8.12 was not P 0 =[ 3 2]. In fact as the reader can easily verify Newton s method diverges for that particular initial point. Newton s method can be modified by treating the expression f (P k 1 )((H f (P k 1 )) 1 ) in formula (10) as a search direction. This is analogous to the use of the search direction S k in the gradient method. As with the gradient method a single parameter minimization (line search) is implemented in the search direction. In general this modified Newton s method will be more reliable than Newton s method. Outline of Modified Newton s Method Suppose that P k has been obtained. (i) Compute the search direction S k = f (P k 1 )((H f (P k 1 )) 1 ). (ii) Perform a single parameter minimization of (γ ) = f (P k +γ S k ) on the interval [0 b] where b is large. This will produce a value γ = h min where a local minimum for (γ ) occurs. The relation (h min ) = f (P k + h min S k ) shows that this is a minimum for f (X) along the search line X = P k + h min S k. (iii) Construct the next point P k+1 = P k + h min S k. (iv) Perform the termination test for minimization; that is are the function values f (P k ) and f (P k+1 ) sufficiently close and the distance P k+1 P k small enough? Repeat the process.
7 Numerical Methods Using Matlab 4 th Edition 2004 John H. Mathews and Kurtis K. Fink ISBN: Prentice-Hall Inc. Upper Saddle River New Jersey USA
p 1 p 0 (p 1, f(p 1 )) (p 0, f(p 0 )) The geometric construction of p 2 for the se- cant method.
80 CHAP. 2 SOLUTION OF NONLINEAR EQUATIONS f (x) = 0 y y = f(x) (p, 0) p 2 p 1 p 0 x (p 1, f(p 1 )) (p 0, f(p 0 )) The geometric construction of p 2 for the se- Figure 2.16 cant method. Secant Method The
More information5.1 Least-Squares Line
252 CHAP. 5 CURVE FITTING 5.1 Least-Squares Line In science and engineering it is often the case that an experiment produces a set of data points (x 1, y 1 ),...,(x N, y N ), where the abscissas {x k }
More information632 CHAP. 11 EIGENVALUES AND EIGENVECTORS. QR Method
632 CHAP 11 EIGENVALUES AND EIGENVECTORS QR Method Suppose that A is a real symmetric matrix In the preceding section we saw how Householder s method is used to construct a similar tridiagonal matrix The
More information9.6 Predictor-Corrector Methods
SEC. 9.6 PREDICTOR-CORRECTOR METHODS 505 Adams-Bashforth-Moulton Method 9.6 Predictor-Corrector Methods The methods of Euler, Heun, Taylor, and Runge-Kutta are called single-step methods because they use
More information434 CHAP. 8 NUMERICAL OPTIMIZATION. Powell's Method. Powell s Method
434 CHAP. 8 NUMERICAL OPTIMIZATION Powell's Method Powell s Method Let X be an initial guess at the location of the minimum of the function z = f (x, x 2,...,x N ). Assume that the partial derivatives
More informationSEC POWER METHOD Power Method
SEC..2 POWER METHOD 599.2 Power Method We now describe the power method for computing the dominant eigenpair. Its extension to the inverse power method is practical for finding any eigenvalue provided
More information9.2 Euler s Method. (1) t k = a + kh for k = 0, 1,..., M where h = b a. The value h is called the step size. We now proceed to solve approximately
464 CHAP. 9 SOLUTION OF DIFFERENTIAL EQUATIONS 9. Euler s Method The reader should be convinced that not all initial value problems can be solved explicitly, and often it is impossible to find a formula
More informationBernstein polynomials of degree N are defined by
SEC. 5.5 BÉZIER CURVES 309 5.5 Bézier Curves Pierre Bézier at Renault and Paul de Casteljau at Citroën independently developed the Bézier curve for CAD/CAM operations, in the 1970s. These parametrically
More informationCalculus 2502A - Advanced Calculus I Fall : Local minima and maxima
Calculus 50A - Advanced Calculus I Fall 014 14.7: Local minima and maxima Martin Frankland November 17, 014 In these notes, we discuss the problem of finding the local minima and maxima of a function.
More informationThe Tangent Parabola The AMATYC Review, Vol. 23, No. 1, Fall 2001, pp
The Tangent Parabola The AMATYC Review, Vol. 3, No., Fall, pp. 5-3. John H. Mathews California State University Fullerton Fullerton, CA 9834 By Russell W. Howell Westmont College Santa Barbara, CA 938
More information9.8 Boundary Value Problems. Another type of differential equation has the form. (1) x = f (t, x, x ) for a t b, with the boundary conditions
528 CHAP. 9 SOLUTION OF DIFFERENTIAL EQUATIONS Linear Shooting Method 9.8 Boundary Value Problems Another type of differential equation has the form (1) x = f (t, x, x ) for a t b, with the boundary conditions
More informationOR MSc Maths Revision Course
OR MSc Maths Revision Course Tom Byrne School of Mathematics University of Edinburgh t.m.byrne@sms.ed.ac.uk 15 September 2017 General Information Today JCMB Lecture Theatre A, 09:30-12:30 Mathematics revision
More informationREVIEW OF DIFFERENTIAL CALCULUS
REVIEW OF DIFFERENTIAL CALCULUS DONU ARAPURA 1. Limits and continuity To simplify the statements, we will often stick to two variables, but everything holds with any number of variables. Let f(x, y) be
More informationTwo Posts to Fill On School Board
Y Y 9 86 4 4 qz 86 x : ( ) z 7 854 Y x 4 z z x x 4 87 88 Y 5 x q x 8 Y 8 x x : 6 ; : 5 x ; 4 ( z ; ( ) ) x ; z 94 ; x 3 3 3 5 94 ; ; ; ; 3 x : 5 89 q ; ; x ; x ; ; x : ; ; ; ; ; ; 87 47% : () : / : 83
More informationMA102: Multivariable Calculus
MA102: Multivariable Calculus Rupam Barman and Shreemayee Bora Department of Mathematics IIT Guwahati Differentiability of f : U R n R m Definition: Let U R n be open. Then f : U R n R m is differentiable
More informationOptimization and Calculus
Optimization and Calculus To begin, there is a close relationship between finding the roots to a function and optimizing a function. In the former case, we solve for x. In the latter, we solve: g(x) =
More informationA DARK GREY P O N T, with a Switch Tail, and a small Star on the Forehead. Any
Y Y Y X X «/ YY Y Y ««Y x ) & \ & & } # Y \#$& / Y Y X» \\ / X X X x & Y Y X «q «z \x» = q Y # % \ & [ & Z \ & { + % ) / / «q zy» / & / / / & x x X / % % ) Y x X Y $ Z % Y Y x x } / % «] «] # z» & Y X»
More informationMTH4101 CALCULUS II REVISION NOTES. 1. COMPLEX NUMBERS (Thomas Appendix 7 + lecture notes) ax 2 + bx + c = 0. x = b ± b 2 4ac 2a. i = 1.
MTH4101 CALCULUS II REVISION NOTES 1. COMPLEX NUMBERS (Thomas Appendix 7 + lecture notes) 1.1 Introduction Types of numbers (natural, integers, rationals, reals) The need to solve quadratic equations:
More informationMath Maximum and Minimum Values, I
Math 213 - Maximum and Minimum Values, I Peter A. Perry University of Kentucky October 8, 218 Homework Re-read section 14.7, pp. 959 965; read carefully pp. 965 967 Begin homework on section 14.7, problems
More informationThe Derivative. Appendix B. B.1 The Derivative of f. Mappings from IR to IR
Appendix B The Derivative B.1 The Derivative of f In this chapter, we give a short summary of the derivative. Specifically, we want to compare/contrast how the derivative appears for functions whose domain
More information3.5 Quadratic Approximation and Convexity/Concavity
3.5 Quadratic Approximation and Convexity/Concavity 55 3.5 Quadratic Approximation and Convexity/Concavity Overview: Second derivatives are useful for understanding how the linear approximation varies
More informationFunctions of Several Variables
Functions of Several Variables Partial Derivatives Philippe B Laval KSU March 21, 2012 Philippe B Laval (KSU) Functions of Several Variables March 21, 2012 1 / 19 Introduction In this section we extend
More informationInvestigation of Tangent Polynomials with a Computer Algebra System The AMATYC Review, Vol. 14, No. 1, Fall 1992, pp
Investigation of Tangent Polynomials wit a Computer Algebra System Te AMATYC Review, Vol. 14, No. 1, Fall 199, pp. -7. Jon H. Matews California State University Fullerton Fullerton, CA 9834 By Russell
More informationMATHEMATICAL EXPOSITION
MATHEMATICAL EXPOSITION An Improved Newton's Method by John H. Mathews California State University, Fullerton, CA 92634 4(oif John earned his Ph.D. in mathematics from Michigan State University under the
More informationThe functions in all models depend on two variables: time t and spatial variable x, (x, y) or (x, y, z).
Review of Multi-variable calculus: The functions in all models depend on two variables: time t and spatial variable x, (x, y) or (x, y, z). The spatial variable represents the environment where the species
More informationNonlinear Equations and Continuous Optimization
Nonlinear Equations and Continuous Optimization Sanzheng Qiao Department of Computing and Software McMaster University March, 2014 Outline 1 Introduction 2 Bisection Method 3 Newton s Method 4 Systems
More informationSOLUTIONS FOR PROBLEMS 1-30
. Answer: 5 Evaluate x x + 9 for x SOLUTIONS FOR PROBLEMS - 0 When substituting x in x be sure to do the exponent before the multiplication by to get (). + 9 5 + When multiplying ( ) so that ( 7) ( ).
More informationPreliminaries Lectures. Dr. Abdulla Eid. Department of Mathematics MATHS 101: Calculus I
Preliminaries 2 1 2 Lectures Department of Mathematics http://www.abdullaeid.net/maths101 MATHS 101: Calculus I (University of Bahrain) Prelim 1 / 35 Pre Calculus MATHS 101: Calculus MATHS 101 is all about
More informationCALCULUS: Math 21C, Fall 2010 Final Exam: Solutions. 1. [25 pts] Do the following series converge or diverge? State clearly which test you use.
CALCULUS: Math 2C, Fall 200 Final Exam: Solutions. [25 pts] Do the following series converge or diverge? State clearly which test you use. (a) (d) n(n + ) ( ) cos n n= n= (e) (b) n= n= [ cos ( ) n n (c)
More informationMath Exam 2, October 14, 2008
Math 96 - Exam 2, October 4, 28 Name: Problem (5 points Find all solutions to the following system of linear equations, check your work: x + x 2 x 3 2x 2 2x 3 2 x x 2 + x 3 2 Solution Let s perform Gaussian
More informationSOLUTIONS to Exercises from Optimization
SOLUTIONS to Exercises from Optimization. Use the bisection method to find the root correct to 6 decimal places: 3x 3 + x 2 = x + 5 SOLUTION: For the root finding algorithm, we need to rewrite the equation
More informationNonlinear Optimization: What s important?
Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global
More information2x (x 2 + y 2 + 1) 2 2y. (x 2 + y 2 + 1) 4. 4xy. (1, 1)(x 1) + (1, 1)(y + 1) (1, 1)(x 1)(y + 1) 81 x y y + 7.
Homework 8 Solutions, November 007. (1 We calculate some derivatives: f x = f y = x (x + y + 1 y (x + y + 1 x = (x + y + 1 4x (x + y + 1 4 y = (x + y + 1 4y (x + y + 1 4 x y = 4xy (x + y + 1 4 Substituting
More information100 CHAPTER 4. SYSTEMS AND ADAPTIVE STEP SIZE METHODS APPENDIX
100 CHAPTER 4. SYSTEMS AND ADAPTIVE STEP SIZE METHODS APPENDIX.1 Norms If we have an approximate solution at a given point and we want to calculate the absolute error, then we simply take the magnitude
More informationINTERPOLATION. and y i = cos x i, i = 0, 1, 2 This gives us the three points. Now find a quadratic polynomial. p(x) = a 0 + a 1 x + a 2 x 2.
INTERPOLATION Interpolation is a process of finding a formula (often a polynomial) whose graph will pass through a given set of points (x, y). As an example, consider defining and x 0 = 0, x 1 = π/4, x
More information3 Applications of partial differentiation
Advanced Calculus Chapter 3 Applications of partial differentiation 37 3 Applications of partial differentiation 3.1 Stationary points Higher derivatives Let U R 2 and f : U R. The partial derivatives
More informationENGI Partial Differentiation Page y f x
ENGI 3424 4 Partial Differentiation Page 4-01 4. Partial Differentiation For functions of one variable, be found unambiguously by differentiation: y f x, the rate of change of the dependent variable can
More informationIntroduction to Unconstrained Optimization: Part 2
Introduction to Unconstrained Optimization: Part 2 James Allison ME 555 January 29, 2007 Overview Recap Recap selected concepts from last time (with examples) Use of quadratic functions Tests for positive
More informationCALCULUS GARRET J. ETGEN SALAS AND HILLE'S. ' MiIIIIIIH. I '////I! li II ii: ONE AND SEVERAL VARIABLES SEVENTH EDITION REVISED BY \
/ / / ' ' ' / / ' '' ' - -'/-' yy xy xy' y- y/ /: - y/ yy y /'}' / >' // yy,y-' 'y '/' /y , I '////I! li II ii: ' MiIIIIIIH IIIIII!l ii r-i: V /- A' /; // ;.1 " SALAS AND HILLE'S
More informationHW3 - Due 02/06. Each answer must be mathematically justified. Don t forget your name. 1 2, A = 2 2
HW3 - Due 02/06 Each answer must be mathematically justified Don t forget your name Problem 1 Find a 2 2 matrix B such that B 3 = A, where A = 2 2 If A was diagonal, it would be easy: we would just take
More informationNon-linear least squares
Non-linear least squares Concept of non-linear least squares We have extensively studied linear least squares or linear regression. We see that there is a unique regression line that can be determined
More informationExtrema of Functions of Several Variables
Extrema of Functions of Several Variables MATH 311, Calculus III J. Robert Buchanan Department of Mathematics Fall 2011 Background (1 of 3) In single-variable calculus there are three important results
More informationQuaternion Dynamics, Part 1 Functions, Derivatives, and Integrals. Gary D. Simpson. rev 00 Dec 27, 2014.
Quaternion Dynamics, Part 1 Functions, Derivatives, and Integrals Gary D. Simpson gsim100887@aol.com rev 00 Dec 27, 2014 Summary Definitions are presented for "quaternion functions" of a quaternion. Polynomial
More information1 Overview. 2 A Characterization of Convex Functions. 2.1 First-order Taylor approximation. AM 221: Advanced Optimization Spring 2016
AM 221: Advanced Optimization Spring 2016 Prof. Yaron Singer Lecture 8 February 22nd 1 Overview In the previous lecture we saw characterizations of optimality in linear optimization, and we reviewed the
More informationBSM510 Numerical Analysis
BSM510 Numerical Analysis Polynomial Interpolation Prof. Manar Mohaisen Department of EEC Engineering Review of Precedent Lecture Polynomial Regression Multiple Linear Regression Nonlinear Regression Lecture
More informationIteration & Fixed Point
Iteration & Fied Point As a method for finding the root of f this method is difficult, but it illustrates some important features of iterstion. We could write f as f g and solve g. Definition.1 (Fied Point)
More information1 Lecture 8: Interpolating polynomials.
1 Lecture 8: Interpolating polynomials. 1.1 Horner s method Before turning to the main idea of this part of the course, we consider how to evaluate a polynomial. Recall that a polynomial is an expression
More informationSection 4.2. Types of Differentiation
42 Types of Differentiation 1 Section 42 Types of Differentiation Note In this section we define differentiation of various structures with respect to a scalar, a vector, and a matrix Definition Let vector
More informationExam 2. Average: 85.6 Median: 87.0 Maximum: Minimum: 55.0 Standard Deviation: Numerical Methods Fall 2011 Lecture 20
Exam 2 Average: 85.6 Median: 87.0 Maximum: 100.0 Minimum: 55.0 Standard Deviation: 10.42 Fall 2011 1 Today s class Multiple Variable Linear Regression Polynomial Interpolation Lagrange Interpolation Newton
More informationECE580 Partial Solution to Problem Set 3
ECE580 Fall 2015 Solution to Problem Set 3 October 23, 2015 1 ECE580 Partial Solution to Problem Set 3 These problems are from the textbook by Chong and Zak, 4th edition, which is the textbook for the
More informationg(t) = f(x 1 (t),..., x n (t)).
Reading: [Simon] p. 313-333, 833-836. 0.1 The Chain Rule Partial derivatives describe how a function changes in directions parallel to the coordinate axes. Now we shall demonstrate how the partial derivatives
More informationMA1032-Numerical Analysis-Analysis Part-14S2- 1 of 8
MA1032-Numerical Analysis-Analysis Part-14S2-www.math.mrt.ac.lk/UCJ-20151221-Page 1 of 8 RIEMANN INTEGRAL Example: Use equal partitions of [0,1] to estimate the area under the curve = using 1. left corner
More informationMath Review for Exam Compute the second degree Taylor polynomials about (0, 0) of the following functions: (a) f(x, y) = e 2x 3y.
Math 35 - Review for Exam 1. Compute the second degree Taylor polynomial of f e x+3y about (, ). Solution. A computation shows that f x(, ), f y(, ) 3, f xx(, ) 4, f yy(, ) 9, f xy(, ) 6. The second degree
More informationChapter 3 Numerical Methods
Chapter 3 Numerical Methods Part 2 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization 1 Outline 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization Summary 2 Outline 3.2
More informationMath 10C - Fall Final Exam
Math 1C - Fall 217 - Final Exam Problem 1. Consider the function f(x, y) = 1 x 2 (y 1) 2. (i) Draw the level curve through the point P (1, 2). Find the gradient of f at the point P and draw the gradient
More informationNewton s Method. Javier Peña Convex Optimization /36-725
Newton s Method Javier Peña Convex Optimization 10-725/36-725 1 Last time: dual correspondences Given a function f : R n R, we define its conjugate f : R n R, f ( (y) = max y T x f(x) ) x Properties and
More informationMultivariable Calculus and Matrix Algebra-Summer 2017
Multivariable Calculus and Matrix Algebra-Summer 017 Homework 4 Solutions Note that the solutions below are for the latest version of the problems posted. For those of you who worked on an earlier version
More informationEmpirical Models Interpolation Polynomial Models
Mathematical Modeling Lia Vas Empirical Models Interpolation Polynomial Models Lagrange Polynomial. Recall that two points (x 1, y 1 ) and (x 2, y 2 ) determine a unique line y = ax + b passing them (obtained
More informationTranspose & Dot Product
Transpose & Dot Product Def: The transpose of an m n matrix A is the n m matrix A T whose columns are the rows of A. So: The columns of A T are the rows of A. The rows of A T are the columns of A. Example:
More information, b = 0. (2) 1 2 The eigenvectors of A corresponding to the eigenvalues λ 1 = 1, λ 2 = 3 are
Quadratic forms We consider the quadratic function f : R 2 R defined by f(x) = 2 xt Ax b T x with x = (x, x 2 ) T, () where A R 2 2 is symmetric and b R 2. We will see that, depending on the eigenvalues
More information12.1 Taylor Polynomials In Two-Dimensions: Nonlinear Approximations of F(x,y) Polynomials in Two-Variables. Taylor Polynomials and Approximations.
Week 12 Lecture Overheads Math2080 Week 12 Page 1 12.1 Taylor Polynomials In Two-Dimensions: Nonlinear Approximations of F(x,y) Polynomials in Two-Variables. Taylor Polynomials and Approximations. 12.2
More informationName: Instructor: Lecture time: TA: Section time:
Math 2220 Prelim 1 February 17, 2009 Name: Instructor: Lecture time: TA: Section time: INSTRUCTIONS READ THIS NOW This test has 6 problems on 7 pages worth a total of 100 points. Look over your test package
More informationSolutions to Homework 7
Solutions to Homework 7 Exercise #3 in section 5.2: A rectangular box is inscribed in a hemisphere of radius r. Find the dimensions of the box of maximum volume. Solution: The base of the rectangular box
More informationChapter 11. Taylor Series. Josef Leydold Mathematical Methods WS 2018/19 11 Taylor Series 1 / 27
Chapter 11 Taylor Series Josef Leydold Mathematical Methods WS 2018/19 11 Taylor Series 1 / 27 First-Order Approximation We want to approximate function f by some simple function. Best possible approximation
More informationDifferentiable Functions
Differentiable Functions Let S R n be open and let f : R n R. We recall that, for x o = (x o 1, x o,, x o n S the partial derivative of f at the point x o with respect to the component x j is defined as
More informationTranspose & Dot Product
Transpose & Dot Product Def: The transpose of an m n matrix A is the n m matrix A T whose columns are the rows of A. So: The columns of A T are the rows of A. The rows of A T are the columns of A. Example:
More informationCalculus. Preliminary Edition. Robert Decker. Dale Varberg. Prentice Hall, Upper Saddle River, New Jersey UNIVERSITY OF HARTFORD
Calculus Preliminary Edition Robert Decker UNIVERSITY OF HARTFORD Dale Varberg HAML1NE UNIVERSITY Prentice Hall, Upper Saddle River, New Jersey 07458 Annotated Table of Contents.For your convenience, we
More informationUnit #24 - Lagrange Multipliers Section 15.3
Unit #24 - Lagrange Multipliers Section 1.3 Some material from Calculus, Single and MultiVariable by Hughes-Hallett, Gleason, McCallum et. al. Copyright 200 by John Wiley & Sons, Inc. This material is
More informationChap. 19: Numerical Differentiation
Chap. 19: Numerical Differentiation Differentiation Definition of difference: y x f x x i x f x i As x is approaching zero, the difference becomes a derivative: dy dx lim x 0 f x i x f x i x 2 High-Accuracy
More informationScientific Computing: Optimization
Scientific Computing: Optimization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 March 8th, 2011 A. Donev (Courant Institute) Lecture
More informationMULTIVARIABLE CALCULUS, LINEAR ALGEBRA, AND DIFFERENTIAL EQUATIONS
T H I R D E D I T I O N MULTIVARIABLE CALCULUS, LINEAR ALGEBRA, AND DIFFERENTIAL EQUATIONS STANLEY I. GROSSMAN University of Montana and University College London SAUNDERS COLLEGE PUBLISHING HARCOURT BRACE
More informationSolving Differential Equations Using Power Series
LECTURE 25 Solving Differential Equations Using Power Series We are now going to employ power series to find solutions to differential equations of the form (25.) y + p(x)y + q(x)y = 0 where the functions
More informationSolving Differential Equations Using Power Series
LECTURE 8 Solving Differential Equations Using Power Series We are now going to employ power series to find solutions to differential equations of the form () y + p(x)y + q(x)y = 0 where the functions
More informationLecture 8: Maxima and Minima
Lecture 8: Maxima and Minima Rafikul Alam Department of Mathematics IIT Guwahati Local extremum of f : R n R Let f : U R n R be continuous, where U is open. Then f has a local maximum at p if there exists
More informationTangent Planes, Linear Approximations and Differentiability
Jim Lambers MAT 80 Spring Semester 009-10 Lecture 5 Notes These notes correspond to Section 114 in Stewart and Section 3 in Marsden and Tromba Tangent Planes, Linear Approximations and Differentiability
More informationNumerical Analysis Fall. Roots: Open Methods
Numerical Analysis 2015 Fall Roots: Open Methods Open Methods Open methods differ from bracketing methods, in that they require only a single starting value or two starting values that do not necessarily
More informationCoordinate Systems. S. F. Ellermeyer. July 10, 2009
Coordinate Systems S F Ellermeyer July 10, 009 These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (rd edition) These notes are
More informationNumerical Method for Solution of Fuzzy Nonlinear Equations
1, Issue 1 (218) 13-18 Journal of Advanced Research in Modelling and Simulations Journal homepage: www.akademiabaru.com/armas.html ISSN: XXXX-XXXX Numerical for Solution of Fuzzy Nonlinear Equations Open
More informationOptimization. Sherif Khalifa. Sherif Khalifa () Optimization 1 / 50
Sherif Khalifa Sherif Khalifa () Optimization 1 / 50 Y f(x 0 ) Y=f(X) X 0 X Sherif Khalifa () Optimization 2 / 50 Y Y=f(X) f(x 0 ) X 0 X Sherif Khalifa () Optimization 3 / 50 A necessary condition for
More informationcha1873x_p02.qxd 3/21/05 1:01 PM Page 104 PART TWO
cha1873x_p02.qxd 3/21/05 1:01 PM Page 104 PART TWO ROOTS OF EQUATIONS PT2.1 MOTIVATION Years ago, you learned to use the quadratic formula x = b ± b 2 4ac 2a to solve f(x) = ax 2 + bx + c = 0 (PT2.1) (PT2.2)
More informationCoordinate Systems. Recall that a basis for a vector space, V, is a set of vectors in V that is linearly independent and spans V.
These notes closely follow the presentation of the material given in David C. Lay s textbook Linear Algebra and its Applications (rd edition). These notes are intended primarily for in-class presentation
More informationThe Newton-Raphson Algorithm
The Newton-Raphson Algorithm David Allen University of Kentucky January 31, 2013 1 The Newton-Raphson Algorithm The Newton-Raphson algorithm, also called Newton s method, is a method for finding the minimum
More information( ) y 2! 4. ( )( y! 2)
1. Dividing: 4x3! 8x 2 + 6x 2x 5.7 Division of Polynomials = 4x3 2x! 8x2 2x + 6x 2x = 2x2! 4 3. Dividing: 1x4 + 15x 3! 2x 2!5x 2 = 1x4!5x 2 + 15x3!5x 2! 2x2!5x 2 =!2x2! 3x + 4 5. Dividing: 8y5 + 1y 3!
More informationX On record with the USOE.
Textbook Alignment to the Utah Core Algebra 1 Name of Company and Individual Conducting Alignment: Chris McHugh, McHugh Inc. A Credential Sheet has been completed on the above company/evaluator and is
More informationGeorgia Department of Education Common Core Georgia Performance Standards Framework CCGPS Advanced Algebra Unit 2
Polynomials Patterns Task 1. To get an idea of what polynomial functions look like, we can graph the first through fifth degree polynomials with leading coefficients of 1. For each polynomial function,
More informationA Generalization of Wilson s Theorem
A Generalization of Wilson s Theorem R. Andrew Ohana June 3, 2009 Contents 1 Introduction 2 2 Background Algebra 2 2.1 Groups................................. 2 2.2 Rings.................................
More informationMath 1314 Test 4 Review Lesson 16 Lesson Use Riemann sums with midpoints and 6 subdivisions to approximate the area between
Math 1314 Test 4 Review Lesson 16 Lesson 24 1. Use Riemann sums with midpoints and 6 subdivisions to approximate the area between and the x-axis on the interval [1, 9]. Recall: RectangleSum[,
More informationVariational Methods & Optimal Control
Variational Methods & Optimal Control lecture 02 Matthew Roughan Discipline of Applied Mathematics School of Mathematical Sciences University of Adelaide April 14, 2016
More informationDifferentiate Early, Differentiate Often!
Differentiate Early, Differentiate Often! Robert Dawson Department of Mathematics and Computing Science Saint Mary s University. Halifax, Nova Scotia Canada B3H 3C3 Phone: 902-420-5796 Fax: 902-420-5035
More informationIntegration, differentiation, and root finding. Phys 420/580 Lecture 7
Integration, differentiation, and root finding Phys 420/580 Lecture 7 Numerical integration Compute an approximation to the definite integral I = b Find area under the curve in the interval Trapezoid Rule:
More information1 Computing with constraints
Notes for 2017-04-26 1 Computing with constraints Recall that our basic problem is minimize φ(x) s.t. x Ω where the feasible set Ω is defined by equality and inequality conditions Ω = {x R n : c i (x)
More informationFinal Exam 2011 Winter Term 2 Solutions
. (a Find the radius of convergence of the series: ( k k+ x k. Solution: Using the Ratio Test, we get: L = lim a k+ a k = lim ( k+ k+ x k+ ( k k+ x k = lim x = x. Note that the series converges for L
More informationMaxima and Minima. Marius Ionescu. November 5, Marius Ionescu () Maxima and Minima November 5, / 7
Maxima and Minima Marius Ionescu November 5, 2012 Marius Ionescu () Maxima and Minima November 5, 2012 1 / 7 Second Derivative Test Fact Suppose the second partial derivatives of f are continuous on a
More information14.7: Maxima and Minima
14.7: Maxima and Minima Marius Ionescu October 29, 2012 Marius Ionescu () 14.7: Maxima and Minima October 29, 2012 1 / 13 Local Maximum and Local Minimum Denition Marius Ionescu () 14.7: Maxima and Minima
More informationNonlinearOptimization
1/35 NonlinearOptimization Pavel Kordík Department of Computer Systems Faculty of Information Technology Czech Technical University in Prague Jiří Kašpar, Pavel Tvrdík, 2011 Unconstrained nonlinear optimization,
More informationMA22S3 Summary Sheet: Ordinary Differential Equations
MA22S3 Summary Sheet: Ordinary Differential Equations December 14, 2017 Kreyszig s textbook is a suitable guide for this part of the module. Contents 1 Terminology 1 2 First order separable 2 2.1 Separable
More information7a3 2. (c) πa 3 (d) πa 3 (e) πa3
1.(6pts) Find the integral x, y, z d S where H is the part of the upper hemisphere of H x 2 + y 2 + z 2 = a 2 above the plane z = a and the normal points up. ( 2 π ) Useful Facts: cos = 1 and ds = ±a sin
More information11 a 12 a 21 a 11 a 22 a 12 a 21. (C.11) A = The determinant of a product of two matrices is given by AB = A B 1 1 = (C.13) and similarly.
C PROPERTIES OF MATRICES 697 to whether the permutation i 1 i 2 i N is even or odd, respectively Note that I =1 Thus, for a 2 2 matrix, the determinant takes the form A = a 11 a 12 = a a 21 a 11 a 22 a
More informationAn Overly Simplified and Brief Review of Differential Equation Solution Methods. 1. Some Common Exact Solution Methods for Differential Equations
An Overly Simplified and Brief Review of Differential Equation Solution Methods We will be dealing with initial or boundary value problems. A typical initial value problem has the form y y 0 y(0) 1 A typical
More informationTangent spaces, normals and extrema
Chapter 3 Tangent spaces, normals and extrema If S is a surface in 3-space, with a point a S where S looks smooth, i.e., without any fold or cusp or self-crossing, we can intuitively define the tangent
More information