446 CHAP. 8 NUMERICAL OPTIMIZATION. Newton's Search for a Minimum of f(x,y) Newton s Method

Size: px
Start display at page:

Download "446 CHAP. 8 NUMERICAL OPTIMIZATION. Newton's Search for a Minimum of f(x,y) Newton s Method"

Transcription

1 446 CHAP. 8 NUMERICAL OPTIMIZATION Newton's Search for a Minimum of f(xy) Newton s Method The quadratic approximation method of Section 8.1 generated a sequence of seconddegree Lagrange polynomials. It was implicitly assumed that near the minimum the shape of the quadratics approximated the shape of the objective function y = f (x). The resulting sequence of minimums of the quadratics produced a sequence converging

2 SEC. 8.3 GRADIENT AND NEWTON S METHODS 447 Table 8.7 Gradient Method for f (x y) = (x y)/(x 2 + y 2 + 2) k x k y k f (x k y k ) y P 1 P 2 P 3 P( 11) P x Figure 8.13 The countour graph of f (x y) = (x y)/(x 2 + y 2 + 2) and the gradient method. to the minimum of the objective function f. Newton s method extends this process to functions of N independent variables: z = f (x 1 x 2...x N ). Starting at an initial point P 0 a sequence of second-degree polynomials in N variables will be constructed recursively. If the objective function is well-behaved and the initial point is near the actual minimum then the sequence of minimums of the quadratics will converge to the minimum of the objective function. The process will use both the first- and second-order partial derivatives of the objective function. Recall that the gradient method used only the first partial derivatives. It is to be expected that Newton s method will be more efficient than the gradient method. Definition 8.5. Let z = f (X) be a function of X such that 2 f (X) x i x j exists for i j =

3 448 CHAP. 8 NUMERICAL OPTIMIZATION N. The Hessian matrix for f at X denoted by H f (X) is the N N matrix 2 f (X) (2) H f (X) = x i x j where i j = N. N N It is appropriate to think of the Hessian matrix of a function f as representing the second derivative of the function (precisely the case when N = 1). It is not difficult to show that the Hessian matrix of a function f equals the Jacobian matrix (see Section 3.7) of the gradient of f ; (3) H f (X) = J f (X). Example Find the Hessian matrix at the point ( 3 2) of the function f (x y) = (x y)/(x 2 + y 2 + 2). From Example 8.8 f x (x y) = x2 + 2xy + y (x 2 + y 2 + 2) 2 and f y (x y) = x2 2xy + y 2 2 (x 2 + y 2 + 2) 2. The second partials are f xx (x y) = 2(x3 3x 2 y 3x(y 2 + 2) + y(y 2 + 2)) (x 2 + y 2 + 2) 3 Evaluating the Hessian matrix at (x y) = ( 3 2) yields f xy (x y) = 2(x3 + 3x 2 y + x(2 3y 2 ) y(y 2 + 2)) (x 2 + y 2 + 2) 3 f yx (x y) = 2(x3 + 3x 2 y + x(2 3y 2 ) y(y 2 + 2)) (x 2 + y 2 + 2) 3 f yy (x y) = 2(2x + x3 6y 3x 2 y 3xy 2 + y 3 ) (x 2 + y 2 + 2) 3. H f (x y) = H f ( 3 2) = [ fxx (x y) f xy (x y) f yx (x y) f yy (x y) [ ] ]. Definition 8.6. The Taylor polynomial of degree two for f (X) centered at A is (4) Q(X) = f (A) + f (A) (X A) (X A)H f (A)(X A). Mathematical descriptions of Taylor polynomials of degree m can be found in most vector or advanced calculus textbooks.

4 SEC. 8.3 GRADIENT AND NEWTON S METHODS 449 Example Calculate the second-degree Taylor polynomial of f (x y) = (x y)/(x 2 + y 2 + 2) centered at the point A = ( 3 2). Treat the gradient of f as a 1 2 matrix. From Examples 8.8 and 8.10 respectively. Thus f ( 3 2) =[f x ( 3 2) f y ( 3 2)] = H f ( 3 2) = [ ] 225 Q(x y) = [ 9 19 ] [ x + 3 y + 2 ] ( ) [ x + 3 y + 2 ] [ x + 3 y + 2 ] = 69x2 61y x 763y 78xy Without ambiguity the matrix notation is dropped from the resultant 1 1 matrix. Assume that the first and second partial derivatives of z = f (x 1 x 2...x N ) exist and are continuous in a region containing the point P 0 and that there is a minimum at the point P. Substituting P 0 for A in formula (4) yields (5) Q(X) = f (P 0 ) + f (P 0 ) (X P 0 ) (X P 0)H f (P 0 )(X P 0 ) a second-degree polynomial in N variables; where X =[x 1 x 2 x N ]. A minimum of Q(X) occurs where (6) Q(X) = 0 or (7) f (P 0 ) + (X P 0 )(H f (P 0 )) = 0. If P 0 is close to the point P (where a minimum of f occurs) then H f (P 0 ) is invertible and equation (7) can be solved for X: (8) X = P 0 f (P 0 )((H f (P 0 )) 1 ). Substituting P 1 for X in formula (8) yields (9) P 1 = P 0 f (P 0 )((H f (P 0 )) 1 ).

5 450 CHAP. 8 NUMERICAL OPTIMIZATION When P k 1 is used in place of P 0 in formula (9) the following general rule is established: (10) P k = P k 1 f (P k 1 )((H f (P k 1 )) 1 ). In equation (7) the inverse of the Hessian matrix was used to solve for X. It would be better to solve the system of linear equations represented by equation (7) with one of the methods from Chapter 3. In general the methods in Chapter 3 are more reliable and efficient. The reader should realize that the inverse is primarily a theoretical tool and the computation and use of inverses is inherently inefficient. Example Use formula (10) to find P 1 and P 2 for the function f (x y) = (x y)/(x 2 + y 2 + 2). Use the initial point P 0 =[ ]. If P 0 =[ ] then f (P 0 ) = [ ] H f (P 0 ) = (H f (P 0 )) = Substituting P 0 f (P 0 ) and (H f (P 0 )) 1 into formula (10) yields P 1 = [ ] [ ] = [ ]. If P 1 = [ ] then f (P 1 ) = [ ] H f (P 1 ) = (H f (P 1 )) = Substituting P 1 f (P 1 ) and (H f (P 1 )) 1 into formula (10) yields P 2 = [ ] [ ] [ = [ ]. The process appears to be converging to the point P =[ 11] where the minimum occurs for the function f.atthefifth interation P 5 =[ 11]. ]

6 SEC. 8.3 GRADIENT AND NEWTON S METHODS 451 It should be noted that formula (9) is equivalent (take the transpose of both sides) to formula (30) in the optional Section 3.7. Formula (10) is also equivalent to step (iv) in the outline of Newton s method in Section 3.7. Thus Program 3.7 (Newton-Raphson method) can be used to produce the sequence {P k } k=0 (without using inverse matrices) that converges to P. Newton s method requires a good initial point if there is to be convergence. This is similar to the situation for the Newton-Raphson method for approximating a root of f (x) = 0. Unlike earlier examples the initial point in Example 8.12 was not P 0 =[ 3 2]. In fact as the reader can easily verify Newton s method diverges for that particular initial point. Newton s method can be modified by treating the expression f (P k 1 )((H f (P k 1 )) 1 ) in formula (10) as a search direction. This is analogous to the use of the search direction S k in the gradient method. As with the gradient method a single parameter minimization (line search) is implemented in the search direction. In general this modified Newton s method will be more reliable than Newton s method. Outline of Modified Newton s Method Suppose that P k has been obtained. (i) Compute the search direction S k = f (P k 1 )((H f (P k 1 )) 1 ). (ii) Perform a single parameter minimization of (γ ) = f (P k +γ S k ) on the interval [0 b] where b is large. This will produce a value γ = h min where a local minimum for (γ ) occurs. The relation (h min ) = f (P k + h min S k ) shows that this is a minimum for f (X) along the search line X = P k + h min S k. (iii) Construct the next point P k+1 = P k + h min S k. (iv) Perform the termination test for minimization; that is are the function values f (P k ) and f (P k+1 ) sufficiently close and the distance P k+1 P k small enough? Repeat the process.

7 Numerical Methods Using Matlab 4 th Edition 2004 John H. Mathews and Kurtis K. Fink ISBN: Prentice-Hall Inc. Upper Saddle River New Jersey USA

p 1 p 0 (p 1, f(p 1 )) (p 0, f(p 0 )) The geometric construction of p 2 for the se- cant method.

p 1 p 0 (p 1, f(p 1 )) (p 0, f(p 0 )) The geometric construction of p 2 for the se- cant method. 80 CHAP. 2 SOLUTION OF NONLINEAR EQUATIONS f (x) = 0 y y = f(x) (p, 0) p 2 p 1 p 0 x (p 1, f(p 1 )) (p 0, f(p 0 )) The geometric construction of p 2 for the se- Figure 2.16 cant method. Secant Method The

More information

5.1 Least-Squares Line

5.1 Least-Squares Line 252 CHAP. 5 CURVE FITTING 5.1 Least-Squares Line In science and engineering it is often the case that an experiment produces a set of data points (x 1, y 1 ),...,(x N, y N ), where the abscissas {x k }

More information

632 CHAP. 11 EIGENVALUES AND EIGENVECTORS. QR Method

632 CHAP. 11 EIGENVALUES AND EIGENVECTORS. QR Method 632 CHAP 11 EIGENVALUES AND EIGENVECTORS QR Method Suppose that A is a real symmetric matrix In the preceding section we saw how Householder s method is used to construct a similar tridiagonal matrix The

More information

9.6 Predictor-Corrector Methods

9.6 Predictor-Corrector Methods SEC. 9.6 PREDICTOR-CORRECTOR METHODS 505 Adams-Bashforth-Moulton Method 9.6 Predictor-Corrector Methods The methods of Euler, Heun, Taylor, and Runge-Kutta are called single-step methods because they use

More information

434 CHAP. 8 NUMERICAL OPTIMIZATION. Powell's Method. Powell s Method

434 CHAP. 8 NUMERICAL OPTIMIZATION. Powell's Method. Powell s Method 434 CHAP. 8 NUMERICAL OPTIMIZATION Powell's Method Powell s Method Let X be an initial guess at the location of the minimum of the function z = f (x, x 2,...,x N ). Assume that the partial derivatives

More information

SEC POWER METHOD Power Method

SEC POWER METHOD Power Method SEC..2 POWER METHOD 599.2 Power Method We now describe the power method for computing the dominant eigenpair. Its extension to the inverse power method is practical for finding any eigenvalue provided

More information

9.2 Euler s Method. (1) t k = a + kh for k = 0, 1,..., M where h = b a. The value h is called the step size. We now proceed to solve approximately

9.2 Euler s Method. (1) t k = a + kh for k = 0, 1,..., M where h = b a. The value h is called the step size. We now proceed to solve approximately 464 CHAP. 9 SOLUTION OF DIFFERENTIAL EQUATIONS 9. Euler s Method The reader should be convinced that not all initial value problems can be solved explicitly, and often it is impossible to find a formula

More information

Bernstein polynomials of degree N are defined by

Bernstein polynomials of degree N are defined by SEC. 5.5 BÉZIER CURVES 309 5.5 Bézier Curves Pierre Bézier at Renault and Paul de Casteljau at Citroën independently developed the Bézier curve for CAD/CAM operations, in the 1970s. These parametrically

More information

Calculus 2502A - Advanced Calculus I Fall : Local minima and maxima

Calculus 2502A - Advanced Calculus I Fall : Local minima and maxima Calculus 50A - Advanced Calculus I Fall 014 14.7: Local minima and maxima Martin Frankland November 17, 014 In these notes, we discuss the problem of finding the local minima and maxima of a function.

More information

The Tangent Parabola The AMATYC Review, Vol. 23, No. 1, Fall 2001, pp

The Tangent Parabola The AMATYC Review, Vol. 23, No. 1, Fall 2001, pp The Tangent Parabola The AMATYC Review, Vol. 3, No., Fall, pp. 5-3. John H. Mathews California State University Fullerton Fullerton, CA 9834 By Russell W. Howell Westmont College Santa Barbara, CA 938

More information

9.8 Boundary Value Problems. Another type of differential equation has the form. (1) x = f (t, x, x ) for a t b, with the boundary conditions

9.8 Boundary Value Problems. Another type of differential equation has the form. (1) x = f (t, x, x ) for a t b, with the boundary conditions 528 CHAP. 9 SOLUTION OF DIFFERENTIAL EQUATIONS Linear Shooting Method 9.8 Boundary Value Problems Another type of differential equation has the form (1) x = f (t, x, x ) for a t b, with the boundary conditions

More information

OR MSc Maths Revision Course

OR MSc Maths Revision Course OR MSc Maths Revision Course Tom Byrne School of Mathematics University of Edinburgh t.m.byrne@sms.ed.ac.uk 15 September 2017 General Information Today JCMB Lecture Theatre A, 09:30-12:30 Mathematics revision

More information

REVIEW OF DIFFERENTIAL CALCULUS

REVIEW OF DIFFERENTIAL CALCULUS REVIEW OF DIFFERENTIAL CALCULUS DONU ARAPURA 1. Limits and continuity To simplify the statements, we will often stick to two variables, but everything holds with any number of variables. Let f(x, y) be

More information

Two Posts to Fill On School Board

Two Posts to Fill On School Board Y Y 9 86 4 4 qz 86 x : ( ) z 7 854 Y x 4 z z x x 4 87 88 Y 5 x q x 8 Y 8 x x : 6 ; : 5 x ; 4 ( z ; ( ) ) x ; z 94 ; x 3 3 3 5 94 ; ; ; ; 3 x : 5 89 q ; ; x ; x ; ; x : ; ; ; ; ; ; 87 47% : () : / : 83

More information

MA102: Multivariable Calculus

MA102: Multivariable Calculus MA102: Multivariable Calculus Rupam Barman and Shreemayee Bora Department of Mathematics IIT Guwahati Differentiability of f : U R n R m Definition: Let U R n be open. Then f : U R n R m is differentiable

More information

Optimization and Calculus

Optimization and Calculus Optimization and Calculus To begin, there is a close relationship between finding the roots to a function and optimizing a function. In the former case, we solve for x. In the latter, we solve: g(x) =

More information

A DARK GREY P O N T, with a Switch Tail, and a small Star on the Forehead. Any

A DARK GREY P O N T, with a Switch Tail, and a small Star on the Forehead. Any Y Y Y X X «/ YY Y Y ««Y x ) & \ & & } # Y \#$& / Y Y X» \\ / X X X x & Y Y X «q «z \x» = q Y # % \ & [ & Z \ & { + % ) / / «q zy» / & / / / & x x X / % % ) Y x X Y $ Z % Y Y x x } / % «] «] # z» & Y X»

More information

MTH4101 CALCULUS II REVISION NOTES. 1. COMPLEX NUMBERS (Thomas Appendix 7 + lecture notes) ax 2 + bx + c = 0. x = b ± b 2 4ac 2a. i = 1.

MTH4101 CALCULUS II REVISION NOTES. 1. COMPLEX NUMBERS (Thomas Appendix 7 + lecture notes) ax 2 + bx + c = 0. x = b ± b 2 4ac 2a. i = 1. MTH4101 CALCULUS II REVISION NOTES 1. COMPLEX NUMBERS (Thomas Appendix 7 + lecture notes) 1.1 Introduction Types of numbers (natural, integers, rationals, reals) The need to solve quadratic equations:

More information

Math Maximum and Minimum Values, I

Math Maximum and Minimum Values, I Math 213 - Maximum and Minimum Values, I Peter A. Perry University of Kentucky October 8, 218 Homework Re-read section 14.7, pp. 959 965; read carefully pp. 965 967 Begin homework on section 14.7, problems

More information

The Derivative. Appendix B. B.1 The Derivative of f. Mappings from IR to IR

The Derivative. Appendix B. B.1 The Derivative of f. Mappings from IR to IR Appendix B The Derivative B.1 The Derivative of f In this chapter, we give a short summary of the derivative. Specifically, we want to compare/contrast how the derivative appears for functions whose domain

More information

3.5 Quadratic Approximation and Convexity/Concavity

3.5 Quadratic Approximation and Convexity/Concavity 3.5 Quadratic Approximation and Convexity/Concavity 55 3.5 Quadratic Approximation and Convexity/Concavity Overview: Second derivatives are useful for understanding how the linear approximation varies

More information

Functions of Several Variables

Functions of Several Variables Functions of Several Variables Partial Derivatives Philippe B Laval KSU March 21, 2012 Philippe B Laval (KSU) Functions of Several Variables March 21, 2012 1 / 19 Introduction In this section we extend

More information

Investigation of Tangent Polynomials with a Computer Algebra System The AMATYC Review, Vol. 14, No. 1, Fall 1992, pp

Investigation of Tangent Polynomials with a Computer Algebra System The AMATYC Review, Vol. 14, No. 1, Fall 1992, pp Investigation of Tangent Polynomials wit a Computer Algebra System Te AMATYC Review, Vol. 14, No. 1, Fall 199, pp. -7. Jon H. Matews California State University Fullerton Fullerton, CA 9834 By Russell

More information

MATHEMATICAL EXPOSITION

MATHEMATICAL EXPOSITION MATHEMATICAL EXPOSITION An Improved Newton's Method by John H. Mathews California State University, Fullerton, CA 92634 4(oif John earned his Ph.D. in mathematics from Michigan State University under the

More information

The functions in all models depend on two variables: time t and spatial variable x, (x, y) or (x, y, z).

The functions in all models depend on two variables: time t and spatial variable x, (x, y) or (x, y, z). Review of Multi-variable calculus: The functions in all models depend on two variables: time t and spatial variable x, (x, y) or (x, y, z). The spatial variable represents the environment where the species

More information

Nonlinear Equations and Continuous Optimization

Nonlinear Equations and Continuous Optimization Nonlinear Equations and Continuous Optimization Sanzheng Qiao Department of Computing and Software McMaster University March, 2014 Outline 1 Introduction 2 Bisection Method 3 Newton s Method 4 Systems

More information

SOLUTIONS FOR PROBLEMS 1-30

SOLUTIONS FOR PROBLEMS 1-30 . Answer: 5 Evaluate x x + 9 for x SOLUTIONS FOR PROBLEMS - 0 When substituting x in x be sure to do the exponent before the multiplication by to get (). + 9 5 + When multiplying ( ) so that ( 7) ( ).

More information

Preliminaries Lectures. Dr. Abdulla Eid. Department of Mathematics MATHS 101: Calculus I

Preliminaries Lectures. Dr. Abdulla Eid. Department of Mathematics   MATHS 101: Calculus I Preliminaries 2 1 2 Lectures Department of Mathematics http://www.abdullaeid.net/maths101 MATHS 101: Calculus I (University of Bahrain) Prelim 1 / 35 Pre Calculus MATHS 101: Calculus MATHS 101 is all about

More information

CALCULUS: Math 21C, Fall 2010 Final Exam: Solutions. 1. [25 pts] Do the following series converge or diverge? State clearly which test you use.

CALCULUS: Math 21C, Fall 2010 Final Exam: Solutions. 1. [25 pts] Do the following series converge or diverge? State clearly which test you use. CALCULUS: Math 2C, Fall 200 Final Exam: Solutions. [25 pts] Do the following series converge or diverge? State clearly which test you use. (a) (d) n(n + ) ( ) cos n n= n= (e) (b) n= n= [ cos ( ) n n (c)

More information

Math Exam 2, October 14, 2008

Math Exam 2, October 14, 2008 Math 96 - Exam 2, October 4, 28 Name: Problem (5 points Find all solutions to the following system of linear equations, check your work: x + x 2 x 3 2x 2 2x 3 2 x x 2 + x 3 2 Solution Let s perform Gaussian

More information

SOLUTIONS to Exercises from Optimization

SOLUTIONS to Exercises from Optimization SOLUTIONS to Exercises from Optimization. Use the bisection method to find the root correct to 6 decimal places: 3x 3 + x 2 = x + 5 SOLUTION: For the root finding algorithm, we need to rewrite the equation

More information

Nonlinear Optimization: What s important?

Nonlinear Optimization: What s important? Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global

More information

2x (x 2 + y 2 + 1) 2 2y. (x 2 + y 2 + 1) 4. 4xy. (1, 1)(x 1) + (1, 1)(y + 1) (1, 1)(x 1)(y + 1) 81 x y y + 7.

2x (x 2 + y 2 + 1) 2 2y. (x 2 + y 2 + 1) 4. 4xy. (1, 1)(x 1) + (1, 1)(y + 1) (1, 1)(x 1)(y + 1) 81 x y y + 7. Homework 8 Solutions, November 007. (1 We calculate some derivatives: f x = f y = x (x + y + 1 y (x + y + 1 x = (x + y + 1 4x (x + y + 1 4 y = (x + y + 1 4y (x + y + 1 4 x y = 4xy (x + y + 1 4 Substituting

More information

100 CHAPTER 4. SYSTEMS AND ADAPTIVE STEP SIZE METHODS APPENDIX

100 CHAPTER 4. SYSTEMS AND ADAPTIVE STEP SIZE METHODS APPENDIX 100 CHAPTER 4. SYSTEMS AND ADAPTIVE STEP SIZE METHODS APPENDIX.1 Norms If we have an approximate solution at a given point and we want to calculate the absolute error, then we simply take the magnitude

More information

INTERPOLATION. and y i = cos x i, i = 0, 1, 2 This gives us the three points. Now find a quadratic polynomial. p(x) = a 0 + a 1 x + a 2 x 2.

INTERPOLATION. and y i = cos x i, i = 0, 1, 2 This gives us the three points. Now find a quadratic polynomial. p(x) = a 0 + a 1 x + a 2 x 2. INTERPOLATION Interpolation is a process of finding a formula (often a polynomial) whose graph will pass through a given set of points (x, y). As an example, consider defining and x 0 = 0, x 1 = π/4, x

More information

3 Applications of partial differentiation

3 Applications of partial differentiation Advanced Calculus Chapter 3 Applications of partial differentiation 37 3 Applications of partial differentiation 3.1 Stationary points Higher derivatives Let U R 2 and f : U R. The partial derivatives

More information

ENGI Partial Differentiation Page y f x

ENGI Partial Differentiation Page y f x ENGI 3424 4 Partial Differentiation Page 4-01 4. Partial Differentiation For functions of one variable, be found unambiguously by differentiation: y f x, the rate of change of the dependent variable can

More information

Introduction to Unconstrained Optimization: Part 2

Introduction to Unconstrained Optimization: Part 2 Introduction to Unconstrained Optimization: Part 2 James Allison ME 555 January 29, 2007 Overview Recap Recap selected concepts from last time (with examples) Use of quadratic functions Tests for positive

More information

CALCULUS GARRET J. ETGEN SALAS AND HILLE'S. ' MiIIIIIIH. I '////I! li II ii: ONE AND SEVERAL VARIABLES SEVENTH EDITION REVISED BY \

CALCULUS GARRET J. ETGEN SALAS AND HILLE'S. ' MiIIIIIIH. I '////I! li II ii: ONE AND SEVERAL VARIABLES SEVENTH EDITION REVISED BY \ / / / ' ' ' / / ' '' ' - -'/-' yy xy xy' y- y/ /: - y/ yy y /'}' / >' // yy,y-' 'y '/' /y , I '////I! li II ii: ' MiIIIIIIH IIIIII!l ii r-i: V /- A' /; // ;.1 " SALAS AND HILLE'S

More information

HW3 - Due 02/06. Each answer must be mathematically justified. Don t forget your name. 1 2, A = 2 2

HW3 - Due 02/06. Each answer must be mathematically justified. Don t forget your name. 1 2, A = 2 2 HW3 - Due 02/06 Each answer must be mathematically justified Don t forget your name Problem 1 Find a 2 2 matrix B such that B 3 = A, where A = 2 2 If A was diagonal, it would be easy: we would just take

More information

Non-linear least squares

Non-linear least squares Non-linear least squares Concept of non-linear least squares We have extensively studied linear least squares or linear regression. We see that there is a unique regression line that can be determined

More information

Extrema of Functions of Several Variables

Extrema of Functions of Several Variables Extrema of Functions of Several Variables MATH 311, Calculus III J. Robert Buchanan Department of Mathematics Fall 2011 Background (1 of 3) In single-variable calculus there are three important results

More information

Quaternion Dynamics, Part 1 Functions, Derivatives, and Integrals. Gary D. Simpson. rev 00 Dec 27, 2014.

Quaternion Dynamics, Part 1 Functions, Derivatives, and Integrals. Gary D. Simpson. rev 00 Dec 27, 2014. Quaternion Dynamics, Part 1 Functions, Derivatives, and Integrals Gary D. Simpson gsim100887@aol.com rev 00 Dec 27, 2014 Summary Definitions are presented for "quaternion functions" of a quaternion. Polynomial

More information

1 Overview. 2 A Characterization of Convex Functions. 2.1 First-order Taylor approximation. AM 221: Advanced Optimization Spring 2016

1 Overview. 2 A Characterization of Convex Functions. 2.1 First-order Taylor approximation. AM 221: Advanced Optimization Spring 2016 AM 221: Advanced Optimization Spring 2016 Prof. Yaron Singer Lecture 8 February 22nd 1 Overview In the previous lecture we saw characterizations of optimality in linear optimization, and we reviewed the

More information

BSM510 Numerical Analysis

BSM510 Numerical Analysis BSM510 Numerical Analysis Polynomial Interpolation Prof. Manar Mohaisen Department of EEC Engineering Review of Precedent Lecture Polynomial Regression Multiple Linear Regression Nonlinear Regression Lecture

More information

Iteration & Fixed Point

Iteration & Fixed Point Iteration & Fied Point As a method for finding the root of f this method is difficult, but it illustrates some important features of iterstion. We could write f as f g and solve g. Definition.1 (Fied Point)

More information

1 Lecture 8: Interpolating polynomials.

1 Lecture 8: Interpolating polynomials. 1 Lecture 8: Interpolating polynomials. 1.1 Horner s method Before turning to the main idea of this part of the course, we consider how to evaluate a polynomial. Recall that a polynomial is an expression

More information

Section 4.2. Types of Differentiation

Section 4.2. Types of Differentiation 42 Types of Differentiation 1 Section 42 Types of Differentiation Note In this section we define differentiation of various structures with respect to a scalar, a vector, and a matrix Definition Let vector

More information

Exam 2. Average: 85.6 Median: 87.0 Maximum: Minimum: 55.0 Standard Deviation: Numerical Methods Fall 2011 Lecture 20

Exam 2. Average: 85.6 Median: 87.0 Maximum: Minimum: 55.0 Standard Deviation: Numerical Methods Fall 2011 Lecture 20 Exam 2 Average: 85.6 Median: 87.0 Maximum: 100.0 Minimum: 55.0 Standard Deviation: 10.42 Fall 2011 1 Today s class Multiple Variable Linear Regression Polynomial Interpolation Lagrange Interpolation Newton

More information

ECE580 Partial Solution to Problem Set 3

ECE580 Partial Solution to Problem Set 3 ECE580 Fall 2015 Solution to Problem Set 3 October 23, 2015 1 ECE580 Partial Solution to Problem Set 3 These problems are from the textbook by Chong and Zak, 4th edition, which is the textbook for the

More information

g(t) = f(x 1 (t),..., x n (t)).

g(t) = f(x 1 (t),..., x n (t)). Reading: [Simon] p. 313-333, 833-836. 0.1 The Chain Rule Partial derivatives describe how a function changes in directions parallel to the coordinate axes. Now we shall demonstrate how the partial derivatives

More information

MA1032-Numerical Analysis-Analysis Part-14S2- 1 of 8

MA1032-Numerical Analysis-Analysis Part-14S2-  1 of 8 MA1032-Numerical Analysis-Analysis Part-14S2-www.math.mrt.ac.lk/UCJ-20151221-Page 1 of 8 RIEMANN INTEGRAL Example: Use equal partitions of [0,1] to estimate the area under the curve = using 1. left corner

More information

Math Review for Exam Compute the second degree Taylor polynomials about (0, 0) of the following functions: (a) f(x, y) = e 2x 3y.

Math Review for Exam Compute the second degree Taylor polynomials about (0, 0) of the following functions: (a) f(x, y) = e 2x 3y. Math 35 - Review for Exam 1. Compute the second degree Taylor polynomial of f e x+3y about (, ). Solution. A computation shows that f x(, ), f y(, ) 3, f xx(, ) 4, f yy(, ) 9, f xy(, ) 6. The second degree

More information

Chapter 3 Numerical Methods

Chapter 3 Numerical Methods Chapter 3 Numerical Methods Part 2 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization 1 Outline 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization Summary 2 Outline 3.2

More information

Math 10C - Fall Final Exam

Math 10C - Fall Final Exam Math 1C - Fall 217 - Final Exam Problem 1. Consider the function f(x, y) = 1 x 2 (y 1) 2. (i) Draw the level curve through the point P (1, 2). Find the gradient of f at the point P and draw the gradient

More information

Newton s Method. Javier Peña Convex Optimization /36-725

Newton s Method. Javier Peña Convex Optimization /36-725 Newton s Method Javier Peña Convex Optimization 10-725/36-725 1 Last time: dual correspondences Given a function f : R n R, we define its conjugate f : R n R, f ( (y) = max y T x f(x) ) x Properties and

More information

Multivariable Calculus and Matrix Algebra-Summer 2017

Multivariable Calculus and Matrix Algebra-Summer 2017 Multivariable Calculus and Matrix Algebra-Summer 017 Homework 4 Solutions Note that the solutions below are for the latest version of the problems posted. For those of you who worked on an earlier version

More information

Empirical Models Interpolation Polynomial Models

Empirical Models Interpolation Polynomial Models Mathematical Modeling Lia Vas Empirical Models Interpolation Polynomial Models Lagrange Polynomial. Recall that two points (x 1, y 1 ) and (x 2, y 2 ) determine a unique line y = ax + b passing them (obtained

More information

Transpose & Dot Product

Transpose & Dot Product Transpose & Dot Product Def: The transpose of an m n matrix A is the n m matrix A T whose columns are the rows of A. So: The columns of A T are the rows of A. The rows of A T are the columns of A. Example:

More information

, b = 0. (2) 1 2 The eigenvectors of A corresponding to the eigenvalues λ 1 = 1, λ 2 = 3 are

, b = 0. (2) 1 2 The eigenvectors of A corresponding to the eigenvalues λ 1 = 1, λ 2 = 3 are Quadratic forms We consider the quadratic function f : R 2 R defined by f(x) = 2 xt Ax b T x with x = (x, x 2 ) T, () where A R 2 2 is symmetric and b R 2. We will see that, depending on the eigenvalues

More information

12.1 Taylor Polynomials In Two-Dimensions: Nonlinear Approximations of F(x,y) Polynomials in Two-Variables. Taylor Polynomials and Approximations.

12.1 Taylor Polynomials In Two-Dimensions: Nonlinear Approximations of F(x,y) Polynomials in Two-Variables. Taylor Polynomials and Approximations. Week 12 Lecture Overheads Math2080 Week 12 Page 1 12.1 Taylor Polynomials In Two-Dimensions: Nonlinear Approximations of F(x,y) Polynomials in Two-Variables. Taylor Polynomials and Approximations. 12.2

More information

Name: Instructor: Lecture time: TA: Section time:

Name: Instructor: Lecture time: TA: Section time: Math 2220 Prelim 1 February 17, 2009 Name: Instructor: Lecture time: TA: Section time: INSTRUCTIONS READ THIS NOW This test has 6 problems on 7 pages worth a total of 100 points. Look over your test package

More information

Solutions to Homework 7

Solutions to Homework 7 Solutions to Homework 7 Exercise #3 in section 5.2: A rectangular box is inscribed in a hemisphere of radius r. Find the dimensions of the box of maximum volume. Solution: The base of the rectangular box

More information

Chapter 11. Taylor Series. Josef Leydold Mathematical Methods WS 2018/19 11 Taylor Series 1 / 27

Chapter 11. Taylor Series. Josef Leydold Mathematical Methods WS 2018/19 11 Taylor Series 1 / 27 Chapter 11 Taylor Series Josef Leydold Mathematical Methods WS 2018/19 11 Taylor Series 1 / 27 First-Order Approximation We want to approximate function f by some simple function. Best possible approximation

More information

Differentiable Functions

Differentiable Functions Differentiable Functions Let S R n be open and let f : R n R. We recall that, for x o = (x o 1, x o,, x o n S the partial derivative of f at the point x o with respect to the component x j is defined as

More information

Transpose & Dot Product

Transpose & Dot Product Transpose & Dot Product Def: The transpose of an m n matrix A is the n m matrix A T whose columns are the rows of A. So: The columns of A T are the rows of A. The rows of A T are the columns of A. Example:

More information

Calculus. Preliminary Edition. Robert Decker. Dale Varberg. Prentice Hall, Upper Saddle River, New Jersey UNIVERSITY OF HARTFORD

Calculus. Preliminary Edition. Robert Decker. Dale Varberg. Prentice Hall, Upper Saddle River, New Jersey UNIVERSITY OF HARTFORD Calculus Preliminary Edition Robert Decker UNIVERSITY OF HARTFORD Dale Varberg HAML1NE UNIVERSITY Prentice Hall, Upper Saddle River, New Jersey 07458 Annotated Table of Contents.For your convenience, we

More information

Unit #24 - Lagrange Multipliers Section 15.3

Unit #24 - Lagrange Multipliers Section 15.3 Unit #24 - Lagrange Multipliers Section 1.3 Some material from Calculus, Single and MultiVariable by Hughes-Hallett, Gleason, McCallum et. al. Copyright 200 by John Wiley & Sons, Inc. This material is

More information

Chap. 19: Numerical Differentiation

Chap. 19: Numerical Differentiation Chap. 19: Numerical Differentiation Differentiation Definition of difference: y x f x x i x f x i As x is approaching zero, the difference becomes a derivative: dy dx lim x 0 f x i x f x i x 2 High-Accuracy

More information

Scientific Computing: Optimization

Scientific Computing: Optimization Scientific Computing: Optimization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 March 8th, 2011 A. Donev (Courant Institute) Lecture

More information

MULTIVARIABLE CALCULUS, LINEAR ALGEBRA, AND DIFFERENTIAL EQUATIONS

MULTIVARIABLE CALCULUS, LINEAR ALGEBRA, AND DIFFERENTIAL EQUATIONS T H I R D E D I T I O N MULTIVARIABLE CALCULUS, LINEAR ALGEBRA, AND DIFFERENTIAL EQUATIONS STANLEY I. GROSSMAN University of Montana and University College London SAUNDERS COLLEGE PUBLISHING HARCOURT BRACE

More information

Solving Differential Equations Using Power Series

Solving Differential Equations Using Power Series LECTURE 25 Solving Differential Equations Using Power Series We are now going to employ power series to find solutions to differential equations of the form (25.) y + p(x)y + q(x)y = 0 where the functions

More information

Solving Differential Equations Using Power Series

Solving Differential Equations Using Power Series LECTURE 8 Solving Differential Equations Using Power Series We are now going to employ power series to find solutions to differential equations of the form () y + p(x)y + q(x)y = 0 where the functions

More information

Lecture 8: Maxima and Minima

Lecture 8: Maxima and Minima Lecture 8: Maxima and Minima Rafikul Alam Department of Mathematics IIT Guwahati Local extremum of f : R n R Let f : U R n R be continuous, where U is open. Then f has a local maximum at p if there exists

More information

Tangent Planes, Linear Approximations and Differentiability

Tangent Planes, Linear Approximations and Differentiability Jim Lambers MAT 80 Spring Semester 009-10 Lecture 5 Notes These notes correspond to Section 114 in Stewart and Section 3 in Marsden and Tromba Tangent Planes, Linear Approximations and Differentiability

More information

Numerical Analysis Fall. Roots: Open Methods

Numerical Analysis Fall. Roots: Open Methods Numerical Analysis 2015 Fall Roots: Open Methods Open Methods Open methods differ from bracketing methods, in that they require only a single starting value or two starting values that do not necessarily

More information

Coordinate Systems. S. F. Ellermeyer. July 10, 2009

Coordinate Systems. S. F. Ellermeyer. July 10, 2009 Coordinate Systems S F Ellermeyer July 10, 009 These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (rd edition) These notes are

More information

Numerical Method for Solution of Fuzzy Nonlinear Equations

Numerical Method for Solution of Fuzzy Nonlinear Equations 1, Issue 1 (218) 13-18 Journal of Advanced Research in Modelling and Simulations Journal homepage: www.akademiabaru.com/armas.html ISSN: XXXX-XXXX Numerical for Solution of Fuzzy Nonlinear Equations Open

More information

Optimization. Sherif Khalifa. Sherif Khalifa () Optimization 1 / 50

Optimization. Sherif Khalifa. Sherif Khalifa () Optimization 1 / 50 Sherif Khalifa Sherif Khalifa () Optimization 1 / 50 Y f(x 0 ) Y=f(X) X 0 X Sherif Khalifa () Optimization 2 / 50 Y Y=f(X) f(x 0 ) X 0 X Sherif Khalifa () Optimization 3 / 50 A necessary condition for

More information

cha1873x_p02.qxd 3/21/05 1:01 PM Page 104 PART TWO

cha1873x_p02.qxd 3/21/05 1:01 PM Page 104 PART TWO cha1873x_p02.qxd 3/21/05 1:01 PM Page 104 PART TWO ROOTS OF EQUATIONS PT2.1 MOTIVATION Years ago, you learned to use the quadratic formula x = b ± b 2 4ac 2a to solve f(x) = ax 2 + bx + c = 0 (PT2.1) (PT2.2)

More information

Coordinate Systems. Recall that a basis for a vector space, V, is a set of vectors in V that is linearly independent and spans V.

Coordinate Systems. Recall that a basis for a vector space, V, is a set of vectors in V that is linearly independent and spans V. These notes closely follow the presentation of the material given in David C. Lay s textbook Linear Algebra and its Applications (rd edition). These notes are intended primarily for in-class presentation

More information

The Newton-Raphson Algorithm

The Newton-Raphson Algorithm The Newton-Raphson Algorithm David Allen University of Kentucky January 31, 2013 1 The Newton-Raphson Algorithm The Newton-Raphson algorithm, also called Newton s method, is a method for finding the minimum

More information

( ) y 2! 4. ( )( y! 2)

( ) y 2! 4. ( )( y! 2) 1. Dividing: 4x3! 8x 2 + 6x 2x 5.7 Division of Polynomials = 4x3 2x! 8x2 2x + 6x 2x = 2x2! 4 3. Dividing: 1x4 + 15x 3! 2x 2!5x 2 = 1x4!5x 2 + 15x3!5x 2! 2x2!5x 2 =!2x2! 3x + 4 5. Dividing: 8y5 + 1y 3!

More information

X On record with the USOE.

X On record with the USOE. Textbook Alignment to the Utah Core Algebra 1 Name of Company and Individual Conducting Alignment: Chris McHugh, McHugh Inc. A Credential Sheet has been completed on the above company/evaluator and is

More information

Georgia Department of Education Common Core Georgia Performance Standards Framework CCGPS Advanced Algebra Unit 2

Georgia Department of Education Common Core Georgia Performance Standards Framework CCGPS Advanced Algebra Unit 2 Polynomials Patterns Task 1. To get an idea of what polynomial functions look like, we can graph the first through fifth degree polynomials with leading coefficients of 1. For each polynomial function,

More information

A Generalization of Wilson s Theorem

A Generalization of Wilson s Theorem A Generalization of Wilson s Theorem R. Andrew Ohana June 3, 2009 Contents 1 Introduction 2 2 Background Algebra 2 2.1 Groups................................. 2 2.2 Rings.................................

More information

Math 1314 Test 4 Review Lesson 16 Lesson Use Riemann sums with midpoints and 6 subdivisions to approximate the area between

Math 1314 Test 4 Review Lesson 16 Lesson Use Riemann sums with midpoints and 6 subdivisions to approximate the area between Math 1314 Test 4 Review Lesson 16 Lesson 24 1. Use Riemann sums with midpoints and 6 subdivisions to approximate the area between and the x-axis on the interval [1, 9]. Recall: RectangleSum[,

More information

Variational Methods & Optimal Control

Variational Methods & Optimal Control Variational Methods & Optimal Control lecture 02 Matthew Roughan Discipline of Applied Mathematics School of Mathematical Sciences University of Adelaide April 14, 2016

More information

Differentiate Early, Differentiate Often!

Differentiate Early, Differentiate Often! Differentiate Early, Differentiate Often! Robert Dawson Department of Mathematics and Computing Science Saint Mary s University. Halifax, Nova Scotia Canada B3H 3C3 Phone: 902-420-5796 Fax: 902-420-5035

More information

Integration, differentiation, and root finding. Phys 420/580 Lecture 7

Integration, differentiation, and root finding. Phys 420/580 Lecture 7 Integration, differentiation, and root finding Phys 420/580 Lecture 7 Numerical integration Compute an approximation to the definite integral I = b Find area under the curve in the interval Trapezoid Rule:

More information

1 Computing with constraints

1 Computing with constraints Notes for 2017-04-26 1 Computing with constraints Recall that our basic problem is minimize φ(x) s.t. x Ω where the feasible set Ω is defined by equality and inequality conditions Ω = {x R n : c i (x)

More information

Final Exam 2011 Winter Term 2 Solutions

Final Exam 2011 Winter Term 2 Solutions . (a Find the radius of convergence of the series: ( k k+ x k. Solution: Using the Ratio Test, we get: L = lim a k+ a k = lim ( k+ k+ x k+ ( k k+ x k = lim x = x. Note that the series converges for L

More information

Maxima and Minima. Marius Ionescu. November 5, Marius Ionescu () Maxima and Minima November 5, / 7

Maxima and Minima. Marius Ionescu. November 5, Marius Ionescu () Maxima and Minima November 5, / 7 Maxima and Minima Marius Ionescu November 5, 2012 Marius Ionescu () Maxima and Minima November 5, 2012 1 / 7 Second Derivative Test Fact Suppose the second partial derivatives of f are continuous on a

More information

14.7: Maxima and Minima

14.7: Maxima and Minima 14.7: Maxima and Minima Marius Ionescu October 29, 2012 Marius Ionescu () 14.7: Maxima and Minima October 29, 2012 1 / 13 Local Maximum and Local Minimum Denition Marius Ionescu () 14.7: Maxima and Minima

More information

NonlinearOptimization

NonlinearOptimization 1/35 NonlinearOptimization Pavel Kordík Department of Computer Systems Faculty of Information Technology Czech Technical University in Prague Jiří Kašpar, Pavel Tvrdík, 2011 Unconstrained nonlinear optimization,

More information

MA22S3 Summary Sheet: Ordinary Differential Equations

MA22S3 Summary Sheet: Ordinary Differential Equations MA22S3 Summary Sheet: Ordinary Differential Equations December 14, 2017 Kreyszig s textbook is a suitable guide for this part of the module. Contents 1 Terminology 1 2 First order separable 2 2.1 Separable

More information

7a3 2. (c) πa 3 (d) πa 3 (e) πa3

7a3 2. (c) πa 3 (d) πa 3 (e) πa3 1.(6pts) Find the integral x, y, z d S where H is the part of the upper hemisphere of H x 2 + y 2 + z 2 = a 2 above the plane z = a and the normal points up. ( 2 π ) Useful Facts: cos = 1 and ds = ±a sin

More information

11 a 12 a 21 a 11 a 22 a 12 a 21. (C.11) A = The determinant of a product of two matrices is given by AB = A B 1 1 = (C.13) and similarly.

11 a 12 a 21 a 11 a 22 a 12 a 21. (C.11) A = The determinant of a product of two matrices is given by AB = A B 1 1 = (C.13) and similarly. C PROPERTIES OF MATRICES 697 to whether the permutation i 1 i 2 i N is even or odd, respectively Note that I =1 Thus, for a 2 2 matrix, the determinant takes the form A = a 11 a 12 = a a 21 a 11 a 22 a

More information

An Overly Simplified and Brief Review of Differential Equation Solution Methods. 1. Some Common Exact Solution Methods for Differential Equations

An Overly Simplified and Brief Review of Differential Equation Solution Methods. 1. Some Common Exact Solution Methods for Differential Equations An Overly Simplified and Brief Review of Differential Equation Solution Methods We will be dealing with initial or boundary value problems. A typical initial value problem has the form y y 0 y(0) 1 A typical

More information

Tangent spaces, normals and extrema

Tangent spaces, normals and extrema Chapter 3 Tangent spaces, normals and extrema If S is a surface in 3-space, with a point a S where S looks smooth, i.e., without any fold or cusp or self-crossing, we can intuitively define the tangent

More information