# Simple Iteration, cont d

Size: px
Start display at page:

## Transcription

1 Jim Lambers MAT 772 Fall Semester Lecture 2 Notes These notes correspond to Section 1.2 in the text. Simple Iteration, cont d In general, nonlinear equations cannot be solved in a finite sequence of steps. As linear equations can be solved using direct methods such as Gaussian elimination, nonlinear equations usually require iterative methods. In iterative methods, an approximate solution is refined with each iteration until it is determined to be sufficiently accurate, at which time the iteration terminates. Since it is desirable for iterative methods to converge to the solution as rapidly as possible, it is necessary to be able to measure the speed with which an iterative method converges. To that end, we assume that an iterative method generates a sequence of iterates x 0, x 1, x 2,... that converges to the exact solution x. Ideally, we would like the error in a given iterate x k+1 to be much smaller than the error in the previous iterate x k. For example, if the error is raised to a power greater than 1 from iteration to iteration, then, because the error is typically less than 1, it will approach zero very rapidly. This leads to the following definition. Definition (Order and Rate of Convergence) Let {x k } k=0 be a sequence in Rn that converges to x R n and assume that x k = x for each k. We say that the order of convergence of {x k } to x is order r, with asymptotic error constant C, if x k+1 x lim k x k x r = C, where r 1. If r = 1, then the number ρ = log 10 C is called the asymptotic rate of convergence. If r = 1, and 0 < C < 1, we say that convergence is linear. If r = 1 and C = 0, or if 1 < r < 2 for any positive C, then we say that convergence is superlinear. If r = 2, then the method converges quadratically, and if r = 3, we say it converges cubically, and so on. Note that the value of C need only be bounded above in the case of linear convergence. When convergence is linear, the asymptotic rate of convergence ρ indicates the number of correct decimal digits obtained in a single iteration. In other words, 1/ρ + 1 iterations are required to obtain an additional correct decimal digit, where x is the floor of x, which is the largest integer that is less than or equal to x. If g satisfies the conditions of the Contraction Mapping Theorem with Lipschitz constant L, then Fixed-point Iteration achieves at least linear convergence, with an asymptotic error constant 1

2 that is bounded above by L. This value can be used to estimate the number of iterations needed to obtain an additional correct decimal digit, but it can also be used to estimate the total number of iterations needed for a specified degree of accuracy. From the Lipschitz condition, we have, for k 1, From we obtain x k x L x k 1 x L k x 0 x. x 0 x x 0 x 1 + x 1 x x 0 x 1 + x 1 x x 0 x 1 + L x 0 x x k x Lk 1 L x 1 x 0. Therefore, in order to satisfy x k x ε, the number of iterations, k, must satisfy k ln x 1 x 0 ln(ε(1 L)). ln(1/l) That is, we can bound the number of iterations after performing a single iteration, as long as the Lipschitz constant L is known. We know that Fixed-point Iteration will converge to the unique fixed point in [a, b] if g satisfies the conditions of the Contraction Mapping Theorem. However, if g is differentiable on [a, b], its derivative can be used to obtain an alternative criterion for convergence that can be more practical than computing the Lipschitz constant L. If we denote the error in x k by e k = x k x, we can see from Taylor s Theorem and the fact that g(x ) = x that e k+1 = x k+1 x = g(x k ) g(x ) = g (x )(x k x ) g (ξ k )(x k x ) 2 = g (x )e k + O(e 2 k ), where ξ k lies between x k and x. Therefore, if g (x ) k, where k < 1, then Fixed-point Iteration is locally convergent; that is, it converges if x 0 is chosen sufficiently close to x. This leads to the following result. Theorem (Fixed-point Theorem) Let g be a continuous function on the interval [a, b], and let g be differentiable on [a, b]. If g(x) [a, b] for each x [a, b], and if there exists a constant k < 1 such that g (x) k, x (a, b), then the sequence of iterates {x k } k=0 converges to the unique fixed point x of g in [a, b], for any initial guess x 0 [a, b]. It can be seen from the preceding discussion why g (x) must be bounded away from 1 on (a, b), as opposed to the weaker condition g (x) < 1 on (a, b). If g (x) is allowed to approach 1 as x 2

3 approaches a point c (a, b), then it is possible that the error e k might not approach zero as k increases, in which case Fixed-point Iteration would not converge. Suppose that g satisfies the conditions of the Fixed-Point Theorem, and that g is also continuously differentiable on [a, b]. We can use the Mean Value Theorem to obtain e k+1 = x k+1 x = g(x k ) g(x ) = g (ξ k )(x k x ) = g (ξ k )e k, where ξ k lies between x k and x. It follows from the continuity of g at x that for any initial iterate x 0 [a, b], Fixed-point Iteration converges linearly with asymptotic error constant g (x ), since, by the definition of ξ k and the continuity of g, e k+1 lim = lim k e k k g (ξ k ) = g (x ). Recall that the conditions we have stated for linear convergence are nearly identical to the conditions for g to have a unique fixed point in [a, b]. The only difference is that now, we also require g to be continuous on [a, b]. The derivative can also be used to indicate why Fixed-point Iteration might not converge. Example The function g(x) = x has two fixed points, x 1 = 1/4 and x 2 = 3/4, as can be determined by solving the quadratic equation x = x. If we consider the interval [0, 3/8], then g satisfies the conditions of the Fixed-point Theorem, as g (x) = 2x < 1 on this interval, and therefore Fixed-point Iteration will converge to x 1 for any x 0 [0, 3/8]. On the other hand, g (3/4) = 2(3/4) = 3/2 > 1. Therefore, it is not possible for g to satisfy the conditions of the Fixed-point Theorem. Furthemore, if x 0 is chosen so that 1/4 < x 0 < 3/4, then Fixed-point Iteration will converge to x 1 = 1/4, whereas if x 0 > 3/4, then Fixed-point Iteration diverges. The fixed point x 2 = 3/4 in the preceding example is an unstable fixed point of g, meaning that no choice of x 0 yields a sequence of iterates that converges to x 2. The fixed point x 1 = 1/4 is a stable fixed point of g, meaning that any choice of x 0 that is sufficiently close to x 1 yields a sequence of iterates that converges to x 1. The preceding example shows that Fixed-point Iteration applied to an equation of the form x = g(x) can fail to converge to a fixed point x if g (x ) > 1. We wish to determine whether this condition indicates non-convergence in general. If g (x ) > 1, and g is continuous in a neighborhood of x, then there exists an interval x x δ such that g (x) > 1 on the interval. If x k lies within this interval, it follows from the Mean Value Theorem that x k+1 x = g(x k ) g(x ) = g (η) x k x, where η lies between x k and x. Because η is also in this interval, we have x k+1 x > x k x. 3

4 In other words, the error in the iterates increases whenever they fall within a sufficiently small interval containing the fixed point. Because of this increase, the iterates must eventually fall outside of the interval. Therefore, it is not possible to find a k 0, for any given δ, such that x k x δ for all k k 0. We have thus proven the following result. Theorem Let g have a fixed point at x, and let g be continuous in a neighborhood of x. If g (x ) > 1, then Fixed-point Iteration does not converge to x for any initial guess x 0 except in a finite number of iterations. Now, suppose that in addition to the conditions of the Fixed-point Theorem, we assume that g (x ) = 0, and that g is twice continuously differentiable on [a, b]. Then, using Taylor s Theorem, we obtain e k+1 = g(x k ) g(x ) = g (x )(x k x ) g (ξ k )(x k x ) 2 = 1 2 g (ξ k )e 2 k, where ξ k lies between x k and x. It follows that for any initial iterate x 0 [a, b], Fixed-point Iteration converges at least quadratically, with asymptotic error constant g (x )/2. Later, this will be exploited to obtain a quadratically convergent method for solving nonlinear equations of the form f(x) = 0. Iterative Solution of Equations Now that we understand the convergence behavior of Fixed-point Iteration, we consider the application of Fixed-point Iteration to the solution of an equation of the form f(x) = 0. When rewriting this equation in the form x = g(x), it is essential to choose the function g wisely. One guideline is to choose g(x) = x + φ(x)f(x), where the function φ(x) is, ideally, nonzero except possibly at a solution of f(x) = 0. This can be satisfied by choosing φ(x) to be constant, but this can fail, as the following example illustrates. Example Consider the equation x + ln x = 0. By the Intermediate Value Theorem, this equation has a solution in the interval [0.5, 0.6]. Furthermore, this solution is unique. To see this, let f(x) = x + ln x. Then f (x) = x + 1/x > 0 on the domain of f, which means that f is increasing on its entire domain. Therefore, it is not possible for f(x) = 0 to have more than one solution. We consider using Fixed-point Iteration to solve the equivalent equation x = x + ( 1)(x + ln x) = ln x. However, with g(x) = ln x, we have g (x) = 1/x > 1 on the interval [0.5, 0.6]. Therefore, by the preceding theorem, Fixed-point Iteration will fail to converge for any initial guess in this 4

5 interval. We therefore apply g 1 (x) = e x to both sides of the equation x = g(x) to obtain which simplifies to g 1 (x) = g 1 (g(x)) = x, x = e x. The function g(x) = e x satisfies g (x) < 1 on [0.5, 0.6], as g (x) = e x, and e x < 1 when the argument x is positive. By narrowing this interval to [0.52, 0.6], which is mapped into itself by this choice of g, we can apply the Fixed-point Theorem to conclude that Fixed-point Iteration will converge to the unique fixed point of g for any choice of x 0 in the interval. 5

### Jim Lambers MAT 460/560 Fall Semester Practice Final Exam

Jim Lambers MAT 460/560 Fall Semester 2009-10 Practice Final Exam 1. Let f(x) = sin 2x + cos 2x. (a) Write down the 2nd Taylor polynomial P 2 (x) of f(x) centered around x 0 = 0. (b) Write down the corresponding

### Numerical Methods in Informatics

Numerical Methods in Informatics Lecture 2, 30.09.2016: Nonlinear Equations in One Variable http://www.math.uzh.ch/binf4232 Tulin Kaman Institute of Mathematics, University of Zurich E-mail: tulin.kaman@math.uzh.ch

### Jim Lambers MAT 460 Fall Semester Lecture 2 Notes

Jim Lambers MAT 460 Fall Semester 2009-10 Lecture 2 Notes These notes correspond to Section 1.1 in the text. Review of Calculus Among the mathematical problems that can be solved using techniques from

### Jim Lambers MAT 169 Fall Semester Lecture 6 Notes. a n. n=1. S = lim s k = lim. n=1. n=1

Jim Lambers MAT 69 Fall Semester 2009-0 Lecture 6 Notes These notes correspond to Section 8.3 in the text. The Integral Test Previously, we have defined the sum of a convergent infinite series to be the

### x 2 x n r n J(x + t(x x ))(x x )dt. For warming-up we start with methods for solving a single equation of one variable.

Maria Cameron 1. Fixed point methods for solving nonlinear equations We address the problem of solving an equation of the form (1) r(x) = 0, where F (x) : R n R n is a vector-function. Eq. (1) can be written

### STOP, a i+ 1 is the desired root. )f(a i) > 0. Else If f(a i+ 1. Set a i+1 = a i+ 1 and b i+1 = b Else Set a i+1 = a i and b i+1 = a i+ 1

53 17. Lecture 17 Nonlinear Equations Essentially, the only way that one can solve nonlinear equations is by iteration. The quadratic formula enables one to compute the roots of p(x) = 0 when p P. Formulas

### PART I Lecture Notes on Numerical Solution of Root Finding Problems MATH 435

PART I Lecture Notes on Numerical Solution of Root Finding Problems MATH 435 Professor Biswa Nath Datta Department of Mathematical Sciences Northern Illinois University DeKalb, IL. 60115 USA E mail: dattab@math.niu.edu

### Numerical Methods I Solving Nonlinear Equations

Numerical Methods I Solving Nonlinear Equations Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 16th, 2014 A. Donev (Courant Institute)

### Numerical Methods in Physics and Astrophysics

Kostas Kokkotas 2 October 20, 2014 2 http://www.tat.physik.uni-tuebingen.de/ kokkotas Kostas Kokkotas 3 TOPICS 1. Solving nonlinear equations 2. Solving linear systems of equations 3. Interpolation, approximation

### Hermite Interpolation

Jim Lambers MAT 77 Fall Semester 010-11 Lecture Notes These notes correspond to Sections 4 and 5 in the text Hermite Interpolation Suppose that the interpolation points are perturbed so that two neighboring

### Numerical Methods in Physics and Astrophysics

Kostas Kokkotas 2 October 17, 2017 2 http://www.tat.physik.uni-tuebingen.de/ kokkotas Kostas Kokkotas 3 TOPICS 1. Solving nonlinear equations 2. Solving linear systems of equations 3. Interpolation, approximation

### Jim Lambers MAT 419/519 Summer Session Lecture 13 Notes

Jim Lambers MAT 419/519 Summer Session 2011-12 Lecture 13 Notes These notes correspond to Section 4.1 in the text. Least Squares Fit One of the most fundamental problems in science and engineering is data

### Chapter 1. Root Finding Methods. 1.1 Bisection method

Chapter 1 Root Finding Methods We begin by considering numerical solutions to the problem f(x) = 0 (1.1) Although the problem above is simple to state it is not always easy to solve analytically. This

### FIXED POINT ITERATION

FIXED POINT ITERATION The idea of the fixed point iteration methods is to first reformulate a equation to an equivalent fixed point problem: f (x) = 0 x = g(x) and then to use the iteration: with an initial

### FIXED POINT ITERATIONS

FIXED POINT ITERATIONS MARKUS GRASMAIR 1. Fixed Point Iteration for Non-linear Equations Our goal is the solution of an equation (1) F (x) = 0, where F : R n R n is a continuous vector valued mapping in

### Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 5. Nonlinear Equations

Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T Heath Chapter 5 Nonlinear Equations Copyright c 2001 Reproduction permitted only for noncommercial, educational

### Chapter 2 Solutions of Equations of One Variable

Chapter 2 Solutions of Equations of One Variable 2.1 Bisection Method In this chapter we consider one of the most basic problems of numerical approximation, the root-finding problem. This process involves

### SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS BISECTION METHOD

BISECTION METHOD If a function f(x) is continuous between a and b, and f(a) and f(b) are of opposite signs, then there exists at least one root between a and b. It is shown graphically as, Let f a be negative

### University of Houston, Department of Mathematics Numerical Analysis, Fall 2005

3 Numerical Solution of Nonlinear Equations and Systems 3.1 Fixed point iteration Reamrk 3.1 Problem Given a function F : lr n lr n, compute x lr n such that ( ) F(x ) = 0. In this chapter, we consider

### Outline. Math Numerical Analysis. Intermediate Value Theorem. Lecture Notes Zeros and Roots. Joseph M. Mahaffy,

Outline Math 541 - Numerical Analysis Lecture Notes Zeros and Roots Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Department of Mathematics and Statistics Dynamical Systems Group Computational Sciences Research

### Math Numerical Analysis

Math 541 - Numerical Analysis Lecture Notes Zeros and Roots Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Department of Mathematics and Statistics Dynamical Systems Group Computational Sciences Research Center

### Nonlinearity Root-finding Bisection Fixed Point Iteration Newton s Method Secant Method Conclusion. Nonlinear Systems

Nonlinear Systems CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Justin Solomon CS 205A: Mathematical Methods Nonlinear Systems 1 / 24 Part III: Nonlinear Problems Not all numerical problems

### Numerical solutions of nonlinear systems of equations

Numerical solutions of nonlinear systems of equations Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan E-mail: min@math.ntnu.edu.tw August 28, 2011 Outline 1 Fixed points

### ROOT FINDING REVIEW MICHELLE FENG

ROOT FINDING REVIEW MICHELLE FENG 1.1. Bisection Method. 1. Root Finding Methods (1) Very naive approach based on the Intermediate Value Theorem (2) You need to be looking in an interval with only one

### Numerical Analysis: Solving Nonlinear Equations

Numerical Analysis: Solving Nonlinear Equations Mirko Navara http://cmp.felk.cvut.cz/ navara/ Center for Machine Perception, Department of Cybernetics, FEE, CTU Karlovo náměstí, building G, office 104a

### CS 450 Numerical Analysis. Chapter 5: Nonlinear Equations

Lecture slides based on the textbook Scientific Computing: An Introductory Survey by Michael T. Heath, copyright c 2018 by the Society for Industrial and Applied Mathematics. http://www.siam.org/books/cl80

### Outline. Scientific Computing: An Introductory Survey. Nonlinear Equations. Nonlinear Equations. Examples: Nonlinear Equations

Methods for Systems of Methods for Systems of Outline Scientific Computing: An Introductory Survey Chapter 5 1 Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign

### Math 273a: Optimization Netwon s methods

Math 273a: Optimization Netwon s methods Instructor: Wotao Yin Department of Mathematics, UCLA Fall 2015 some material taken from Chong-Zak, 4th Ed. Main features of Newton s method Uses both first derivatives

### Convergence Rates on Root Finding

Convergence Rates on Root Finding Com S 477/577 Oct 5, 004 A sequence x i R converges to ξ if for each ǫ > 0, there exists an integer Nǫ) such that x l ξ > ǫ for all l Nǫ). The Cauchy convergence criterion

### Polynomial Approximations and Power Series

Polynomial Approximations and Power Series June 24, 206 Tangent Lines One of the first uses of the derivatives is the determination of the tangent as a linear approximation of a differentiable function

### Root Finding For NonLinear Equations Bisection Method

Root Finding For NonLinear Equations Bisection Method P. Sam Johnson November 17, 2014 P. Sam Johnson (NITK) Root Finding For NonLinear Equations Bisection MethodNovember 17, 2014 1 / 26 Introduction The

### Section 6.3 Richardson s Extrapolation. Extrapolation (To infer or estimate by extending or projecting known information.)

Section 6.3 Richardson s Extrapolation Key Terms: Extrapolation (To infer or estimate by extending or projecting known information.) Illustrated using Finite Differences The difference between Interpolation

### Vectors in Function Spaces

Jim Lambers MAT 66 Spring Semester 15-16 Lecture 18 Notes These notes correspond to Section 6.3 in the text. Vectors in Function Spaces We begin with some necessary terminology. A vector space V, also

### Economics 204 Summer/Fall 2011 Lecture 5 Friday July 29, 2011

Economics 204 Summer/Fall 2011 Lecture 5 Friday July 29, 2011 Section 2.6 (cont.) Properties of Real Functions Here we first study properties of functions from R to R, making use of the additional structure

### Unconstrained optimization I Gradient-type methods

Unconstrained optimization I Gradient-type methods Antonio Frangioni Department of Computer Science University of Pisa www.di.unipi.it/~frangio frangio@di.unipi.it Computational Mathematics for Learning

### Nonlinearity Root-finding Bisection Fixed Point Iteration Newton s Method Secant Method Conclusion. Nonlinear Systems

Nonlinear Systems CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Nonlinear Systems 1 / 27 Part III: Nonlinear Problems Not

### a factors The exponential 0 is a special case. If b is any nonzero real number, then

0.1 Exponents The expression x a is an exponential expression with base x and exponent a. If the exponent a is a positive integer, then the expression is simply notation that counts how many times the

### Lecture 39: Root Finding via Newton s Method

Lecture 39: Root Finding via Newton s Method We have studied two braceting methods for finding zeros of a function, bisection and regula falsi. These methods have certain virtues (most importantly, they

### Mathematics 1a, Section 4.3 Solutions

Mathematics 1a, Section 4.3 Solutions Alexander Ellis November 30, 2004 1. f(8) f(0) 8 0 = 6 4 8 = 1 4 The values of c which satisfy f (c) = 1/4 seem to be about c = 0.8, 3.2, 4.4, and 6.1. 2. a. g is

### Goals for This Lecture:

Goals for This Lecture: Learn the Newton-Raphson method for finding real roots of real functions Learn the Bisection method for finding real roots of a real function Look at efficient implementations of

### Homework and Computer Problems for Math*2130 (W17).

Homework and Computer Problems for Math*2130 (W17). MARCUS R. GARVIE 1 December 21, 2016 1 Department of Mathematics & Statistics, University of Guelph NOTES: These questions are a bare minimum. You should

### 3.1 Introduction. Solve non-linear real equation f(x) = 0 for real root or zero x. E.g. x x 1.5 =0, tan x x =0.

3.1 Introduction Solve non-linear real equation f(x) = 0 for real root or zero x. E.g. x 3 +1.5x 1.5 =0, tan x x =0. Practical existence test for roots: by intermediate value theorem, f C[a, b] & f(a)f(b)

### you expect to encounter difficulties when trying to solve A x = b? 4. A composite quadrature rule has error associated with it in the following form

Qualifying exam for numerical analysis (Spring 2017) Show your work for full credit. If you are unable to solve some part, attempt the subsequent parts. 1. Consider the following finite difference: f (0)

### Consistency and Convergence

Jim Lambers MAT 77 Fall Semester 010-11 Lecture 0 Notes These notes correspond to Sections 1.3, 1.4 and 1.5 in the text. Consistency and Convergence We have learned that the numerical solution obtained

### E5295/5B5749 Convex optimization with engineering applications. Lecture 8. Smooth convex unconstrained and equality-constrained minimization

E5295/5B5749 Convex optimization with engineering applications Lecture 8 Smooth convex unconstrained and equality-constrained minimization A. Forsgren, KTH 1 Lecture 8 Convex optimization 2006/2007 Unconstrained

### ECE133A Applied Numerical Computing Additional Lecture Notes

Winter Quarter 2018 ECE133A Applied Numerical Computing Additional Lecture Notes L. Vandenberghe ii Contents 1 LU factorization 1 1.1 Definition................................. 1 1.2 Nonsingular sets

### MATH 614 Dynamical Systems and Chaos Lecture 3: Classification of fixed points.

MATH 614 Dynamical Systems and Chaos Lecture 3: Classification of fixed points. Periodic points Definition. A point x X is called a fixed point of a map f : X X if f(x) = x. A point x X is called a periodic

### Lyapunov Stability Theory

Lyapunov Stability Theory Peter Al Hokayem and Eduardo Gallestey March 16, 2015 1 Introduction In this lecture we consider the stability of equilibrium points of autonomous nonlinear systems, both in continuous

### 4 Stability analysis of finite-difference methods for ODEs

MATH 337, by T. Lakoba, University of Vermont 36 4 Stability analysis of finite-difference methods for ODEs 4.1 Consistency, stability, and convergence of a numerical method; Main Theorem In this Lecture

### NONLINEAR DC ANALYSIS

ECE 552 Numerical Circuit Analysis Chapter Six NONLINEAR DC ANALYSIS OR: Solution of Nonlinear Algebraic Equations I. Hajj 2017 Nonlinear Algebraic Equations A system of linear equations Ax = b has a

### Queens College, CUNY, Department of Computer Science Numerical Methods CSCI 361 / 761 Spring 2018 Instructor: Dr. Sateesh Mane.

Queens College, CUNY, Department of Computer Science Numerical Methods CSCI 361 / 761 Spring 2018 Instructor: Dr. Sateesh Mane c Sateesh R. Mane 2018 3 Lecture 3 3.1 General remarks March 4, 2018 This

11.10 Taylor and Maclaurin Series Copyright Cengage Learning. All rights reserved. We start by supposing that f is any function that can be represented by a power series f(x)= c 0 +c 1 (x a)+c 2 (x a)

### Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Chapter 5 Nonlinear Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction

### Infinite series, improper integrals, and Taylor series

Chapter 2 Infinite series, improper integrals, and Taylor series 2. Introduction to series In studying calculus, we have explored a variety of functions. Among the most basic are polynomials, i.e. functions

### THE SECANT METHOD. q(x) = a 0 + a 1 x. with

THE SECANT METHOD Newton s method was based on using the line tangent to the curve of y = f (x), with the point of tangency (x 0, f (x 0 )). When x 0 α, the graph of the tangent line is approximately the

### MAT 460: Numerical Analysis I. James V. Lambers

MAT 460: Numerical Analysis I James V. Lambers January 31, 2013 2 Contents 1 Mathematical Preliminaries and Error Analysis 7 1.1 Introduction............................ 7 1.1.1 Error Analysis......................

### Solution of Nonlinear Equations

Solution of Nonlinear Equations (Com S 477/577 Notes) Yan-Bin Jia Sep 14, 017 One of the most frequently occurring problems in scientific work is to find the roots of equations of the form f(x) = 0. (1)

### c 2007 Society for Industrial and Applied Mathematics

SIAM J. OPTIM. Vol. 18, No. 1, pp. 106 13 c 007 Society for Industrial and Applied Mathematics APPROXIMATE GAUSS NEWTON METHODS FOR NONLINEAR LEAST SQUARES PROBLEMS S. GRATTON, A. S. LAWLESS, AND N. K.

### Introduction to Numerical Analysis

Introduction to Numerical Analysis S. Baskar and S. Sivaji Ganesh Department of Mathematics Indian Institute of Technology Bombay Powai, Mumbai 400 076. Introduction to Numerical Analysis Lecture Notes

### 1. Nonlinear Equations. This lecture note excerpted parts from Michael Heath and Max Gunzburger. f(x) = 0

Numerical Analysis 1 1. Nonlinear Equations This lecture note excerpted parts from Michael Heath and Max Gunzburger. Given function f, we seek value x for which where f : D R n R n is nonlinear. f(x) =

### Solving nonlinear equations

Chapter Solving nonlinear equations Bisection Introduction Linear equations are of the form: find x such that ax + b = 0 Proposition Let f be a real-valued function that is defined and continuous on a

### Numerical Methods for Large-Scale Nonlinear Systems

Numerical Methods for Large-Scale Nonlinear Systems Handouts by Ronald H.W. Hoppe following the monograph P. Deuflhard Newton Methods for Nonlinear Problems Springer, Berlin-Heidelberg-New York, 2004 Num.

### Chapter 3: Root Finding. September 26, 2005

Chapter 3: Root Finding September 26, 2005 Outline 1 Root Finding 2 3.1 The Bisection Method 3 3.2 Newton s Method: Derivation and Examples 4 3.3 How To Stop Newton s Method 5 3.4 Application: Division

### Unit 2: Solving Scalar Equations. Notes prepared by: Amos Ron, Yunpeng Li, Mark Cowlishaw, Steve Wright Instructor: Steve Wright

cs416: introduction to scientific computing 01/9/07 Unit : Solving Scalar Equations Notes prepared by: Amos Ron, Yunpeng Li, Mark Cowlishaw, Steve Wright Instructor: Steve Wright 1 Introduction We now

### Numerical Methods. King Saud University

Numerical Methods King Saud University Aims In this lecture, we will... Introduce the topic of numerical methods Consider the Error analysis and sources of errors Introduction A numerical method which

### Math 471. Numerical methods Root-finding algorithms for nonlinear equations

Math 471. Numerical methods Root-finding algorithms for nonlinear equations overlap Section.1.5 of Bradie Our goal in this chapter is to find the root(s) for f(x) = 0..1 Bisection Method Intermediate value

### SYSTEMS OF NONLINEAR EQUATIONS

SYSTEMS OF NONLINEAR EQUATIONS Widely used in the mathematical modeling of real world phenomena. We introduce some numerical methods for their solution. For better intuition, we examine systems of two

### 5.6 Logarithmic and Exponential Equations

SECTION 5.6 Logarithmic and Exponential Equations 305 5.6 Logarithmic and Exponential Equations PREPARING FOR THIS SECTION Before getting started, review the following: Solving Equations Using a Graphing

### MA 8019: Numerical Analysis I Solution of Nonlinear Equations

MA 8019: Numerical Analysis I Solution of Nonlinear Equations Suh-Yuh Yang ( 楊肅煜 ) Department of Mathematics, National Central University Jhongli District, Taoyuan City 32001, Taiwan syyang@math.ncu.edu.tw

### MATH 2053 Calculus I Review for the Final Exam

MATH 05 Calculus I Review for the Final Exam (x+ x) 9 x 9 1. Find the limit: lim x 0. x. Find the limit: lim x + x x (x ).. Find lim x (x 5) = L, find such that f(x) L < 0.01 whenever 0 < x

### Linear Algebra and Matrix Inversion

Jim Lambers MAT 46/56 Spring Semester 29- Lecture 2 Notes These notes correspond to Section 63 in the text Linear Algebra and Matrix Inversion Vector Spaces and Linear Transformations Matrices are much

### Autonomous Systems and Stability

LECTURE 8 Autonomous Systems and Stability An autonomous system is a system of ordinary differential equations of the form 1 1 ( 1 ) 2 2 ( 1 ). ( 1 ) or, in vector notation, x 0 F (x) That is to say, an

### C.7. Numerical series. Pag. 147 Proof of the converging criteria for series. Theorem 5.29 (Comparison test) Let a k and b k be positive-term series

C.7 Numerical series Pag. 147 Proof of the converging criteria for series Theorem 5.29 (Comparison test) Let and be positive-term series such that 0, for any k 0. i) If the series converges, then also

### 1 Solutions to selected problems

Solutions to selected problems Section., #a,c,d. a. p x = n for i = n : 0 p x = xp x + i end b. z = x, y = x for i = : n y = y + x i z = zy end c. y = (t x ), p t = a for i = : n y = y(t x i ) p t = p

### COURSE Iterative methods for solving linear systems

COURSE 0 4.3. Iterative methods for solving linear systems Because of round-off errors, direct methods become less efficient than iterative methods for large systems (>00 000 variables). An iterative scheme

### Jim Lambers MAT 772 Fall Semester Lecture 21 Notes

Jim Lambers MAT 772 Fall Semester 21-11 Lecture 21 Notes These notes correspond to Sections 12.6, 12.7 and 12.8 in the text. Multistep Methods All of the numerical methods that we have developed for solving

### Nonlinear equations and optimization

Notes for 2017-03-29 Nonlinear equations and optimization For the next month or so, we will be discussing methods for solving nonlinear systems of equations and multivariate optimization problems. We will

### 1. Method 1: bisection. The bisection methods starts from two points a 0 and b 0 such that

Chapter 4 Nonlinear equations 4.1 Root finding Consider the problem of solving any nonlinear relation g(x) = h(x) in the real variable x. We rephrase this problem as one of finding the zero (root) of a

### INTRODUCTION, FOUNDATIONS

1 INTRODUCTION, FOUNDATIONS ELM1222 Numerical Analysis Some of the contents are adopted from Laurene V. Fausett, Applied Numerical Analysis using MATLAB. Prentice Hall Inc., 1999 2 Today s lecture Information

### Continuous functions that are nowhere differentiable

Continuous functions that are nowhere differentiable S. Kesavan The Institute of Mathematical Sciences, CIT Campus, Taramani, Chennai - 600113. e-mail: kesh @imsc.res.in Abstract It is shown that the existence

### Series Solutions. 8.1 Taylor Polynomials

8 Series Solutions 8.1 Taylor Polynomials Polynomial functions, as we have seen, are well behaved. They are continuous everywhere, and have continuous derivatives of all orders everywhere. It also turns

### MATH ASSIGNMENT 03 SOLUTIONS

MATH444.0 ASSIGNMENT 03 SOLUTIONS 4.3 Newton s method can be used to compute reciprocals, without division. To compute /R, let fx) = x R so that fx) = 0 when x = /R. Write down the Newton iteration for

### Completion Date: Monday February 11, 2008

MATH 4 (R) Winter 8 Intermediate Calculus I Solutions to Problem Set #4 Completion Date: Monday February, 8 Department of Mathematical and Statistical Sciences University of Alberta Question. [Sec..9,

### Introductory Numerical Analysis

Introductory Numerical Analysis Lecture Notes December 16, 017 Contents 1 Introduction to 1 11 Floating Point Numbers 1 1 Computational Errors 13 Algorithm 3 14 Calculus Review 3 Root Finding 5 1 Bisection

### On sequential optimality conditions for constrained optimization. José Mario Martínez martinez

On sequential optimality conditions for constrained optimization José Mario Martínez www.ime.unicamp.br/ martinez UNICAMP, Brazil 2011 Collaborators This talk is based in joint papers with Roberto Andreani

### Roundoff Analysis of Gaussian Elimination

Jim Lambers MAT 60 Summer Session 2009-0 Lecture 5 Notes These notes correspond to Sections 33 and 34 in the text Roundoff Analysis of Gaussian Elimination In this section, we will perform a detailed error

### Numerical Methods for Differential Equations Mathematical and Computational Tools

Numerical Methods for Differential Equations Mathematical and Computational Tools Gustaf Söderlind Numerical Analysis, Lund University Contents V4.16 Part 1. Vector norms, matrix norms and logarithmic

### Chapter 4. Solution of Non-linear Equation. Module No. 1. Newton s Method to Solve Transcendental Equation

Numerical Analysis by Dr. Anita Pal Assistant Professor Department of Mathematics National Institute of Technology Durgapur Durgapur-713209 email: anita.buie@gmail.com 1 . Chapter 4 Solution of Non-linear

### Analysis II: The Implicit and Inverse Function Theorems

Analysis II: The Implicit and Inverse Function Theorems Jesse Ratzkin November 17, 2009 Let f : R n R m be C 1. When is the zero set Z = {x R n : f(x) = 0} the graph of another function? When is Z nicely

### 11.10a Taylor and Maclaurin Series

11.10a 1 11.10a Taylor and Maclaurin Series Let y = f(x) be a differentiable function at x = a. In first semester calculus we saw that (1) f(x) f(a)+f (a)(x a), for all x near a The right-hand side of

### Optimization. Charles J. Geyer School of Statistics University of Minnesota. Stat 8054 Lecture Notes

Optimization Charles J. Geyer School of Statistics University of Minnesota Stat 8054 Lecture Notes 1 One-Dimensional Optimization Look at a graph. Grid search. 2 One-Dimensional Zero Finding Zero finding

### Numerical Methods

Numerical Methods 263-2014 Prof. M. K. Banda Botany Building: 2-10. Prof. M. K. Banda (Tuks) WTW263 Semester II 1 / 18 Topic 1: Solving Nonlinear Equations Prof. M. K. Banda (Tuks) WTW263 Semester II 2

### Variable. Peter W. White Fall 2018 / Numerical Analysis. Department of Mathematics Tarleton State University

Newton s Iterative s Peter W. White white@tarleton.edu Department of Mathematics Tarleton State University Fall 2018 / Numerical Analysis Overview Newton s Iterative s Newton s Iterative s Newton s Iterative

### Topic 5: The Difference Equation

Topic 5: The Difference Equation Yulei Luo Economics, HKU October 30, 2017 Luo, Y. (Economics, HKU) ME October 30, 2017 1 / 42 Discrete-time, Differences, and Difference Equations When time is taken to

### Euler s Method, cont d

Jim Lambers MAT 461/561 Spring Semester 009-10 Lecture 3 Notes These notes correspond to Sections 5. and 5.4 in the text. Euler s Method, cont d We conclude our discussion of Euler s method with an example

### Applications of the Maximum Principle

Jim Lambers MAT 606 Spring Semester 2015-16 Lecture 26 Notes These notes correspond to Sections 7.4-7.6 in the text. Applications of the Maximum Principle The maximum principle for Laplace s equation is

### CS 323: Numerical Analysis and Computing

CS 323: Numerical Analysis and Computing MIDTERM #2 Instructions: This is an open notes exam, i.e., you are allowed to consult any textbook, your class notes, homeworks, or any of the handouts from us.

### CLASS NOTES Models, Algorithms and Data: Introduction to computing 2018

CLASS NOTES Models, Algorithms and Data: Introduction to computing 2018 Petros Koumoutsakos, Jens Honore Walther (Last update: April 16, 2018) IMPORTANT DISCLAIMERS 1. REFERENCES: Much of the material