Applied Computational Economics Workshop. Part 3: Nonlinear Equations
|
|
- Giles McKinney
- 6 years ago
- Views:
Transcription
1 Applied Computational Economics Workshop Part 3: Nonlinear Equations 1
2 Overview Introduction Function iteration Newton s method Quasi-Newton methods Practical example Practical issues 2
3 Introduction Nonlinear equations take one of two forms Root-finding problem Given a function : R R, compute an -vector, called a root of, such that Fixed-point problem ( )=0 Given a function : R R, compute an -vector, called a fixed-point of, such that ( )= 3
4 Introduction The two forms of nonlinear equations are equivalent A root of is a fixed-point of A fixed-point of is root of 4
5 Introduction Nonlinear equations arise naturally in economics multi-commodity market equilibrium models multi-person static game models unconstrained optimization models Nonlinear equations also arise indirectly when numerically solving economic models involving functional equations dynamic optimization models rational expectations models arbitrage pricing models 5
6 Function Iteration Function iteration is an algorithm for computing a fixed-point of a function Guess an initial value and successively form the iterates until the iterates converge 6
7 Computing Fixed-Point of Iteration Using Function 7
8 Function Iteration: Example To compute the fixed-point of ( ) = employs the iteration rule +0.2using function iteration, one = ( ) = +0.2 In MATLAB x = 0.4; for it=1:50 xold = x x = sqrt(x+0.2); if abs(x-xold)<1.e-10, break, end end After 28 iterations, x converges to
9 Function Iteration When is function iteration guaranteed to converge? By the Contraction Mapping Theorem, if for some, for all and, then possesses an unique fixed-point and function iteration will converge to it from any initial value 9
10 Newton s Method Newton s method is an algorithm for computing a root of a function Guess an initial value and successively form the iterates until the iterates converge 10
11 Newton s Method: Example To compute the root of ( ) = 2using Newton s method, one employs the iteration rule = = 2 4 In MATLAB x = 2.3; for it=1:50 s = -(x^4-2)/(4*x^3); x = x + s; if abs(s)<1.e-10, break, end end After 8 iterations, converges to
12 Newton s Method Newton s method employs a strategy of successive linearization The strategy calls for the nonlinear function to be approximated by a sequence of linear functions whose roots are easily computed and, ideally, converge to the root of In particular, the +1 iterate = ( ) is the root of the Taylor linear approximation of around the preceding iterate, viz. ( )+ ( )( ) 12
13 Computing Root of Method Using Newton s 13
14 Newton s Method When is Newton s method guaranteed to converge? In theory, Newton s method will converge if the initial value is close to a root of at which is non-singular Theory, however, provides no practical definition of close Moreover, in practice, Newton s method will fail to converge to if is ill-conditioned, i.e. nearly singular 14
15 Quasi-Newton Methods Quasi-Newton methods replace the Jacobian in Newton s method with an estimate that is easier to compute Specifically, quasi-newton methods use an iteration rule where is an estimate of the Jacobian 15
16 Secant Method The quasi-newton method for univariate root-finding problems is called the secant method The secant method replaces the derivative in Newton s method with the estimate which leads to the iteration rule, = ( ) The secant method is so called because it approximates the function using secant lines drawn through successive pairs of points on its graph 16
17 Computing Root of Using Secant Method 17
18 Broyden s Method Broyden s method is the most popular multivariate generalization of the univariate secant method Broyden s method replaces the Jacobian in Newton s method with an estimate that is updated by making the smallest possible change that is consistent with the secant condition This yields a complicated iteration rule that is omitted here, but which may found in many textbooks, including Miranda and Fackler (2002) 18
19 Quasi-Newton Methods When are quasi-newton methods guaranteed to converge? In theory, a quasi-newton (e.g. Broyden s) method will converge if the initial value is close to a root of at which the Jacobian is non-singular and if the initial Jacobian estimate is close to Theory, however, provides no practical definition of close Moreover, in practice, Broyden s method will fail to converge to if the Jacobian estimates become illconditioned, i.e. nearly singular 19
20 Numerical Examples The CompEcon Toolbox provides two utilities for computing the root of function Utility newton uses Newton s method Utility broyden uses Broyden s method 20
21 Numerical Examples The calling protocol for newton is Input [x,fval] = newton(f,x,varargin) f x varargin Output x fval function of form [fval,fjac]=f(x,varargin) where fval and fjac are the function value and the Jacobian value at x, respectively initial guess for a root of f additional arguments for f (optional) a root of f value of f at x 21
22 Numerical Examples The calling protocol for broyden is Input f x [x,fval] = broyden(f,x,varargin) function of form fval=f(x,varargin) where fval is the function value at x initial guess for a root of f varargin additional arguments for f (optional) Output x fval a root of f value of f at x 22
23 Numerical Examples Let us compute the root of a function : R R given by ( ) = exp( ) 2 To use broyden, first compose a MATLAB function function fval = f(x) fval = [x(2)*exp(x(1))-2*x(2);x(1)*x(2)-x(2)^3]; and save it as f.m Then, on the MATLAB command screen, execute x = [1.0;0.5]; optset('broyden','showiters',1); [x,fval] = broyden(@f,x) After 11 iterations, this produces x=(0.6931,0.8326) 23
24 Numerical Examples To use newton, edit f.m so that it computes the analytic Jacobian, viz. function [fval,fjac] = f(x) fval = [x(2)*exp(x(1))-2*x(2);x(1)*x(2)-x(2)^3]; fjac = [x(2)*exp(x(1)) exp(x(1)) 2; x(2) x(1)-3*x(2)^2]; On the MATLAB command screen, execute x = [1.0;0.5]; checkjac(@f,x) This checks the internal consistency of your function file by comparing the analytic derivative to a numerical derivative 24
25 Numerical Examples Then, on the MATLAB command screen, execute x = [1.0;0.5]; optset('newton','showiters',1); [x,fval] = newton(@f,x) After 5 iterations, this produces x=(0.6931,0.8326) 25
26 Convergence Path for Newton and Broyden Methods 26
27 Numerical Examples (Cournot Duopoly) Consider a market with two firms producing the same good Firm s total cost of production is a function of the quantity it produces. The market clearing price is a function of the total quantity produced by both firms 27
28 Numerical Examples (Cournot Duopoly) Firm chooses production so as to maximize its profit taking the other firm s output as given Thus, in equilibrium, for 28
29 Numerical Examples (Cournot Duopoly) Suppose,, and To compute the equilibrium using Broyden s method, open the file exampcournot.m and uncomment alpha = 0.6; beta = [ ]; P Pder f P(q)+Pder(q)*q(2)-beta(2)*q(2)]; Here, f computes the marginal profits of both firms 29
30 Numerical Examples (Cournot Duopoly) Then type the following and execute q = [0.2;0.2]; optset('broyden','showiters',1); q = broyden(f,q) After 10 iterations, converges to 30
31 Practical Issues Failure to converge Execution speed Choosing a solution method 31
32 Failure to Converge In practice, nonlinear equation algorithms can fail to converge for various reasons human error bad initial value ill-conditioning 32
33 Failure to Converge: Human Error Math errors: analyst incorrectly derives function or Jacobian Coding errors: analyst incorrectly codes function or Jacobian Coding errors are less likely with function iteration and Broyden s method because they are derivative-free 33
34 Failure to Converge: Bad Initial Value Nonlinear equation algorithms require initial values If initial value is far from desired root, algorithm can diverge or converge to a wrong root Theory provides no guidance on how to specify initial value Analyst must supply good guess from knowledge of model If algorithm diverges, try another initial value Well behaved functions are more robust to initial value Poorly behaved functions are more sensitive to initial value 34
35 Failure to Converge: Ill-Conditioning Computing iteration step in Newton s and Broyden s methods requires solution to a linear equation involving the Jacobian or its estimate If the Jacobian or estimate is ill-conditioned near solution, the iteration step cannot be accurately computed Very little can be done about this It arises more often than we like 35
36 Execution Speed Two factors determine the speed with which a properly coded and initiated algorithm will converge to a solution asymptotic rate of convergence computational effort per iteration 36
37 Execution Speed The number of iterations required for a properly coded and initiated algorithm to converge is closely tied to its theoretical asymptotic rate of convergence Function iteration converges at a linear rate relatively slow Broyden s method converges at a superlinear rate relatively fast Newton s method converges at a quadratic rate extremely fast 37
38 Rate of Convergence When Computing Fixed-Point of Using Various Methods, 38
39 Execution Speed However, algorithms differ in computations per iteration function iteration requires a function evaluation Broyden s method additionally requires a linear solve Newton s method additionally requires a Jacobian evaluation Thus, a faster rate of convergence typically can be achieved only by investing greater computational effort per iteration The optimal tradeoff between rate of convergence and computational effort per iteration varies across applications 39
40 Choosing a Solution Method Concerns about execution speed, however, are exaggerated The time that must be invested by the analyst to write and debug code typically is far more important Derivative-free methods such as function iteration and Broyden s method can be implemented faster in real time and more reliably than Newton s method Newton s method should be used only if dimension is low or derivatives are simple other methods have failed to converge, or general purpose, re-usable code is needed 40
Solving Nonlinear Equations
Solving Nonlinear Equations Jijian Fan Department of Economics University of California, Santa Cruz Oct 13 2014 Overview NUMERICALLY solving nonlinear equation Four methods Bisection Function iteration
More informationMATH 3795 Lecture 13. Numerical Solution of Nonlinear Equations in R N.
MATH 3795 Lecture 13. Numerical Solution of Nonlinear Equations in R N. Dmitriy Leykekhman Fall 2008 Goals Learn about different methods for the solution of F (x) = 0, their advantages and disadvantages.
More informationNonlinear equations can take one of two forms. In the nonlinear root nding n n
Chapter 3 Nonlinear Equations Nonlinear equations can take one of two forms. In the nonlinear root nding n n problem, a function f from < to < is given, and one must compute an n-vector x, called a root
More informationLecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 5. Nonlinear Equations
Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T Heath Chapter 5 Nonlinear Equations Copyright c 2001 Reproduction permitted only for noncommercial, educational
More informationCS 450 Numerical Analysis. Chapter 5: Nonlinear Equations
Lecture slides based on the textbook Scientific Computing: An Introductory Survey by Michael T. Heath, copyright c 2018 by the Society for Industrial and Applied Mathematics. http://www.siam.org/books/cl80
More informationKey Concepts: Economic Computation, Part III
Key Concepts: Economic Computation, Part III Brent Hickman Summer, 8 1 Using Newton s Method to Find Roots of Real- Valued Functions The intuition behind Newton s method is that finding zeros of non-linear
More informationNumerical Methods I Solving Nonlinear Equations
Numerical Methods I Solving Nonlinear Equations Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 16th, 2014 A. Donev (Courant Institute)
More informationOutline. Scientific Computing: An Introductory Survey. Nonlinear Equations. Nonlinear Equations. Examples: Nonlinear Equations
Methods for Systems of Methods for Systems of Outline Scientific Computing: An Introductory Survey Chapter 5 1 Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 5 Nonlinear Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More informationCS 323: Numerical Analysis and Computing
CS 323: Numerical Analysis and Computing MIDTERM #2 Instructions: This is an open notes exam, i.e., you are allowed to consult any textbook, your class notes, homeworks, or any of the handouts from us.
More informationEAD 115. Numerical Solution of Engineering and Scientific Problems. David M. Rocke Department of Applied Science
EAD 115 Numerical Solution of Engineering and Scientific Problems David M. Rocke Department of Applied Science Multidimensional Unconstrained Optimization Suppose we have a function f() of more than one
More information5 Quasi-Newton Methods
Unconstrained Convex Optimization 26 5 Quasi-Newton Methods If the Hessian is unavailable... Notation: H = Hessian matrix. B is the approximation of H. C is the approximation of H 1. Problem: Solve min
More informationx 2 x n r n J(x + t(x x ))(x x )dt. For warming-up we start with methods for solving a single equation of one variable.
Maria Cameron 1. Fixed point methods for solving nonlinear equations We address the problem of solving an equation of the form (1) r(x) = 0, where F (x) : R n R n is a vector-function. Eq. (1) can be written
More informationLine Search Methods. Shefali Kulkarni-Thaker
1 BISECTION METHOD Line Search Methods Shefali Kulkarni-Thaker Consider the following unconstrained optimization problem min f(x) x R Any optimization algorithm starts by an initial point x 0 and performs
More informationSolving non-linear systems of equations
Solving non-linear systems of equations Felix Kubler 1 1 DBF, University of Zurich and Swiss Finance Institute October 7, 2017 Felix Kubler Comp.Econ. Gerzensee, Ch2 October 7, 2017 1 / 38 The problem
More informationNumerical Methods in Informatics
Numerical Methods in Informatics Lecture 2, 30.09.2016: Nonlinear Equations in One Variable http://www.math.uzh.ch/binf4232 Tulin Kaman Institute of Mathematics, University of Zurich E-mail: tulin.kaman@math.uzh.ch
More informationNumerical solutions of nonlinear systems of equations
Numerical solutions of nonlinear systems of equations Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan E-mail: min@math.ntnu.edu.tw August 28, 2011 Outline 1 Fixed points
More informationLecture 7: Numerical Tools
Lecture 7: Numerical Tools Fatih Guvenen January 10, 2016 Fatih Guvenen Lecture 7: Numerical Tools January 10, 2016 1 / 18 Overview Three Steps: V (k, z) =max c,k 0 apple u(c)+ Z c + k 0 =(1 + r)k + z
More informationCS 323: Numerical Analysis and Computing
CS 323: Numerical Analysis and Computing MIDTERM #2 Instructions: This is an open notes exam, i.e., you are allowed to consult any textbook, your class notes, homeworks, or any of the handouts from us.
More informationNumerical Analysis Fall. Roots: Open Methods
Numerical Analysis 2015 Fall Roots: Open Methods Open Methods Open methods differ from bracketing methods, in that they require only a single starting value or two starting values that do not necessarily
More informationLecture V. Numerical Optimization
Lecture V Numerical Optimization Gianluca Violante New York University Quantitative Macroeconomics G. Violante, Numerical Optimization p. 1 /19 Isomorphism I We describe minimization problems: to maximize
More informationMath 411 Preliminaries
Math 411 Preliminaries Provide a list of preliminary vocabulary and concepts Preliminary Basic Netwon s method, Taylor series expansion (for single and multiple variables), Eigenvalue, Eigenvector, Vector
More informationMATH 350: Introduction to Computational Mathematics
MATH 350: Introduction to Computational Mathematics Chapter IV: Locating Roots of Equations Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Spring 2011 fasshauer@iit.edu
More informationMATH 350: Introduction to Computational Mathematics
MATH 350: Introduction to Computational Mathematics Chapter IV: Locating Roots of Equations Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Spring 2011 fasshauer@iit.edu
More informationLecture 8. Root finding II
1 Introduction Lecture 8 Root finding II In the previous lecture we considered the bisection root-bracketing algorithm. It requires only that the function be continuous and that we have a root bracketed
More informationECS550NFB Introduction to Numerical Methods using Matlab Day 2
ECS550NFB Introduction to Numerical Methods using Matlab Day 2 Lukas Laffers lukas.laffers@umb.sk Department of Mathematics, University of Matej Bel June 9, 2015 Today Root-finding: find x that solves
More informationChapter 1. Root Finding Methods. 1.1 Bisection method
Chapter 1 Root Finding Methods We begin by considering numerical solutions to the problem f(x) = 0 (1.1) Although the problem above is simple to state it is not always easy to solve analytically. This
More informationUnconstrained Multivariate Optimization
Unconstrained Multivariate Optimization Multivariate optimization means optimization of a scalar function of a several variables: and has the general form: y = () min ( ) where () is a nonlinear scalar-valued
More information17 Solution of Nonlinear Systems
17 Solution of Nonlinear Systems We now discuss the solution of systems of nonlinear equations. An important ingredient will be the multivariate Taylor theorem. Theorem 17.1 Let D = {x 1, x 2,..., x m
More informationApplied Mathematics 205. Unit I: Data Fitting. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit I: Data Fitting Lecturer: Dr. David Knezevic Unit I: Data Fitting Chapter I.4: Nonlinear Least Squares 2 / 25 Nonlinear Least Squares So far we have looked at finding a best
More informationMath 409/509 (Spring 2011)
Math 409/509 (Spring 2011) Instructor: Emre Mengi Study Guide for Homework 2 This homework concerns the root-finding problem and line-search algorithms for unconstrained optimization. Please don t hesitate
More informationQueens College, CUNY, Department of Computer Science Numerical Methods CSCI 361 / 761 Spring 2018 Instructor: Dr. Sateesh Mane.
Queens College, CUNY, Department of Computer Science Numerical Methods CSCI 361 / 761 Spring 2018 Instructor: Dr. Sateesh Mane c Sateesh R. Mane 2018 3 Lecture 3 3.1 General remarks March 4, 2018 This
More informationAdvanced Microeconomics
Advanced Microeconomics Leonardo Felli EC441: Room D.106, Z.332, D.109 Lecture 8 bis: 24 November 2004 Monopoly Consider now the pricing behavior of a profit maximizing monopolist: a firm that is the only
More information(One Dimension) Problem: for a function f(x), find x 0 such that f(x 0 ) = 0. f(x)
Solving Nonlinear Equations & Optimization One Dimension Problem: or a unction, ind 0 such that 0 = 0. 0 One Root: The Bisection Method This one s guaranteed to converge at least to a singularity, i not
More information, b = 0. (2) 1 2 The eigenvectors of A corresponding to the eigenvalues λ 1 = 1, λ 2 = 3 are
Quadratic forms We consider the quadratic function f : R 2 R defined by f(x) = 2 xt Ax b T x with x = (x, x 2 ) T, () where A R 2 2 is symmetric and b R 2. We will see that, depending on the eigenvalues
More informationLine Search Algorithms
Lab 1 Line Search Algorithms Investigate various Line-Search algorithms for numerical opti- Lab Objective: mization. Overview of Line Search Algorithms Imagine you are out hiking on a mountain, and you
More information1 The best of all possible worlds
Notes for 2017-03-18 1 The best of all possible worlds Last time, we discussed three methods of solving f(x) = 0: Newton, modified Newton, and bisection. Newton is potentially faster than bisection; bisection
More information1. Method 1: bisection. The bisection methods starts from two points a 0 and b 0 such that
Chapter 4 Nonlinear equations 4.1 Root finding Consider the problem of solving any nonlinear relation g(x) = h(x) in the real variable x. We rephrase this problem as one of finding the zero (root) of a
More informationChapter 3: Root Finding. September 26, 2005
Chapter 3: Root Finding September 26, 2005 Outline 1 Root Finding 2 3.1 The Bisection Method 3 3.2 Newton s Method: Derivation and Examples 4 3.3 How To Stop Newton s Method 5 3.4 Application: Division
More informationSOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS BISECTION METHOD
BISECTION METHOD If a function f(x) is continuous between a and b, and f(a) and f(b) are of opposite signs, then there exists at least one root between a and b. It is shown graphically as, Let f a be negative
More informationTHE SECANT METHOD. q(x) = a 0 + a 1 x. with
THE SECANT METHOD Newton s method was based on using the line tangent to the curve of y = f (x), with the point of tangency (x 0, f (x 0 )). When x 0 α, the graph of the tangent line is approximately the
More informationLecture 5. September 4, 2018 Math/CS 471: Introduction to Scientific Computing University of New Mexico
Lecture 5 September 4, 2018 Math/CS 471: Introduction to Scientific Computing University of New Mexico 1 Review: Office hours at regularly scheduled times this week Tuesday: 9:30am-11am Wed: 2:30pm-4:00pm
More informationNon-polynomial Least-squares fitting
Applied Math 205 Last time: piecewise polynomial interpolation, least-squares fitting Today: underdetermined least squares, nonlinear least squares Homework 1 (and subsequent homeworks) have several parts
More informationOptimization 2. CS5240 Theoretical Foundations in Multimedia. Leow Wee Kheng
Optimization 2 CS5240 Theoretical Foundations in Multimedia Leow Wee Kheng Department of Computer Science School of Computing National University of Singapore Leow Wee Kheng (NUS) Optimization 2 1 / 38
More informationMATHEMATICS FOR COMPUTER VISION WEEK 8 OPTIMISATION PART 2. Dr Fabio Cuzzolin MSc in Computer Vision Oxford Brookes University Year
MATHEMATICS FOR COMPUTER VISION WEEK 8 OPTIMISATION PART 2 1 Dr Fabio Cuzzolin MSc in Computer Vision Oxford Brookes University Year 2013-14 OUTLINE OF WEEK 8 topics: quadratic optimisation, least squares,
More informationGoals for This Lecture:
Goals for This Lecture: Learn the Newton-Raphson method for finding real roots of real functions Learn the Bisection method for finding real roots of a real function Look at efficient implementations of
More informationSolution of Nonlinear Equations
Solution of Nonlinear Equations (Com S 477/577 Notes) Yan-Bin Jia Sep 14, 017 One of the most frequently occurring problems in scientific work is to find the roots of equations of the form f(x) = 0. (1)
More informationOptimization: Nonlinear Optimization without Constraints. Nonlinear Optimization without Constraints 1 / 23
Optimization: Nonlinear Optimization without Constraints Nonlinear Optimization without Constraints 1 / 23 Nonlinear optimization without constraints Unconstrained minimization min x f(x) where f(x) is
More informationMaria Cameron. f(x) = 1 n
Maria Cameron 1. Local algorithms for solving nonlinear equations Here we discuss local methods for nonlinear equations r(x) =. These methods are Newton, inexact Newton and quasi-newton. We will show that
More informationBindel, Spring 2016 Numerical Analysis (CS 4220) Notes for
Life beyond Newton Notes for 2016-04-08 Newton s method has many attractive properties, particularly when we combine it with a globalization strategy. Unfortunately, Newton steps are not cheap. At each
More informationShiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 3. Gradient Method
Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 3 Gradient Method Shiqian Ma, MAT-258A: Numerical Optimization 2 3.1. Gradient method Classical gradient method: to minimize a differentiable convex
More informationMotivation: We have already seen an example of a system of nonlinear equations when we studied Gaussian integration (p.8 of integration notes)
AMSC/CMSC 460 Computational Methods, Fall 2007 UNIT 5: Nonlinear Equations Dianne P. O Leary c 2001, 2002, 2007 Solving Nonlinear Equations and Optimization Problems Read Chapter 8. Skip Section 8.1.1.
More informationUnconstrained optimization
Chapter 4 Unconstrained optimization An unconstrained optimization problem takes the form min x Rnf(x) (4.1) for a target functional (also called objective function) f : R n R. In this chapter and throughout
More informationCHEE 222: PROCESS DYNAMICS AND NUMERICAL METHODS
CHEE 222: PROCESS DYNAMICS AND NUMERICAL METHODS Winter 2017 Implementation of Numerical Methods via MATLAB Instructor: Xiang Li 1 Outline 1. Introduction - Command, script and function - MATLAB function
More informationSimple Iteration, cont d
Jim Lambers MAT 772 Fall Semester 2010-11 Lecture 2 Notes These notes correspond to Section 1.2 in the text. Simple Iteration, cont d In general, nonlinear equations cannot be solved in a finite sequence
More informationTvestlanka Karagyozova University of Connecticut
September, 005 CALCULUS REVIEW Tvestlanka Karagyozova University of Connecticut. FUNCTIONS.. Definition: A function f is a rule that associates each value of one variable with one and only one value of
More informationOctober 16, 2018 Notes on Cournot. 1. Teaching Cournot Equilibrium
October 1, 2018 Notes on Cournot 1. Teaching Cournot Equilibrium Typically Cournot equilibrium is taught with identical zero or constant-mc cost functions for the two firms, because that is simpler. I
More informationOptimization. Totally not complete this is...don't use it yet...
Optimization Totally not complete this is...don't use it yet... Bisection? Doing a root method is akin to doing a optimization method, but bi-section would not be an effective method - can detect sign
More informationBindel, Fall 2011 Intro to Scientific Computing (CS 3220) Week 6: Monday, Mar 7. e k+1 = 1 f (ξ k ) 2 f (x k ) e2 k.
Problem du jour Week 6: Monday, Mar 7 Show that for any initial guess x 0 > 0, Newton iteration on f(x) = x 2 a produces a decreasing sequence x 1 x 2... x n a. What is the rate of convergence if a = 0?
More informationScientific Computing: Optimization
Scientific Computing: Optimization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 March 8th, 2011 A. Donev (Courant Institute) Lecture
More informationA nonlinear equation is any equation of the form. f(x) = 0. A nonlinear equation can have any number of solutions (finite, countable, uncountable)
Nonlinear equations Definition A nonlinear equation is any equation of the form where f is a nonlinear function. Nonlinear equations x 2 + x + 1 = 0 (f : R R) f(x) = 0 (x cos y, 2y sin x) = (0, 0) (f :
More informationMath 551 Homework Assignment 3 Page 1 of 6
Math 551 Homework Assignment 3 Page 1 of 6 Name and section: ID number: E-mail: 1. Consider Newton s method for finding + α with α > 0 by finding the positive root of f(x) = x 2 α = 0. Assuming that x
More informationQuadratic function and equations Quadratic function/equations, supply, demand, market equilibrium
Exercises 8 Quadratic function and equations Quadratic function/equations, supply, demand, market equilibrium Objectives - know and understand the relation between a quadratic function and a quadratic
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted
More informationChapter 4 AD AS. O. Afonso, P. B. Vasconcelos. Computational Economics: a concise introduction
Chapter 4 AD AS O. Afonso, P. B. Vasconcelos Computational Economics: a concise introduction O. Afonso, P. B. Vasconcelos Computational Economics 1 / 32 Overview 1 Introduction 2 Economic model 3 Numerical
More informationNumerical Analysis: Solving Nonlinear Equations
Numerical Analysis: Solving Nonlinear Equations Mirko Navara http://cmp.felk.cvut.cz/ navara/ Center for Machine Perception, Department of Cybernetics, FEE, CTU Karlovo náměstí, building G, office 104a
More informationmin f(x). (2.1) Objectives consisting of a smooth convex term plus a nonconvex regularization term;
Chapter 2 Gradient Methods The gradient method forms the foundation of all of the schemes studied in this book. We will provide several complementary perspectives on this algorithm that highlight the many
More informationCLASS NOTES Models, Algorithms and Data: Introduction to computing 2018
CLASS NOTES Models, Algorithms and Data: Introduction to computing 2018 Petros Koumoutsakos, Jens Honore Walther (Last update: April 16, 2018) IMPORTANT DISCLAIMERS 1. REFERENCES: Much of the material
More informationChapter 6. Nonlinear Equations. 6.1 The Problem of Nonlinear Root-finding. 6.2 Rate of Convergence
Chapter 6 Nonlinear Equations 6. The Problem of Nonlinear Root-finding In this module we consider the problem of using numerical techniques to find the roots of nonlinear equations, f () =. Initially we
More informationInvestigating Limits in MATLAB
MTH229 Investigating Limits in MATLAB Project 5 Exercises NAME: SECTION: INSTRUCTOR: Exercise 1: Use the graphical approach to find the following right limit of f(x) = x x, x > 0 lim x 0 + xx What is the
More informationNumerical Solution of f(x) = 0
Numerical Solution of f(x) = 0 Gerald W. Recktenwald Department of Mechanical Engineering Portland State University gerry@pdx.edu ME 350: Finding roots of f(x) = 0 Overview Topics covered in these slides
More informationOligopoly Theory 2 Bertrand Market Games
1/10 Oligopoly Theory 2 Bertrand Market Games May 4, 2014 2/10 Outline 1 Bertrand Market Game 2 Bertrand Paradox 3 Asymmetric Firms 3/10 Bertrand Duopoly Market Game Discontinuous Payoff Functions (1 p
More information6.254 : Game Theory with Engineering Applications Lecture 7: Supermodular Games
6.254 : Game Theory with Engineering Applications Lecture 7: Asu Ozdaglar MIT February 25, 2010 1 Introduction Outline Uniqueness of a Pure Nash Equilibrium for Continuous Games Reading: Rosen J.B., Existence
More informationQuasi-Newton Methods
Newton s Method Pros and Cons Quasi-Newton Methods MA 348 Kurt Bryan Newton s method has some very nice properties: It s extremely fast, at least once it gets near the minimum, and with the simple modifications
More informationFixed Point Theorems
Fixed Point Theorems Definition: Let X be a set and let f : X X be a function that maps X into itself. (Such a function is often called an operator, a transformation, or a transform on X, and the notation
More informationChapter 4. Unconstrained optimization
Chapter 4. Unconstrained optimization Version: 28-10-2012 Material: (for details see) Chapter 11 in [FKS] (pp.251-276) A reference e.g. L.11.2 refers to the corresponding Lemma in the book [FKS] PDF-file
More informationLecture 15. Dynamic Stochastic General Equilibrium Model. Randall Romero Aguilar, PhD I Semestre 2017 Last updated: July 3, 2017
Lecture 15 Dynamic Stochastic General Equilibrium Model Randall Romero Aguilar, PhD I Semestre 2017 Last updated: July 3, 2017 Universidad de Costa Rica EC3201 - Teoría Macroeconómica 2 Table of contents
More informationNumerical Study of Some Iterative Methods for Solving Nonlinear Equations
International Journal of Engineering Science Invention ISSN (Online): 2319 6734, ISSN (Print): 2319 6726 Volume 5 Issue 2 February 2016 PP.0110 Numerical Study of Some Iterative Methods for Solving Nonlinear
More informationReview for Exam 2 Ben Wang and Mark Styczynski
Review for Exam Ben Wang and Mark Styczynski This is a rough approximation of what we went over in the review session. This is actually more detailed in portions than what we went over. Also, please note
More informationOptimization II: Unconstrained Multivariable
Optimization II: Unconstrained Multivariable CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Justin Solomon CS 205A: Mathematical Methods Optimization II: Unconstrained Multivariable 1
More informationSolving Dynamic Games with Newton s Method
Michael Ferris 1 Kenneth L. Judd 2 Karl Schmedders 3 1 Department of Computer Science, University of Wisconsin at Madison 2 Hoover Institution, Stanford University 3 Institute for Operations Research,
More informationNumerical Methods Lecture 3
Numerical Methods Lecture 3 Nonlinear Equations by Pavel Ludvík Introduction Definition (Root or zero of a function) A root (or a zero) of a function f is a solution of an equation f (x) = 0. We learn
More informationINTRODUCTION TO NUMERICAL ANALYSIS
INTRODUCTION TO NUMERICAL ANALYSIS Cho, Hyoung Kyu Department of Nuclear Engineering Seoul National University 3. SOLVING NONLINEAR EQUATIONS 3.1 Background 3.2 Estimation of errors in numerical solutions
More informationDISCRETE-TIME DYNAMICS OF AN
Chapter 1 DISCRETE-TIME DYNAMICS OF AN OLIGOPOLY MODEL WITH DIFFERENTIATED GOODS K. Andriopoulos, T. Bountis and S. Dimas * Department of Mathematics, University of Patras, Patras, GR-26500, Greece Abstract
More information446 CHAP. 8 NUMERICAL OPTIMIZATION. Newton's Search for a Minimum of f(x,y) Newton s Method
446 CHAP. 8 NUMERICAL OPTIMIZATION Newton's Search for a Minimum of f(xy) Newton s Method The quadratic approximation method of Section 8.1 generated a sequence of seconddegree Lagrange polynomials. It
More informationMath 4329: Numerical Analysis Chapter 03: Fixed Point Iteration and Ill behaving problems. Natasha S. Sharma, PhD
Why another root finding technique? iteration gives us the freedom to design our own root finding algorithm. The design of such algorithms is motivated by the need to improve the speed and accuracy of
More informationMA 8019: Numerical Analysis I Solution of Nonlinear Equations
MA 8019: Numerical Analysis I Solution of Nonlinear Equations Suh-Yuh Yang ( 楊肅煜 ) Department of Mathematics, National Central University Jhongli District, Taoyuan City 32001, Taiwan syyang@math.ncu.edu.tw
More informationCHAPTER 10 Zeros of Functions
CHAPTER 10 Zeros of Functions An important part of the maths syllabus in secondary school is equation solving. This is important for the simple reason that equations are important a wide range of problems
More information1 Functions and Graphs
1 Functions and Graphs 1.1 Functions Cartesian Coordinate System A Cartesian or rectangular coordinate system is formed by the intersection of a horizontal real number line, usually called the x axis,
More informationOutline. Scientific Computing: An Introductory Survey. Optimization. Optimization Problems. Examples: Optimization Problems
Outline Scientific Computing: An Introductory Survey Chapter 6 Optimization 1 Prof. Michael. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More informationNonlinear Optimization for Optimal Control
Nonlinear Optimization for Optimal Control Pieter Abbeel UC Berkeley EECS Many slides and figures adapted from Stephen Boyd [optional] Boyd and Vandenberghe, Convex Optimization, Chapters 9 11 [optional]
More informationStatistics 580 Optimization Methods
Statistics 580 Optimization Methods Introduction Let fx be a given real-valued function on R p. The general optimization problem is to find an x ɛ R p at which fx attain a maximum or a minimum. It is of
More informationAM 205: lecture 19. Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods Optimality Conditions: Equality Constrained Case As another example of equality
More informationMATH 3795 Lecture 12. Numerical Solution of Nonlinear Equations.
MATH 3795 Lecture 12. Numerical Solution of Nonlinear Equations. Dmitriy Leykekhman Fall 2008 Goals Learn about different methods for the solution of f(x) = 0, their advantages and disadvantages. Convergence
More information4 damped (modified) Newton methods
4 damped (modified) Newton methods 4.1 damped Newton method Exercise 4.1 Determine with the damped Newton method the unique real zero x of the real valued function of one variable f(x) = x 3 +x 2 using
More informationNonlinear dynamics in the Cournot duopoly game with heterogeneous players
arxiv:nlin/0210035v1 [nlin.cd] 16 Oct 2002 Nonlinear dynamics in the Cournot duopoly game with heterogeneous players H. N. Agiza and A. A. Elsadany Department of Mathematics, Faculty of Science Mansoura
More information2.4 - Convergence of the Newton Method and Modified Newton Method
2.4 - Convergence of the Newton Method and Modified Newton Method Consider the problem of finding the solution of the equation: for in Assume that is continuous and for in 1. Newton's Method: Suppose that
More informationCS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares
CS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares Robert Bridson October 29, 2008 1 Hessian Problems in Newton Last time we fixed one of plain Newton s problems by introducing line search
More informationGame theory and market power
Game theory and market power Josh Taylor Section 6.1.3, 6.3 in Convex Optimization of Power Systems. 1 Market weaknesses Recall Optimal power flow: minimize p,θ subject to λ i : χ ij 0 : f i (p i ) i p
More information