Roots of equations, minimization, numerical integration
|
|
- Georgia Morris
- 5 years ago
- Views:
Transcription
1 Roots of equations, minimization, numerical integration Alexander Khanov PHYS6260: Experimental Methods is HEP Oklahoma State University November 1, 2017
2 Roots of equations Find the roots solve equation f (x) = 0 find x, a < x < b, such that f (x) = 0 Multidimensional case: solve f (x) = 0 inside some region of x values In general not a trivial problem: usually not possible to be done analytically except for simple cases experimentalists hate to solve anything more complicated than a quadratic equation impractical if one needs to probe many different functions (e.g. polynomials of different degrees) even less practical if the function value is a result of some numerical calculation (no explicit formula) A. Khanov (PHYS6260, OSU) PHYS /1/17 2 / 13
3 Roots of equations (2) Before looking for roots, one needs to make sure they exist if f (x) is continuous and f (a) > 0, f (b) < 0, then it has a root between a and b This is not necessarily trivial multidimensional case: continuity in each argument does not imply multivariate continuity numerically computed functions: difficult to prove anything It s OK to find the root(s) approximately: instead of finding x such that f (x) = 0, find x such that x x < ε for a reasonably small ε whatever comes out of a computer usually has an uncertainty due to finite number of digits in the floating point number representation it s important not to set ε smaller than computing precision (true for any numerical method) A. Khanov (PHYS6260, OSU) PHYS /1/17 3 / 13
4 Simple methods to find the root Bisection: divide the current interval in half, pick the half where f (x) has opposite signs on its ends simple, the error on x decreases by a factor of 2 on each step, not very fast False position (variation of secant method): draw a straight line through (a, f (a)) and (b, f (b)), find the intersection with the y = 0 axis, proceed as in bisection Converges faster than bisection, but the interval width may or may not approach zero (if it doesn t, it s not clear when to stop) Both methods can be easily extended to multidimensional case A. Khanov (PHYS6260, OSU) PHYS /1/17 4 / 13
5 Roots and minimization The two problems can be solved in the same way: finding a root of f (x) is equivalent of minimizing f (x), or (f (x)) 2 if analytic behavior is desirable x is not analytic at 0 (no derivative, no Taylor series) Example: f (x) = x has a root x = 0 and (f (x)) 2 = x 2 has a minimum at x = 0 (equal to 0) The advantages of replacing root finding with minimum finding become significant in the multidimensional case A. Khanov (PHYS6260, OSU) PHYS /1/17 5 / 13
6 Newton s method Start from zero approximation x 0, proceed according to the formula x k+1 = x k f (x k) f (x k ) Advantages: converges really fast usually the number of correct digits doubles at every step this is called quadratic convergence (not true for roots with multiplicity > 1) cf. bisection method, a new correct (binary) digit is added at every step linear convergence Disadvantages: need to know the first derivative (may prove nontrivial for computed function); sometimes it does not converge at all f (x) = x 3 2x + 2, f (x) = 3x 2 2 x 0 = 0, f (0) = 2, f (0) = 2, x 1 = = 1 x 1 = 1, f (1) = 1, f (1) = 1, x 2 = = 0 A. Khanov (PHYS6260, OSU) PHYS /1/17 6 / 13
7 Newton s algorithm: multidimensional case Newton s method can be generalized to miltidimensional case of n functions f = f 1,..., f n of n variables x = x 1,..., x n Start from zero approximation x 0, proceed according to the formula x k+1 = x k J(x k ) 1 f(x k ) ( ) fi where J = is the jacobian x j A modification of the method (Newton-Gauss algorithm) can be used in case when the number of functions n is larger than the number of variables m, in this case the procedure x k+1 = x k (J(x k ) T J(x k )) 1 J(x k ) T f(x k ) finds the minimum of n (f i (x)) 2 non-linear least squares problem i=1 A. Khanov (PHYS6260, OSU) PHYS /1/17 7 / 13
8 Problems of root finding All algorithms are in trouble when there are many roots No really good advice except trying several points and seeing where it converges f(x)=x -x A. Khanov (PHYS6260, OSU) PHYS /1/17 8 / 13
9 Minimization Minimization of f (x) = maximization of f (x) (the same problem) Since at minimum f (x) = 0, minimization = finding the root of f (x) = 0 Gradient descent (multidimensional case): make steps proportional to the megative of the gradient A lot of other methods, more or less heuristic A. Khanov (PHYS6260, OSU) PHYS /1/17 9 / 13
10 MINUIT The only program you need to do the minimization The rules of the game are the following: 1 The function F (x) is assumed not to be known analytically, but is specified by giving its value at any point x. 2 The allowed values of the variable x may be restricted to a certain range, in which case one speaks of constrained minimization. Here only unconstrained problems are refered to. 3 In some cases additional information about the function F may be available, such as the numerical values of the derivative F / x at any point x. Such knowledge cannot in general be assumed, but should be used when possible. 4 The function F (x) is repeatedly evaluated at different points x until its minimum value is attained. (from the MINUIT tutorial) A. Khanov (PHYS6260, OSU) PHYS /1/17 10 / 13
11 1d numerical integration Do it analytically best, but not always possible Interpolate + integrate the interpolation Linear interpolation trapezoidal rule good: ( b f (a) + f (b) f (x) dx = x 2 better: b a a f (x) dx = x (x 1 = a, x n = b) ( f (a) + f (b) 2 n 1 + ) f (x i ) + O( x 2 ) n 1 + ) f (x i ) i=2 i=2 + x 2 12 (f (a) f (b))+o( x 3 ) Any interpolation works, but beware of interpolation problems (Runge s effect etc.) A. Khanov (PHYS6260, OSU) PHYS /1/17 11 / 13
12 Improper integrals Integrals with infinite limits cut the integration at some point where the function is small small can be usually estimated from the fact that f (x) dx < φ(x) dx if f (x) < φ(x) for all x > x 0 x 0 x 0 change variables to transform to finite f (a) but b a f (x) dx exists and is finite integrate by parts, change variables to transform to finite if not possible, do the numerical integration but use uneven spacing dx = 2 x = ξ, x = ξ 2, dx = 2ξ dξ 2dξ 0 x 0 A. Khanov (PHYS6260, OSU) PHYS /1/17 12 / 13
13 Multidimensional integrals Usually the function is not difficult to calculate, but the integration limits are tricky Typical approach: Monte Carlo techinique A. Khanov (PHYS6260, OSU) PHYS /1/17 13 / 13
Interpolation and extrapolation
Interpolation and extrapolation Alexander Khanov PHYS6260: Experimental Methods is HEP Oklahoma State University October 30, 207 Interpolation/extrapolation vs fitting Formulation of the problem: there
More informationNumerical solutions of nonlinear systems of equations
Numerical solutions of nonlinear systems of equations Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan E-mail: min@math.ntnu.edu.tw August 28, 2011 Outline 1 Fixed points
More informationReview. Numerical Methods Lecture 22. Prof. Jinbo Bi CSE, UConn
Review Taylor Series and Error Analysis Roots of Equations Linear Algebraic Equations Optimization Numerical Differentiation and Integration Ordinary Differential Equations Partial Differential Equations
More informationROOT FINDING REVIEW MICHELLE FENG
ROOT FINDING REVIEW MICHELLE FENG 1.1. Bisection Method. 1. Root Finding Methods (1) Very naive approach based on the Intermediate Value Theorem (2) You need to be looking in an interval with only one
More information1. Method 1: bisection. The bisection methods starts from two points a 0 and b 0 such that
Chapter 4 Nonlinear equations 4.1 Root finding Consider the problem of solving any nonlinear relation g(x) = h(x) in the real variable x. We rephrase this problem as one of finding the zero (root) of a
More informationOutline. Scientific Computing: An Introductory Survey. Nonlinear Equations. Nonlinear Equations. Examples: Nonlinear Equations
Methods for Systems of Methods for Systems of Outline Scientific Computing: An Introductory Survey Chapter 5 1 Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign
More informationMotivation: We have already seen an example of a system of nonlinear equations when we studied Gaussian integration (p.8 of integration notes)
AMSC/CMSC 460 Computational Methods, Fall 2007 UNIT 5: Nonlinear Equations Dianne P. O Leary c 2001, 2002, 2007 Solving Nonlinear Equations and Optimization Problems Read Chapter 8. Skip Section 8.1.1.
More informationOptimization. Next: Curve Fitting Up: Numerical Analysis for Chemical Previous: Linear Algebraic and Equations. Subsections
Next: Curve Fitting Up: Numerical Analysis for Chemical Previous: Linear Algebraic and Equations Subsections One-dimensional Unconstrained Optimization Golden-Section Search Quadratic Interpolation Newton's
More informationLecture 7: Minimization or maximization of functions (Recipes Chapter 10)
Lecture 7: Minimization or maximization of functions (Recipes Chapter 10) Actively studied subject for several reasons: Commonly encountered problem: e.g. Hamilton s and Lagrange s principles, economics
More informationNewton s Method and Linear Approximations
Newton s Method and Linear Approximations Curves are tricky. Lines aren t. Newton s Method and Linear Approximations Newton s Method for finding roots Goal: Where is f (x) = 0? f (x) = x 7 + 3x 3 + 7x
More informationCS 450 Numerical Analysis. Chapter 5: Nonlinear Equations
Lecture slides based on the textbook Scientific Computing: An Introductory Survey by Michael T. Heath, copyright c 2018 by the Society for Industrial and Applied Mathematics. http://www.siam.org/books/cl80
More informationNumerical Analysis: Solving Nonlinear Equations
Numerical Analysis: Solving Nonlinear Equations Mirko Navara http://cmp.felk.cvut.cz/ navara/ Center for Machine Perception, Department of Cybernetics, FEE, CTU Karlovo náměstí, building G, office 104a
More informationSOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS BISECTION METHOD
BISECTION METHOD If a function f(x) is continuous between a and b, and f(a) and f(b) are of opposite signs, then there exists at least one root between a and b. It is shown graphically as, Let f a be negative
More informationLecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 5. Nonlinear Equations
Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T Heath Chapter 5 Nonlinear Equations Copyright c 2001 Reproduction permitted only for noncommercial, educational
More informationGENG2140, S2, 2012 Week 7: Curve fitting
GENG2140, S2, 2012 Week 7: Curve fitting Curve fitting is the process of constructing a curve, or mathematical function, f(x) that has the best fit to a series of data points Involves fitting lines and
More informationNumerical Analysis. EE, NCKU Tien-Hao Chang (Darby Chang)
Numerical Analysis EE, NCKU Tien-Hao Chang (Darby Chang) 1 In the previous slide Error (motivation) Floating point number system difference to real number system problem of roundoff Introduced/propagated
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 5 Nonlinear Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More informationLecture 8. Root finding II
1 Introduction Lecture 8 Root finding II In the previous lecture we considered the bisection root-bracketing algorithm. It requires only that the function be continuous and that we have a root bracketed
More informationChapter 3: Root Finding. September 26, 2005
Chapter 3: Root Finding September 26, 2005 Outline 1 Root Finding 2 3.1 The Bisection Method 3 3.2 Newton s Method: Derivation and Examples 4 3.3 How To Stop Newton s Method 5 3.4 Application: Division
More informationPreliminary check: are the terms that we are adding up go to zero or not? If not, proceed! If the terms a n are going to zero, pick another test.
Throughout these templates, let series. be a series. We hope to determine the convergence of this Divergence Test: If lim is not zero or does not exist, then the series diverges. Preliminary check: are
More informationPHYS 410/555 Computational Physics Solution of Non Linear Equations (a.k.a. Root Finding) (Reference Numerical Recipes, 9.0, 9.1, 9.
PHYS 410/555 Computational Physics Solution of Non Linear Equations (a.k.a. Root Finding) (Reference Numerical Recipes, 9.0, 9.1, 9.4) We will consider two cases 1. f(x) = 0 1-dimensional 2. f(x) = 0 d-dimensional
More informationTABLE OF CONTENTS INTRODUCTION, APPROXIMATION & ERRORS 1. Chapter Introduction to numerical methods 1 Multiple-choice test 7 Problem set 9
TABLE OF CONTENTS INTRODUCTION, APPROXIMATION & ERRORS 1 Chapter 01.01 Introduction to numerical methods 1 Multiple-choice test 7 Problem set 9 Chapter 01.02 Measuring errors 11 True error 11 Relative
More informationExact and Approximate Numbers:
Eact and Approimate Numbers: The numbers that arise in technical applications are better described as eact numbers because there is not the sort of uncertainty in their values that was described above.
More informationMATH 350: Introduction to Computational Mathematics
MATH 350: Introduction to Computational Mathematics Chapter IV: Locating Roots of Equations Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Spring 2011 fasshauer@iit.edu
More informationNewton s Method and Linear Approximations 10/19/2011
Newton s Method and Linear Approximations 10/19/2011 Curves are tricky. Lines aren t. Newton s Method and Linear Approximations 10/19/2011 Newton s Method Goal: Where is f (x) =0? f (x) =x 7 +3x 3 +7x
More informationNUMERICAL MATHEMATICS AND COMPUTING
NUMERICAL MATHEMATICS AND COMPUTING Fourth Edition Ward Cheney David Kincaid The University of Texas at Austin 9 Brooks/Cole Publishing Company I(T)P An International Thomson Publishing Company Pacific
More informationMidterm Review. Igor Yanovsky (Math 151A TA)
Midterm Review Igor Yanovsky (Math 5A TA) Root-Finding Methods Rootfinding methods are designed to find a zero of a function f, that is, to find a value of x such that f(x) =0 Bisection Method To apply
More informationNumerical Methods I Solving Nonlinear Equations
Numerical Methods I Solving Nonlinear Equations Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 16th, 2014 A. Donev (Courant Institute)
More informationZeros of Functions. Chapter 10
Chapter 10 Zeros of Functions An important part of the mathematics syllabus in secondary school is equation solving. This is important for the simple reason that equations are important a wide range of
More informationSolution of Nonlinear Equations
Solution of Nonlinear Equations (Com S 477/577 Notes) Yan-Bin Jia Sep 14, 017 One of the most frequently occurring problems in scientific work is to find the roots of equations of the form f(x) = 0. (1)
More informationNUMERICAL AND STATISTICAL COMPUTING (MCA-202-CR)
NUMERICAL AND STATISTICAL COMPUTING (MCA-202-CR) Autumn Session UNIT 1 Numerical analysis is the study of algorithms that uses, creates and implements algorithms for obtaining numerical solutions to problems
More information3.1 Introduction. Solve non-linear real equation f(x) = 0 for real root or zero x. E.g. x x 1.5 =0, tan x x =0.
3.1 Introduction Solve non-linear real equation f(x) = 0 for real root or zero x. E.g. x 3 +1.5x 1.5 =0, tan x x =0. Practical existence test for roots: by intermediate value theorem, f C[a, b] & f(a)f(b)
More informationMATH 350: Introduction to Computational Mathematics
MATH 350: Introduction to Computational Mathematics Chapter IV: Locating Roots of Equations Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Spring 2011 fasshauer@iit.edu
More information17 Solution of Nonlinear Systems
17 Solution of Nonlinear Systems We now discuss the solution of systems of nonlinear equations. An important ingredient will be the multivariate Taylor theorem. Theorem 17.1 Let D = {x 1, x 2,..., x m
More informationCS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares
CS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares Robert Bridson October 29, 2008 1 Hessian Problems in Newton Last time we fixed one of plain Newton s problems by introducing line search
More informationRoot Finding (and Optimisation)
Root Finding (and Optimisation) M.Sc. in Mathematical Modelling & Scientific Computing, Practical Numerical Analysis Michaelmas Term 2018, Lecture 4 Root Finding The idea of root finding is simple we want
More informationChapter 4. Solution of Non-linear Equation. Module No. 1. Newton s Method to Solve Transcendental Equation
Numerical Analysis by Dr. Anita Pal Assistant Professor Department of Mathematics National Institute of Technology Durgapur Durgapur-713209 email: anita.buie@gmail.com 1 . Chapter 4 Solution of Non-linear
More informationNumerical Methods in Physics and Astrophysics
Kostas Kokkotas 2 October 20, 2014 2 http://www.tat.physik.uni-tuebingen.de/ kokkotas Kostas Kokkotas 3 TOPICS 1. Solving nonlinear equations 2. Solving linear systems of equations 3. Interpolation, approximation
More informationCHAPTER 10 Zeros of Functions
CHAPTER 10 Zeros of Functions An important part of the maths syllabus in secondary school is equation solving. This is important for the simple reason that equations are important a wide range of problems
More informationNumerical Methods in Physics and Astrophysics
Kostas Kokkotas 2 October 17, 2017 2 http://www.tat.physik.uni-tuebingen.de/ kokkotas Kostas Kokkotas 3 TOPICS 1. Solving nonlinear equations 2. Solving linear systems of equations 3. Interpolation, approximation
More informationLecture 5: Random numbers and Monte Carlo (Numerical Recipes, Chapter 7) Motivations for generating random numbers
Lecture 5: Random numbers and Monte Carlo (Numerical Recipes, Chapter 7) Motivations for generating random numbers To sample a function in a statistically controlled manner (i.e. for Monte Carlo integration)
More informationJim Lambers MAT 460/560 Fall Semester Practice Final Exam
Jim Lambers MAT 460/560 Fall Semester 2009-10 Practice Final Exam 1. Let f(x) = sin 2x + cos 2x. (a) Write down the 2nd Taylor polynomial P 2 (x) of f(x) centered around x 0 = 0. (b) Write down the corresponding
More informationRoot Finding and Optimization
Root Finding and Optimization Ramses van Zon SciNet, University o Toronto Scientiic Computing Lecture 11 February 11, 2014 Root Finding It is not uncommon in scientiic computing to want solve an equation
More informationNumerical optimization
THE UNIVERSITY OF WESTERN ONTARIO LONDON ONTARIO Paul Klein Office: SSC 408 Phone: 661-111 ext. 857 Email: paul.klein@uwo.ca URL: www.ssc.uwo.ca/economics/faculty/klein/ Numerical optimization In these
More informationOptimization. Totally not complete this is...don't use it yet...
Optimization Totally not complete this is...don't use it yet... Bisection? Doing a root method is akin to doing a optimization method, but bi-section would not be an effective method - can detect sign
More information8 Numerical methods for unconstrained problems
8 Numerical methods for unconstrained problems Optimization is one of the important fields in numerical computation, beside solving differential equations and linear systems. We can see that these fields
More informationSolving Non-Linear Equations (Root Finding)
Solving Non-Linear Equations (Root Finding) Root finding Methods What are root finding methods? Methods for determining a solution of an equation. Essentially finding a root of a function, that is, a zero
More informationOutline. Math Numerical Analysis. Intermediate Value Theorem. Lecture Notes Zeros and Roots. Joseph M. Mahaffy,
Outline Math 541 - Numerical Analysis Lecture Notes Zeros and Roots Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Department of Mathematics and Statistics Dynamical Systems Group Computational Sciences Research
More informationMath Numerical Analysis
Math 541 - Numerical Analysis Lecture Notes Zeros and Roots Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Department of Mathematics and Statistics Dynamical Systems Group Computational Sciences Research Center
More information4 damped (modified) Newton methods
4 damped (modified) Newton methods 4.1 damped Newton method Exercise 4.1 Determine with the damped Newton method the unique real zero x of the real valued function of one variable f(x) = x 3 +x 2 using
More informationError propagation. Alexander Khanov. October 4, PHYS6260: Experimental Methods is HEP Oklahoma State University
Error propagation Alexander Khanov PHYS660: Experimental Methods is HEP Oklahoma State University October 4, 017 Why error propagation? In many cases we measure one thing and want to know something else
More informationNumerical Methods in Informatics
Numerical Methods in Informatics Lecture 2, 30.09.2016: Nonlinear Equations in One Variable http://www.math.uzh.ch/binf4232 Tulin Kaman Institute of Mathematics, University of Zurich E-mail: tulin.kaman@math.uzh.ch
More informationChapter 4. Unconstrained optimization
Chapter 4. Unconstrained optimization Version: 28-10-2012 Material: (for details see) Chapter 11 in [FKS] (pp.251-276) A reference e.g. L.11.2 refers to the corresponding Lemma in the book [FKS] PDF-file
More informationLine Search Algorithms
Lab 1 Line Search Algorithms Investigate various Line-Search algorithms for numerical opti- Lab Objective: mization. Overview of Line Search Algorithms Imagine you are out hiking on a mountain, and you
More informationComputational Methods
Numerical Computational Methods Revised Edition P. B. Patil U. P. Verma Alpha Science International Ltd. Oxford, U.K. CONTENTS Preface List ofprograms v vii 1. NUMER1CAL METHOD, ERROR AND ALGORITHM 1 1.1
More informationScientific Computing. Roots of Equations
ECE257 Numerical Methods and Scientific Computing Roots of Equations Today s s class: Roots of Equations Polynomials Polynomials A polynomial is of the form: ( x) = a 0 + a 1 x + a 2 x 2 +L+ a n x n f
More informationTHE SECANT METHOD. q(x) = a 0 + a 1 x. with
THE SECANT METHOD Newton s method was based on using the line tangent to the curve of y = f (x), with the point of tangency (x 0, f (x 0 )). When x 0 α, the graph of the tangent line is approximately the
More informationby Martin Mendez, UASLP Copyright 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Chapter 5 by Martin Mendez, 1 Roots of Equations Part Why? b m b a + b + c = 0 = aa 4ac But a 5 4 3 + b + c + d + e + f = 0 sin + = 0 =? =? by Martin Mendez, Nonlinear Equation Solvers Bracketing Graphical
More informationMonte Carlo programs
Monte Carlo programs Alexander Khanov PHYS6260: Experimental Methods is HEP Oklahoma State University November 13, 2017 Monte Carlo methods: the idea In classical physics, if we need to measure something
More informationASSIGNMENT BOOKLET. Numerical Analysis (MTE-10) (Valid from 1 st July, 2011 to 31 st March, 2012)
ASSIGNMENT BOOKLET MTE-0 Numerical Analysis (MTE-0) (Valid from st July, 0 to st March, 0) It is compulsory to submit the assignment before filling in the exam form. School of Sciences Indira Gandhi National
More informationLine Search Methods. Shefali Kulkarni-Thaker
1 BISECTION METHOD Line Search Methods Shefali Kulkarni-Thaker Consider the following unconstrained optimization problem min f(x) x R Any optimization algorithm starts by an initial point x 0 and performs
More informationIntegration, differentiation, and root finding. Phys 420/580 Lecture 7
Integration, differentiation, and root finding Phys 420/580 Lecture 7 Numerical integration Compute an approximation to the definite integral I = b Find area under the curve in the interval Trapezoid Rule:
More informationTHE UNIVERSITY OF NEW SOUTH WALES SCHOOL OF PHYSICS PHYS2020 COMPUTATIONAL PHYSICS FINAL EXAM SESSION I Answer all questions
THE UNIVERSITY OF NEW SOUTH WALES SCHOOL OF PHYSICS PHYS2020 COMPUTATIONAL PHYSICS FINAL EXAM SESSION I 2007 Answer all questions Time allowed =2 hours Total number of questions =5 Marks =40 The questions
More informationEAD 115. Numerical Solution of Engineering and Scientific Problems. David M. Rocke Department of Applied Science
EAD 115 Numerical Solution of Engineering and Scientific Problems David M. Rocke Department of Applied Science Taylor s Theorem Can often approximate a function by a polynomial The error in the approximation
More informationChapter 1. Root Finding Methods. 1.1 Bisection method
Chapter 1 Root Finding Methods We begin by considering numerical solutions to the problem f(x) = 0 (1.1) Although the problem above is simple to state it is not always easy to solve analytically. This
More informationSection 6.6 Gaussian Quadrature
Section 6.6 Gaussian Quadrature Key Terms: Method of undetermined coefficients Nonlinear systems Gaussian quadrature Error Legendre polynomials Inner product Adapted from http://pathfinder.scar.utoronto.ca/~dyer/csca57/book_p/node44.html
More informationStochastic Subgradient Method
Stochastic Subgradient Method Lingjie Weng, Yutian Chen Bren School of Information and Computer Science UC Irvine Subgradient Recall basic inequality for convex differentiable f : f y f x + f x T (y x)
More information1-D Optimization. Lab 16. Overview of Line Search Algorithms. Derivative versus Derivative-Free Methods
Lab 16 1-D Optimization Lab Objective: Many high-dimensional optimization algorithms rely on onedimensional optimization methods. In this lab, we implement four line search algorithms for optimizing scalar-valued
More informationGradient Descent. Dr. Xiaowei Huang
Gradient Descent Dr. Xiaowei Huang https://cgi.csc.liv.ac.uk/~xiaowei/ Up to now, Three machine learning algorithms: decision tree learning k-nn linear regression only optimization objectives are discussed,
More informationMath Numerical Analysis Mid-Term Test Solutions
Math 400 - Numerical Analysis Mid-Term Test Solutions. Short Answers (a) A sufficient and necessary condition for the bisection method to find a root of f(x) on the interval [a,b] is f(a)f(b) < 0 or f(a)
More informationNewton s Method and Linear Approximations
Newton s Method and Linear Approximations Newton s Method for finding roots Goal: Where is f (x) =0? f (x) =x 7 +3x 3 +7x 2 1 2-1 -0.5 0.5-2 Newton s Method for finding roots Goal: Where is f (x) =0? f
More informationX. Numerical Methods
X. Numerical Methods. Taylor Approximation Suppose that f is a function defined in a neighborhood of a point c, and suppose that f has derivatives of all orders near c. In section 5 of chapter 9 we introduced
More informationSolution of Algebric & Transcendental Equations
Page15 Solution of Algebric & Transcendental Equations Contents: o Introduction o Evaluation of Polynomials by Horner s Method o Methods of solving non linear equations o Bracketing Methods o Bisection
More informationToday. Introduction to optimization Definition and motivation 1-dimensional methods. Multi-dimensional methods. General strategies, value-only methods
Optimization Last time Root inding: deinition, motivation Algorithms: Bisection, alse position, secant, Newton-Raphson Convergence & tradeos Eample applications o Newton s method Root inding in > 1 dimension
More informationmin f(x). (2.1) Objectives consisting of a smooth convex term plus a nonconvex regularization term;
Chapter 2 Gradient Methods The gradient method forms the foundation of all of the schemes studied in this book. We will provide several complementary perspectives on this algorithm that highlight the many
More informationMain topics for the First Midterm Exam
Main topics for the First Midterm Exam The final will cover Sections.-.0, 2.-2.5, and 4.. This is roughly the material from first three homeworks and three quizzes, in addition to the lecture on Monday,
More informationBACHELOR OF COMPUTER APPLICATIONS (BCA) (Revised) Term-End Examination December, 2015 BCS-054 : COMPUTER ORIENTED NUMERICAL TECHNIQUES
No. of Printed Pages : 5 BCS-054 BACHELOR OF COMPUTER APPLICATIONS (BCA) (Revised) Term-End Examination December, 2015 058b9 BCS-054 : COMPUTER ORIENTED NUMERICAL TECHNIQUES Time : 3 hours Maximum Marks
More informationBindel, Fall 2011 Intro to Scientific Computing (CS 3220) Week 6: Monday, Mar 7. e k+1 = 1 f (ξ k ) 2 f (x k ) e2 k.
Problem du jour Week 6: Monday, Mar 7 Show that for any initial guess x 0 > 0, Newton iteration on f(x) = x 2 a produces a decreasing sequence x 1 x 2... x n a. What is the rate of convergence if a = 0?
More informationCLASS NOTES Models, Algorithms and Data: Introduction to computing 2018
CLASS NOTES Models, Algorithms and Data: Introduction to computing 2018 Petros Koumoutsakos, Jens Honore Walther (Last update: April 16, 2018) IMPORTANT DISCLAIMERS 1. REFERENCES: Much of the material
More informationComputational Methods CMSC/AMSC/MAPL 460. Solving nonlinear equations and zero finding. Finding zeroes of functions
Computational Methods CMSC/AMSC/MAPL 460 Solving nonlinear equations and zero finding Ramani Duraiswami, Dept. of Computer Science Where does it arise? Finding zeroes of functions Solving functional equations
More informationQuantitative Techniques (Finance) 203. Polynomial Functions
Quantitative Techniques (Finance) 03 Polynomial Functions Felix Chan October 006 Introduction This topic discusses the properties and the applications of polynomial functions, specifically, linear and
More informationLecture 7: Numerical Tools
Lecture 7: Numerical Tools Fatih Guvenen January 10, 2016 Fatih Guvenen Lecture 7: Numerical Tools January 10, 2016 1 / 18 Overview Three Steps: V (k, z) =max c,k 0 apple u(c)+ Z c + k 0 =(1 + r)k + z
More informationExam in TMA4215 December 7th 2012
Norwegian University of Science and Technology Department of Mathematical Sciences Page of 9 Contact during the exam: Elena Celledoni, tlf. 7359354, cell phone 48238584 Exam in TMA425 December 7th 22 Allowed
More information1.1: The bisection method. September 2017
(1/11) 1.1: The bisection method Solving nonlinear equations MA385/530 Numerical Analysis September 2017 3 2 f(x)= x 2 2 x axis 1 0 1 x [0] =a x [2] =1 x [3] =1.5 x [1] =b 2 0.5 0 0.5 1 1.5 2 2.5 1 Solving
More informationRoot Finding Methods
Root Finding Methods Let's take a simple problem ax 2 +bx+c=0 Can calculate roots using the quadratic equation or just by factoring Easy and does not need any numerical methods How about solving for the
More informationChapter 1 Review of Equations and Inequalities
Chapter 1 Review of Equations and Inequalities Part I Review of Basic Equations Recall that an equation is an expression with an equal sign in the middle. Also recall that, if a question asks you to solve
More informationLecture 34 Minimization and maximization of functions
Lecture 34 Minimization and maximization of functions Introduction Golden section search Parabolic interpolation Search with first derivatives Downhill simplex method Introduction In a nutshell, you are
More informationToday s class. Numerical differentiation Roots of equation Bracketing methods. Numerical Methods, Fall 2011 Lecture 4. Prof. Jinbo Bi CSE, UConn
Today s class Numerical differentiation Roots of equation Bracketing methods 1 Numerical Differentiation Finite divided difference First forward difference First backward difference Lecture 3 2 Numerical
More informationQR Decomposition. When solving an overdetermined system by projection (or a least squares solution) often the following method is used:
(In practice not Gram-Schmidt, but another process Householder Transformations are used.) QR Decomposition When solving an overdetermined system by projection (or a least squares solution) often the following
More information(One Dimension) Problem: for a function f(x), find x 0 such that f(x 0 ) = 0. f(x)
Solving Nonlinear Equations & Optimization One Dimension Problem: or a unction, ind 0 such that 0 = 0. 0 One Root: The Bisection Method This one s guaranteed to converge at least to a singularity, i not
More informationUnconstrained optimization I Gradient-type methods
Unconstrained optimization I Gradient-type methods Antonio Frangioni Department of Computer Science University of Pisa www.di.unipi.it/~frangio frangio@di.unipi.it Computational Mathematics for Learning
More informationWarm up: Recall we can approximate R b
Warm up: Recall we can approximate R b a f(x) dx using rectangles as follows: i. Pick a number n and divide [a, b] into n equal intervals. Note that x =(b a)/n is the length of each of these intervals.
More informationLecture 17: Numerical Optimization October 2014
Lecture 17: Numerical Optimization 36-350 22 October 2014 Agenda Basics of optimization Gradient descent Newton s method Curve-fitting R: optim, nls Reading: Recipes 13.1 and 13.2 in The R Cookbook Optional
More informationCS 323: Numerical Analysis and Computing
CS 323: Numerical Analysis and Computing MIDTERM #2 Instructions: This is an open notes exam, i.e., you are allowed to consult any textbook, your class notes, homeworks, or any of the handouts from us.
More informationCISE-301: Numerical Methods Topic 1:
CISE-3: Numerical Methods Topic : Introduction to Numerical Methods and Taylor Series Lectures -4: KFUPM Term 9 Section 8 CISE3_Topic KFUPM - T9 - Section 8 Lecture Introduction to Numerical Methods What
More informationCS 323: Numerical Analysis and Computing
CS 323: Numerical Analysis and Computing MIDTERM #2 Instructions: This is an open notes exam, i.e., you are allowed to consult any textbook, your class notes, homeworks, or any of the handouts from us.
More informationCS412: Introduction to Numerical Methods
CS412: Introduction to Numerical Methods MIDTERM #1 2:30PM - 3:45PM, Tuesday, 03/10/2015 Instructions: This exam is a closed book and closed notes exam, i.e., you are not allowed to consult any textbook,
More informationMVE165/MMG631 Linear and integer optimization with applications Lecture 13 Overview of nonlinear programming. Ann-Brith Strömberg
MVE165/MMG631 Overview of nonlinear programming Ann-Brith Strömberg 2015 05 21 Areas of applications, examples (Ch. 9.1) Structural optimization Design of aircraft, ships, bridges, etc Decide on the material
More informationNotes for Numerical Analysis Math 5465 by S. Adjerid Virginia Polytechnic Institute and State University. (A Rough Draft)
Notes for Numerical Analysis Math 5465 by S. Adjerid Virginia Polytechnic Institute and State University (A Rough Draft) 1 2 Contents 1 Error Analysis 5 2 Nonlinear Algebraic Equations 7 2.1 Convergence
More informationUnconstrained Optimization
1 / 36 Unconstrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University February 2, 2015 2 / 36 3 / 36 4 / 36 5 / 36 1. preliminaries 1.1 local approximation
More information