Newton s Method for One - Dimensional Optimization - Theory

Similar documents
Chapter Newton s Method

Single Variable Optimization

Lecture Notes on Linear Regression

Chapter 4: Root Finding

OPTIMISATION. Introduction Single Variable Unconstrained Optimisation Multivariable Unconstrained Optimisation Linear Programming

Topic 5: Non-Linear Regression

Lecture 21: Numerical methods for pricing American type derivatives

Lecture 10 Support Vector Machines II

: Numerical Analysis Topic 2: Solution of Nonlinear Equations Lectures 5-11:

THE VIBRATIONS OF MOLECULES II THE CARBON DIOXIDE MOLECULE Student Instructions

Curve Fitting with the Least Square Method

Summary with Examples for Root finding Methods -Bisection -Newton Raphson -Secant

VARIATION OF CONSTANT SUM CONSTRAINT FOR INTEGER MODEL WITH NON UNIFORM VARIABLES

Lecture 2 Solution of Nonlinear Equations ( Root Finding Problems )

Report on Image warping

Physics 5153 Classical Mechanics. D Alembert s Principle and The Lagrangian-1

IV. Performance Optimization

Classification as a Regression Problem

Natural Language Processing and Information Retrieval

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification

Integrals and Invariants of Euler-Lagrange Equations

xp(x µ) = 0 p(x = 0 µ) + 1 p(x = 1 µ) = µ

ACTM State Calculus Competition Saturday April 30, 2011

Chapter 12. Ordinary Differential Equation Boundary Value (BV) Problems

Problem Set 9 Solutions

The Study of Teaching-learning-based Optimization Algorithm

CHAPTER 4d. ROOTS OF EQUATIONS

PHYS 705: Classical Mechanics. Calculus of Variations II

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Review of Taylor Series. Read Section 1.2

CSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography

A Local Variational Problem of Second Order for a Class of Optimal Control Problems with Nonsmooth Objective Function

CIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M

Winter 2008 CS567 Stochastic Linear/Integer Programming Guest Lecturer: Xu, Huan

Maximum Likelihood Estimation

On an Extension of Stochastic Approximation EM Algorithm for Incomplete Data Problems. Vahid Tadayon 1

14 Lagrange Multipliers

2. PROBLEM STATEMENT AND SOLUTION STRATEGIES. L q. Suppose that we have a structure with known geometry (b, h, and L) and material properties (EA).

CS : Algorithms and Uncertainty Lecture 17 Date: October 26, 2016

Some modelling aspects for the Matlab implementation of MMA

CISE301: Numerical Methods Topic 2: Solution of Nonlinear Equations

ME 501A Seminar in Engineering Analysis Page 1

CHAPTER 7 CONSTRAINED OPTIMIZATION 2: SQP AND GRG

Numerical Solutions of a Generalized Nth Order Boundary Value Problems Using Power Series Approximation Method

ECE559VV Project Report

Chapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems

Finite Element Modelling of truss/cable structures

Nice plotting of proteins II

% & 5.3 PRACTICAL APPLICATIONS. Given system, (49) , determine the Boolean Function, , in such a way that we always have expression: " Y1 = Y2

A new Approach for Solving Linear Ordinary Differential Equations

MATH 829: Introduction to Data Mining and Analysis The EM algorithm (part 2)

AP Physics 1 & 2 Summer Assignment

Econ107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)

Calculation of time complexity (3%)

Common loop optimizations. Example to improve locality. Why Dependence Analysis. Data Dependence in Loops. Goal is to find best schedule:

Dynamic Programming. Preview. Dynamic Programming. Dynamic Programming. Dynamic Programming (Example: Fibonacci Sequence)

Why Bayesian? 3. Bayes and Normal Models. State of nature: class. Decision rule. Rev. Thomas Bayes ( ) Bayes Theorem (yes, the famous one)

Singular Value Decomposition: Theory and Applications

NON-CENTRAL 7-POINT FORMULA IN THE METHOD OF LINES FOR PARABOLIC AND BURGERS' EQUATIONS

THE ARIMOTO-BLAHUT ALGORITHM FOR COMPUTATION OF CHANNEL CAPACITY. William A. Pearlman. References: S. Arimoto - IEEE Trans. Inform. Thy., Jan.

Sharp integral inequalities involving high-order partial derivatives. Journal Of Inequalities And Applications, 2008, v. 2008, article no.

For now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results.

Credit Card Pricing and Impact of Adverse Selection

A Hybrid Variational Iteration Method for Blasius Equation

The Minimum Universal Cost Flow in an Infeasible Flow Network

Finite Mixture Models and Expectation Maximization. Most slides are from: Dr. Mario Figueiredo, Dr. Anil Jain and Dr. Rong Jin

Tracking with Kalman Filter

EEE 241: Linear Systems

PETER HENR IC I TECHNICAL REPORT NO. CS 137 JULY COMPUTER SCIENCE DEPARTMENT School of Humanities and Sciences STANFORD UN IVERS ITY

Review: Fit a line to N data points

Module 2. Random Processes. Version 2 ECE IIT, Kharagpur

An identification algorithm of model kinetic parameters of the interfacial layer growth in fiber composites

Design and Optimization of Fuzzy Controller for Inverse Pendulum System Using Genetic Algorithm

Chapter - 2. Distribution System Power Flow Analysis

Case A. P k = Ni ( 2L i k 1 ) + (# big cells) 10d 2 P k.

Assortment Optimization under MNL

Introduction to the R Statistical Computing Environment R Programming

Section 8.3 Polar Form of Complex Numbers

10.34 Numerical Methods Applied to Chemical Engineering Fall Homework #3: Systems of Nonlinear Equations and Optimization

PART 8. Partial Differential Equations PDEs

An Interactive Optimisation Tool for Allocation Problems

Appendix B. The Finite Difference Scheme

1 Matrix representations of canonical matrices

ORIGIN 1. PTC_CE_BSD_3.2_us_mp.mcdx. Mathcad Enabled Content 2011 Knovel Corp.

SOLVING NON-LINEAR SYSTEMS BY NEWTON s METHOD USING SPREADSHEET EXCEL Tay Kim Gaik Universiti Tun Hussein Onn Malaysia

ANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)

Estimation: Part 2. Chapter GREG estimation

Additional Codes using Finite Difference Method. 1 HJB Equation for Consumption-Saving Problem Without Uncertainty

Logistic Regression. CAP 5610: Machine Learning Instructor: Guo-Jun QI

Grover s Algorithm + Quantum Zeno Effect + Vaidman

Introduction to Vapor/Liquid Equilibrium, part 2. Raoult s Law:

Computer Based Porosity Design by Multi Phase Topology Optimization

Pattern Classification

Research on Route guidance of logistic scheduling problem under fuzzy time window

6) Derivatives, gradients and Hessian matrices

RELIABILITY ASSESSMENT

CSE 546 Midterm Exam, Fall 2014(with Solution)

A MODIFIED METHOD FOR SOLVING SYSTEM OF NONLINEAR EQUATIONS

Solutions HW #2. minimize. Ax = b. Give the dual problem, and make the implicit equality constraints explicit. Solution.

Linear Approximation with Regularization and Moving Least Squares

Transcription:

Numercal Methods Newton s Method for One - Dmensonal Optmzaton - Theory

For more detals on ths topc Go to Clck on Keyword Clck on Newton s Method for One- Dmensonal Optmzaton

You are free to Share to copy, dstrbute, dsplay and perform the work to Remx to make dervatve works

Under the followng condtons Attrbuton You must attrbute the work n the manner specfed by the author or lcensor (but not n any way that suggests that they endorse you or your use of the work. Noncommercal You may not use ths work for commercal purposes. Share Alke If you alter, transform, or buld upon ths work, you may dstrbute the resultng work only under the same or smlar lcense to ths one.

Newton s Method-Overvew Open search method A good ntal estmate of the soluton s requred The objectve functon must be twce dfferentable Unlke Golden Secton Search method Lower and upper search boundares are not requred (open vs. bracketng May not converge to the optmal soluton 5

Newton s Method-How t works The dervatve of the functon f Opt. x,nonlnear ' root fndng equaton f ( x = 0 = F( x at the functon s maxmum and mnmum. The mnma and the maxma can be found by applyng the Newton-Raphson method to the dervatve, essentally obtanng x + 1 = x f f Next slde wll explan how to get/derve the above formula ' '' ( x ( x ( 6

F(x F(x F(x +1 Newton s Method-To fnd root of a Slope = F ( E A B x +1 x Hence: nonlnear equaton C F x x +1 D F(x - 0 + 1 = x Slope @ pt. C We wsh that n the next teraton x +1 wll be the root, or. Thus: Slope @ pt. C = Or F( F ( I F( 1 = 0 F ( + = F( + 1 F( F( + 1 N-R Equaton F( 0 + 1 + 1 7

Newton s Method-To fnd root of a nonlneat equaton f ( f ( If F ( x f ( x,then + 1 =. For Mult-varable case,then N-R method becomes = [ f ( ] f ( 1 + 1 8

Newton s Method-Algorthm Intalzaton: Determne a reasonably good estmate for the maxma or the mnma of the functon f ( x. Step 1. Determne f ' ( x and f '' ( x. Step 2. Substtute x (ntal estmate x0 for the frst teraton and f ' ( x nto f '' ( x ' f ( x x+ 1 = x '' f ( x to determne x +1 and the functon value n teraton. Step 3.If the value of the frst dervatve of the functon s zero then you have reached the optmum (maxma or mnma. Otherwse, repeat Step 2 wth the new value of x 9

THE END

Acknowledgement Ths nstructonal power pont brought to you by Numercal Methods for STEM undergraduate Commtted to brngng numercal methods to the undergraduate

For nstructonal vdeos on other topcs, go to Ths materal s based upon work supported by the Natonal Scence Foundaton under Grant # 0717624. Any opnons, fndngs, and conclusons or recommendatons expressed n ths materal are those of the author(s and do not necessarly reflect the vews of the Natonal Scence Foundaton.

The End - Really

Numercal Methods Newton s Method for One - Dmensonal Optmzaton - Example

For more detals on ths topc Go to Clck on Keyword Clck on Newton s Method for One- Dmensonal Optmzaton

You are free to Share to copy, dstrbute, dsplay and perform the work to Remx to make dervatve works

Under the followng condtons Attrbuton You must attrbute the work n the manner specfed by the author or lcensor (but not n any way that suggests that they endorse you or your use of the work. Noncommercal You may not use ths work for commercal purposes. Share Alke If you alter, transform, or buld upon ths work, you may dstrbute the resultng work only under the same or smlar lcense to ths one.

Example. 2 2 θ 2 θ The cross-sectonal area A of a gutter wth equal base and edge length of 2 s gven by A = 4snθ (1 + cosθ Fnd the angle θ whch maxmzes the cross-sectonal area of the gutter. 18

Soluton The functon to be maxmzed s f ( θ = 4snθ (1 + cosθ f ( θ = 4(cosθ + cos 2 θ sn 2 θ f ( θ = 4snθ (1 + 4 cosθ Iteraton 1: Use the soluton θ = π 4 0 = 0.7854rad π 2 π 2 π 4(cos + cos sn π θ = 4 4 4 4 π π 4sn (1 + 4cos 4 4 1 = f ( 1.0466 = as the ntal estmate of 5.196151 1.0466 19

Soluton Cont. Iteraton 2: 2 2 4(cos1.0466 + cos 1.0466 sn 1.0466 θ2 = 1.0466 = 1.0472 4sn1.0466(1 + 4 cos1.0466 Summary of teratons Iteraton θ f '( θ f ''( θ θ estmate f (θ 1 0.7854 2.8284-10.8284 1.0466 5.1962 2 1.0466 0.0062-10.3959 1.0472 5.1962 3 1.0472 1.06E-06-10.3923 1.0472 5.1962 4 1.0472 3.06E-14-10.3923 1.0472 5.1962 5 1.0472 1.3322E-15-10.3923 1.0472 5.1962 Remember that the actual soluton to the problem s at 60 degrees or 1.0472 radans. 20

THE END

Acknowledgement Ths nstructonal power pont brought to you by Numercal Methods for STEM undergraduate Commtted to brngng numercal methods to the undergraduate

For nstructonal vdeos on other topcs, go to Ths materal s based upon work supported by the Natonal Scence Foundaton under Grant # 0717624. Any opnons, fndngs, and conclusons or recommendatons expressed n ths materal are those of the author(s and do not necessarly reflect the vews of the Natonal Scence Foundaton.

The End - Really