Trilateration: The Mathematics Behind a Local Positioning System. Willy Hereman

Size: px
Start display at page:

Download "Trilateration: The Mathematics Behind a Local Positioning System. Willy Hereman"

Transcription

1 Trilateration: The Mathematics Behind a Local Positioning System Willy Hereman Department of Mathematical and Computer Sciences Colorado School of Mines Golden, Colorado, U.S.A. whereman@mines.edu whereman/ Department of Computer Engineering Turgut Özal University, Keçiören, Ankara, Turkey Tuesday, June 21, 2011, 11:00

2 Acknowledgements William Murphy (M.S. Dissertation, CSM, 1992) William Navidi (Department of MCS, CSM) Atlantic Richfield Company (ARCO) Thunder Basin Coal Company, Wright, Wyoming This presentation was made in TeXpower

3 Outline Problem statement: What is trilateration? How did we get into this? Applications of our algorithm An exact linearization Linear least squares method Nonlinear least squares method Mathematica demonstration Simulation Results of experiments Conclusions

4 Global Positioning System (GPS)

5

6 How did we get into this?

7 Accidents Happen

8 Bulldozer with Beacons

9 Applications of our Algorithm Thunder Basin Coal Mine locating bulldozers Surveying without triangulation (Mining) Mobile computing sensor networks Geosensing networks (SmartGeo) Precision manufacturing Positioning systems for medical applications (Electrical Engineering) Ignite program blasting rockets

10

11

12 Problem Statement and Setup

13 Notations θ = (x, y, z) : spatial coordinates of target point θ. B i = (x i, y i, z i ) : exact location of beacon B i. i = 1, 2,..., n with n 4. d i (θ) = (x x i ) 2 + (y y i ) 2 + (z z i ) 2 : true distance between beacon B i and target θ. (x r, y r, z r ) : exact coordinates of a reference point. d ir = (x i x r ) 2 + (y i y r ) 2 + (z i z r ) 2 : true distance between reference point and beacon B i. d r (θ) = (x x r ) 2 + (y y r ) 2 + (z z r ) 2 : true distance between the reference point and the target θ.

14 Derivation of an Exact Linear Model Apply a simple trick (the cosine rule!) d i (θ) 2 = (x x i ) 2 + (y y i ) 2 + (z z i ) 2 = (x x r + x r x i ) 2 + (y y r + y r y i ) 2 +(z z r + z r z i ) 2 = (x x r ) (x r x i )(x x r ) + (x r x i ) 2 +(y y r ) (y r y i )(y y r ) + (y r y i ) 2 +(z z r ) (z r z i )(z z r ) + (z r z i ) 2

15 Keep the double product terms on the left hand side. ( ) 2 (x i x r )(x x r ) + (y i y r )(y y r ) + (z i z r )(z z r ) = (x x r ) 2 + (y y r ) 2 + (z z r ) 2 +(x r x i ) 2 + (y r y i ) 2 + (z r z i ) 2 d i (θ) 2 = d r (θ) 2 + d ir 2 d i (θ) 2 where i = 1, 2,..., n with n 4.

16 Use any beacon (say, B 1 ) as reference point. Replace exact distances by measured distances. (x 2 x 1 )(x x 1 ) + (y 2 y 1 )(y y 1 ) + (z 2 z 1 )(z z 1 ) 2[ 1 2 r1 r d 2 21] := b21 (x 3 x 1 )(x x 1 ) + (y 3 y 1 )(y y 1 ) + (z 3 z 1 )(z z 1 ) 2[ 1 2 r1 r d 2 31] := b31. (x n x 1 )(x x 1 ) + (y n y 1 )(y y 1 ) + (z n z 1 )(z z 1 ) 2[ 1 2 r1 r 2 n + d 2 n1] := bn1 Linear system of (n 1) equations in 3 unknowns.

17 Linear Least Squares (LSQ) Model Write the linear system in matrix form: Ax b with x 2 x 1 y 2 y 1 z 2 z 1 x A= 3 x 1 y 3 y 1 z 3 z 1... x n x 1 y n y 1 z n z 1 x x 1, x= y y 1 z z 1, b= b 21 b 31. b n1

18 Minimizing the sum of the squares of the residuals S = (b Ax) T (b Ax) requires solving the normal equation A T Ax = A T b Solution method depends on the condition number of A T A. If A T A is non-singular and well-conditioned then x = (A T A) 1 A T b

19 If A T A is nearly-singular (poorly conditioned): Compute A = QR Q is orthonormal matrix, R is upper-triangular matrix. Solve Rx = Q T b by back substitution when A is full rank. x x 1 The target θ is then θ = y = x + y 1. z z 1

20 Nonlinear Least Squares (NLSQ) Model Minimize the sum of the squares of the errors on the distances: n F (θ) = F (x, y, z) = f i (x, y, z) 2 where i=1 f i (x, y, z) = f i (θ) := d i (θ) r i = (x x i ) 2 + (y y i ) 2 + (z z i ) 2 r i. Recall: r i are the measured distances between the target θ = (x, y, z) and beacon B i = (x i, y i, z i ), and n is the number of beacons.

21 Differentiating F with respect to x yields F (θ) x = 2 n i=1 f i f i (θ) x = 2 n i=1 f i d i (θ) x. The formulae for Let f(θ) = F (θ) y and F (θ) z f 1 (θ) f 2 (θ), F (θ) =. f n (θ) are similar. F (θ) x F (θ) y F (θ) z

22 and define the Jacobian as J(θ) = d 1 (θ) x d 2 (θ) x. d n (θ) x d 1 (θ) y d 2 (θ) y. d n (θ) y d 1 (θ) z d 2 (θ) z. d n (θ) z

23 We must solve F (θ) = 2J(θ) T f(θ) = 0 where J(θ) T f(θ) = n i=1 n i=1 n i=1 (x x i )f i (θ) d i (θ) (y y i )f i (θ) d i (θ) (z z i )f i (θ) d i (θ).

24 Newton-Raphson Method Iterative Solver Problem: Solve the scalar problem f(x) = 0 Solution: Newton s method: x k+1 = x k f(x k) f (x k ) Problem: Solve the vector problem: f(x) = 0 simplest case: n equations, n unknowns. Solution: Newton s method: x k+1 = x k [J(x k )] 1 f(x k )

25 Apply Newton s method to g(θ) = J(θ) T f(θ) = 0. Solution: θ {k+1} = θ {k} [J(θ {k} ) T J(θ {k} )] 1 J(θ {k} ) T f(θ {k} ) where θ {k} denotes the kth estimate of the target. A reasonably accurate initial guess, θ {1}, could be computed with the LSQ method. Starting with θ {1}, iterate until the change θ {k+1} θ {k} is sufficiently small.

26 The expression for J(θ) T J(θ) is n (x x i ) 2 n (x x i )(y y i ) n (x x i )(z z i ) i=1 d i (θ) 2 i=1 d i (θ) 2 i=1 d i (θ) 2 n (x x i )(y y i ) n (y y i ) 2 n (y y i )(z z i ) i=1 d i (θ) 2 i=1 d i (θ) 2 i=1 d i (θ) 2. n (x x i )(z z i ) n (y y i )(z z i ) n (z z i ) 2 i=1 d i (θ) 2 i=1 d i (θ) 2 i=1 d i (θ) 2

27 Mathematica Demonstration 1 Computation of Target using the NLSQ Method Mathematica s NMinimize Function

28 Mathematica Demonstration 2 Computation of Target using the NLSQ Method Newton Iteration

29 Mathematica Demonstration 3 Computation of Target using the LSQ Method

30 Simulation Results of Experiments Beacon coordinates (8 beacons were used) X Y Z

31 Location of 8 Beacons

32 Test Grid of 1000 Points

33 Requirement: determine target within 2 feet (distances measured within 1 2 foot). One thousand target points on a rectangular grid. Top of box is 5 feet below lowest beacon. For each target point, 10, 000 data sets were generated. Each data set consisted of one measurement from each beacon. Each measurement was obtained by adding to the true distance a random error distributed uniformly on ( 0.5, 0.5).

34 Methods were implemented in Macsyma and C++ Horizontal coordinates were accurate (98% of test points). Vertical coordinate (height) was imprecise (off by several feet for 5% of test points). Trouble with hardware (AccuTrack, Canada).

35 Conclusions Exact linearization for nonlinear problem. LSQ method is reliable even with small samples. NLSQ method gives best performance. Methods are easy to implement. Good alternative for applications where GPS cannot be used. Publications are on the Internet: URL: whereman/

36 Thank You

37 Publications 1. W. Navidi, W. Murphy, Jr., and W. Hereman, Statistical methods in surveying by trilateration, Computational Statistics and Data Analysis, 27(2), pp (1998). 2. W. Murphy and W. Hereman, Determination of a position in three dimensions using trilateration and approximate distances, Technical Report MCS-95-07, Department of Mathematical and Computer Sciences, Colorado School of Mines, Golden, Colorado (1995), 19 pages. 3. W. Murphy, Determination of a Position Using Approximate Distances and Trilateration, M.S. Thesis, Colorado School of Mines, May 1992.

Symbolic Computation of Recursion Operators of Nonlinear Differential-Difference Equations

Symbolic Computation of Recursion Operators of Nonlinear Differential-Difference Equations Symbolic Computation of Recursion Operators of Nonlinear Differential-Difference Equations Ünal Göktaş and Willy Hereman Department of Computer Engineering Turgut Özal University, Keçiören, Ankara, Turkey

More information

Lecture 6. Regularized least-squares and minimum-norm methods 6 1

Lecture 6. Regularized least-squares and minimum-norm methods 6 1 Regularized least-squares and minimum-norm methods 6 1 Lecture 6 Regularized least-squares and minimum-norm methods EE263 Autumn 2004 multi-objective least-squares regularized least-squares nonlinear least-squares

More information

Presentation for the Visiting Committee & Graduate Seminar SYMBOLIC COMPUTING IN RESEARCH. Willy Hereman

Presentation for the Visiting Committee & Graduate Seminar SYMBOLIC COMPUTING IN RESEARCH. Willy Hereman Presentation for the Visiting Committee & Graduate Seminar SYMBOLIC COMPUTING IN RESEARCH Willy Hereman Dept. of Mathematical and Computer Sciences Colorado School of Mines Golden, CO 80401-1887 Monday,

More information

Linear Models Review

Linear Models Review Linear Models Review Vectors in IR n will be written as ordered n-tuples which are understood to be column vectors, or n 1 matrices. A vector variable will be indicted with bold face, and the prime sign

More information

j=1 r 1 x 1 x n. r m r j (x) r j r j (x) r j (x). r j x k

j=1 r 1 x 1 x n. r m r j (x) r j r j (x) r j (x). r j x k Maria Cameron Nonlinear Least Squares Problem The nonlinear least squares problem arises when one needs to find optimal set of parameters for a nonlinear model given a large set of data The variables x,,

More information

Abstract Trilateration techniques use distance measurements to survey the spatial coordinates of unknown positions. In practice, distances are measure

Abstract Trilateration techniques use distance measurements to survey the spatial coordinates of unknown positions. In practice, distances are measure Statistical Methods in Surveying by Trilateration William Navidi y, William S. Murphy Jr. z, and Willy Hereman y ydepartment of Mathematical and Computer Sciences Colorado School of Mines Golden, Colorado

More information

NIST Electromagnetic Field Division Seminar SYMBOLIC COMPUTING IN RESEARCH. Willy Hereman

NIST Electromagnetic Field Division Seminar SYMBOLIC COMPUTING IN RESEARCH. Willy Hereman . NIST Electromagnetic Field Division Seminar SYMBOLIC COMPUTING IN RESEARCH Willy Hereman Dept. of Mathematical and Computer Sciences Colorado School of Mines Golden, CO 80401-1887 Tuesday, March 10,

More information

Numerical Methods I Solving Nonlinear Equations

Numerical Methods I Solving Nonlinear Equations Numerical Methods I Solving Nonlinear Equations Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 16th, 2014 A. Donev (Courant Institute)

More information

Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Scientific Computing: An Introductory Survey Chapter 5 Nonlinear Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction

More information

Lecture Note 8: Inverse Kinematics

Lecture Note 8: Inverse Kinematics ECE5463: Introduction to Robotics Lecture Note 8: Inverse Kinematics Prof. Wei Zhang Department of Electrical and Computer Engineering Ohio State University Columbus, Ohio, USA Spring 2018 Lecture 8 (ECE5463

More information

Symbolic Computation of Conservation Laws of Nonlinear Partial Differential Equations

Symbolic Computation of Conservation Laws of Nonlinear Partial Differential Equations Symbolic Computation of Conservation Laws of Nonlinear Partial Differential Equations Willy Hereman Department of Mathematical and Computer Sciences Colorado School of Mines Golden, Colorado whereman@mines.edu

More information

Lecture Note 8: Inverse Kinematics

Lecture Note 8: Inverse Kinematics ECE5463: Introduction to Robotics Lecture Note 8: Inverse Kinematics Prof. Wei Zhang Department of Electrical and Computer Engineering Ohio State University Columbus, Ohio, USA Spring 2018 Lecture 8 (ECE5463

More information

Numerical solutions of nonlinear systems of equations

Numerical solutions of nonlinear systems of equations Numerical solutions of nonlinear systems of equations Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan E-mail: min@math.ntnu.edu.tw August 28, 2011 Outline 1 Fixed points

More information

Continuous Optimization

Continuous Optimization Continuous Optimization Sanzheng Qiao Department of Computing and Software McMaster University March, 2009 Outline 1 Introduction 2 Golden Section Search 3 Multivariate Functions Steepest Descent Method

More information

Stabilization and Acceleration of Algebraic Multigrid Method

Stabilization and Acceleration of Algebraic Multigrid Method Stabilization and Acceleration of Algebraic Multigrid Method Recursive Projection Algorithm A. Jemcov J.P. Maruszewski Fluent Inc. October 24, 2006 Outline 1 Need for Algorithm Stabilization and Acceleration

More information

Application of an Optimal Stochastic Newton- Raphson Technique to Triangulation-Based Localization Systems

Application of an Optimal Stochastic Newton- Raphson Technique to Triangulation-Based Localization Systems Application of an Optimal Stochastic Newton- Raphson Technique to Triangulation-Based Localization Systems Khaled Kamal Saab Department of Electrical and Computer Engineering Georgia Institute of Technology

More information

Least Squares. Tom Lyche. October 26, Centre of Mathematics for Applications, Department of Informatics, University of Oslo

Least Squares. Tom Lyche. October 26, Centre of Mathematics for Applications, Department of Informatics, University of Oslo Least Squares Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo October 26, 2010 Linear system Linear system Ax = b, A C m,n, b C m, x C n. under-determined

More information

x 2 x n r n J(x + t(x x ))(x x )dt. For warming-up we start with methods for solving a single equation of one variable.

x 2 x n r n J(x + t(x x ))(x x )dt. For warming-up we start with methods for solving a single equation of one variable. Maria Cameron 1. Fixed point methods for solving nonlinear equations We address the problem of solving an equation of the form (1) r(x) = 0, where F (x) : R n R n is a vector-function. Eq. (1) can be written

More information

S.F. Xu (Department of Mathematics, Peking University, Beijing)

S.F. Xu (Department of Mathematics, Peking University, Beijing) Journal of Computational Mathematics, Vol.14, No.1, 1996, 23 31. A SMALLEST SINGULAR VALUE METHOD FOR SOLVING INVERSE EIGENVALUE PROBLEMS 1) S.F. Xu (Department of Mathematics, Peking University, Beijing)

More information

Lecture 10: Powers of Matrices, Difference Equations

Lecture 10: Powers of Matrices, Difference Equations Lecture 10: Powers of Matrices, Difference Equations Difference Equations A difference equation, also sometimes called a recurrence equation is an equation that defines a sequence recursively, i.e. each

More information

Preliminary Examination, Numerical Analysis, August 2016

Preliminary Examination, Numerical Analysis, August 2016 Preliminary Examination, Numerical Analysis, August 2016 Instructions: This exam is closed books and notes. The time allowed is three hours and you need to work on any three out of questions 1-4 and any

More information

Outline. Scientific Computing: An Introductory Survey. Nonlinear Equations. Nonlinear Equations. Examples: Nonlinear Equations

Outline. Scientific Computing: An Introductory Survey. Nonlinear Equations. Nonlinear Equations. Examples: Nonlinear Equations Methods for Systems of Methods for Systems of Outline Scientific Computing: An Introductory Survey Chapter 5 1 Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign

More information

CS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares

CS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares CS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares Robert Bridson October 29, 2008 1 Hessian Problems in Newton Last time we fixed one of plain Newton s problems by introducing line search

More information

Math 211 Business Calculus TEST 3. Question 1. Section 2.2. Second Derivative Test.

Math 211 Business Calculus TEST 3. Question 1. Section 2.2. Second Derivative Test. Math 211 Business Calculus TEST 3 Question 1. Section 2.2. Second Derivative Test. p. 1/?? Math 211 Business Calculus TEST 3 Question 1. Section 2.2. Second Derivative Test. Question 2. Section 2.3. Graph

More information

1. Method 1: bisection. The bisection methods starts from two points a 0 and b 0 such that

1. Method 1: bisection. The bisection methods starts from two points a 0 and b 0 such that Chapter 4 Nonlinear equations 4.1 Root finding Consider the problem of solving any nonlinear relation g(x) = h(x) in the real variable x. We rephrase this problem as one of finding the zero (root) of a

More information

PHYS 410/555 Computational Physics Solution of Non Linear Equations (a.k.a. Root Finding) (Reference Numerical Recipes, 9.0, 9.1, 9.

PHYS 410/555 Computational Physics Solution of Non Linear Equations (a.k.a. Root Finding) (Reference Numerical Recipes, 9.0, 9.1, 9. PHYS 410/555 Computational Physics Solution of Non Linear Equations (a.k.a. Root Finding) (Reference Numerical Recipes, 9.0, 9.1, 9.4) We will consider two cases 1. f(x) = 0 1-dimensional 2. f(x) = 0 d-dimensional

More information

Lecture 6, Sci. Comp. for DPhil Students

Lecture 6, Sci. Comp. for DPhil Students Lecture 6, Sci. Comp. for DPhil Students Nick Trefethen, Thursday 1.11.18 Today II.3 QR factorization II.4 Computation of the QR factorization II.5 Linear least-squares Handouts Quiz 4 Householder s 4-page

More information

Maria Cameron. f(x) = 1 n

Maria Cameron. f(x) = 1 n Maria Cameron 1. Local algorithms for solving nonlinear equations Here we discuss local methods for nonlinear equations r(x) =. These methods are Newton, inexact Newton and quasi-newton. We will show that

More information

13. Nonlinear least squares

13. Nonlinear least squares L. Vandenberghe ECE133A (Fall 2018) 13. Nonlinear least squares definition and examples derivatives and optimality condition Gauss Newton method Levenberg Marquardt method 13.1 Nonlinear least squares

More information

EAD 115. Numerical Solution of Engineering and Scientific Problems. David M. Rocke Department of Applied Science

EAD 115. Numerical Solution of Engineering and Scientific Problems. David M. Rocke Department of Applied Science EAD 115 Numerical Solution of Engineering and Scientific Problems David M. Rocke Department of Applied Science Taylor s Theorem Can often approximate a function by a polynomial The error in the approximation

More information

Computational Methods. Least Squares Approximation/Optimization

Computational Methods. Least Squares Approximation/Optimization Computational Methods Least Squares Approximation/Optimization Manfred Huber 2011 1 Least Squares Least squares methods are aimed at finding approximate solutions when no precise solution exists Find the

More information

The N k Problem using AC Power Flows

The N k Problem using AC Power Flows The N k Problem using AC Power Flows Sean Harnett 5-19-2011 Outline Introduction AC power flow model The optimization problem Some results Goal: find a small set of lines whose removal will cause the power

More information

An Overly Simplified and Brief Review of Differential Equation Solution Methods. 1. Some Common Exact Solution Methods for Differential Equations

An Overly Simplified and Brief Review of Differential Equation Solution Methods. 1. Some Common Exact Solution Methods for Differential Equations An Overly Simplified and Brief Review of Differential Equation Solution Methods We will be dealing with initial or boundary value problems. A typical initial value problem has the form y y 0 y(0) 1 A typical

More information

17 Solution of Nonlinear Systems

17 Solution of Nonlinear Systems 17 Solution of Nonlinear Systems We now discuss the solution of systems of nonlinear equations. An important ingredient will be the multivariate Taylor theorem. Theorem 17.1 Let D = {x 1, x 2,..., x m

More information

NONLINEAR DC ANALYSIS

NONLINEAR DC ANALYSIS ECE 552 Numerical Circuit Analysis Chapter Six NONLINEAR DC ANALYSIS OR: Solution of Nonlinear Algebraic Equations I. Hajj 2017 Nonlinear Algebraic Equations A system of linear equations Ax = b has a

More information

Iterative Methods. Splitting Methods

Iterative Methods. Splitting Methods Iterative Methods Splitting Methods 1 Direct Methods Solving Ax = b using direct methods. Gaussian elimination (using LU decomposition) Variants of LU, including Crout and Doolittle Other decomposition

More information

CS 323: Numerical Analysis and Computing

CS 323: Numerical Analysis and Computing CS 323: Numerical Analysis and Computing MIDTERM #1 Instructions: This is an open notes exam, i.e., you are allowed to consult any textbook, your class notes, homeworks, or any of the handouts from us.

More information

Lecture 5. September 4, 2018 Math/CS 471: Introduction to Scientific Computing University of New Mexico

Lecture 5. September 4, 2018 Math/CS 471: Introduction to Scientific Computing University of New Mexico Lecture 5 September 4, 2018 Math/CS 471: Introduction to Scientific Computing University of New Mexico 1 Review: Office hours at regularly scheduled times this week Tuesday: 9:30am-11am Wed: 2:30pm-4:00pm

More information

Quiescent Steady State (DC) Analysis The Newton-Raphson Method

Quiescent Steady State (DC) Analysis The Newton-Raphson Method Quiescent Steady State (DC) Analysis The Newton-Raphson Method J. Roychowdhury, University of California at Berkeley Slide 1 Solving the System's DAEs DAEs: many types of solutions useful DC steady state:

More information

Jim Lambers MAT 419/519 Summer Session Lecture 11 Notes

Jim Lambers MAT 419/519 Summer Session Lecture 11 Notes Jim Lambers MAT 49/59 Summer Session 20-2 Lecture Notes These notes correspond to Section 34 in the text Broyden s Method One of the drawbacks of using Newton s Method to solve a system of nonlinear equations

More information

Application of Euler and Homotopy Operators to Integrability Testing

Application of Euler and Homotopy Operators to Integrability Testing Application of Euler and Homotopy Operators to Integrability Testing Willy Hereman Department of Applied Mathematics and Statistics Colorado School of Mines Golden, Colorado, U.S.A. http://www.mines.edu/fs

More information

Lecture 10. Neural networks and optimization. Machine Learning and Data Mining November Nando de Freitas UBC. Nonlinear Supervised Learning

Lecture 10. Neural networks and optimization. Machine Learning and Data Mining November Nando de Freitas UBC. Nonlinear Supervised Learning Lecture 0 Neural networks and optimization Machine Learning and Data Mining November 2009 UBC Gradient Searching for a good solution can be interpreted as looking for a minimum of some error (loss) function

More information

COMP 558 lecture 18 Nov. 15, 2010

COMP 558 lecture 18 Nov. 15, 2010 Least squares We have seen several least squares problems thus far, and we will see more in the upcoming lectures. For this reason it is good to have a more general picture of these problems and how to

More information

ECEN 615 Methods of Electric Power Systems Analysis Lecture 18: Least Squares, State Estimation

ECEN 615 Methods of Electric Power Systems Analysis Lecture 18: Least Squares, State Estimation ECEN 615 Methods of Electric Power Systems Analysis Lecture 18: Least Squares, State Estimation Prof. om Overbye Dept. of Electrical and Computer Engineering exas A&M University overbye@tamu.edu Announcements

More information

Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted

More information

Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted

More information

Symbolic Computation of Conservation Laws, Generalized Symmetries, and Recursion Operators for Nonlinear Differential-Difference Equations

Symbolic Computation of Conservation Laws, Generalized Symmetries, and Recursion Operators for Nonlinear Differential-Difference Equations Symbolic Computation of Conservation Laws, Generalized Symmetries, and Recursion Operators for Nonlinear Differential-Difference Equations Ünal Götaş and Willy Hereman Abstract Algorithms for the symbolic

More information

Integration, differentiation, and root finding. Phys 420/580 Lecture 7

Integration, differentiation, and root finding. Phys 420/580 Lecture 7 Integration, differentiation, and root finding Phys 420/580 Lecture 7 Numerical integration Compute an approximation to the definite integral I = b Find area under the curve in the interval Trapezoid Rule:

More information

Estimating the Pure Characteristics Demand Model: A Computational Note

Estimating the Pure Characteristics Demand Model: A Computational Note Estimating the Pure Characteristics Demand Model: A Computational Note Minjae Song School of Economics, Georgia Institute of Technology April, 2006 Abstract This paper provides details of the computational

More information

Orthogonality. Orthonormal Bases, Orthogonal Matrices. Orthogonality

Orthogonality. Orthonormal Bases, Orthogonal Matrices. Orthogonality Orthonormal Bases, Orthogonal Matrices The Major Ideas from Last Lecture Vector Span Subspace Basis Vectors Coordinates in different bases Matrix Factorization (Basics) The Major Ideas from Last Lecture

More information

Numerical solution of Least Squares Problems 1/32

Numerical solution of Least Squares Problems 1/32 Numerical solution of Least Squares Problems 1/32 Linear Least Squares Problems Suppose that we have a matrix A of the size m n and the vector b of the size m 1. The linear least square problem is to find

More information

Topic 5: Non-Linear Relationships and Non-Linear Least Squares

Topic 5: Non-Linear Relationships and Non-Linear Least Squares Topic 5: Non-Linear Relationships and Non-Linear Least Squares Non-linear Relationships Many relationships between variables are non-linear. (Examples) OLS may not work (recall A.1). It may be biased and

More information

Outline. Scientific Computing: An Introductory Survey. Optimization. Optimization Problems. Examples: Optimization Problems

Outline. Scientific Computing: An Introductory Survey. Optimization. Optimization Problems. Examples: Optimization Problems Outline Scientific Computing: An Introductory Survey Chapter 6 Optimization 1 Prof. Michael. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction

More information

Two hours. To be provided by Examinations Office: Mathematical Formula Tables. THE UNIVERSITY OF MANCHESTER. 29 May :45 11:45

Two hours. To be provided by Examinations Office: Mathematical Formula Tables. THE UNIVERSITY OF MANCHESTER. 29 May :45 11:45 Two hours MATH20602 To be provided by Examinations Office: Mathematical Formula Tables. THE UNIVERSITY OF MANCHESTER NUMERICAL ANALYSIS 1 29 May 2015 9:45 11:45 Answer THREE of the FOUR questions. If more

More information

1 Lecture 8: Interpolating polynomials.

1 Lecture 8: Interpolating polynomials. 1 Lecture 8: Interpolating polynomials. 1.1 Horner s method Before turning to the main idea of this part of the course, we consider how to evaluate a polynomial. Recall that a polynomial is an expression

More information

Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 5. Nonlinear Equations

Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 5. Nonlinear Equations Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T Heath Chapter 5 Nonlinear Equations Copyright c 2001 Reproduction permitted only for noncommercial, educational

More information

Lecture 7 Unconstrained nonlinear programming

Lecture 7 Unconstrained nonlinear programming Lecture 7 Unconstrained nonlinear programming Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University,

More information

The Newton-Raphson Algorithm

The Newton-Raphson Algorithm The Newton-Raphson Algorithm David Allen University of Kentucky January 31, 2013 1 The Newton-Raphson Algorithm The Newton-Raphson algorithm, also called Newton s method, is a method for finding the minimum

More information

Accurate Thermal Analysis Considering Nonlinear Thermal Conductivity

Accurate Thermal Analysis Considering Nonlinear Thermal Conductivity Accurate Thermal Analysis Considering Nonlinear Thermal Conductivity Anand Ramalingam 1 Frank Liu 2 Sani R. Nassif 2 David Z. Pan 1 1 Department of Electrical and Computer Engineering, The University of

More information

Numerical Methods I: Orthogonalization and Newton s method

Numerical Methods I: Orthogonalization and Newton s method 1/42 Numerical Methods I: Orthogonalization and Newton s method Georg Stadler Courant Institute, NYU stadler@cims.nyu.edu October 5, 2017 Overview 2/42 Linear least squares and orthogonalization methods

More information

Power System Analysis Prof. A. K. Sinha Department of Electrical Engineering Indian Institute of Technology, Kharagpur. Lecture - 21 Power Flow VI

Power System Analysis Prof. A. K. Sinha Department of Electrical Engineering Indian Institute of Technology, Kharagpur. Lecture - 21 Power Flow VI Power System Analysis Prof. A. K. Sinha Department of Electrical Engineering Indian Institute of Technology, Kharagpur Lecture - 21 Power Flow VI (Refer Slide Time: 00:57) Welcome to lesson 21. In this

More information

CS 450 Numerical Analysis. Chapter 5: Nonlinear Equations

CS 450 Numerical Analysis. Chapter 5: Nonlinear Equations Lecture slides based on the textbook Scientific Computing: An Introductory Survey by Michael T. Heath, copyright c 2018 by the Society for Industrial and Applied Mathematics. http://www.siam.org/books/cl80

More information

6.1 Polynomial Functions

6.1 Polynomial Functions 6.1 Polynomial Functions Definition. A polynomial function is any function p(x) of the form p(x) = p n x n + p n 1 x n 1 + + p 2 x 2 + p 1 x + p 0 where all of the exponents are non-negative integers and

More information

Symbolic Computation of Conservation Laws of Nonlinear Partial Differential Equations in Multiple Space Dimensions

Symbolic Computation of Conservation Laws of Nonlinear Partial Differential Equations in Multiple Space Dimensions Symbolic Computation of Conservation Laws of Nonlinear Partial Differential Equations in Multiple Space Dimensions Willy Hereman and Douglas Poole Department of Mathematical and Computer Sciences Colorado

More information

11 a 12 a 21 a 11 a 22 a 12 a 21. (C.11) A = The determinant of a product of two matrices is given by AB = A B 1 1 = (C.13) and similarly.

11 a 12 a 21 a 11 a 22 a 12 a 21. (C.11) A = The determinant of a product of two matrices is given by AB = A B 1 1 = (C.13) and similarly. C PROPERTIES OF MATRICES 697 to whether the permutation i 1 i 2 i N is even or odd, respectively Note that I =1 Thus, for a 2 2 matrix, the determinant takes the form A = a 11 a 12 = a a 21 a 11 a 22 a

More information

2t t dt.. So the distance is (t2 +6) 3/2

2t t dt.. So the distance is (t2 +6) 3/2 Math 8, Solutions to Review for the Final Exam Question : The distance is 5 t t + dt To work that out, integrate by parts with u t +, so that t dt du The integral is t t + dt u du u 3/ (t +) 3/ So the

More information

arxiv: v1 [math.na] 5 May 2011

arxiv: v1 [math.na] 5 May 2011 ITERATIVE METHODS FOR COMPUTING EIGENVALUES AND EIGENVECTORS MAYSUM PANJU arxiv:1105.1185v1 [math.na] 5 May 2011 Abstract. We examine some numerical iterative methods for computing the eigenvalues and

More information

CS 323: Numerical Analysis and Computing

CS 323: Numerical Analysis and Computing CS 323: Numerical Analysis and Computing MIDTERM #2 Instructions: This is an open notes exam, i.e., you are allowed to consult any textbook, your class notes, homeworks, or any of the handouts from us.

More information

Root Finding (and Optimisation)

Root Finding (and Optimisation) Root Finding (and Optimisation) M.Sc. in Mathematical Modelling & Scientific Computing, Practical Numerical Analysis Michaelmas Term 2018, Lecture 4 Root Finding The idea of root finding is simple we want

More information

ENGG5781 Matrix Analysis and Computations Lecture 8: QR Decomposition

ENGG5781 Matrix Analysis and Computations Lecture 8: QR Decomposition ENGG5781 Matrix Analysis and Computations Lecture 8: QR Decomposition Wing-Kin (Ken) Ma 2017 2018 Term 2 Department of Electronic Engineering The Chinese University of Hong Kong Lecture 8: QR Decomposition

More information

Written Homework 7 Solutions

Written Homework 7 Solutions Written Homework 7 Solutions MATH 0 - CSM Assignment: pp 5-56 problem 6, 8, 0,,, 5, 7, 8, 20. 6. Find formulas for the derivatives of the following functions; that is, differentiate them. Solution: (a)

More information

Interpreting Derivatives, Local Linearity, Newton s

Interpreting Derivatives, Local Linearity, Newton s Unit #4 : Method Interpreting Derivatives, Local Linearity, Newton s Goals: Review inverse trigonometric functions and their derivatives. Create and use linearization/tangent line formulas. Investigate

More information

Numerical Methods for PDE-Constrained Optimization

Numerical Methods for PDE-Constrained Optimization Numerical Methods for PDE-Constrained Optimization Richard H. Byrd 1 Frank E. Curtis 2 Jorge Nocedal 2 1 University of Colorado at Boulder 2 Northwestern University Courant Institute of Mathematical Sciences,

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 7: More on Householder Reflectors; Least Squares Problems Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 15 Outline

More information

Chapter 3 Numerical Methods

Chapter 3 Numerical Methods Chapter 3 Numerical Methods Part 2 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization 1 Outline 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization Summary 2 Outline 3.2

More information

Unit 7: Introduction to Functions

Unit 7: Introduction to Functions Section 7.1: Relations and Functions Section 7.2: Function Notation Section 7.3: Domain and Range Section 7.4: Practical Domain and Range Section 7.5: Applications KEY TERMS AND CONCEPTS Look for the following

More information

Orthogonality. 6.1 Orthogonal Vectors and Subspaces. Chapter 6

Orthogonality. 6.1 Orthogonal Vectors and Subspaces. Chapter 6 Chapter 6 Orthogonality 6.1 Orthogonal Vectors and Subspaces Recall that if nonzero vectors x, y R n are linearly independent then the subspace of all vectors αx + βy, α, β R (the space spanned by x and

More information

Lecture 2: Linear Algebra Review

Lecture 2: Linear Algebra Review EE 227A: Convex Optimization and Applications January 19 Lecture 2: Linear Algebra Review Lecturer: Mert Pilanci Reading assignment: Appendix C of BV. Sections 2-6 of the web textbook 1 2.1 Vectors 2.1.1

More information

Introduction - Motivation. Many phenomena (physical, chemical, biological, etc.) are model by differential equations. f f(x + h) f(x) (x) = lim

Introduction - Motivation. Many phenomena (physical, chemical, biological, etc.) are model by differential equations. f f(x + h) f(x) (x) = lim Introduction - Motivation Many phenomena (physical, chemical, biological, etc.) are model by differential equations. Recall the definition of the derivative of f(x) f f(x + h) f(x) (x) = lim. h 0 h Its

More information

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column

More information

Non-polynomial Least-squares fitting

Non-polynomial Least-squares fitting Applied Math 205 Last time: piecewise polynomial interpolation, least-squares fitting Today: underdetermined least squares, nonlinear least squares Homework 1 (and subsequent homeworks) have several parts

More information

EE263 homework 3 solutions

EE263 homework 3 solutions EE263 Prof. S. Boyd EE263 homework 3 solutions 2.17 Gradient of some common functions. Recall that the gradient of a differentiable function f : R n R, at a point x R n, is defined as the vector f(x) =

More information

Linear Algebra Methods for Data Mining

Linear Algebra Methods for Data Mining Linear Algebra Methods for Data Mining Saara Hyvönen, Saara.Hyvonen@cs.helsinki.fi Spring 2007 1. Basic Linear Algebra Linear Algebra Methods for Data Mining, Spring 2007, University of Helsinki Example

More information

Statistics 580 Optimization Methods

Statistics 580 Optimization Methods Statistics 580 Optimization Methods Introduction Let fx be a given real-valued function on R p. The general optimization problem is to find an x ɛ R p at which fx attain a maximum or a minimum. It is of

More information

CS137 Introduction to Scientific Computing Winter Quarter 2004 Solutions to Homework #3

CS137 Introduction to Scientific Computing Winter Quarter 2004 Solutions to Homework #3 CS137 Introduction to Scientific Computing Winter Quarter 2004 Solutions to Homework #3 Felix Kwok February 27, 2004 Written Problems 1. (Heath E3.10) Let B be an n n matrix, and assume that B is both

More information

Constrained optimization. Unconstrained optimization. One-dimensional. Multi-dimensional. Newton with equality constraints. Active-set method.

Constrained optimization. Unconstrained optimization. One-dimensional. Multi-dimensional. Newton with equality constraints. Active-set method. Optimization Unconstrained optimization One-dimensional Multi-dimensional Newton s method Basic Newton Gauss- Newton Quasi- Newton Descent methods Gradient descent Conjugate gradient Constrained optimization

More information

SIMPLEX LIKE (aka REDUCED GRADIENT) METHODS. REDUCED GRADIENT METHOD (Wolfe)

SIMPLEX LIKE (aka REDUCED GRADIENT) METHODS. REDUCED GRADIENT METHOD (Wolfe) 19 SIMPLEX LIKE (aka REDUCED GRADIENT) METHODS The REDUCED GRADIENT algorithm and its variants such as the CONVEX SIMPLEX METHOD (CSM) and the GENERALIZED REDUCED GRADIENT (GRG) algorithm are approximation

More information

DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University

DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University January 17 2019 Logistics HW 1 is on Piazza and Gradescope Deadline: Friday, Jan. 25, 2019 Office

More information

Chapter 6: Derivative-Based. optimization 1

Chapter 6: Derivative-Based. optimization 1 Chapter 6: Derivative-Based Optimization Introduction (6. Descent Methods (6. he Method of Steepest Descent (6.3 Newton s Methods (NM (6.4 Step Size Determination (6.5 Nonlinear Least-Squares Problems

More information

converges to a root, it may not always be the root you have in mind.

converges to a root, it may not always be the root you have in mind. Math 1206 Calculus Sec. 4.9: Newton s Method I. Introduction For linear and quadratic equations there are simple formulas for solving for the roots. For third- and fourth-degree equations there are also

More information

Matrix decompositions

Matrix decompositions Matrix decompositions Zdeněk Dvořák May 19, 2015 Lemma 1 (Schur decomposition). If A is a symmetric real matrix, then there exists an orthogonal matrix Q and a diagonal matrix D such that A = QDQ T. The

More information

Analysis Methods in Atmospheric and Oceanic Science

Analysis Methods in Atmospheric and Oceanic Science Analysis Methods in Atmospheric and Oceanic Science Copyright 2016 University of Maryland. 1 AOSC 652 Root Finding & Function Minimization Week 7, Day 2 12 Oct 2016 2 Method #3: Newton-Raphson Root is

More information

MecE 390 Final examination, Winter 2014

MecE 390 Final examination, Winter 2014 MecE 390 Final examination, Winter 2014 Directions: (i) a double-sided 8.5 11 formula sheet is permitted, (ii) no calculators are permitted, (iii) the exam is 80 minutes in duration; please turn your paper

More information

Numerical Linear Algebra

Numerical Linear Algebra Numerical Linear Algebra The two principal problems in linear algebra are: Linear system Given an n n matrix A and an n-vector b, determine x IR n such that A x = b Eigenvalue problem Given an n n matrix

More information

Lecture 3: QR-Factorization

Lecture 3: QR-Factorization Lecture 3: QR-Factorization This lecture introduces the Gram Schmidt orthonormalization process and the associated QR-factorization of matrices It also outlines some applications of this factorization

More information

Methods for Computing Periodic Steady-State Jacob White

Methods for Computing Periodic Steady-State Jacob White Introduction to Simulation - Lecture 15 Methods for Computing Periodic Steady-State Jacob White Thanks to Deepak Ramaswamy, Michal Rewienski, and Karen Veroy Outline Periodic Steady-state problems Application

More information

The Normal Equations. For A R m n with m > n, A T A is singular if and only if A is rank-deficient. 1 Proof:

The Normal Equations. For A R m n with m > n, A T A is singular if and only if A is rank-deficient. 1 Proof: Applied Math 205 Homework 1 now posted. Due 5 PM on September 26. Last time: piecewise polynomial interpolation, least-squares fitting Today: least-squares, nonlinear least-squares The Normal Equations

More information

CLASS NOTES Models, Algorithms and Data: Introduction to computing 2018

CLASS NOTES Models, Algorithms and Data: Introduction to computing 2018 CLASS NOTES Models, Algorithms and Data: Introduction to computing 2018 Petros Koumoutsakos, Jens Honore Walther (Last update: April 16, 2018) IMPORTANT DISCLAIMERS 1. REFERENCES: Much of the material

More information

Math 128A: Homework 2 Solutions

Math 128A: Homework 2 Solutions Math 128A: Homework 2 Solutions Due: June 28 1. In problems where high precision is not needed, the IEEE standard provides a specification for single precision numbers, which occupy 32 bits of storage.

More information

Lecture Note 7: Velocity Kinematics and Jacobian

Lecture Note 7: Velocity Kinematics and Jacobian ECE5463: Introduction to Robotics Lecture Note 7: Velocity Kinematics and Jacobian Prof. Wei Zhang Department of Electrical and Computer Engineering Ohio State University Columbus, Ohio, USA Spring 2018

More information