Nonlinear equations. Norms for R n. Convergence orders for iterative methods
|
|
- Dustin Nicholson
- 5 years ago
- Views:
Transcription
1 Nonlinear equations Norms for R n Assume that X is a vector space. A norm is a mapping X R with x such that for all x, y X, α R x = = x = αx = α x x + y x + y We define the following norms on the vector space R n : x 1 = x x n x 2 = ( x x n 2) 1/2 x = max { x 1,..., x n } A matrix A R n n corresponds to a linear mapping from R n to R n. If x denotes a vector norm for x R n we can define the matrix norm A as follows: Ax A = sup x R n x = sup Ax x R n x =1 Note that S = {x R n x = 1} is a compact set, hence there exists a point x S with Ax = sup x R n Ax, x =1 and we can write max instead of sup. Lemma 1. For E R n n with E < 1 the matrix I + E is invertible and (I + E) E. Proof: x = (1 + E)x Ex (1 + E)x + E x, hence (I + E)x (1 E ) x. Convergence orders for iterative methods We want to approximate x R n. An iterative method gives a sequence x, x 1, x 2, x 3,.... We say the method converges if looing at quotients lim x = x. We can specify more precisely how quicly it converges by x +1 x x x α for α 1. This leads to so-called q-orders of convergence (where q stands for quotient): the method converges with q-order 1 or q-linearly if there exists C < 1 such that for all x +1 x C x x the method converges with q-order 2 or q-quadratically if lim x = x and there exists C such that for all x +1 x C x x 2 the method converges q-superlinearly if x +1 x lim x x = Example: We consider a nonlinear equation f(x) = where x D R. Assume 1
2 f(x ) = the derivatives f and f exist around x and are continuous f (x ) Then there exists ɛ > such that for x x < ɛ the Newton method converges q-quadratically for x x < ɛ and x 1 x < ɛ the secant method converges with q-order α = Hence it converges q-superlinearly. In many cases we cannot prove that the error improves for every single step from to + 1, but we still have upper bounds x x E where E converges with a certain q-order to zero. This leads to so-called r-orders of convergence: a method converges with r-order 1 or r-linearly if x x E where E converges with q-order 1 to a method converges with r-order 2 or r-quadratically if x x E where E converges with q-order 2 to a method converges r-superlinearly if x x E where E converges with superlinearly to Example: We consider a nonlinear equation f(x) = where f is continuous on the interval [a, b ] f(a ) f(b ) < Then the bisection method gives a sequence of intervals [a, b ]. Let c = (a + b )/2. Then the sequence c converges r-linearly to a point x with f(x ) = : c x E = b a 2 +1 Note that in general we do not have q-linear convergence of c for the bisection method. Derivatives Let X and Y denote normed vector spaces. Consider a mapping f from D X to Y. We say that f is Frechet differentiable at a point x D if there exists a linear mapping A: X Y such that f(x) f(x ) A(x x ) x x x x Lipschitz conditions We say a function f satisfies a Lipschitz condition with constant L on D if f(x) f(y) y x for all x, y D Lemma 2. Assume D is convex f(x) exists and is continuous on D f(x) L for x D Then f satisfies a Lipschitz condition on D with constant L on D. 2
3 Taylor remainder terms For a function f : D R with D R we have estimates ( ) f(x + h) f(x) max f (u) h u conv(x,y) if f C 1 (D) ( ) 1 f(x + h) f(x) f (x)h max f (u) u conv(x,y) 2 h 2 if f C 2 (D). We would lie to have analogous estimates for a function f : D R n with D R n. Assume thatf C 1 (D). Then the derivative exists in the Frechet sense: f(x + h) = f(x) + Df(x) h + R, R = o( h ) for h where Df(x): R n R n is a linear mapping h Df(x) h. Since linear mappings correspond to matrices in R n n we use the notation f 1 (x) 1 f 1 (x) n f 1 (x) f(x) =., Df(x) = f (x) =.. f n (x) 1 f n (x) n f n (x) y = Df(x) h y i = j f i (x)h j j=1 y i j f i (x) h, y max j=1 Df(x) = max i=1,...,n i=1,...,n j f i (x) h j=1 j f i (x) with i = x i. Assume that D is convex and x, y D. Let h := y x and conv(x, y) D denote the line segment betwen x and y. We have by the fundamental theorem of calculus and the chain rule for j=1 F (t) := f(x + th), F (t) = Df(x + th) h f(y) f(x) = F (1) F () = f(y) f(x) F (t)dt = f (x + th)h dt ( ) f (x + th h dt max f (u) y x. u conv(x,y) This shows that the Lipschitz constant for f in D is bounded by max u D f (u). Note that we first choose a vector norm, e.g., v = max j=1,...,n v j. For A R n n this induces a matrix norm: A = max Av = max A ij. v =1 i f (x) = max f (x)v = max j f i. v =1 i We can read this as an estimate for the Taylor remainder term of order 1: f(y) = f(x) + R 1, R 1. Now we want to consider the next Taylor term and remainder: f(y) = f(x) + f (x)(y x) + R 2 : R 2 = f(y) f(x) f (x)(y x) = Assume that f satisfies a Lipschitz condition for x, y D: j j [f (x + t(y x)) f (x)] (y x) dt. f (y) f (x) γ y x for x, y D, 3
4 then we obtain R 2 γt y x 2 dt = γ 2 y x 2. Now we assume that f C 2 (D): For a fixed u R n let G(x) := Df(x) u. Then DG(x) v = D 2 f(x) u, v where y = D 2 f(x) u, v y i = ( j f i ) u j v y i j,=1 Now we have for any u R n and h = y x Df(y) Df(x) u j,=1 j f i u v, y max D 2 f(x) = max i=1,...,n i=1,...,n j,=1 j,=1 j f i. G(t) := Df(x + th) u, G (t) = D 2 f(x + th) u, h [Df(y) Df(x)] u = G(1) G() = D 2 f(x + th) u h dt, G (t)dt = Local convergence results for fixed point iteration Assume that D is open so that x D implies that a ball B δ D. j f i u v D 2 f(x + th) u, h dt ( Df(y) Df(x) max z D D 2 f(z) ) y x. Theorem 3. Assume g : D R n with D R n, and g(x ) = x for x D. If g C 1 (D) and g (x ) < 1 then the fixed point iteration x +1 := g(x ) is locally convergent: There exists δ > so that for x x < δ there holds lim x = x. Proof: Since g is continuous there exists δ >, q < 1 so that g (x) q for x B δ := {x x x < δ}. Assume that x B δ. Then x +1 x = g(x ) g(x ) ( ) max g (u) x x q x x. u B δ Hence x +1 B δ. Then x B δ implies x x q x x and therefore lim x = x. Remar: For a chosen vector norm (e.g., ) and the induced matrix norm one may have g (x ) > 1, but for a different vector norm and the induced matrix norm one could still have g (x ) > 1. The spectral radius ρ(a) := max{ λ j (A) } denotes the maximum absolute values of eigenvalues of A and satisfies [see e.g. Stoer-Bulirsch Theorem (6.9.2)] ρ(a) A for any choice of vector norm and induced matrix norm for any ɛ > there exists a vector norm so that its induced matrix norm satisfies A ρ(a) + ɛ. Therefore we can replace the condition g (x ) < 1 by ρ(g (x )) < 1 and the conclusion of the theorem still holds. Theorem: Assume g : D R n with D R n, and g(x ) = x for x D. If g (x ) = and g is Lipschitz in D then the fixed point iteration x +1 := g(x ) is locally convergent of order 2: There exists δ > so that for x x < δ there holds lim x = x and x +1 x C x x 2. Proof: Assume that we have f (y) f (x) γ y x for x, y D. x +1 x = g(x ) g(x ) g (x )(x x ) γ x x
5 Newton method We have local convergence of order 2 as long as f(x ) is a nonsingular matrix: Theorem 4. Assume that f(x ) = and f exists and satisfies a Lipschitz condition in a neighborhood of x f(x ) is nonsingular Then there exists δ > such that for an initial guess x with x x δ the Newton iterates x exist for = 1, 2, 3,... since f(x 1 ) is nonsingular lim x = x the convergence is of q-order 2 Note that we have with d = x +1 x = f(x ) 1 f(x ) and using f(x )d = f(x ) that implying f(x +1 ) = f(x +1 ) f(x +1 ) f(x ) = f(x +1 ) = f(x + t d )d dt [ f(x + t d ) f(x )] d dt f(x + t d ) f(x ) d dt L }{{} 2 d 2 Lt d [ f(x + t d ) f(x )] f(x ) 1 f(x )dt = f(x +1 ) = Newton-Kantorovich theorem The Newton-Kantorovich theorem does not assume that a solution x with f(x ) exists. Theorem 5. Assume [ f(x + t d ) f(x ) 1 I ] f(x )dt f(x ) C, f(x ) 1 C1 f(x) is Lipschitz with constant L for x x ρ where Then h := C C 2 1L 1 2, ρ := 1 1 2h C 1 L there exists a unique solution x B ρ (x ) with f(x ) = the Newton iterates x are well-defined and converge to x the convergence is r-quadratic for h < 1 2, and r-linear for h =
6 In preparation for the proof we consider the Newton method for the quadratic equation f(t) = t 2 2t + 2h = : t t 1 t 2 t 3 t t + Lemma 6. Assume ω and h 1 2 and define ω, h for = 1, 2,... by Then ω +1 := ω h 2, h +1 := 1 h 2(1 h ) 2 (1) j= h = 1 1 2h ω ω and the sum converges q-quadratically for h < 1 2, and q-linearly for h = 1 2. Proof: It is sufficient to consider the case ω = 1. Consider the quadratic equation f(t) = t 2 2t + 2h = which has the two roots t ± = 1 ± 1 2h with < t < 1 < t + for h < 1 2 and < t = t + = 1 for h = 1 2. We define the sequence t using the Newton method with t =. We have t < t +1 < t for all, this implies that lim = t. In the case of h < 1 2 we have quadratic convergence, in the case of h = 1 2 we have linear convergence. Note that for a quadratic function f(t) = t 2 + a 1 t + a and its tangent line p(t) at t = c we have f(t) p(t) = (t c) 2 (since it must be a quadratic function with leading coefficient 1 and minimum at t = c). Hence for c = t we have by the definition of the Newton method p(t +1 ) = and (t +1 t ) 2 = f(t +1 ) p(t +1 ) = f(t +1 ) for all so that We now want to show that for =, 1, 2,... we have We use induction: For = we have t =, and 1 t = 1 = ω 1 as ω = 1. t +1 t = f(t ) f (t ) = (t t 1 ) 2 = 1 (t t 1 ) 2. (2) 2t t t +1 t = ω 1 h, 1 t = ω 1 (3) t 1 t = f()/f () = 2h /( 2) = h = ω h 6
7 Induction step: Assume that (3) holds. We have ω 1 (D) +1 = ω 1 (1 h ) (I) = ω 1 which is the second equation in (3) with + 1 instead of. We get from (2) that (t +1 t ) (I) = (1 t ) (t +1 t ) = 1 t +1 t +2 t +1 = 1 (t +1 t ) 2 = 1 (h /ω ) t +1 2 ω 1 = ω 1 1 h (ω +1 /ω +1 ) 2 = 1 h 2 ω (1 h ) 2 = ω 1 +1 h +1 Note that N = h ω = N (t +1 t ) = t N+1 t = 1 1 2h as N = We use the following notation for open and closed balls: B r (x ) := {x x x < r}, B r (x ) := {x x x r} The Newton method uses the iteration x +1 = x + d with d := f (x ) 1 f(x ) Assumption: There are constants α, ω > such that h := αω 1 2 and f (x ) 1 f(x ) α (4) f (x ) 1 (f (y) f (x)) ω y x for x, y D (5) Then: B ρ (x ) D with ρ := 1 1 2h ω there exists a unique solution x B ρ (x ) with f(x ) = the Newton iterates x are well-defined and converge to x the convergence is r-quadratic for h < 1 2, and r-linear for h = 1 2. Induction: We claim that for =, 1, 2,...: (B ) : (A ) : ω d h (6) f (x ) 1 (f (y) f (x))) ω y x for x, y D (7) where ω +1 := x B ρ (x ) (8) ω h 2, h +1 := 1 h 2(1 h ) 2 (9) Proof: For = the statements (6), (7) are just the assumptions (4), (5). Induction step: Assume that (6), (7), (8) and we have to prove the corresponding statements for + 1. We first prove (8): From (A ) and the Lemma we obtain that x +1 x h j d j = ρ. (1) ω j j= j= 7
8 We then prove (6) for + 1: Let M := f (x ) 1 f (x +1 ), then f (x +1 ) 1 = M 1 f (x ) 1 and since f (x +1 ) 1 (f (y) f (x)) M 1 f (x ) 1 (f (y) f (x)) (B ) M 1 ω y x 1 ω y x = ω +1 y x 1 h M = I + E, E := f (x ) 1 (f (x +1 ) f (x )), E (B ) ω d (A ) h M 1 = (I + E) E 1 1 h We now prove (6) for + 1: With x +1 = x + d and f(x ) = f (x )d we have d +1 = f (x +1 ) 1 f(x +1 ) = f (x +1 ) 1 [f(x +1 ) f(x ) + f(x )] = d +1 (B +1) ω +1 t d d dt = 1 2 ω +1 h 2 ω 2 f (x +1 ) 1 [f (x + td ) f (x )] d dt h 2 ω +1 d (ω /ω +1 ) 2 = 1 2 (1 h ) 2 = h +1. This concludes the induction proof. We now obtain from (1) that x is a Cauchy sequence and therefore has a limit x B ρ (x ) D. By taing the limit in f (x )(x +1 x ) = f(x ) we obtain by the continuity of f and f that f(x ) =. We sip the proof of the uniqueness. h 2 8
1. Nonlinear Equations. This lecture note excerpted parts from Michael Heath and Max Gunzburger. f(x) = 0
Numerical Analysis 1 1. Nonlinear Equations This lecture note excerpted parts from Michael Heath and Max Gunzburger. Given function f, we seek value x for which where f : D R n R n is nonlinear. f(x) =
More informationTHE INVERSE FUNCTION THEOREM
THE INVERSE FUNCTION THEOREM W. PATRICK HOOPER The implicit function theorem is the following result: Theorem 1. Let f be a C 1 function from a neighborhood of a point a R n into R n. Suppose A = Df(a)
More information1. Bounded linear maps. A linear map T : E F of real Banach
DIFFERENTIABLE MAPS 1. Bounded linear maps. A linear map T : E F of real Banach spaces E, F is bounded if M > 0 so that for all v E: T v M v. If v r T v C for some positive constants r, C, then T is bounded:
More informationX. Linearization and Newton s Method
163 X. Linearization and Newton s Method ** linearization ** X, Y nls s, f : G X Y. Given y Y, find z G s.t. fz = y. Since there is no assumption about f being linear, we might as well assume that y =.
More informationMathematical Analysis Outline. William G. Faris
Mathematical Analysis Outline William G. Faris January 8, 2007 2 Chapter 1 Metric spaces and continuous maps 1.1 Metric spaces A metric space is a set X together with a real distance function (x, x ) d(x,
More informationx 2 x n r n J(x + t(x x ))(x x )dt. For warming-up we start with methods for solving a single equation of one variable.
Maria Cameron 1. Fixed point methods for solving nonlinear equations We address the problem of solving an equation of the form (1) r(x) = 0, where F (x) : R n R n is a vector-function. Eq. (1) can be written
More informationf(x) f(z) c x z > 0 1
INVERSE AND IMPLICIT FUNCTION THEOREMS I use df x for the linear transformation that is the differential of f at x.. INVERSE FUNCTION THEOREM Definition. Suppose S R n is open, a S, and f : S R n is a
More informationFIXED POINT ITERATIONS
FIXED POINT ITERATIONS MARKUS GRASMAIR 1. Fixed Point Iteration for Non-linear Equations Our goal is the solution of an equation (1) F (x) = 0, where F : R n R n is a continuous vector valued mapping in
More informationUniversity of Houston, Department of Mathematics Numerical Analysis, Fall 2005
3 Numerical Solution of Nonlinear Equations and Systems 3.1 Fixed point iteration Reamrk 3.1 Problem Given a function F : lr n lr n, compute x lr n such that ( ) F(x ) = 0. In this chapter, we consider
More informationA nonlinear equation is any equation of the form. f(x) = 0. A nonlinear equation can have any number of solutions (finite, countable, uncountable)
Nonlinear equations Definition A nonlinear equation is any equation of the form where f is a nonlinear function. Nonlinear equations x 2 + x + 1 = 0 (f : R R) f(x) = 0 (x cos y, 2y sin x) = (0, 0) (f :
More informationMaria Cameron. f(x) = 1 n
Maria Cameron 1. Local algorithms for solving nonlinear equations Here we discuss local methods for nonlinear equations r(x) =. These methods are Newton, inexact Newton and quasi-newton. We will show that
More informationMath 421, Homework #9 Solutions
Math 41, Homework #9 Solutions (1) (a) A set E R n is said to be path connected if for any pair of points x E and y E there exists a continuous function γ : [0, 1] R n satisfying γ(0) = x, γ(1) = y, and
More informationThe Inverse Function Theorem via Newton s Method. Michael Taylor
The Inverse Function Theorem via Newton s Method Michael Taylor We aim to prove the following result, known as the inverse function theorem. Theorem 1. Let F be a C k map (k 1) from a neighborhood of p
More informationMost Continuous Functions are Nowhere Differentiable
Most Continuous Functions are Nowhere Differentiable Spring 2004 The Space of Continuous Functions Let K = [0, 1] and let C(K) be the set of all continuous functions f : K R. Definition 1 For f C(K) we
More informationCourse 224: Geometry - Continuity and Differentiability
Course 224: Geometry - Continuity and Differentiability Lecture Notes by Prof. David Simms L A TEXed by Chris Blair 1 Continuity Consider M, N finite dimensional real or complex vector spaces. What does
More informationNonlinear Programming Algorithms Handout
Nonlinear Programming Algorithms Handout Michael C. Ferris Computer Sciences Department University of Wisconsin Madison, Wisconsin 5376 September 9 1 Eigenvalues The eigenvalues of a matrix A C n n are
More informationMetric Spaces and Topology
Chapter 2 Metric Spaces and Topology From an engineering perspective, the most important way to construct a topology on a set is to define the topology in terms of a metric on the set. This approach underlies
More informationd(x n, x) d(x n, x nk ) + d(x nk, x) where we chose any fixed k > N
Problem 1. Let f : A R R have the property that for every x A, there exists ɛ > 0 such that f(t) > ɛ if t (x ɛ, x + ɛ) A. If the set A is compact, prove there exists c > 0 such that f(x) > c for all x
More informationSTOP, a i+ 1 is the desired root. )f(a i) > 0. Else If f(a i+ 1. Set a i+1 = a i+ 1 and b i+1 = b Else Set a i+1 = a i and b i+1 = a i+ 1
53 17. Lecture 17 Nonlinear Equations Essentially, the only way that one can solve nonlinear equations is by iteration. The quadratic formula enables one to compute the roots of p(x) = 0 when p P. Formulas
More informationNumerical Methods for Differential Equations Mathematical and Computational Tools
Numerical Methods for Differential Equations Mathematical and Computational Tools Gustaf Söderlind Numerical Analysis, Lund University Contents V4.16 Part 1. Vector norms, matrix norms and logarithmic
More informationUnconstrained minimization of smooth functions
Unconstrained minimization of smooth functions We want to solve min x R N f(x), where f is convex. In this section, we will assume that f is differentiable (so its gradient exists at every point), and
More informationINVERSE FUNCTION THEOREM and SURFACES IN R n
INVERSE FUNCTION THEOREM and SURFACES IN R n Let f C k (U; R n ), with U R n open. Assume df(a) GL(R n ), where a U. The Inverse Function Theorem says there is an open neighborhood V U of a in R n so that
More informationSelçuk Demir WS 2017 Functional Analysis Homework Sheet
Selçuk Demir WS 2017 Functional Analysis Homework Sheet 1. Let M be a metric space. If A M is non-empty, we say that A is bounded iff diam(a) = sup{d(x, y) : x.y A} exists. Show that A is bounded iff there
More informationReview of Multi-Calculus (Study Guide for Spivak s CHAPTER ONE TO THREE)
Review of Multi-Calculus (Study Guide for Spivak s CHPTER ONE TO THREE) This material is for June 9 to 16 (Monday to Monday) Chapter I: Functions on R n Dot product and norm for vectors in R n : Let X
More informationLecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 5. Nonlinear Equations
Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T Heath Chapter 5 Nonlinear Equations Copyright c 2001 Reproduction permitted only for noncommercial, educational
More informationLecture 5. Ch. 5, Norms for vectors and matrices. Norms for vectors and matrices Why?
KTH ROYAL INSTITUTE OF TECHNOLOGY Norms for vectors and matrices Why? Lecture 5 Ch. 5, Norms for vectors and matrices Emil Björnson/Magnus Jansson/Mats Bengtsson April 27, 2016 Problem: Measure size of
More informationImplicit Functions, Curves and Surfaces
Chapter 11 Implicit Functions, Curves and Surfaces 11.1 Implicit Function Theorem Motivation. In many problems, objects or quantities of interest can only be described indirectly or implicitly. It is then
More information8 Numerical methods for unconstrained problems
8 Numerical methods for unconstrained problems Optimization is one of the important fields in numerical computation, beside solving differential equations and linear systems. We can see that these fields
More informationChapter 1. Optimality Conditions: Unconstrained Optimization. 1.1 Differentiable Problems
Chapter 1 Optimality Conditions: Unconstrained Optimization 1.1 Differentiable Problems Consider the problem of minimizing the function f : R n R where f is twice continuously differentiable on R n : P
More informationNATIONAL BOARD FOR HIGHER MATHEMATICS. Research Scholarships Screening Test. Saturday, February 2, Time Allowed: Two Hours Maximum Marks: 40
NATIONAL BOARD FOR HIGHER MATHEMATICS Research Scholarships Screening Test Saturday, February 2, 2008 Time Allowed: Two Hours Maximum Marks: 40 Please read, carefully, the instructions on the following
More informationMath Introduction to Numerical Analysis - Class Notes. Fernando Guevara Vasquez. Version Date: January 17, 2012.
Math 5620 - Introduction to Numerical Analysis - Class Notes Fernando Guevara Vasquez Version 1990. Date: January 17, 2012. 3 Contents 1. Disclaimer 4 Chapter 1. Iterative methods for solving linear systems
More informationMath 127C, Spring 2006 Final Exam Solutions. x 2 ), g(y 1, y 2 ) = ( y 1 y 2, y1 2 + y2) 2. (g f) (0) = g (f(0))f (0).
Math 27C, Spring 26 Final Exam Solutions. Define f : R 2 R 2 and g : R 2 R 2 by f(x, x 2 (sin x 2 x, e x x 2, g(y, y 2 ( y y 2, y 2 + y2 2. Use the chain rule to compute the matrix of (g f (,. By the chain
More information1.1. The category of Banach spaces. A Banach space E over F is a complete normed vector space where F = R or C. This means that
BANACH CALCULUS CHIN-LUNG WANG Abstract. This is part of my lecture notes for Honored Advanced Calculus at National Taiwan University in 211-212. We prove the inverse function theorem for Banach spaces
More informationChapter 0. Mathematical Preliminaries. 0.1 Norms
Chapter 0 Mathematical Preliminaries 0.1 Norms Throughout this course we will be working with the vector space R n. For this reason we begin with a brief review of its metric space properties Definition
More information2tdt 1 y = t2 + C y = which implies C = 1 and the solution is y = 1
Lectures - Week 11 General First Order ODEs & Numerical Methods for IVPs In general, nonlinear problems are much more difficult to solve than linear ones. Unfortunately many phenomena exhibit nonlinear
More informationEconomics 204 Fall 2013 Problem Set 5 Suggested Solutions
Economics 204 Fall 2013 Problem Set 5 Suggested Solutions 1. Let A and B be n n matrices such that A 2 = A and B 2 = B. Suppose that A and B have the same rank. Prove that A and B are similar. Solution.
More informationWe denote the derivative at x by DF (x) = L. With respect to the standard bases of R n and R m, DF (x) is simply the matrix of partial derivatives,
The derivative Let O be an open subset of R n, and F : O R m a continuous function We say F is differentiable at a point x O, with derivative L, if L : R n R m is a linear transformation such that, for
More informationTHE INVERSE FUNCTION THEOREM FOR LIPSCHITZ MAPS
THE INVERSE FUNCTION THEOREM FOR LIPSCHITZ MAPS RALPH HOWARD DEPARTMENT OF MATHEMATICS UNIVERSITY OF SOUTH CAROLINA COLUMBIA, S.C. 29208, USA HOWARD@MATH.SC.EDU Abstract. This is an edited version of a
More informationEconomics 204 Fall 2011 Problem Set 2 Suggested Solutions
Economics 24 Fall 211 Problem Set 2 Suggested Solutions 1. Determine whether the following sets are open, closed, both or neither under the topology induced by the usual metric. (Hint: think about limit
More informationSolutions Final Exam May. 14, 2014
Solutions Final Exam May. 14, 2014 1. Determine whether the following statements are true or false. Justify your answer (i.e., prove the claim, derive a contradiction or give a counter-example). (a) (10
More informationNORMS ON SPACE OF MATRICES
NORMS ON SPACE OF MATRICES. Operator Norms on Space of linear maps Let A be an n n real matrix and x 0 be a vector in R n. We would like to use the Picard iteration method to solve for the following system
More informationAnalysis Finite and Infinite Sets The Real Numbers The Cantor Set
Analysis Finite and Infinite Sets Definition. An initial segment is {n N n n 0 }. Definition. A finite set can be put into one-to-one correspondence with an initial segment. The empty set is also considered
More informationProblem set 5, Real Analysis I, Spring, otherwise. (a) Verify that f is integrable. Solution: Compute since f is even, 1 x (log 1/ x ) 2 dx 1
Problem set 5, Real Analysis I, Spring, 25. (5) Consider the function on R defined by f(x) { x (log / x ) 2 if x /2, otherwise. (a) Verify that f is integrable. Solution: Compute since f is even, R f /2
More informationChapter 3. Differentiable Mappings. 1. Differentiable Mappings
Chapter 3 Differentiable Mappings 1 Differentiable Mappings Let V and W be two linear spaces over IR A mapping L from V to W is called a linear mapping if L(u + v) = Lu + Lv for all u, v V and L(λv) =
More informationFunctions of Several Variables (Rudin)
Functions of Several Variables (Rudin) Definition: Let X and Y be finite-dimensional real vector spaces. Then L(X, Y ) denotes the set of all linear transformations from X to Y and L(X) denotes the set
More informationAPPLICATIONS OF DIFFERENTIABILITY IN R n.
APPLICATIONS OF DIFFERENTIABILITY IN R n. MATANIA BEN-ARTZI April 2015 Functions here are defined on a subset T R n and take values in R m, where m can be smaller, equal or greater than n. The (open) ball
More informationLMI Methods in Optimal and Robust Control
LMI Methods in Optimal and Robust Control Matthew M. Peet Arizona State University Lecture 15: Nonlinear Systems and Lyapunov Functions Overview Our next goal is to extend LMI s and optimization to nonlinear
More informationElements of Linear Algebra, Topology, and Calculus
Appendix A Elements of Linear Algebra, Topology, and Calculus A.1 LINEAR ALGEBRA We follow the usual conventions of matrix computations. R n p is the set of all n p real matrices (m rows and p columns).
More informationMultivariable Calculus
2 Multivariable Calculus 2.1 Limits and Continuity Problem 2.1.1 (Fa94) Let the function f : R n R n satisfy the following two conditions: (i) f (K ) is compact whenever K is a compact subset of R n. (ii)
More informationCALCULUS JIA-MING (FRANK) LIOU
CALCULUS JIA-MING (FRANK) LIOU Abstract. Contents. Power Series.. Polynomials and Formal Power Series.2. Radius of Convergence 2.3. Derivative and Antiderivative of Power Series 4.4. Power Series Expansion
More informationNumerical Method for Solving Fuzzy Nonlinear Equations
Applied Mathematical Sciences, Vol. 2, 2008, no. 24, 1191-1203 Numerical Method for Solving Fuzzy Nonlinear Equations Javad Shokri Department of Mathematics, Urmia University P.O.Box 165, Urmia, Iran j.shokri@mail.urmia.ac.ir
More information17 Solution of Nonlinear Systems
17 Solution of Nonlinear Systems We now discuss the solution of systems of nonlinear equations. An important ingredient will be the multivariate Taylor theorem. Theorem 17.1 Let D = {x 1, x 2,..., x m
More informationEcon Lecture 14. Outline
Econ 204 2010 Lecture 14 Outline 1. Differential Equations and Solutions 2. Existence and Uniqueness of Solutions 3. Autonomous Differential Equations 4. Complex Exponentials 5. Linear Differential Equations
More informationSeveral variables. x 1 x 2. x n
Several variables Often we have not only one, but several variables in a problem The issues that come up are somewhat more complex than for one variable Let us first start with vector spaces and linear
More information1 Overview. 2 A Characterization of Convex Functions. 2.1 First-order Taylor approximation. AM 221: Advanced Optimization Spring 2016
AM 221: Advanced Optimization Spring 2016 Prof. Yaron Singer Lecture 8 February 22nd 1 Overview In the previous lecture we saw characterizations of optimality in linear optimization, and we reviewed the
More information3.1 Introduction. Solve non-linear real equation f(x) = 0 for real root or zero x. E.g. x x 1.5 =0, tan x x =0.
3.1 Introduction Solve non-linear real equation f(x) = 0 for real root or zero x. E.g. x 3 +1.5x 1.5 =0, tan x x =0. Practical existence test for roots: by intermediate value theorem, f C[a, b] & f(a)f(b)
More informationInexact Newton Methods Applied to Under Determined Systems. Joseph P. Simonis. A Dissertation. Submitted to the Faculty
Inexact Newton Methods Applied to Under Determined Systems by Joseph P. Simonis A Dissertation Submitted to the Faculty of WORCESTER POLYTECHNIC INSTITUTE in Partial Fulfillment of the Requirements for
More informationCOMPLETE METRIC SPACES AND THE CONTRACTION MAPPING THEOREM
COMPLETE METRIC SPACES AND THE CONTRACTION MAPPING THEOREM A metric space (M, d) is a set M with a metric d(x, y), x, y M that has the properties d(x, y) = d(y, x), x, y M d(x, y) d(x, z) + d(z, y), x,
More informationIterative Solution of a Matrix Riccati Equation Arising in Stochastic Control
Iterative Solution of a Matrix Riccati Equation Arising in Stochastic Control Chun-Hua Guo Dedicated to Peter Lancaster on the occasion of his 70th birthday We consider iterative methods for finding the
More informationEconomics 204 Summer/Fall 2011 Lecture 11 Monday August 8, f(x + h) (f(x) + ah) = a = 0 = 0. f(x + h) (f(x) + T x (h))
Economics 204 Summer/Fall 2011 Lecture 11 Monday August 8, 2011 Sections 41-43 (Unified) Definition 1 Let f : I R, where I R is an open interval f is differentiable at x I if for some a R f(x + h) f lim
More informationOutline. Scientific Computing: An Introductory Survey. Nonlinear Equations. Nonlinear Equations. Examples: Nonlinear Equations
Methods for Systems of Methods for Systems of Outline Scientific Computing: An Introductory Survey Chapter 5 1 Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign
More informationChapter 8: Taylor s theorem and L Hospital s rule
Chapter 8: Taylor s theorem and L Hospital s rule Theorem: [Inverse Mapping Theorem] Suppose that a < b and f : [a, b] R. Given that f (x) > 0 for all x (a, b) then f 1 is differentiable on (f(a), f(b))
More informationProblem Set 6: Solutions Math 201A: Fall a n x n,
Problem Set 6: Solutions Math 201A: Fall 2016 Problem 1. Is (x n ) n=0 a Schauder basis of C([0, 1])? No. If f(x) = a n x n, n=0 where the series converges uniformly on [0, 1], then f has a power series
More informationx 3y 2z = 6 1.2) 2x 4y 3z = 8 3x + 6y + 8z = 5 x + 3y 2z + 5t = 4 1.5) 2x + 8y z + 9t = 9 3x + 5y 12z + 17t = 7
Linear Algebra and its Applications-Lab 1 1) Use Gaussian elimination to solve the following systems x 1 + x 2 2x 3 + 4x 4 = 5 1.1) 2x 1 + 2x 2 3x 3 + x 4 = 3 3x 1 + 3x 2 4x 3 2x 4 = 1 x + y + 2z = 4 1.4)
More informationMath 140A - Fall Final Exam
Math 140A - Fall 2014 - Final Exam Problem 1. Let {a n } n 1 be an increasing sequence of real numbers. (i) If {a n } has a bounded subsequence, show that {a n } is itself bounded. (ii) If {a n } has a
More informationBootcamp. Christoph Thiele. Summer As in the case of separability we have the following two observations: Lemma 1 Finite sets are compact.
Bootcamp Christoph Thiele Summer 212.1 Compactness Definition 1 A metric space is called compact, if every cover of the space has a finite subcover. As in the case of separability we have the following
More informationconverges as well if x < 1. 1 x n x n 1 1 = 2 a nx n
Solve the following 6 problems. 1. Prove that if series n=1 a nx n converges for all x such that x < 1, then the series n=1 a n xn 1 x converges as well if x < 1. n For x < 1, x n 0 as n, so there exists
More informationHigher-Order Methods
Higher-Order Methods Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. PCMI, July 2016 Stephen Wright (UW-Madison) Higher-Order Methods PCMI, July 2016 1 / 25 Smooth
More informationUnconstrained optimization
Chapter 4 Unconstrained optimization An unconstrained optimization problem takes the form min x Rnf(x) (4.1) for a target functional (also called objective function) f : R n R. In this chapter and throughout
More informationNotes on Numerical Optimization
Notes on Numerical Optimization University of Chicago, 2014 Viva Patel October 18, 2014 1 Contents Contents 2 List of Algorithms 4 I Fundamentals of Optimization 5 1 Overview of Numerical Optimization
More informationNonlinear Systems Theory
Nonlinear Systems Theory Matthew M. Peet Arizona State University Lecture 2: Nonlinear Systems Theory Overview Our next goal is to extend LMI s and optimization to nonlinear systems analysis. Today we
More informationg 2 (x) (1/3)M 1 = (1/3)(2/3)M.
COMPACTNESS If C R n is closed and bounded, then by B-W it is sequentially compact: any sequence of points in C has a subsequence converging to a point in C Conversely, any sequentially compact C R n is
More information1. Method 1: bisection. The bisection methods starts from two points a 0 and b 0 such that
Chapter 4 Nonlinear equations 4.1 Root finding Consider the problem of solving any nonlinear relation g(x) = h(x) in the real variable x. We rephrase this problem as one of finding the zero (root) of a
More informationDifferentiable Functions
Differentiable Functions Let S R n be open and let f : R n R. We recall that, for x o = (x o 1, x o,, x o n S the partial derivative of f at the point x o with respect to the component x j is defined as
More informationNOTES ON MULTIVARIABLE CALCULUS: DIFFERENTIAL CALCULUS
NOTES ON MULTIVARIABLE CALCULUS: DIFFERENTIAL CALCULUS SAMEER CHAVAN Abstract. This is the first part of Notes on Multivariable Calculus based on the classical texts [6] and [5]. We present here the geometric
More informationSelf-Concordant Barrier Functions for Convex Optimization
Appendix F Self-Concordant Barrier Functions for Convex Optimization F.1 Introduction In this Appendix we present a framework for developing polynomial-time algorithms for the solution of convex optimization
More informationJournal of Computational and Applied Mathematics
Journal of Computational Applied Mathematics 236 (212) 3174 3185 Contents lists available at SciVerse ScienceDirect Journal of Computational Applied Mathematics journal homepage: wwwelseviercom/locate/cam
More informationMAT 570 REAL ANALYSIS LECTURE NOTES. Contents. 1. Sets Functions Countability Axiom of choice Equivalence relations 9
MAT 570 REAL ANALYSIS LECTURE NOTES PROFESSOR: JOHN QUIGG SEMESTER: FALL 204 Contents. Sets 2 2. Functions 5 3. Countability 7 4. Axiom of choice 8 5. Equivalence relations 9 6. Real numbers 9 7. Extended
More informationYou should be able to...
Lecture Outline Gradient Projection Algorithm Constant Step Length, Varying Step Length, Diminishing Step Length Complexity Issues Gradient Projection With Exploration Projection Solving QPs: active set
More informationHOMEWORK FOR , FALL 2007 ASSIGNMENT 1 SOLUTIONS. T = sup T(x) Ax A := sup
HOMEWORK FOR 18.101, FALL 007 ASSIGNMENT 1 SOLUTIONS (1) Given a linear map T : R m R n Define the operator norm of T as follows: T(x) T := sup x 0 x Similarly, if A is a matrix, define the operator norm
More informationMath 361: Homework 1 Solutions
January 3, 4 Math 36: Homework Solutions. We say that two norms and on a vector space V are equivalent or comparable if the topology they define on V are the same, i.e., for any sequence of vectors {x
More informationMATH 205C: STATIONARY PHASE LEMMA
MATH 205C: STATIONARY PHASE LEMMA For ω, consider an integral of the form I(ω) = e iωf(x) u(x) dx, where u Cc (R n ) complex valued, with support in a compact set K, and f C (R n ) real valued. Thus, I(ω)
More informationNumerical Methods for Large-Scale Nonlinear Systems
Numerical Methods for Large-Scale Nonlinear Systems Handouts by Ronald H.W. Hoppe following the monograph P. Deuflhard Newton Methods for Nonlinear Problems Springer, Berlin-Heidelberg-New York, 2004 Num.
More informationReal Analysis Problems
Real Analysis Problems Cristian E. Gutiérrez September 14, 29 1 1 CONTINUITY 1 Continuity Problem 1.1 Let r n be the sequence of rational numbers and Prove that f(x) = 1. f is continuous on the irrationals.
More informationChapter 4. Inverse Function Theorem. 4.1 The Inverse Function Theorem
Chapter 4 Inverse Function Theorem d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d dd d d d d This chapter
More informationDetermine for which real numbers s the series n>1 (log n)s /n converges, giving reasons for your answer.
Problem A. Determine for which real numbers s the series n> (log n)s /n converges, giving reasons for your answer. Solution: It converges for s < and diverges otherwise. To see this use the integral test,
More informationAnalysis II: The Implicit and Inverse Function Theorems
Analysis II: The Implicit and Inverse Function Theorems Jesse Ratzkin November 17, 2009 Let f : R n R m be C 1. When is the zero set Z = {x R n : f(x) = 0} the graph of another function? When is Z nicely
More informationPRACTICE PROBLEMS FOR MIDTERM I
Problem. Find the limits or explain why they do not exist (i) lim x,y 0 x +y 6 x 6 +y ; (ii) lim x,y,z 0 x 6 +y 6 +z 6 x +y +z. (iii) lim x,y 0 sin(x +y ) x +y Problem. PRACTICE PROBLEMS FOR MIDTERM I
More informationCS 450 Numerical Analysis. Chapter 5: Nonlinear Equations
Lecture slides based on the textbook Scientific Computing: An Introductory Survey by Michael T. Heath, copyright c 2018 by the Society for Industrial and Applied Mathematics. http://www.siam.org/books/cl80
More informationSpectral Mapping Theorem
Spectral Mapping Theorem Dan Sievewright Definition of a Hilbert space Let H be a Hilbert space over C. 1) H is a vector space over C. 2) H has an inner product, : H H C with the following properties.
More informationMATHEMATICAL ECONOMICS: OPTIMIZATION. Contents
MATHEMATICAL ECONOMICS: OPTIMIZATION JOÃO LOPES DIAS Contents 1. Introduction 2 1.1. Preliminaries 2 1.2. Optimal points and values 2 1.3. The optimization problems 3 1.4. Existence of optimal points 4
More informationFIXED POINT METHODS IN NONLINEAR ANALYSIS
FIXED POINT METHODS IN NONLINEAR ANALYSIS ZACHARY SMITH Abstract. In this paper we present a selection of fixed point theorems with applications in nonlinear analysis. We begin with the Banach fixed point
More informationLecture 13 Newton-type Methods A Newton Method for VIs. October 20, 2008
Lecture 13 Newton-type Methods A Newton Method for VIs October 20, 2008 Outline Quick recap of Newton methods for composite functions Josephy-Newton methods for VIs A special case: mixed complementarity
More informationNonlinearity Root-finding Bisection Fixed Point Iteration Newton s Method Secant Method Conclusion. Nonlinear Systems
Nonlinear Systems CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Justin Solomon CS 205A: Mathematical Methods Nonlinear Systems 1 / 24 Part III: Nonlinear Problems Not all numerical problems
More informationMATH 3795 Lecture 13. Numerical Solution of Nonlinear Equations in R N.
MATH 3795 Lecture 13. Numerical Solution of Nonlinear Equations in R N. Dmitriy Leykekhman Fall 2008 Goals Learn about different methods for the solution of F (x) = 0, their advantages and disadvantages.
More informationNumerical Methods in Informatics
Numerical Methods in Informatics Lecture 2, 30.09.2016: Nonlinear Equations in One Variable http://www.math.uzh.ch/binf4232 Tulin Kaman Institute of Mathematics, University of Zurich E-mail: tulin.kaman@math.uzh.ch
More informationLECTURE NOTES ELEMENTARY NUMERICAL METHODS. Eusebius Doedel
LECTURE NOTES on ELEMENTARY NUMERICAL METHODS Eusebius Doedel TABLE OF CONTENTS Vector and Matrix Norms 1 Banach Lemma 20 The Numerical Solution of Linear Systems 25 Gauss Elimination 25 Operation Count
More informationKantorovich s Majorants Principle for Newton s Method
Kantorovich s Majorants Principle for Newton s Method O. P. Ferreira B. F. Svaiter January 17, 2006 Abstract We prove Kantorovich s theorem on Newton s method using a convergence analysis which makes clear,
More informationChapter 3: Root Finding. September 26, 2005
Chapter 3: Root Finding September 26, 2005 Outline 1 Root Finding 2 3.1 The Bisection Method 3 3.2 Newton s Method: Derivation and Examples 4 3.3 How To Stop Newton s Method 5 3.4 Application: Division
More informationChapter 4. Unconstrained optimization
Chapter 4. Unconstrained optimization Version: 28-10-2012 Material: (for details see) Chapter 11 in [FKS] (pp.251-276) A reference e.g. L.11.2 refers to the corresponding Lemma in the book [FKS] PDF-file
More information