# 1 Linear Least Squares

Size: px
Start display at page:

Transcription

1 Lest Squres Pge 1 1 Liner Lest Squres I will try to be consistent in nottion, with n being the number of dt points, nd m < n being the number of prmeters in model function. We re interested in solving n inconsistent set of equtions, Ax = b, where A is n n m-mtrix with components i,j, x is n m-vector with m < n nd components x i, b is n n-vector with components b i. Notice tht if m = n this could be solved providing solution exists using Gussin elimintion. This system is inconsistent hs no solution since the number of unknown prmeters m is less thn the number of equtions n nd it is unlikely tht x cn be chosen such tht Ax = b. If m = n, we would wnt to determine x such tht Ax b 2 = 0 where we will work with the Eucliden norm. Gol: If m < n, we would wnt to determine x such tht Ax b 2 is minimized. Nottion: Given A = [ i,j ] n m, we define the trnspose of A s A T = [ j,i ] m n. Useful results from liner lgebr: A + B T = A T + B T AB T = B T A T x y 2 = x i y i 2 = x y T x y ith row of Ax is ith row of A T b is i,k x k k,i b k Definition: Two vectors x nd y in R n re orthogonl if x T y = 0 I wnt to ttck this in different mnner thn the text, using directionl derivtives. Ax b 2 = Ax b T Ax b = Ax T b T Ax b = x T A T b T Ax b = x T A T Ax b T Ax x T A T b + b T b Now, ech term in the bove is sclr, since the norm is sclr. For this to be minimum, the directionl derivtive in ny direction u must be zero from Clculus III. D u Ax b 2 = x T A T Ax b T Ax x T A T b + b T b u = 0 x

2 Lest Squres Pge 2 where 1 α α x = α x 1 α x m T Let s work out ech of these derivtives in turn, strting with the esy ones, nd looking t the jth row α. We will use x i = δ ij. α = b T b = α = 0 b 2 i x bt b = 0 α = α = α = x T A T b = k,j b k = α = b T Ax = b i i,j = x i k,i b k j,k T b k jth row of A T b b i i,k x k j,i T b i jth row of A T b x xt A T b = A T b x bt Ax = A T b α = x T A T Ax = Ax T Ax = i,q x q i,k x k q=1 α = i,j i,q x q + i,k x k i,j product rule of derivtives q=1 = 2 i,j i,k x k = 2 j,i T i,k x k = 2 jth row of A T Ax product of ith row of Ax x xt A T Ax = 2A T Ax Substituting these results bck, we find tht 2A T Ax 2A T b u = 0 1 Note this is the denomintor lyout convention which you cn lern more bout here: clculus

3 Lest Squres Pge 3 nd for this to be true for ll directions u, we must hve A T Ax = A T b the norml equtions The solution x to the norml equtions will minimize the vlue of the residul r = Ax b. You cn solve the norml equtions using Gussin elimintion, or finding n inverse of mtrix: x = A T A 1 A T b where A T A 1 A T is the pseudoinverse of the mtrix A. Fitting Models to Dt If we re given 1 i n dt points x i, y i nd hve liner model we wish to fit fx = c j f j x j=1 where m < n then we cn use the norml equtions to determine the vlues of the prmeters c j tht best fit the dt. Best fit is tken to men minimizing the quntity Ac b, where substituting the dt points into the model results in the equtions: fx i = y i which is written s Ac = b. You should note tht this process is ctully quite flexible, nd you cn construct whtever model might best fit the dt, even something like: fx, y = c 1 + c 2 cos x + c 3 sin y + c 4 xy You simply use this eqution to crete the ssocited norml equtions for the relted system Ac = b. The process is liner in terms of being liner in the prmeters c i tht re being determined, not in terms of liner model function f. Section 4.2 shows quite few models, so mke sure to red it. Improvements When Solving The Norml Equtions Fil: QR Algorithm Sometimes solving the norml equtions fils, since the Gussin elimintion is not successful. In those cses it is useful to employ the QR fctoriztion.

4 Lest Squres Pge 4 Grm-Schmidt Orthogonliztion I think the text switched the mening of m nd n here, so I will mintin my ide tht n is the number of dt points t m < n is the number of prmeters this is how the Mthemtic notebook is set up. Grm-Schmidt orthogonliztion is used to tke set of m linerly independent vectors nd crete new set of m vectors tht is orthogonlized ech vector is perpendiculr to ll the other vectors. The lgorithm is firly strightforwrd to element, nd is discussed in the Mthemtic file. The full QR decomposition is A [n m] = Q [n n] R [n m] where Q T = Q 1 this is why we need the full decomposition, so the inverse mtrix exists since the inverse is only defined for squre mtrices. Q T Q = Q 1 Q = I Once we hve the full QR decomposition of A we do liner lest squres on the m n system Ac = b for which we wish Ac b 2 to be minimum in the following mnner: Ac = b QRc = b Q T QRc = Q T b left multiply by Q T Rc = Q T b since Q T Q = I Rc = d where R is the upper m m prt of R nd d is the upper m entries of Q T b. Solve this system for the model prmeters c. 2 Nonliner Regression: Guss-Newton Method If the model function does not linerly depend on the prmeters, then the previous liner lgebr bsed methods will not work. We cn exmine the problem insted from the point of view of minimizing the error by tking the grdient. Generl Set Up We hve n dt points x i, y i, i = 1, 2,..., n which we wnt to fit to model function which hs m djustble prmeters α j,, j = 1, 2,..., m: fx = fx; α 1,..., α n. We ctully hve gret del of choice in wht type of function we wnt to minimize. It cn be nything tht will mesure the reltion of the dt to the model function. The vector which compres the dt to the model function t ech point is given by y 1 fx 1 ; α 1,..., α m y 2 fx 2 ; α 1,..., α m. y n fx n ; α 1,..., α m

5 Lest Squres Pge 5 We cn minimize this vector bsed on vriety of different norms: l 1 norm: l p norm: l norm: y i fx i ; α 1,..., α m 1/p y i fx i ; α 1,..., α m p mx y i fx i ; α 1,..., α m,..,n Wht is typiclly done is tht the l 2 norm is used, since it is the Eucliden spce norm, nd the squre of the norm is minimized: minimize Eα 1,..., α m = y i fx i ; α 1,..., α m 2 Minimizing is just multivrible unconstrined minimiztion procedure, which yields the system of equtions y i fx i ; α 1,..., α m fx i ; α 1,..., α m, k = 1,..., m 1 α k which must be solved for the m unknowns α j. You cn solve the equtions numericlly using Newton s method for systems. Applying this technique to Liner Regression For now we re interested in fitting to polynomil of degree m which mens we hve m + 1 prmeters. fx; α 0,..., α m = α j x j The system of equtions we solve becomes y i α j x j i α j x j i, k = 0,..., m α k m y i α j x j i δ jk x j i, k = 0,..., m α j x j+k i = y i α j x j i x k i, y i x k i y i x k i, k = 0,..., m α j x j i xk i, k = 0,..., m k = 0,..., m This is system of m + 1 equtions in the m + 1 unknowns α j. It is liner system, nd cn be solved using Crmer s rule, resulting in the fits you my hve seen before especilly for fx = α 0 + α 1 x.

6 Lest Squres Pge 6 3 Aside: Orthogonl Polynomils nd Approximting Functions Now, we consider hving function fx nd fitting curve P x; α 0,..., α n to the function. We wnt to minimize the error between the two, nd gin choose the lest squres pproximtion: 2 minimize Eα 0,..., α n = wt ft P t; α 0,..., α n dt where wt is weight function. The weight function gives greter weight to certin sections of the region tht it is defined on. Minimizing this eqution is just multivrible unconstrined minimiztion procedure, which yields Eα 0,..., α n b = 2 wt ft P t; α 0,..., α n P t; α 0,..., α n dt, α k α k We choose the function P t; α 0,..., α n = n α jφ j t, where φ j t is set of linerly independent functions on [, b]. Therefore, we get wtftφ k t dt = wt ft wt ft wt ft α j φ j t α k α j φ j t dt, α j φ j t δ kj φ j t dt, α j wtφ j tφ k t dt, α j φ j t φ k t dt, If the functions φ k t cn be chosen so tht they re orthogonl, { 0 j k wtφ j tφ k t dt = j = k then we get wtftφ k t dt = = β j α j wtφ j tφ k t dt, α j δ jk β j, = α k β k, α k = 1 wtftφ k t dt β k nd we hve determined the α j which minimize the function. The Grm-Schmidt process describes how to construct orthogonl polynomils given specific weight function wx.

### Orthogonal Polynomials

Mth 4401 Gussin Qudrture Pge 1 Orthogonl Polynomils Orthogonl polynomils rise from series solutions to differentil equtions, lthough they cn be rrived t in vriety of different mnners. Orthogonl polynomils

### Best Approximation in the 2-norm

Jim Lmbers MAT 77 Fll Semester 1-11 Lecture 1 Notes These notes correspond to Sections 9. nd 9.3 in the text. Best Approximtion in the -norm Suppose tht we wish to obtin function f n (x) tht is liner combintion

### Matrices, Moments and Quadrature, cont d

Jim Lmbers MAT 285 Summer Session 2015-16 Lecture 2 Notes Mtrices, Moments nd Qudrture, cont d We hve described how Jcobi mtrices cn be used to compute nodes nd weights for Gussin qudrture rules for generl

### MATH34032: Green s Functions, Integral Equations and the Calculus of Variations 1

MATH34032: Green s Functions, Integrl Equtions nd the Clculus of Vritions 1 Section 1 Function spces nd opertors Here we gives some brief detils nd definitions, prticulrly relting to opertors. For further

### Numerical Linear Algebra Assignment 008

Numericl Liner Algebr Assignment 008 Nguyen Qun B Hong Students t Fculty of Mth nd Computer Science, Ho Chi Minh University of Science, Vietnm emil. nguyenqunbhong@gmil.com blog. http://hongnguyenqunb.wordpress.com

### A Matrix Algebra Primer

A Mtrix Algebr Primer Mtrices, Vectors nd Sclr Multipliction he mtrix, D, represents dt orgnized into rows nd columns where the rows represent one vrible, e.g. time, nd the columns represent second vrible,

### dt. However, we might also be curious about dy

Section 0. The Clculus of Prmetric Curves Even though curve defined prmetricly my not be function, we cn still consider concepts such s rtes of chnge. However, the concepts will need specil tretment. For

Qudrtic Forms Recll the Simon & Blume excerpt from n erlier lecture which sid tht the min tsk of clculus is to pproximte nonliner functions with liner functions. It s ctully more ccurte to sy tht we pproximte

### Chapter 3 Polynomials

Dr M DRAIEF As described in the introduction of Chpter 1, pplictions of solving liner equtions rise in number of different settings In prticulr, we will in this chpter focus on the problem of modelling

### Math 270A: Numerical Linear Algebra

Mth 70A: Numericl Liner Algebr Instructor: Michel Holst Fll Qurter 014 Homework Assignment #3 Due Give to TA t lest few dys before finl if you wnt feedbck. Exercise 3.1. (The Bsic Liner Method for Liner

### Matrices and Determinants

Nme Chpter 8 Mtrices nd Determinnts Section 8.1 Mtrices nd Systems of Equtions Objective: In this lesson you lerned how to use mtrices, Gussin elimintion, nd Guss-Jordn elimintion to solve systems of liner

### Best Approximation. Chapter The General Case

Chpter 4 Best Approximtion 4.1 The Generl Cse In the previous chpter, we hve seen how n interpolting polynomil cn be used s n pproximtion to given function. We now wnt to find the best pproximtion to given

### 1 The Lagrange interpolation formula

Notes on Qudrture 1 The Lgrnge interpoltion formul We briefly recll the Lgrnge interpoltion formul. The strting point is collection of N + 1 rel points (x 0, y 0 ), (x 1, y 1 ),..., (x N, y N ), with x

### Things to Memorize: A Partial List. January 27, 2017

Things to Memorize: A Prtil List Jnury 27, 2017 Chpter 2 Vectors - Bsic Fcts A vector hs mgnitude (lso clled size/length/norm) nd direction. It does not hve fixed position, so the sme vector cn e moved

### The Algebra (al-jabr) of Matrices

Section : Mtri lgebr nd Clculus Wshkewicz College of Engineering he lgebr (l-jbr) of Mtrices lgebr s brnch of mthemtics is much broder thn elementry lgebr ll of us studied in our high school dys. In sense

### Review of Gaussian Quadrature method

Review of Gussin Qudrture method Nsser M. Asi Spring 006 compiled on Sundy Decemer 1, 017 t 09:1 PM 1 The prolem To find numericl vlue for the integrl of rel vlued function of rel vrile over specific rnge

### ( dg. ) 2 dt. + dt. dt j + dh. + dt. r(t) dt. Comparing this equation with the one listed above for the length of see that

Arc Length of Curves in Three Dimensionl Spce If the vector function r(t) f(t) i + g(t) j + h(t) k trces out the curve C s t vries, we cn mesure distnces long C using formul nerly identicl to one tht we

### CSCI 5525 Machine Learning

CSCI 555 Mchine Lerning Some Deini*ons Qudrtic Form : nn squre mtri R n n : n vector R n the qudrtic orm: It is sclr vlue. We oten implicitly ssume tht is symmetric since / / I we write it s the elements

### Module 6: LINEAR TRANSFORMATIONS

Module 6: LINEAR TRANSFORMATIONS. Trnsformtions nd mtrices Trnsformtions re generliztions of functions. A vector x in some set S n is mpped into m nother vector y T( x). A trnsformtion is liner if, for

### n f(x i ) x. i=1 In section 4.2, we defined the definite integral of f from x = a to x = b as n f(x i ) x; f(x) dx = lim i=1

The Fundmentl Theorem of Clculus As we continue to study the re problem, let s think bck to wht we know bout computing res of regions enclosed by curves. If we wnt to find the re of the region below the

### Lecture Note 9: Orthogonal Reduction

MATH : Computtionl Methods of Liner Algebr 1 The Row Echelon Form Lecture Note 9: Orthogonl Reduction Our trget is to solve the norml eution: Xinyi Zeng Deprtment of Mthemticl Sciences, UTEP A t Ax = A

### Discrete Least-squares Approximations

Discrete Lest-squres Approximtions Given set of dt points (x, y ), (x, y ),, (x m, y m ), norml nd useful prctice in mny pplictions in sttistics, engineering nd other pplied sciences is to construct curve

### Lecture 19: Continuous Least Squares Approximation

Lecture 19: Continuous Lest Squres Approximtion 33 Continuous lest squres pproximtion We begn 31 with the problem of pproximting some f C[, b] with polynomil p P n t the discrete points x, x 1,, x m for

### AP Calculus Multiple Choice: BC Edition Solutions

AP Clculus Multiple Choice: BC Edition Solutions J. Slon Mrch 8, 04 ) 0 dx ( x) is A) B) C) D) E) Divergent This function inside the integrl hs verticl symptotes t x =, nd the integrl bounds contin this

### September 13 Homework Solutions

College of Engineering nd Computer Science Mechnicl Engineering Deprtment Mechnicl Engineering 5A Seminr in Engineering Anlysis Fll Ticket: 5966 Instructor: Lrry Cretto Septemer Homework Solutions. Are

### Recitation 3: More Applications of the Derivative

Mth 1c TA: Pdric Brtlett Recittion 3: More Applictions of the Derivtive Week 3 Cltech 2012 1 Rndom Question Question 1 A grph consists of the following: A set V of vertices. A set E of edges where ech

### How do we solve these things, especially when they get complicated? How do we know when a system has a solution, and when is it unique?

XII. LINEAR ALGEBRA: SOLVING SYSTEMS OF EQUATIONS Tody we re going to tlk bout solving systems of liner equtions. These re problems tht give couple of equtions with couple of unknowns, like: 6 2 3 7 4

### Properties of Integrals, Indefinite Integrals. Goals: Definition of the Definite Integral Integral Calculations using Antiderivatives

Block #6: Properties of Integrls, Indefinite Integrls Gols: Definition of the Definite Integrl Integrl Clcultions using Antiderivtives Properties of Integrls The Indefinite Integrl 1 Riemnn Sums - 1 Riemnn

### CMDA 4604: Intermediate Topics in Mathematical Modeling Lecture 19: Interpolation and Quadrature

CMDA 4604: Intermedite Topics in Mthemticl Modeling Lecture 19: Interpoltion nd Qudrture In this lecture we mke brief diversion into the res of interpoltion nd qudrture. Given function f C[, b], we sy

### B.Sc. in Mathematics (Ordinary)

R48/0 DUBLIN INSTITUTE OF TECHNOLOGY KEVIN STREET, DUBLIN 8 B.Sc. in Mthemtics (Ordinry) SUPPLEMENTAL EXAMINATIONS 01 Numericl Methods Dr. D. Mckey Dr. C. Hills Dr. E.A. Cox Full mrks for complete nswers

### HW3, Math 307. CSUF. Spring 2007.

HW, Mth 7. CSUF. Spring 7. Nsser M. Abbsi Spring 7 Compiled on November 5, 8 t 8:8m public Contents Section.6, problem Section.6, problem Section.6, problem 5 Section.6, problem 7 6 5 Section.6, problem

### How can we approximate the area of a region in the plane? What is an interpretation of the area under the graph of a velocity function?

Mth 125 Summry Here re some thoughts I ws hving while considering wht to put on the first midterm. The core of your studying should be the ssigned homework problems: mke sure you relly understnd those

### Theoretical foundations of Gaussian quadrature

Theoreticl foundtions of Gussin qudrture 1 Inner product vector spce Definition 1. A vector spce (or liner spce) is set V = {u, v, w,...} in which the following two opertions re defined: (A) Addition of

### DEFINITION The inner product of two functions f 1 and f 2 on an interval [a, b] is the number. ( f 1, f 2 ) b DEFINITION 11.1.

398 CHAPTER 11 ORTHOGONAL FUNCTIONS AND FOURIER SERIES 11.1 ORTHOGONAL FUNCTIONS REVIEW MATERIAL The notions of generlized vectors nd vector spces cn e found in ny liner lger text. INTRODUCTION The concepts

### 1. Gauss-Jacobi quadrature and Legendre polynomials. p(t)w(t)dt, p {p(x 0 ),...p(x n )} p(t)w(t)dt = w k p(x k ),

1. Guss-Jcobi qudrture nd Legendre polynomils Simpson s rule for evluting n integrl f(t)dt gives the correct nswer with error of bout O(n 4 ) (with constnt tht depends on f, in prticulr, it depends on

### MATRICES AND VECTORS SPACE

MATRICES AND VECTORS SPACE MATRICES AND MATRIX OPERATIONS SYSTEM OF LINEAR EQUATIONS DETERMINANTS VECTORS IN -SPACE AND -SPACE GENERAL VECTOR SPACES INNER PRODUCT SPACES EIGENVALUES, EIGENVECTORS LINEAR

### ODE: Existence and Uniqueness of a Solution

Mth 22 Fll 213 Jerry Kzdn ODE: Existence nd Uniqueness of Solution The Fundmentl Theorem of Clculus tells us how to solve the ordinry differentil eqution (ODE) du = f(t) dt with initil condition u() =

### Inner-product spaces

Inner-product spces Definition: Let V be rel or complex liner spce over F (here R or C). An inner product is n opertion between two elements of V which results in sclr. It is denoted by u, v nd stisfies:

### Orthogonal Polynomials and Least-Squares Approximations to Functions

Chpter Orthogonl Polynomils nd Lest-Squres Approximtions to Functions **4/5/3 ET. Discrete Lest-Squres Approximtions Given set of dt points (x,y ), (x,y ),..., (x m,y m ), norml nd useful prctice in mny

### Chapter 3 MATRIX. In this chapter: 3.1 MATRIX NOTATION AND TERMINOLOGY

Chpter 3 MTRIX In this chpter: Definition nd terms Specil Mtrices Mtrix Opertion: Trnspose, Equlity, Sum, Difference, Sclr Multipliction, Mtrix Multipliction, Determinnt, Inverse ppliction of Mtrix in

### CAAM 453 NUMERICAL ANALYSIS I Examination There are four questions, plus a bonus. Do not look at them until you begin the exam.

Exmintion 1 Posted 23 October 2002. Due no lter thn 5pm on Mondy, 28 October 2002. Instructions: 1. Time limit: 3 uninterrupted hours. 2. There re four questions, plus bonus. Do not look t them until you

### Abstract inner product spaces

WEEK 4 Abstrct inner product spces Definition An inner product spce is vector spce V over the rel field R equipped with rule for multiplying vectors, such tht the product of two vectors is sclr, nd the

### Bases for Vector Spaces

Bses for Vector Spces 2-26-25 A set is independent if, roughly speking, there is no redundncy in the set: You cn t uild ny vector in the set s liner comintion of the others A set spns if you cn uild everything

### Geometric Sequences. Geometric Sequence a sequence whose consecutive terms have a common ratio.

Geometric Sequences Geometric Sequence sequence whose consecutive terms hve common rtio. Geometric Sequence A sequence is geometric if the rtios of consecutive terms re the sme. 2 3 4... 2 3 The number

### Elementary Linear Algebra

Elementry Liner Algebr Anton & Rorres, 1 th Edition Lecture Set 5 Chpter 4: Prt II Generl Vector Spces 163 คณ ตศาสตร ว ศวกรรม 3 สาขาว ชาว ศวกรรมคอมพ วเตอร ป การศ กษา 1/2555 163 คณตศาสตรวศวกรรม 3 สาขาวชาวศวกรรมคอมพวเตอร

### Construction of Gauss Quadrature Rules

Jim Lmbers MAT 772 Fll Semester 2010-11 Lecture 15 Notes These notes correspond to Sections 10.2 nd 10.3 in the text. Construction of Guss Qudrture Rules Previously, we lerned tht Newton-Cotes qudrture

### Integrals - Motivation

Integrls - Motivtion When we looked t function s rte of chnge If f(x) is liner, the nswer is esy slope If f(x) is non-liner, we hd to work hrd limits derivtive A relted question is the re under f(x) (but

### Numerical Methods I Orthogonal Polynomials

Numericl Methods I Orthogonl Polynomils Aleksndr Donev Cournt Institute, NYU 1 donev@cournt.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fll 2014 Nov 6th, 2014 A. Donev (Cournt Institute) Lecture IX

### We partition C into n small arcs by forming a partition of [a, b] by picking s i as follows: a = s 0 < s 1 < < s n = b.

Mth 255 - Vector lculus II Notes 4.2 Pth nd Line Integrls We begin with discussion of pth integrls (the book clls them sclr line integrls). We will do this for function of two vribles, but these ides cn

### Matrix Algebra. Matrix Addition, Scalar Multiplication and Transposition. Linear Algebra I 24

Mtrix lger Mtrix ddition, Sclr Multipliction nd rnsposition Mtrix lger Section.. Mtrix ddition, Sclr Multipliction nd rnsposition rectngulr rry of numers is clled mtrix ( the plurl is mtrices ) nd the

### Math 520 Final Exam Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008

Mth 520 Finl Exm Topic Outline Sections 1 3 (Xio/Dums/Liw) Spring 2008 The finl exm will be held on Tuesdy, My 13, 2-5pm in 117 McMilln Wht will be covered The finl exm will cover the mteril from ll of

### Math 231E, Lecture 33. Parametric Calculus

Mth 31E, Lecture 33. Prmetric Clculus 1 Derivtives 1.1 First derivtive Now, let us sy tht we wnt the slope t point on prmetric curve. Recll the chin rule: which exists s long s /. = / / Exmple 1.1. Reconsider

### Partial Derivatives. Limits. For a single variable function f (x), the limit lim

Limits Prtil Derivtives For single vrible function f (x), the limit lim x f (x) exists only if the right-hnd side limit equls to the left-hnd side limit, i.e., lim f (x) = lim f (x). x x + For two vribles

### Numerical Analysis: Trapezoidal and Simpson s Rule

nd Simpson s Mthemticl question we re interested in numericlly nswering How to we evlute I = f (x) dx? Clculus tells us tht if F(x) is the ntiderivtive of function f (x) on the intervl [, b], then I =

Lecture notes on Vritionl nd Approximte Methods in Applied Mthemtics - A Peirce UBC Lecture 6: Singulr Integrls, Open Qudrture rules, nd Guss Qudrture (Compiled 6 August 7) In this lecture we discuss the

### Lecture 2e Orthogonal Complement (pages )

Lecture 2e Orthogonl Complement (pges -) We hve now seen tht n orthonorml sis is nice wy to descrie suspce, ut knowing tht we wnt n orthonorml sis doesn t mke one fll into our lp. In theory, the process

### Chapter 0. What is the Lebesgue integral about?

Chpter 0. Wht is the Lebesgue integrl bout? The pln is to hve tutoril sheet ech week, most often on Fridy, (to be done during the clss) where you will try to get used to the ides introduced in the previous

### Math Lecture 23

Mth 8 - Lecture 3 Dyln Zwick Fll 3 In our lst lecture we delt with solutions to the system: x = Ax where A is n n n mtrix with n distinct eigenvlues. As promised, tody we will del with the question of

### Lecture Solution of a System of Linear Equation

ChE Lecture Notes, Dept. of Chemicl Engineering, Univ. of TN, Knoville - D. Keffer, 5/9/98 (updted /) Lecture 8- - Solution of System of Liner Eqution 8. Why is it importnt to e le to solve system of liner

### Multivariate problems and matrix algebra

University of Ferrr Stefno Bonnini Multivrite problems nd mtrix lgebr Multivrite problems Multivrite sttisticl nlysis dels with dt contining observtions on two or more chrcteristics (vribles) ech mesured

### NUMERICAL INTEGRATION. The inverse process to differentiation in calculus is integration. Mathematically, integration is represented by.

NUMERICAL INTEGRATION 1 Introduction The inverse process to differentition in clculus is integrtion. Mthemticlly, integrtion is represented by f(x) dx which stnds for the integrl of the function f(x) with

### Lecture 21: Order statistics

Lecture : Order sttistics Suppose we hve N mesurements of sclr, x i =, N Tke ll mesurements nd sort them into scending order x x x 3 x N Define the mesured running integrl S N (x) = 0 for x < x = i/n for

### Recitation 3: Applications of the Derivative. 1 Higher-Order Derivatives and their Applications

Mth 1c TA: Pdric Brtlett Recittion 3: Applictions of the Derivtive Week 3 Cltech 013 1 Higher-Order Derivtives nd their Applictions Another thing we could wnt to do with the derivtive, motivted by wht

### Unit #9 : Definite Integral Properties; Fundamental Theorem of Calculus

Unit #9 : Definite Integrl Properties; Fundmentl Theorem of Clculus Gols: Identify properties of definite integrls Define odd nd even functions, nd reltionship to integrl vlues Introduce the Fundmentl

### Part IB Numerical Analysis

Prt IB Numericl Anlysis Theorems with proof Bsed on lectures by G. Moore Notes tken by Dexter Chu Lent 2016 These notes re not endorsed by the lecturers, nd I hve modified them (often significntly) fter

### The Islamic University of Gaza Faculty of Engineering Civil Engineering Department. Numerical Analysis ECIV Chapter 11

The Islmic University of Gz Fculty of Engineering Civil Engineering Deprtment Numericl Anlysis ECIV 6 Chpter Specil Mtrices nd Guss-Siedel Associte Prof Mzen Abultyef Civil Engineering Deprtment, The Islmic

### Before we can begin Ch. 3 on Radicals, we need to be familiar with perfect squares, cubes, etc. Try and do as many as you can without a calculator!!!

Nme: Algebr II Honors Pre-Chpter Homework Before we cn begin Ch on Rdicls, we need to be fmilir with perfect squres, cubes, etc Try nd do s mny s you cn without clcultor!!! n The nth root of n n Be ble

### ECON 331 Lecture Notes: Ch 4 and Ch 5

Mtrix Algebr ECON 33 Lecture Notes: Ch 4 nd Ch 5. Gives us shorthnd wy of writing lrge system of equtions.. Allows us to test for the existnce of solutions to simultneous systems. 3. Allows us to solve

### Instructor: Marios M. Fyrillas HOMEWORK ASSIGNMENT ON INTERPOLATION

AMAT 34 Numericl Methods Instructor: Mrios M. Fyrills Emil: m.fyrills@fit.c.cy Office Tel.: 34559/6 Et. 3 HOMEWORK ASSIGNMENT ON INTERPOATION QUESTION Using interpoltion by colloction-polynomil fit method

### The graphs of Rational Functions

Lecture 4 5A: The its of Rtionl Functions s x nd s x + The grphs of Rtionl Functions The grphs of rtionl functions hve severl differences compred to power functions. One of the differences is the behvior

### Main topics for the First Midterm

Min topics for the First Midterm The Midterm will cover Section 1.8, Chpters 2-3, Sections 4.1-4.8, nd Sections 5.1-5.3 (essentilly ll of the mteril covered in clss). Be sure to know the results of the

### ( ) Same as above but m = f x = f x - symmetric to y-axis. find where f ( x) Relative: Find where f ( x) x a + lim exists ( lim f exists.

AP Clculus Finl Review Sheet solutions When you see the words This is wht you think of doing Find the zeros Set function =, fctor or use qudrtic eqution if qudrtic, grph to find zeros on clcultor Find

### Introduction to Determinants. Remarks. Remarks. The determinant applies in the case of square matrices

Introduction to Determinnts Remrks The determinnt pplies in the cse of squre mtrices squre mtrix is nonsingulr if nd only if its determinnt not zero, hence the term determinnt Nonsingulr mtrices re sometimes

### Matrix Eigenvalues and Eigenvectors September 13, 2017

Mtri Eigenvlues nd Eigenvectors September, 7 Mtri Eigenvlues nd Eigenvectors Lrry Cretto Mechnicl Engineering 5A Seminr in Engineering Anlysis September, 7 Outline Review lst lecture Definition of eigenvlues

### Lecture 1: Introduction to integration theory and bounded variation

Lecture 1: Introduction to integrtion theory nd bounded vrition Wht is this course bout? Integrtion theory. The first question you might hve is why there is nything you need to lern bout integrtion. You

### NUMERICAL INTEGRATION

NUMERICAL INTEGRATION How do we evlute I = f (x) dx By the fundmentl theorem of clculus, if F (x) is n ntiderivtive of f (x), then I = f (x) dx = F (x) b = F (b) F () However, in prctice most integrls

### Jim Lambers MAT 169 Fall Semester Lecture 4 Notes

Jim Lmbers MAT 169 Fll Semester 2009-10 Lecture 4 Notes These notes correspond to Section 8.2 in the text. Series Wht is Series? An infinte series, usully referred to simply s series, is n sum of ll of

### Exam 2, Mathematics 4701, Section ETY6 6:05 pm 7:40 pm, March 31, 2016, IH-1105 Instructor: Attila Máté 1

Exm, Mthemtics 471, Section ETY6 6:5 pm 7:4 pm, Mrch 1, 16, IH-115 Instructor: Attil Máté 1 17 copies 1. ) Stte the usul sufficient condition for the fixed-point itertion to converge when solving the eqution

### Contents. Outline. Structured Rank Matrices Lecture 2: The theorem Proofs Examples related to structured ranks References. Structure Transport

Contents Structured Rnk Mtrices Lecture 2: Mrc Vn Brel nd Rf Vndebril Dept. of Computer Science, K.U.Leuven, Belgium Chemnitz, Germny, 26-30 September 2011 1 Exmples relted to structured rnks 2 2 / 26

### Analytical Methods Exam: Preparatory Exercises

Anlyticl Methods Exm: Preprtory Exercises Question. Wht does it men tht (X, F, µ) is mesure spce? Show tht µ is monotone, tht is: if E F re mesurble sets then µ(e) µ(f). Question. Discuss if ech of the

### Section 14.3 Arc Length and Curvature

Section 4.3 Arc Length nd Curvture Clculus on Curves in Spce In this section, we ly the foundtions for describing the movement of n object in spce.. Vector Function Bsics In Clc, formul for rc length in

### Math 8 Winter 2015 Applications of Integration

Mth 8 Winter 205 Applictions of Integrtion Here re few importnt pplictions of integrtion. The pplictions you my see on n exm in this course include only the Net Chnge Theorem (which is relly just the Fundmentl

### Lecture 3. In this lecture, we will discuss algorithms for solving systems of linear equations.

Lecture 3 3 Solving liner equtions In this lecture we will discuss lgorithms for solving systems of liner equtions Multiplictive identity Let us restrict ourselves to considering squre mtrices since one

### MTH 5102 Linear Algebra Practice Exam 1 - Solutions Feb. 9, 2016

Nme (Lst nme, First nme): MTH 502 Liner Algebr Prctice Exm - Solutions Feb 9, 206 Exm Instructions: You hve hour & 0 minutes to complete the exm There re totl of 6 problems You must show your work Prtil

### Chapters 4 & 5 Integrals & Applications

Contents Chpters 4 & 5 Integrls & Applictions Motivtion to Chpters 4 & 5 2 Chpter 4 3 Ares nd Distnces 3. VIDEO - Ares Under Functions............................................ 3.2 VIDEO - Applictions

### INTRODUCTION TO LINEAR ALGEBRA

ME Applied Mthemtics for Mechnicl Engineers INTRODUCTION TO INEAR AGEBRA Mtrices nd Vectors Prof. Dr. Bülent E. Pltin Spring Sections & / ME Applied Mthemtics for Mechnicl Engineers INTRODUCTION TO INEAR

### Elements of Matrix Algebra

Elements of Mtrix Algebr Klus Neusser Kurt Schmidheiny September 30, 2015 Contents 1 Definitions 2 2 Mtrix opertions 3 3 Rnk of Mtrix 5 4 Specil Functions of Qudrtic Mtrices 6 4.1 Trce of Mtrix.........................

### How do we solve these things, especially when they get complicated? How do we know when a system has a solution, and when is it unique?

XII. LINEAR ALGEBRA: SOLVING SYSTEMS OF EQUATIONS Tody we re going to tlk out solving systems of liner equtions. These re prolems tht give couple of equtions with couple of unknowns, like: 6= x + x 7=

### Lecture 17. Integration: Gauss Quadrature. David Semeraro. University of Illinois at Urbana-Champaign. March 20, 2014

Lecture 17 Integrtion: Guss Qudrture Dvid Semerro University of Illinois t Urbn-Chmpign Mrch 0, 014 Dvid Semerro (NCSA) CS 57 Mrch 0, 014 1 / 9 Tody: Objectives identify the most widely used qudrture method

### State space systems analysis (continued) Stability. A. Definitions A system is said to be Asymptotically Stable (AS) when it satisfies

Stte spce systems nlysis (continued) Stbility A. Definitions A system is sid to be Asymptoticlly Stble (AS) when it stisfies ut () = 0, t > 0 lim xt () 0. t A system is AS if nd only if the impulse response

### along the vector 5 a) Find the plane s coordinate after 1 hour. b) Find the plane s coordinate after 2 hours. c) Find the plane s coordinate

L8 VECTOR EQUATIONS OF LINES HL Mth - Sntowski Vector eqution of line 1 A plne strts journey t the point (4,1) moves ech hour long the vector. ) Find the plne s coordinte fter 1 hour. b) Find the plne

### Line and Surface Integrals: An Intuitive Understanding

Line nd Surfce Integrls: An Intuitive Understnding Joseph Breen Introduction Multivrible clculus is ll bout bstrcting the ides of differentition nd integrtion from the fmilir single vrible cse to tht of

### Operations with Matrices

Section. Equlit of Mtrices Opertions with Mtrices There re three ws to represent mtri.. A mtri cn be denoted b n uppercse letter, such s A, B, or C.. A mtri cn be denoted b representtive element enclosed

### AMATH 731: Applied Functional Analysis Fall Additional notes on Fréchet derivatives

AMATH 731: Applied Functionl Anlysis Fll 214 Additionl notes on Fréchet derivtives (To ccompny Section 3.1 of the AMATH 731 Course Notes) Let X,Y be normed liner spces. The Fréchet derivtive of n opertor

### Math& 152 Section Integration by Parts

Mth& 5 Section 7. - Integrtion by Prts Integrtion by prts is rule tht trnsforms the integrl of the product of two functions into other (idelly simpler) integrls. Recll from Clculus I tht given two differentible

### Student Handbook for MATH 3300

Student Hndbook for MATH 3300 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 0.5 0 0.5 0.5 0 0.5 If people do not believe tht mthemtics is simple, it is only becuse they do not relize how complicted life is. John Louis

### A. Limits - L Hopital s Rule ( ) How to find it: Try and find limits by traditional methods (plugging in). If you get 0 0 or!!, apply C.! 1 6 C.

A. Limits - L Hopitl s Rule Wht you re finding: L Hopitl s Rule is used to find limits of the form f ( x) lim where lim f x x! c g x ( ) = or lim f ( x) = limg( x) = ". ( ) x! c limg( x) = 0 x! c x! c

### Bernoulli Numbers Jeff Morton

Bernoulli Numbers Jeff Morton. We re interested in the opertor e t k d k t k, which is to sy k tk. Applying this to some function f E to get e t f d k k tk d k f f + d k k tk dk f, we note tht since f