The Method of Least Squares. To understand least squares fitting of data.
|
|
- Milton Hodge
- 5 years ago
- Views:
Transcription
1 The Method of Least Squares KEY WORDS Curve fittig, least square GOAL To uderstad least squares fittig of data To uderstad the least squares solutio of icosistet systems of liear equatios 1 Motivatio Curve fittig is a process of fidig a relatively simple fuctio to approximate a set of data The data are ofte from experimetal measuremets that may cotai errors or have a limited umber of sigificat digits The approximatig fuctio cotais parameters that ca be adjusted to give agreemet with the data The quality of the fit is most ofte measured by a distace characterizatio betwee values of the approximatig fuctio ad the give data The least squares priciple asserts that the best fit is the oe that miimizes this distace For example, cosider the problem of fittig the data set x x 0 = 0 x 1 = 2 x 2 = 1 x 3 = 4 y y 0 = 2 y 1 = 1 y 2 = 1 y 3 = 3 I stadard polyomial iterpolatio, we would eed to fid a polyomial of degree 3, deoted by p 3 (x), such that p 3 (x i ) = y i, i = 0, 1, 2, 3 If the data ivolve error, this error will be carried over by the iterpolatig polyomial p 3 (x) Now we ask if we ca fit the give data set by usig a straight lie, a polyomial of degree 1 This meas that we eed to fid a polyomial such that p 1 (x) = a 0 + a 1 x (1) p 1 (x i ) = y i, i = 0, 1, 2, 3 (2) Sice a liear fuctio has oly two coefficiets (which ca be cosidered as adjustable parameters), it is geerally impossible to satisfy (2) I other words, i geeral, it is impossible to fid a liear fuctio that iterpolates four poits simultaeously The least squares method fids a liear fuctio p 1 (x) that fits the four poits i a best possible way This procedure ca be described as follows 1
2 First, we defie the error betwee the straight lie p 1 (x) of (1) ad the give set of data as follows: 3 σ(p 1 ) [p 1 (x i ) y i ] 2 i=0 Next, we adjust the parameters a 0 ad a 1 to miimize the error fuctio σ(p 1 ) To carry out the details for our example, we otice that the error fuctio is really give by 3 σ(p 1 ) = (a 0 + a 1 x i y i ) 2, which ca be expressed as i=0 σ(p 1 ) = (a 0 2) 2 + (a 0 + 2a 1 1) 2 + (a 0 a 1 + 1) 2 + (a 0 + 4a 1 3) 2 The fuctio σ(p 1 ) attais its miimum value at a poit (a 0, a 1 ) where the gradiet of σ(p 1 ) vaishes Therefore, we eed to solve the followig system of liear equatios: σ(p 1 ) a 0 2(a 0 2) + 2(a 0 + 2a 1 1) + 2(a 0 a 1 + 1) + 2(a 0 + 4a 1 3) = 8a a 1 10 = 0, σ(p 1 ) a 1 4(a 0 + 2a 1 1) 2(a 0 a 1 + 1) + 8(a 0 + 4a 1 3) = 10a a 1 30 = 0 This system of liear equatios ca the be rewritte as 4a 0 + 5a 1 = 5, 5a a 1 = 15, which ca be solved usig Gaussia elimiatio to give a 0 = 30 59, a 1 = (You ca check this usig the backslash operator i MATLAB) As a result, the straight lie that best fits the give data set i the least squares sese is y = x 2 Least Squares Fittig I this sectio, we preset a geeral discussio of least squares fittig of the data {(x r, f r )} As before, we assume that f r = f(x r ) 2
3 Let q m (x) be a polyomial of degree m with m Observe that q m (x r ) f r is the error i acceptig q m (x r ) as a approximatio to f r Thus the sum of the squares of these errors, σ(q m ) [q m (x r ) f r ] 2, gives a measure of how well q m (x) fits f(x) The idea is that the smaller the value of σ(q m ), the closer the polyomial q m (x) fits the data We say p m (x) is a least squares polyomial of degree m if p m (x) is a polyomial of degree m with the property that σ(p m ) σ(q m ) for all polyomials q m (x) of degree m; usually we oly have equality if q m (x) p m (x) As show i a advaced course i umerical aalysis or liear algebra, if the poits {x r } r=1 are distict ad if m, there is oe ad oly oe least squares polyomial of degree m for these data; thus we say p m (x) is the least squares polyomial of degree m So, the polyomial p m (x) that produces the smallest value σ(p m ) yields the least squares fit of the data While p m (x) produces the best fit of the data i the least squares sese, it may ot produce a very useful fit For example, cosider the case m = The the least squares fit p (x) is the same as the iterpolatig polyomial We have see already that the iterpolatig polyomial ca be a poor fit i the sese of havig a large ad highly oscillatory error So, a close fit i a least squares sese does ot ecessarily imply a very good fit ad, i some cases, the closer the fit is to a iterpolat the less useful it might be Sice the least squares criterio relaxes the fittig coditio from iterpolatio to a weaker coditio o the coefficiets of the polyomial, we eed fewer coefficiets (that is, a lower degree polyomial) i the represetatio As we have idicated, for the problem to be well posed, it is sufficiet that all the data poits be distict ad that m EXAMPLE 1 Cosider the least squares fit to the data i x i f i by a straight lie: p 1 (x) = a 0 + a 1 x; that is, the coefficiets a 0 ad a 1 are to be determied We have σ(p 1 ) = {p 1 (1) 2} 2 + {p 1 (3) 4} 2 + {p 1 (4) 3} 2 + {p 1 (5) 1} 2 = {a 0 + a 1 2} 2 + {a 0 + 3a 1 4} 2 + {a 0 + 4a 1 3} 2 + {a 0 + 5a 1 1} 2 3
4 Observe that σ(p 1 ) is quadratic i the ukow coefficiets a 0 ad a 1 For a miimum of σ(p 1 ), the ukows a 0 ad a 1 must satisfy the liear equatios σ(p 1 ) a 0 2 {a 0 + a 1 2} + 2 {a 0 + 3a 1 4} + 2 {a 0 + 4a 1 3} + 2 {a 0 + 5a 1 1} = a a 1 = 0, σ(p 1 ) a 1 2 {a 0 + a 1 2} + 6 {a 0 + 3a 1 4} + 8 {a 0 + 4a 1 3} + 10 {a 0 + 5a 1 1} = a a 1 = 0; that is, Gaussia elimiatio yields the solutio 4a a 1 = 10, (3) 13a a 1 = 31 (4) a 0 = , a 1 = 6 35 (Agai you ca check this usig the backslash operator i MATLAB) Therefore, the straight lie that best fits the give data set i the least squares sese is p 1 (x) = x It ca be see that the method of least-squares ivolves two mai steps: Formulate the total error Determie parameters (coefficiets) to miimize the total error Geerally, if we cosider fittig data usig a polyomial writte i power series form p m (x) = a 0 + a 1 x + + a m x m, (5) the σ(p m ) is quadratic i the ukow coefficiets a 0, a 1,, a m For the data {(x r, f r )}, we have σ(p m ) = {p m (x r ) f r } 2 (6) = {p m (x 0 ) f 0 } 2 + {p m (x 1 ) f 1 } {p m (x ) f } 2 (7) 4
5 The coefficiets a 0, a 1,, a m are determied by solvig the liear system For each value j = 0, 1,, m, the liear equatio is formed as follows Observe that The, by the chai rule, σ(p m ) a 0 = 0, (8) σ(p m ) a 1 = 0, (9) (10) σ(p m ) a m = 0 (11) σ(p m ) a j = 0 p m (x r ) a j = x j r σ(p m ) [ = {p m (x 0 ) f 0 } 2 + {p m (x 1 ) f 1 } {p m (x ) f } 2] (12) a j a j [ = 2 {p m (x 0 ) f 0 } p m(x 0 ) + {p m (x 1 ) f 1 } p m(x 1 ) + + {p m (x ) f } p ] m(x ) (13) a j a j a j [ ] = 2 {p m (x 0 ) f 0 } x j 0 + {p m(x 1 ) f 1 } x j {p m(x ) f } x j (14) = 2 {p m (x r ) f r } x j r (15) = 2 x j rp m (x r ) 2 f r x j r (16) Substitutig the power series form for the polyomial p m (x r ), (5), leads to [ ] σ(p m ) = 2 x j r {a 0 + a 1 x r + + a m x m r } f r x j r (17) a j [ ] = 2 a 0 x j r + a 1 x j+1 r + + a m x j+m r f r x j r (18) Therefore, the equatios σ(pm) a j ormal equatios: a 0 x j r + a 1 x j+1 r + + a m = 0, j = 0, 1,, m, may be rewritte as the 5 x j+m r = f r x j r, j = 0, 1,, m
6 I matrix form, the ormal equatios may be writte as 1 x r x r x2 r xm r xm+1 r xm r xm+1 r x2m r a 0 a 1 a m = f r f rx r f rx m r (19) The coefficiet matrix of this system is symmetric ad positive defiite, permittig use of a accurate, efficiet modified versio of Gaussia elimiatio which exploits these properties, without the eed for partial pivotig by rows for size This ca be doe usig the backslash operator \ i MATLAB (Whe \ is called with a symmetric coefficiet matrix that has positive diagoal elemets, MATLAB determies if the matrix is positive defiite ad, if it is, solves the system i a optimal way Otherwise, it uses Gaussia elimiatio with partial pivotig) EXAMPLE 2 To compute a straight lie fit a 0 +a 1 x to the data {(x r, f r )}, we set m = 1 i the ormal equatios to give a 0 a 0 x r + a a 1 x r = x 2 r = f r, (20) f r x r (21) Substitutig the data i x i f i from Example 1, we have the ormal equatios 4a a 1 = 10, a 1 = 31, which give the same result as i Example 1 The ext example shows how a quadratic polyomial ca be determied i the least squares sese for a give set of data 6
7 EXAMPLE 3 To compute a quadratic fit a 0 + a 1 x + a 2 x 2 to the data {(x r, f r )}, we set m = 2 i the ormal equatios to give a a 1 x r + a 2 a 0 x r + a 1 x 2 r + a 2 a 0 x 2 r + a 1 x 3 r + a 2 x 2 r = x 3 r = x 4 r = f r, (22) f r x r, (23) f r x 2 r (24) The least squares formulatio permits more geeral fuctios p m (x) tha simply polyomials, but the ukow coefficiets i p m (x) must still occur liearly The most geeral form is p m (x) = a 0 φ 0 (x) + a 1 φ 1 (x) + + a m φ m (x) = m a r φ r (x), where the basis fuctios {φ r (x)} m could be, for example, a liear polyomial splie basis, a cubic polyomial B splie basis, or a set of liearly idepedet trigoometric fuctios By aalogy with the power series case, the liear system of ormal equatios is a 0 φ 0 (x r )φ j (x r )+a 1 φ 1 (x r )φ j (x r )+ +a m φ m (x r )φ j (x r ) = f r φ j (x r ), for j = 0, 1,, m Agai, the coefficiet matrix of this liear system φ 0(x r ) 2 φ 0(x r )φ 1 (x r ) φ 0(x r )φ m (x r ) φ 1(x r )φ 0 (x r ) φ 1(x r ) 2 φ 1(x r )φ m (x r ) φ m(x r )φ 0 (x r ) φ m(x r )φ 1 (x r ) φ m(x r ) 2 is symmetric ad positive defiite 3 Least Squares Solutio of Liear Systems The priciple of least squares ca be used i other situatios As a example, let us attempt to solve a icosistet system of liear equatios of the form: a ij x j = b i, i = 1, 2,, m; (25) j=1 that is Ax = b, 7
8 where A = (a ij ) m, x = [x 1,, x ] T, b = [b 1,, b m ] T Here, m > ; that is, the umber of equatios is more tha the umber of ukows i the system of liear equatios (25) Geerally speakig, this system of liear equatios is icosistet i the sese that there is o vector x = (x 1,, x ) T which satisfies all the equatios i (25) I this case, a alterative is to miimize the sum of the squares of the residuals to obtai the least squares solutio of the liear system This solutio ca be determied i the followig way The sum of the squares of the residuals is give by 2 m σ(x 1, x 2,, x ) = a ij x j b i To fid the miimizatio poit for the error fuctio σ(x 1, x 2,, x ), we take the partial derivative of σ with respect to each variable x k, yieldig σ m = 2 a ij x j b i a ik, x k j=1 for k = 1, 2, Therefore, the miimizatio poit x = (x 1,, x ) must be a solutio of the liear system By settig j=1 j=1 m m a ij a ik x j = a ik b i, k = 1, 2,, m m c kj = a ij a ik, d k = a ik b i, (26) we see that the miimizatio poit x = (x 1, x 2,, x ) is the solutio of the system of liear equatios: c kj x j = d k, k = 1, 2,, m j=1 EXAMPLE 4 As a example of the least squares solutio of icosistet liear systems, cosider the three equatios i two ukows: x 1 + x 2 = 1, x 1 x 2 = 2, x 1 + 2x 2 = 1, 8
9 or ( x1 x 2 ) = This system of liear equatios is icosistet because it has o solutio at all Now, from (26), we see that c 11 = 3, c 12 = 2, c 21 = 2, c 22 = 6, d 1 = 2, d 2 = 3 Therefore, the least squares solutio must solve the 2 2 liear system The solutio is give by 3x 1 + 2x 2 = 2, (27) 2x 1 + 6x 2 = 3 (28) x 1 = 9 7, x 2 = There is a short cut to the least squares formulatio for icosistet liear systems Notice that the matrix C = (c ij ), where c ij is give by (26), is othig but the product of A T ad A; that is, C = A T A, ad the right-had side vector d = (d 1,, d ) T is give by d = A T b Therefore, the least squares solutio of the matrix problem Ax = b is give by the solutio of the ormal equatios A T Ax = A T b The followig summarizes the procedure i the least squares approach for icosistet liear systems of the form Ax = b: Fid the traspose of A, deoted by A T Multiply the system Ax = b by A T : A T Ax = A T b Fid the solutio for the ormal system: A T Ax = A T b The solutio to the ormal system is exactly the least squares solutio of the origial icosistet liear system 9
A widely used display of protein shapes is based on the coordinates of the alpha carbons - - C α
Nice plottig of proteis: I A widely used display of protei shapes is based o the coordiates of the alpha carbos - - C α -s. The coordiates of the C α -s are coected by a cotiuous curve that roughly follows
More informationZeros of Polynomials
Math 160 www.timetodare.com 4.5 4.6 Zeros of Polyomials I these sectios we will study polyomials algebraically. Most of our work will be cocered with fidig the solutios of polyomial equatios of ay degree
More informationLecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
More informationLecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationDefinitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients.
Defiitios ad Theorems Remember the scalar form of the liear programmig problem, Miimize, Subject to, f(x) = c i x i a 1i x i = b 1 a mi x i = b m x i 0 i = 1,2,, where x are the decisio variables. c, b,
More informationLinear Regression Demystified
Liear Regressio Demystified Liear regressio is a importat subject i statistics. I elemetary statistics courses, formulae related to liear regressio are ofte stated without derivatio. This ote iteds to
More information3.2 Properties of Division 3.3 Zeros of Polynomials 3.4 Complex and Rational Zeros of Polynomials
Math 60 www.timetodare.com 3. Properties of Divisio 3.3 Zeros of Polyomials 3.4 Complex ad Ratioal Zeros of Polyomials I these sectios we will study polyomials algebraically. Most of our work will be cocered
More informationCHAPTER 5. Theory and Solution Using Matrix Techniques
A SERIES OF CLASS NOTES FOR 2005-2006 TO INTRODUCE LINEAR AND NONLINEAR PROBLEMS TO ENGINEERS, SCIENTISTS, AND APPLIED MATHEMATICIANS DE CLASS NOTES 3 A COLLECTION OF HANDOUTS ON SYSTEMS OF ORDINARY DIFFERENTIAL
More informationChapter 2 The Solution of Numerical Algebraic and Transcendental Equations
Chapter The Solutio of Numerical Algebraic ad Trascedetal Equatios Itroductio I this chapter we shall discuss some umerical methods for solvig algebraic ad trascedetal equatios. The equatio f( is said
More informationChapter 4. Fourier Series
Chapter 4. Fourier Series At this poit we are ready to ow cosider the caoical equatios. Cosider, for eample the heat equatio u t = u, < (4.) subject to u(, ) = si, u(, t) = u(, t) =. (4.) Here,
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationInverse Matrix. A meaning that matrix B is an inverse of matrix A.
Iverse Matrix Two square matrices A ad B of dimesios are called iverses to oe aother if the followig holds, AB BA I (11) The otio is dual but we ofte write 1 B A meaig that matrix B is a iverse of matrix
More informationSeptember 2012 C1 Note. C1 Notes (Edexcel) Copyright - For AS, A2 notes and IGCSE / GCSE worksheets 1
September 0 s (Edecel) Copyright www.pgmaths.co.uk - For AS, A otes ad IGCSE / GCSE worksheets September 0 Copyright www.pgmaths.co.uk - For AS, A otes ad IGCSE / GCSE worksheets September 0 Copyright
More information[ 11 ] z of degree 2 as both degree 2 each. The degree of a polynomial in n variables is the maximum of the degrees of its terms.
[ 11 ] 1 1.1 Polyomial Fuctios 1 Algebra Ay fuctio f ( x) ax a1x... a1x a0 is a polyomial fuctio if ai ( i 0,1,,,..., ) is a costat which belogs to the set of real umbers ad the idices,, 1,...,1 are atural
More informationRecurrence Relations
Recurrece Relatios Aalysis of recursive algorithms, such as: it factorial (it ) { if (==0) retur ; else retur ( * factorial(-)); } Let t be the umber of multiplicatios eeded to calculate factorial(). The
More informationU8L1: Sec Equations of Lines in R 2
MCVU Thursda Ma, Review of Equatios of a Straight Lie (-D) U8L Sec. 8.9. Equatios of Lies i R Cosider the lie passig through A (-,) with slope, as show i the diagram below. I poit slope form, the equatio
More informationU8L1: Sec Equations of Lines in R 2
MCVU U8L: Sec. 8.9. Equatios of Lies i R Review of Equatios of a Straight Lie (-D) Cosider the lie passig through A (-,) with slope, as show i the diagram below. I poit slope form, the equatio of the lie
More informationNotes on iteration and Newton s method. Iteration
Notes o iteratio ad Newto s method Iteratio Iteratio meas doig somethig over ad over. I our cotet, a iteratio is a sequece of umbers, vectors, fuctios, etc. geerated by a iteratio rule of the type 1 f
More informationREGRESSION (Physics 1210 Notes, Partial Modified Appendix A)
REGRESSION (Physics 0 Notes, Partial Modified Appedix A) HOW TO PERFORM A LINEAR REGRESSION Cosider the followig data poits ad their graph (Table I ad Figure ): X Y 0 3 5 3 7 4 9 5 Table : Example Data
More informationProperties and Tests of Zeros of Polynomial Functions
Properties ad Tests of Zeros of Polyomial Fuctios The Remaider ad Factor Theorems: Sythetic divisio ca be used to fid the values of polyomials i a sometimes easier way tha substitutio. This is show by
More informationEfficient GMM LECTURE 12 GMM II
DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet
More informationMath 155 (Lecture 3)
Math 55 (Lecture 3) September 8, I this lecture, we ll cosider the aswer to oe of the most basic coutig problems i combiatorics Questio How may ways are there to choose a -elemet subset of the set {,,,
More informationAlgebra of Least Squares
October 19, 2018 Algebra of Least Squares Geometry of Least Squares Recall that out data is like a table [Y X] where Y collects observatios o the depedet variable Y ad X collects observatios o the k-dimesioal
More informationIn algebra one spends much time finding common denominators and thus simplifying rational expressions. For example:
74 The Method of Partial Fractios I algebra oe speds much time fidig commo deomiators ad thus simplifyig ratioal epressios For eample: + + + 6 5 + = + = = + + + + + ( )( ) 5 It may the seem odd to be watig
More informationFeedback in Iterative Algorithms
Feedback i Iterative Algorithms Charles Byre (Charles Byre@uml.edu), Departmet of Mathematical Scieces, Uiversity of Massachusetts Lowell, Lowell, MA 01854 October 17, 2005 Abstract Whe the oegative system
More informationCALCULUS BASIC SUMMER REVIEW
CALCULUS BASIC SUMMER REVIEW NAME rise y y y Slope of a o vertical lie: m ru Poit Slope Equatio: y y m( ) The slope is m ad a poit o your lie is, ). ( y Slope-Itercept Equatio: y m b slope= m y-itercept=
More informationSNAP Centre Workshop. Basic Algebraic Manipulation
SNAP Cetre Workshop Basic Algebraic Maipulatio 8 Simplifyig Algebraic Expressios Whe a expressio is writte i the most compact maer possible, it is cosidered to be simplified. Not Simplified: x(x + 4x)
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More informationStudy the bias (due to the nite dimensional approximation) and variance of the estimators
2 Series Methods 2. Geeral Approach A model has parameters (; ) where is ite-dimesioal ad is oparametric. (Sometimes, there is o :) We will focus o regressio. The fuctio is approximated by a series a ite
More informationECON 3150/4150, Spring term Lecture 3
Itroductio Fidig the best fit by regressio Residuals ad R-sq Regressio ad causality Summary ad ext step ECON 3150/4150, Sprig term 2014. Lecture 3 Ragar Nymoe Uiversity of Oslo 21 Jauary 2014 1 / 30 Itroductio
More informationLast time, we talked about how Equation (1) can simulate Equation (2). We asserted that Equation (2) can also simulate Equation (1).
6896 Quatum Complexity Theory Sept 23, 2008 Lecturer: Scott Aaroso Lecture 6 Last Time: Quatum Error-Correctio Quatum Query Model Deutsch-Jozsa Algorithm (Computes x y i oe query) Today: Berstei-Vazirii
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More informationThe Discrete Fourier Transform
The Discrete Fourier Trasform Complex Fourier Series Represetatio Recall that a Fourier series has the form a 0 + a k cos(kt) + k=1 b k si(kt) This represetatio seems a bit awkward, sice it ivolves two
More informationOptimization Methods MIT 2.098/6.255/ Final exam
Optimizatio Methods MIT 2.098/6.255/15.093 Fial exam Date Give: December 19th, 2006 P1. [30 pts] Classify the followig statemets as true or false. All aswers must be well-justified, either through a short
More informationA sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as
More informationIntroduction to Machine Learning DIS10
CS 189 Fall 017 Itroductio to Machie Learig DIS10 1 Fu with Lagrage Multipliers (a) Miimize the fuctio such that f (x,y) = x + y x + y = 3. Solutio: The Lagragia is: L(x,y,λ) = x + y + λ(x + y 3) Takig
More informationUnit 4: Polynomial and Rational Functions
48 Uit 4: Polyomial ad Ratioal Fuctios Polyomial Fuctios A polyomial fuctio y px ( ) is a fuctio of the form p( x) ax + a x + a x +... + ax + ax+ a 1 1 1 0 where a, a 1,..., a, a1, a0are real costats ad
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationEigenvalues and Eigenvectors
5 Eigevalues ad Eigevectors 5.3 DIAGONALIZATION DIAGONALIZATION Example 1: Let. Fid a formula for A k, give that P 1 1 = 1 2 ad, where Solutio: The stadard formula for the iverse of a 2 2 matrix yields
More informationMost text will write ordinary derivatives using either Leibniz notation 2 3. y + 5y= e and y y. xx tt t
Itroductio to Differetial Equatios Defiitios ad Termiolog Differetial Equatio: A equatio cotaiig the derivatives of oe or more depedet variables, with respect to oe or more idepedet variables, is said
More informationIt is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial.
Taylor Polyomials ad Taylor Series It is ofte useful to approximate complicated fuctios usig simpler oes We cosider the task of approximatig a fuctio by a polyomial If f is at least -times differetiable
More informationSection A assesses the Units Numerical Analysis 1 and 2 Section B assesses the Unit Mathematics for Applied Mathematics
X0/70 NATIONAL QUALIFICATIONS 005 MONDAY, MAY.00 PM 4.00 PM APPLIED MATHEMATICS ADVANCED HIGHER Numerical Aalysis Read carefully. Calculators may be used i this paper.. Cadidates should aswer all questios.
More informationALGEBRAIC GEOMETRY COURSE NOTES, LECTURE 5: SINGULARITIES.
ALGEBRAIC GEOMETRY COURSE NOTES, LECTURE 5: SINGULARITIES. ANDREW SALCH 1. The Jacobia criterio for osigularity. You have probably oticed by ow that some poits o varieties are smooth i a sese somethig
More informationA NEW CLASS OF 2-STEP RATIONAL MULTISTEP METHODS
Jural Karya Asli Loreka Ahli Matematik Vol. No. (010) page 6-9. Jural Karya Asli Loreka Ahli Matematik A NEW CLASS OF -STEP RATIONAL MULTISTEP METHODS 1 Nazeeruddi Yaacob Teh Yua Yig Norma Alias 1 Departmet
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationANALYSIS OF EXPERIMENTAL ERRORS
ANALYSIS OF EXPERIMENTAL ERRORS All physical measuremets ecoutered i the verificatio of physics theories ad cocepts are subject to ucertaities that deped o the measurig istrumets used ad the coditios uder
More information3. Z Transform. Recall that the Fourier transform (FT) of a DT signal xn [ ] is ( ) [ ] = In order for the FT to exist in the finite magnitude sense,
3. Z Trasform Referece: Etire Chapter 3 of text. Recall that the Fourier trasform (FT) of a DT sigal x [ ] is ω ( ) [ ] X e = j jω k = xe I order for the FT to exist i the fiite magitude sese, S = x [
More informationLinear Programming and the Simplex Method
Liear Programmig ad the Simplex ethod Abstract This article is a itroductio to Liear Programmig ad usig Simplex method for solvig LP problems i primal form. What is Liear Programmig? Liear Programmig is
More informationChapter 10: Power Series
Chapter : Power Series 57 Chapter Overview: Power Series The reaso series are part of a Calculus course is that there are fuctios which caot be itegrated. All power series, though, ca be itegrated because
More informationMath 475, Problem Set #12: Answers
Math 475, Problem Set #12: Aswers A. Chapter 8, problem 12, parts (b) ad (d). (b) S # (, 2) = 2 2, sice, from amog the 2 ways of puttig elemets ito 2 distiguishable boxes, exactly 2 of them result i oe
More informationNUMERICAL METHODS FOR SOLVING EQUATIONS
Mathematics Revisio Guides Numerical Methods for Solvig Equatios Page 1 of 11 M.K. HOME TUITION Mathematics Revisio Guides Level: GCSE Higher Tier NUMERICAL METHODS FOR SOLVING EQUATIONS Versio:. Date:
More informationOutline. Linear regression. Regularization functions. Polynomial curve fitting. Stochastic gradient descent for regression. MLE for regression
REGRESSION 1 Outlie Liear regressio Regularizatio fuctios Polyomial curve fittig Stochastic gradiet descet for regressio MLE for regressio Step-wise forward regressio Regressio methods Statistical techiques
More information, then cv V. Differential Equations Elements of Lineaer Algebra Name: Consider the differential equation. and y2 cos( kx)
Cosider the differetial equatio y '' k y 0 has particular solutios y1 si( kx) ad y cos( kx) I geeral, ay liear combiatio of y1 ad y, cy 1 1 cy where c1, c is also a solutio to the equatio above The reaso
More informationEDEXCEL NATIONAL CERTIFICATE UNIT 4 MATHEMATICS FOR TECHNICIANS OUTCOME 4 - CALCULUS
EDEXCEL NATIONAL CERTIFICATE UNIT 4 MATHEMATICS FOR TECHNICIANS OUTCOME 4 - CALCULUS TUTORIAL 1 - DIFFERENTIATION Use the elemetary rules of calculus arithmetic to solve problems that ivolve differetiatio
More informationThe Growth of Functions. Theoretical Supplement
The Growth of Fuctios Theoretical Supplemet The Triagle Iequality The triagle iequality is a algebraic tool that is ofte useful i maipulatig absolute values of fuctios. The triagle iequality says that
More informationFind quadratic function which pass through the following points (0,1),(1,1),(2, 3)... 11
Adrew Powuk - http://www.powuk.com- Math 49 (Numerical Aalysis) Iterpolatio... 4. Polyomial iterpolatio (system of equatio)... 4.. Lier iterpolatio... 5... Fid a lie which pass through (,) (,)... 8...
More informationPolynomial Functions. New Section 1 Page 1. A Polynomial function of degree n is written is the form:
New Sectio 1 Page 1 A Polyomial fuctio of degree is writte is the form: 1 P x a x a x a x a x a x a 1 1 0 where is a o-egative iteger expoet ad a 0 ca oly take o values where a, a, a,..., a, a. 0 1 1.
More informationFor a 3 3 diagonal matrix we find. Thus e 1 is a eigenvector corresponding to eigenvalue λ = a 11. Thus matrix A has eigenvalues 2 and 3.
Closed Leotief Model Chapter 6 Eigevalues I a closed Leotief iput-output-model cosumptio ad productio coicide, i.e. V x = x = x Is this possible for the give techology matrix V? This is a special case
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More information10.6 ALTERNATING SERIES
0.6 Alteratig Series Cotemporary Calculus 0.6 ALTERNATING SERIES I the last two sectios we cosidered tests for the covergece of series whose terms were all positive. I this sectio we examie series whose
More informationTEACHER CERTIFICATION STUDY GUIDE
COMPETENCY 1. ALGEBRA SKILL 1.1 1.1a. ALGEBRAIC STRUCTURES Kow why the real ad complex umbers are each a field, ad that particular rigs are ot fields (e.g., itegers, polyomial rigs, matrix rigs) Algebra
More informationMath 113 Exam 3 Practice
Math Exam Practice Exam will cover.-.9. This sheet has three sectios. The first sectio will remid you about techiques ad formulas that you should kow. The secod gives a umber of practice questios for you
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationLinear Regression Models
Liear Regressio Models Dr. Joh Mellor-Crummey Departmet of Computer Sciece Rice Uiversity johmc@cs.rice.edu COMP 528 Lecture 9 15 February 2005 Goals for Today Uderstad how to Use scatter diagrams to ispect
More information18.01 Calculus Jason Starr Fall 2005
Lecture 18. October 5, 005 Homework. Problem Set 5 Part I: (c). Practice Problems. Course Reader: 3G 1, 3G, 3G 4, 3G 5. 1. Approximatig Riema itegrals. Ofte, there is o simpler expressio for the atiderivative
More informationThe picture in figure 1.1 helps us to see that the area represents the distance traveled. Figure 1: Area represents distance travelled
1 Lecture : Area Area ad distace traveled Approximatig area by rectagles Summatio The area uder a parabola 1.1 Area ad distace Suppose we have the followig iformatio about the velocity of a particle, how
More informationMachine Learning for Data Science (CS 4786)
Machie Learig for Data Sciece CS 4786) Lecture & 3: Pricipal Compoet Aalysis The text i black outlies high level ideas. The text i blue provides simple mathematical details to derive or get to the algorithm
More informationM A T H F A L L CORRECTION. Algebra I 1 4 / 1 0 / U N I V E R S I T Y O F T O R O N T O
M A T H 2 4 0 F A L L 2 0 1 4 HOMEWORK ASSIGNMENT #4 CORRECTION Algebra I 1 4 / 1 0 / 2 0 1 4 U N I V E R S I T Y O F T O R O N T O P r o f e s s o r : D r o r B a r - N a t a Correctio Homework Assigmet
More informationSlide Set 13 Linear Model with Endogenous Regressors and the GMM estimator
Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday
More informationSummary and Discussion on Simultaneous Analysis of Lasso and Dantzig Selector
Summary ad Discussio o Simultaeous Aalysis of Lasso ad Datzig Selector STAT732, Sprig 28 Duzhe Wag May 4, 28 Abstract This is a discussio o the work i Bickel, Ritov ad Tsybakov (29). We begi with a short
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationSection 11.8: Power Series
Sectio 11.8: Power Series 1. Power Series I this sectio, we cosider geeralizig the cocept of a series. Recall that a series is a ifiite sum of umbers a. We ca talk about whether or ot it coverges ad i
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationTHE UNIVERSITY OF TORONTO UNDERGRADUATE MATHEMATICS COMPETITION. In Memory of Robert Barrington Leigh. March 9, 2014
THE UNIVERSITY OF TORONTO UNDERGRADUATE MATHEMATICS COMPETITION I Memory of Robert Barrigto Leigh March 9, 4 Time: 3 hours No aids or calculators permitted. The gradig is desiged to ecourage oly the stroger
More informationMATH 304: MIDTERM EXAM SOLUTIONS
MATH 304: MIDTERM EXAM SOLUTIONS [The problems are each worth five poits, except for problem 8, which is worth 8 poits. Thus there are 43 possible poits.] 1. Use the Euclidea algorithm to fid the greatest
More information5.1. The Rayleigh s quotient. Definition 49. Let A = A be a self-adjoint matrix. quotient is the function. R(x) = x,ax, for x = 0.
40 RODICA D. COSTIN 5. The Rayleigh s priciple ad the i priciple for the eigevalues of a self-adjoit matrix Eigevalues of self-adjoit matrices are easy to calculate. This sectio shows how this is doe usig
More informationGeometry of LS. LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT
OCTOBER 7, 2016 LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT Geometry of LS We ca thik of y ad the colums of X as members of the -dimesioal Euclidea space R Oe ca
More information2 Geometric interpretation of complex numbers
2 Geometric iterpretatio of complex umbers 2.1 Defiitio I will start fially with a precise defiitio, assumig that such mathematical object as vector space R 2 is well familiar to the studets. Recall that
More informationWe are mainly going to be concerned with power series in x, such as. (x)} converges - that is, lims N n
Review of Power Series, Power Series Solutios A power series i x - a is a ifiite series of the form c (x a) =c +c (x a)+(x a) +... We also call this a power series cetered at a. Ex. (x+) is cetered at
More informationTMA4205 Numerical Linear Algebra. The Poisson problem in R 2 : diagonalization methods
TMA4205 Numerical Liear Algebra The Poisso problem i R 2 : diagoalizatio methods September 3, 2007 c Eiar M Røquist Departmet of Mathematical Scieces NTNU, N-749 Trodheim, Norway All rights reserved A
More informationPAPER : IIT-JAM 2010
MATHEMATICS-MA (CODE A) Q.-Q.5: Oly oe optio is correct for each questio. Each questio carries (+6) marks for correct aswer ad ( ) marks for icorrect aswer.. Which of the followig coditios does NOT esure
More informationIP Reference guide for integer programming formulations.
IP Referece guide for iteger programmig formulatios. by James B. Orli for 15.053 ad 15.058 This documet is iteded as a compact (or relatively compact) guide to the formulatio of iteger programs. For more
More informationx x x 2x x N ( ) p NUMERICAL METHODS UNIT-I-SOLUTION OF EQUATIONS AND EIGENVALUE PROBLEMS By Newton-Raphson formula
NUMERICAL METHODS UNIT-I-SOLUTION OF EQUATIONS AND EIGENVALUE PROBLEMS. If g( is cotiuous i [a,b], te uder wat coditio te iterative (or iteratio metod = g( as a uique solutio i [a,b]? '( i [a,b].. Wat
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationMath 113 Exam 3 Practice
Math Exam Practice Exam 4 will cover.-., 0. ad 0.. Note that eve though. was tested i exam, questios from that sectios may also be o this exam. For practice problems o., refer to the last review. This
More informationPhysics 324, Fall Dirac Notation. These notes were produced by David Kaplan for Phys. 324 in Autumn 2001.
Physics 324, Fall 2002 Dirac Notatio These otes were produced by David Kapla for Phys. 324 i Autum 2001. 1 Vectors 1.1 Ier product Recall from liear algebra: we ca represet a vector V as a colum vector;
More informationAnalysis of Experimental Measurements
Aalysis of Experimetal Measuremets Thik carefully about the process of makig a measuremet. A measuremet is a compariso betwee some ukow physical quatity ad a stadard of that physical quatity. As a example,
More information~j=zhax~ 6=0, t,2... ), k=~
Numerische Mathematik 6, 35 -- 39 (964) Exactess Coditios i Numerical Quadrature* By HERBERT S. WILl* The itegral b () I--f t(x)dx is usually computed umerically by meas of a formula of the type (2) I
More informationThe Phi Power Series
The Phi Power Series I did this work i about 0 years while poderig the relatioship betwee the golde mea ad the Madelbrot set. I have fially decided to make it available from my blog at http://semresearch.wordpress.com/.
More informationA multivariate rational interpolation with no poles in R m
NTMSCI 3, No., 9-8 (05) 9 New Treds i Mathematical Scieces http://www.tmsci.com A multivariate ratioal iterpolatio with o poles i R m Osma Rasit Isik, Zekeriya Guey ad Mehmet Sezer Departmet of Mathematics,
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationCALCULATION OF FIBONACCI VECTORS
CALCULATION OF FIBONACCI VECTORS Stuart D. Aderso Departmet of Physics, Ithaca College 953 Daby Road, Ithaca NY 14850, USA email: saderso@ithaca.edu ad Dai Novak Departmet of Mathematics, Ithaca College
More informationBINOMIAL COEFFICIENT AND THE GAUSSIAN
BINOMIAL COEFFICIENT AND THE GAUSSIAN The biomial coefficiet is defied as-! k!(! ad ca be writte out i the form of a Pascal Triagle startig at the zeroth row with elemet 0,0) ad followed by the two umbers,
More informationChapter 9 - CD companion 1. A Generic Implementation; The Common-Merge Amplifier. 1 τ is. ω ch. τ io
Chapter 9 - CD compaio CHAPTER NINE CD-9.2 CD-9.2. Stages With Voltage ad Curret Gai A Geeric Implemetatio; The Commo-Merge Amplifier The advaced method preseted i the text for approximatig cutoff frequecies
More information6.867 Machine learning, lecture 7 (Jaakkola) 1
6.867 Machie learig, lecture 7 (Jaakkola) 1 Lecture topics: Kerel form of liear regressio Kerels, examples, costructio, properties Liear regressio ad kerels Cosider a slightly simpler model where we omit
More informationMath 312 Lecture Notes One Dimensional Maps
Math 312 Lecture Notes Oe Dimesioal Maps Warre Weckesser Departmet of Mathematics Colgate Uiversity 21-23 February 25 A Example We begi with the simplest model of populatio growth. Suppose, for example,
More information( a) ( ) 1 ( ) 2 ( ) ( ) 3 3 ( ) =!
.8,.9: Taylor ad Maclauri Series.8. Although we were able to fid power series represetatios for a limited group of fuctios i the previous sectio, it is ot immediately obvious whether ay give fuctio has
More informationMIDTERM 3 CALCULUS 2. Monday, December 3, :15 PM to 6:45 PM. Name PRACTICE EXAM SOLUTIONS
MIDTERM 3 CALCULUS MATH 300 FALL 08 Moday, December 3, 08 5:5 PM to 6:45 PM Name PRACTICE EXAM S Please aswer all of the questios, ad show your work. You must explai your aswers to get credit. You will
More information