Least Squares Fitting of Data
|
|
- Eugene Jayson Short
- 6 years ago
- Views:
Transcription
1 Least Squares Fttng of Data Davd Eberly Geoetrc Tools, LLC Copyrght c All Rghts Reserved. Created: July 15, 1999 Last Modfed: January 5, 2015 Contents 1 Lnear Fttng of 2D Ponts of For x, fx)) 2 2 Lnear Fttng of nd Ponts Usng Orthogonal Regresson 2 3 Planar Fttng of 3D Ponts of For x, y, fx, y)) 3 4 Hyperplanar Fttng of nd Ponts Usng Orthogonal Regresson 4 5 Fttng a Crcle to 2D Ponts 5 6 Fttng a Sphere to 3D Ponts 6 7 Fttng an Ellpse to 2D Ponts 7 8 Fttng an Ellpsod to 3D Ponts 8 9 Fttng a Parabolod to 3D Ponts of the For x, y, fx, y)) 8 1
2 Ths docuent descrbes soe algorths for fttng 2D or 3D pont sets by lnear or quadratc structures usng least squares nzaton. 1 Lnear Fttng of 2D Ponts of For x, fx)) Ths s the usual ntroducton to least squares ft by a lne when the data represents easureents where the y-coponent s assued to be functonally dependent on the x-coponent. Gven a set of saples {x, y )}, deterne A and B so that the lne y = Ax + B best fts the saples n the sense that the su of the squared errors between the y and the lne values Ax + B s nzed. Note that the error s easured only n the y-drecton. Defne EA, B) = [Ax + B) y ] 2. Ths functon s nonnegatve and ts graph s a parabolod whose vertex occurs when the gradent satstfes E = 0, 0). Ths leads to a syste of two lnear equatons n A and B whch can be easly solved. Precsely, 0, 0) = E = 2 [Ax + B) y ]x, 1) and so x2 x x 1 A B = x y y. The soluton provdes the least squares soluton y = Ax + B. If pleented drectly, ths forulaton can lead to an ll-condtoned lnear syste. To avod ths, you should frst copute the averages x = x and ȳ = y and subtract the fro the data, x x x and y y ȳ. The ftted lne s of the for y ȳ = Ax x). 2 Lnear Fttng of nd Ponts Usng Orthogonal Regresson It s also possble to ft a lne usng least squares where the errors are easured orthogonally to the proposed lne rather than easured vertcally. The followng arguent holds for saple ponts and lnes n n densons. Let the lne be Lt) = td + A where D s unt length. Defne X to be the saple ponts; then X = A + d D + p D where d = D X A) and D s soe unt length vector perpendcular to D wth approprate coeffcent p. Defne Y = X A. The vector fro X to ts projecton onto the lne s Y d D = p D. The squared length of ths vector s p 2 = Y d D) 2. The energy functon for the least squares nzaton s EA, D) = p2. Two alternate fors for ths functon are EA, D) = Y T 2 [I DD T] Y )
3 and EA, D) = D T [ ] ) Y Y )I Y Y T D = D T MA)D. Usng the frst for of E n the prevous equaton, take the dervatve wth respect to A to get [ A = 2 I DD T] Y. Ths partal dervatve s zero whenever Y = 0 n whch case A = 1/) X the average of the saple ponts). Gven A, the atrx MA) s deterned n the second for of the energy functon. The quantty D T MA)D s a quadratc for whose nu s the sallest egenvalue of MA). Ths can be found by standard egensyste solvers. A correspondng unt length egenvector D copletes our constructon of the least squares lne. For n = 2, f A = a, b), then atrx MA) s gven by ) n MA) = x a) 2 + y b) x a) 2 x a)y b) x. a)y b) y b) 2 For n = 3, f A = a, b, c), then atrx MA) s gven by x a) 2 x a)y b) x a)z c) MA) = δ x a)y b) y b) 2 y b)z c) x a)z c) y b)z c) z c) 2 where δ = x a) 2 + y b) 2 + z c) 2. 3 Planar Fttng of 3D Ponts of For x, y, fx, y)) The assupton s that the z-coponent of the data s functonally dependent on the x- and y-coponents. Gven a set of saples {x, y, z )}, deterne A, B, and C so that the plane z = Ax + By + C best fts the saples n the sense that the su of the squared errors between the z and the plane values Ax +By +C s nzed. Note that the error s easured only n the z-drecton. Defne EA, B, C) = [Ax + By + C) z ] 2. Ths functon s nonnegatve and ts graph s a hyperparabolod whose vertex occurs when the gradent satstfes E = 0, 0, 0). Ths leads to a syste of three lnear equatons n A, B, and C whch can be easly solved. Precsely, 0, 0, 0) = E = 2 [Ax + By + C) z ]x, y, 1) 3
4 and so x2 x y x x y y2 y x y 1 A B C = x z y z z. The soluton provdes the least squares soluton z = Ax + By + C. If pleented drectly, ths forulaton can lead to an ll-condtoned lnear syste. To avod ths, you should frst copute the averages x = x, ȳ = y, and z = z, and then subtract the fro the data, x x x, y y ȳ, and z z z. The ftted plane s of the for z z = Ax x)+by ȳ). 4 Hyperplanar Fttng of nd Ponts Usng Orthogonal Regresson It s also possble to ft a plane usng least squares where the errors are easured orthogonally to the proposed plane rather than easured vertcally. The followng arguent holds for saple ponts and hyperplanes n n densons. Let the hyperplane be N X A) = 0 where N s a unt length noral to the hyperplane and A s a pont on the hyperplane. Defne X to be the saple ponts; then X = A + λ N + p N where λ = N X A) and N s soe unt length vector perpendcular to N wth approprate coeffcent p. Defne Y = X A. The vector fro X to ts projecton onto the hyperplane s λ N. The squared length of ths vector s λ 2 = N Y ) 2. The energy functon for the least squares nzaton s EA, N) = λ2. Two alternate fors for ths functon are EA, N) = Y T [NN T] Y ) and ) EA, N) = N T Y Y T N = N T MA)N. Usng the frst for of E n the prevous equaton, take the dervatve wth respect to A to get [ A = 2 NN T] Y. Ths partal dervatve s zero whenever Y = 0 n whch case A = 1/) X the average of the saple ponts). Gven A, the atrx MA) s deterned n the second for of the energy functon. The quantty N T MA)N s a quadratc for whose nu s the sallest egenvalue of MA). Ths can be found by standard egensyste solvers. A correspondng unt length egenvector N copletes our constructon of the least squares hyperplane. 4
5 For n = 3, f A = a, b, c), then atrx MA) s gven by x a) 2 x a)y b) x a)z c) MA) = x a)y b) y b) 2 y b)z c) x a)z c) y b)z c) z c) 2. 5 Fttng a Crcle to 2D Ponts Gven a set of ponts {x, y )}, 3, ft the wth a crcle x a)2 + y b) 2 = r 2 where a, b) s the crcle center and r s the crcle radus. An assupton of ths algorth s that not all the ponts are collnear. The energy functon to be nzed s Ea, b, r) = L r) 2 where L = x a) 2 + y b) 2. Take the partal dervatve wth respect to r to obtan Settng equal to zero yelds r = 2 L r). r = 1 L. Take the partal dervatve wth respect to a to obtan a = 2 L r) L a = 2 x a) + r L ) a and take the partal dervatve wth respect to b to obtan b = 2 L r) L b = 2 y b) + r L ). b Settng these two dervatves equal to zero yelds a = 1 x + r 1 and b = 1 y + r 1 Replacng r by ts equvalent fro / r = 0 and usng L / a = a x )/L and L / b = b y )/L, we get two nonlnear equatons n a and b: a = x + L L a =: F a, b) b = ȳ + L L b =: Ga, b) L a L b. 5
6 where x = 1 x ȳ = 1 y L = 1 L L a = 1 L b = 1 Fxed pont teraton can be appled to solvng these equatons: a 0 = x, b 0 = ȳ, and a +1 = F a, b ) and b +1 = Ga, b ) for 0. Warnng. I have not analyzed the convergence propertes of ths algorth. In a few experents t sees to converge just fne. a x L b y L 6 Fttng a Sphere to 3D Ponts Gven a set of ponts {x, y, z )}, 4, ft the wth a sphere x a)2 + y b) 2 + z c) 2 = r 2 where a, b, c) s the sphere center and r s the sphere radus. An assupton of ths algorth s that not all the ponts are coplanar. The energy functon to be nzed s Ea, b, c, r) = L r) 2 where L = x a) 2 + y b) 2 + z c). Take the partal dervatve wth respect to r to obtan Settng equal to zero yelds r = 2 L r). r = 1 L. Take the partal dervatve wth respect to a to obtan a = 2 L r) L a = 2 x a) + r L ), a take the partal dervatve wth respect to b to obtan b = 2 L r) L b = 2 y b) + r L ), b and take the partal dervatve wth respect to c to obtan c = 2 L r) L c = 2 z c) + r L ). c 6
7 Settng these three dervatves equal to zero yelds and and a = 1 b = 1 c = 1 x + r 1 y + r 1 z + r 1 Replacng r by ts equvalent fro / r = 0 and usng L / a = a x )/L, L / b = b y )/L, and L / c = c z )/L, we get three nonlnear equatons n a, b, and c: where L a L b. L c. a = x + L L a =: F a, b, c) b = ȳ + L L b =: Ga, b, c) c = z + L L c =: Ha, b, c) x = 1 x ȳ = 1 y z = 1 z L = 1 L L a = 1 L b = 1 L c = 1 Fxed pont teraton can be appled to solvng these equatons: a 0 = x, b 0 = ȳ, c 0 = z, and a +1 = F a, b, c ), b +1 = Ga, b, c ), and c +1 = Ha, b, c ) for 0. Warnng. I have not analyzed the convergence propertes of ths algorth. In a few experents t sees to converge just fne. a x L b y L c z L 7 Fttng an Ellpse to 2D Ponts Gven a set of ponts {X }, 3, ft the wth an ellpse X U)T R T DRX U) = 1 where U s the ellpse center, R s an orthonoral atrx representng the ellpse orentaton, and D s a dagonal atrx whose dagonal entres represent the recprocal of the squares of the half-lengths lengths of the axes of the ellpse. An axs-algned ellpse wth center at the orgn has equaton x/a) 2 + y/b) 2 = 1. In ths settng, U = 0, 0), R = I the dentty atrx), and D = dag1/a 2, 1/b 2 ). The energy functon to be nzed s EU, R, D) = 7 L r) 2
8 where L s the dstance fro X to the ellpse wth the gven paraeters. Ths proble s ore dffcult than that of fttng crcles. The dstance L s coputed accordng to the algorth descrbed n Dstance fro a Pont to an Ellpse, an Ellpsod, or a Hyperellpsod. The functon E s nzed teratvely usng Powell s drecton-set ethod to search for a nu. An pleentaton s GteApprEllpse2.h. 8 Fttng an Ellpsod to 3D Ponts Gven a set of ponts {X }, 3, ft the wth an ellpsod X U)T R T DRX U) = 1 where U s the ellpsod center and R s an orthonoral atrx representng the ellpsod orentaton. The atrx D s a dagonal atrx whose dagonal entres represent the recprocal of the squares of the half-lengths of the axes of the ellpsod. An axs-algned ellpsod wth center at the orgn has equaton x/a) 2 + y/b) 2 + z/c) 2 = 1. In ths settng, U = 0, 0, 0), R = I the dentty atrx), and D = dag1/a 2, 1/b 2, 1/c 2 ). The energy functon to be nzed s EU, R, D) = L r) 2 where L s the dstance fro X to the ellpse wth the gven paraeters. Ths proble s ore dffcult than that of fttng spheres. The dstance L s coputed accordng to the algorth descrbed n Dstance fro a Pont to an Ellpse, an Ellpsod, or a Hyperellpsod. The functon E s nzed teratvely usng Powell s drecton-set ethod to search for a nu. An pleentaton s GteApprEllpsod3.h. 9 Fttng a Parabolod to 3D Ponts of the For x, y, fx, y)) Gven a set of saples {x, y, z )} and assung that the true values le on a parabolod z = fx, y) = p 1 x 2 + p 2 xy + p 3 y 2 + p 4 x + p 5 y + p 6 = P Qx, y) where P = p 1, p 2, p 3, p 4, p 5, p 6 ) and Qx, y) = x 2, xy, y 2, x, y, 1), select P to nze the su of squared errors EP) = P Q z ) 2 where Q = Qx, y ). The nu occurs when the gradent of E s the zero vector, E = 2 P Q z )Q = 0. Soe algebra converts ths to a syste of 6 equatons n 6 unknowns: ) Q Q T P = z Q. 8
9 The product Q Q T s a product of the 6 1 atrx Q wth the 1 6 atrx Q T, the result beng a 6 6 atrx. Defne the 6 6 syetrc atrx A = Q Q T and the 6 1 vector B = z Q. The choce for P s the soluton to the lnear syste of equatons AP = B. The entres of A and B ndcate suatons over the approprate product of varables. For exaple, sx 3 y) = x3 y : sx 4 ) sx 3 y) sx 2 y 2 ) sx 3 ) sx 2 y) sx 2 ) p 1 szx 2 ) sx 3 y) sx 2 y 2 ) sxy 3 ) sx 2 y) sxy 2 ) sxy) p 2 szxy) sx 2 y 2 ) sxy 3 ) sy 4 ) sxy 2 ) sy 3 ) sy 2 ) p 3 szy sx 3 ) sx 2 y) sxy 2 ) sx 2 = 2 ) ) sxy) sx) p 4 szx) sx 2 y) sxy 2 ) sy 3 ) sxy) sy 2 ) sy) p 5 szy) sx 2 ) sxy) sy 2 ) sx) sy) s1) sz) p 6 9
Least Squares Fitting of Data
Least Squares Fttng of Data Davd Eberly Geoetrc Tools, LLC http://www.geoetrctools.co/ Copyrght c 1998-2014. All Rghts Reserved. Created: July 15, 1999 Last Modfed: February 9, 2008 Contents 1 Lnear Fttng
More informationLeast Squares Fitting of Data
Least Squares Fitting of Data David Eberly, Geoetric Tools, Redond WA 98052 https://www.geoetrictools.co/ This work is licensed under the Creative Coons Attribution 4.0 International License. To view a
More informationApplied Mathematics Letters
Appled Matheatcs Letters 2 (2) 46 5 Contents lsts avalable at ScenceDrect Appled Matheatcs Letters journal hoepage: wwwelseverco/locate/al Calculaton of coeffcents of a cardnal B-splne Gradr V Mlovanovć
More informationXII.3 The EM (Expectation-Maximization) Algorithm
XII.3 The EM (Expectaton-Maxzaton) Algorth Toshnor Munaata 3/7/06 The EM algorth s a technque to deal wth varous types of ncoplete data or hdden varables. It can be appled to a wde range of learnng probles
More informationLECTURE :FACTOR ANALYSIS
LCUR :FACOR ANALYSIS Rta Osadchy Based on Lecture Notes by A. Ng Motvaton Dstrbuton coes fro MoG Have suffcent aount of data: >>n denson Use M to ft Mture of Gaussans nu. of tranng ponts If
More informationDenote the function derivatives f(x) in given points. x a b. Using relationships (1.2), polynomials (1.1) are written in the form
SET OF METHODS FO SOUTION THE AUHY POBEM FO STIFF SYSTEMS OF ODINAY DIFFEENTIA EUATIONS AF atypov and YuV Nulchev Insttute of Theoretcal and Appled Mechancs SB AS 639 Novosbrs ussa Introducton A constructon
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:
More informationExcess Error, Approximation Error, and Estimation Error
E0 370 Statstcal Learnng Theory Lecture 10 Sep 15, 011 Excess Error, Approxaton Error, and Estaton Error Lecturer: Shvan Agarwal Scrbe: Shvan Agarwal 1 Introducton So far, we have consdered the fnte saple
More informationITERATIVE ESTIMATION PROCEDURE FOR GEOSTATISTICAL REGRESSION AND GEOSTATISTICAL KRIGING
ESE 5 ITERATIVE ESTIMATION PROCEDURE FOR GEOSTATISTICAL REGRESSION AND GEOSTATISTICAL KRIGING Gven a geostatstcal regresson odel: k Y () s x () s () s x () s () s, s R wth () unknown () E[ ( s)], s R ()
More information= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the right-hand side of the system.
Chapter Matlab Exercses Chapter Matlab Exercses. Consder the lnear system of Example n Secton.. x x x y z y y z (a) Use the MATLAB command rref to solve the system. (b) Let A be the coeffcent matrx and
More informationLecture 12: Discrete Laplacian
Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly
More informationBAYESIAN CURVE FITTING USING PIECEWISE POLYNOMIALS. Dariusz Biskup
BAYESIAN CURVE FITTING USING PIECEWISE POLYNOMIALS Darusz Bskup 1. Introducton The paper presents a nonparaetrc procedure for estaton of an unknown functon f n the regresson odel y = f x + ε = N. (1) (
More informationy new = M x old Feature Selection: Linear Transformations Constraint Optimization (insertion)
Feature Selecton: Lnear ransforatons new = M x old Constrant Optzaton (nserton) 3 Proble: Gven an objectve functon f(x) to be optzed and let constrants be gven b h k (x)=c k, ovng constants to the left,
More informationXiangwen Li. March 8th and March 13th, 2001
CS49I Approxaton Algorths The Vertex-Cover Proble Lecture Notes Xangwen L March 8th and March 3th, 00 Absolute Approxaton Gven an optzaton proble P, an algorth A s an approxaton algorth for P f, for an
More informationAn Optimal Bound for Sum of Square Roots of Special Type of Integers
The Sxth Internatonal Syposu on Operatons Research and Its Applcatons ISORA 06 Xnang, Chna, August 8 12, 2006 Copyrght 2006 ORSC & APORC pp. 206 211 An Optal Bound for Su of Square Roots of Specal Type
More informationWhat is LP? LP is an optimization technique that allocates limited resources among competing activities in the best possible manner.
(C) 998 Gerald B Sheblé, all rghts reserved Lnear Prograng Introducton Contents I. What s LP? II. LP Theor III. The Splex Method IV. Refneents to the Splex Method What s LP? LP s an optzaton technque that
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationSalmon: Lectures on partial differential equations. Consider the general linear, second-order PDE in the form. ,x 2
Salmon: Lectures on partal dfferental equatons 5. Classfcaton of second-order equatons There are general methods for classfyng hgher-order partal dfferental equatons. One s very general (applyng even to
More informationPreference and Demand Examples
Dvson of the Huantes and Socal Scences Preference and Deand Exaples KC Border October, 2002 Revsed Noveber 206 These notes show how to use the Lagrange Karush Kuhn Tucker ultpler theores to solve the proble
More informationMinimization of l 2 -Norm of the KSOR Operator
ournal of Matheatcs and Statstcs 8 (): 6-70, 0 ISSN 59-36 0 Scence Publcatons do:0.38/jssp.0.6.70 Publshed Onlne 8 () 0 (http://www.thescpub.co/jss.toc) Mnzaton of l -Nor of the KSOR Operator Youssef,
More information1.3 Hence, calculate a formula for the force required to break the bond (i.e. the maximum value of F)
EN40: Dynacs and Vbratons Hoework 4: Work, Energy and Lnear Moentu Due Frday March 6 th School of Engneerng Brown Unversty 1. The Rydberg potental s a sple odel of atoc nteractons. It specfes the potental
More informationSlobodan Lakić. Communicated by R. Van Keer
Serdca Math. J. 21 (1995), 335-344 AN ITERATIVE METHOD FOR THE MATRIX PRINCIPAL n-th ROOT Slobodan Lakć Councated by R. Van Keer In ths paper we gve an teratve ethod to copute the prncpal n-th root and
More information,..., k N. , k 2. ,..., k i. The derivative with respect to temperature T is calculated by using the chain rule: & ( (5) dj j dt = "J j. k i.
Suppleentary Materal Dervaton of Eq. 1a. Assue j s a functon of the rate constants for the N coponent reactons: j j (k 1,,..., k,..., k N ( The dervatve wth respect to teperature T s calculated by usng
More informationPolynomial Regression Models
LINEAR REGRESSION ANALYSIS MODULE XII Lecture - 6 Polynomal Regresson Models Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur Test of sgnfcance To test the sgnfcance
More informationSolutions to exam in SF1811 Optimization, Jan 14, 2015
Solutons to exam n SF8 Optmzaton, Jan 4, 25 3 3 O------O -4 \ / \ / The network: \/ where all lnks go from left to rght. /\ / \ / \ 6 O------O -5 2 4.(a) Let x = ( x 3, x 4, x 23, x 24 ) T, where the varable
More informationSolutions for Homework #9
Solutons for Hoewor #9 PROBEM. (P. 3 on page 379 n the note) Consder a sprng ounted rgd bar of total ass and length, to whch an addtonal ass s luped at the rghtost end. he syste has no dapng. Fnd the natural
More informationGeometric Camera Calibration
Geoetrc Caera Calbraton EECS 598-8 Fall 24! Foundatons of Coputer Vson!! Instructor: Jason Corso (jjcorso)! web.eecs.uch.edu/~jjcorso/t/598f4!! Readngs: F.; SZ 6. (FL 4.6; extra notes)! Date: 9/7/4!! Materals
More informationAN ANALYSIS OF A FRACTAL KINETICS CURVE OF SAVAGEAU
AN ANALYI OF A FRACTAL KINETIC CURE OF AAGEAU by John Maloney and Jack Hedel Departent of Matheatcs Unversty of Nebraska at Oaha Oaha, Nebraska 688 Eal addresses: aloney@unoaha.edu, jhedel@unoaha.edu Runnng
More informationChapter 11: Simple Linear Regression and Correlation
Chapter 11: Smple Lnear Regresson and Correlaton 11-1 Emprcal Models 11-2 Smple Lnear Regresson 11-3 Propertes of the Least Squares Estmators 11-4 Hypothess Test n Smple Lnear Regresson 11-4.1 Use of t-tests
More informationOn the number of regions in an m-dimensional space cut by n hyperplanes
6 On the nuber of regons n an -densonal space cut by n hyperplanes Chungwu Ho and Seth Zeran Abstract In ths note we provde a unfor approach for the nuber of bounded regons cut by n hyperplanes n general
More informationBy M. O'Neill,* I. G. Sinclairf and Francis J. Smith
52 Polynoal curve fttng when abscssas and ordnates are both subject to error By M. O'Nell,* I. G. Snclarf and Francs J. Sth Departents of Coputer Scence and Appled Matheatcs, School of Physcs and Appled
More informationC/CS/Phy191 Problem Set 3 Solutions Out: Oct 1, 2008., where ( 00. ), so the overall state of the system is ) ( ( ( ( 00 ± 11 ), Φ ± = 1
C/CS/Phy9 Problem Set 3 Solutons Out: Oct, 8 Suppose you have two qubts n some arbtrary entangled state ψ You apply the teleportaton protocol to each of the qubts separately What s the resultng state obtaned
More informationReport on Image warping
Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.
More informationCHAPTER 10 ROTATIONAL MOTION
CHAPTER 0 ROTATONAL MOTON 0. ANGULAR VELOCTY Consder argd body rotates about a fxed axs through pont O n x-y plane as shown. Any partcle at pont P n ths rgd body rotates n a crcle of radus r about O. The
More informationOn the Calderón-Zygmund lemma for Sobolev functions
arxv:0810.5029v1 [ath.ca] 28 Oct 2008 On the Calderón-Zygund lea for Sobolev functons Pascal Auscher october 16, 2008 Abstract We correct an naccuracy n the proof of a result n [Aus1]. 2000 MSC: 42B20,
More informationOn Pfaff s solution of the Pfaff problem
Zur Pfaff scen Lösung des Pfaff scen Probles Mat. Ann. 7 (880) 53-530. On Pfaff s soluton of te Pfaff proble By A. MAYER n Lepzg Translated by D. H. Delpenc Te way tat Pfaff adopted for te ntegraton of
More informationGradient Descent Learning and Backpropagation
Artfcal Neural Networks (art 2) Chrstan Jacob Gradent Descent Learnng and Backpropagaton CSC 533 Wnter 200 Learnng by Gradent Descent Defnton of the Learnng roble Let us start wth the sple case of lnear
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationThe Geometry of Logit and Probit
The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.
More informationtotal If no external forces act, the total linear momentum of the system is conserved. This occurs in collisions and explosions.
Lesson 0: Collsons, Rotatonal netc Energy, Torque, Center o Graty (Sectons 7.8 Last te we used ewton s second law to deelop the pulse-oentu theore. In words, the theore states that the change n lnear oentu
More informationSolution of Linear System of Equations and Matrix Inversion Gauss Seidel Iteration Method
Soluton of Lnear System of Equatons and Matr Inverson Gauss Sedel Iteraton Method It s another well-known teratve method for solvng a system of lnear equatons of the form a + a22 + + ann = b a2 + a222
More informationChapter 9: Statistical Inference and the Relationship between Two Variables
Chapter 9: Statstcal Inference and the Relatonshp between Two Varables Key Words The Regresson Model The Sample Regresson Equaton The Pearson Correlaton Coeffcent Learnng Outcomes After studyng ths chapter,
More informationPerron Vectors of an Irreducible Nonnegative Interval Matrix
Perron Vectors of an Irreducble Nonnegatve Interval Matrx Jr Rohn August 4 2005 Abstract As s well known an rreducble nonnegatve matrx possesses a unquely determned Perron vector. As the man result of
More informationComputational and Statistical Learning theory Assignment 4
Coputatonal and Statstcal Learnng theory Assgnent 4 Due: March 2nd Eal solutons to : karthk at ttc dot edu Notatons/Defntons Recall the defnton of saple based Radeacher coplexty : [ ] R S F) := E ɛ {±}
More informationOur focus will be on linear systems. A system is linear if it obeys the principle of superposition and homogenity, i.e.
SSTEM MODELLIN In order to solve a control syste proble, the descrptons of the syste and ts coponents ust be put nto a for sutable for analyss and evaluaton. The followng ethods can be used to odel physcal
More informationLinear Feature Engineering 11
Lnear Feature Engneerng 11 2 Least-Squares 2.1 Smple least-squares Consder the followng dataset. We have a bunch of nputs x and correspondng outputs y. The partcular values n ths dataset are x y 0.23 0.19
More informationThe Impact of the Earth s Movement through the Space on Measuring the Velocity of Light
Journal of Appled Matheatcs and Physcs, 6, 4, 68-78 Publshed Onlne June 6 n ScRes http://wwwscrporg/journal/jap http://dxdoorg/436/jap646 The Ipact of the Earth s Moeent through the Space on Measurng the
More informationModified parallel multisplitting iterative methods for non-hermitian positive definite systems
Adv Coput ath DOI 0.007/s0444-0-9262-8 odfed parallel ultsplttng teratve ethods for non-hertan postve defnte systes Chuan-Long Wang Guo-Yan eng Xue-Rong Yong Receved: Septeber 20 / Accepted: 4 Noveber
More informationOn the Eigenspectrum of the Gram Matrix and the Generalisation Error of Kernel PCA (Shawe-Taylor, et al. 2005) Ameet Talwalkar 02/13/07
On the Egenspectru of the Gra Matr and the Generalsaton Error of Kernel PCA Shawe-aylor, et al. 005 Aeet alwalar 0/3/07 Outlne Bacground Motvaton PCA, MDS Isoap Kernel PCA Generalsaton Error of Kernel
More informationGravitational Acceleration: A case of constant acceleration (approx. 2 hr.) (6/7/11)
Gravtatonal Acceleraton: A case of constant acceleraton (approx. hr.) (6/7/11) Introducton The gravtatonal force s one of the fundamental forces of nature. Under the nfluence of ths force all objects havng
More informationAPPENDIX 2 FITTING A STRAIGHT LINE TO OBSERVATIONS
Unversty of Oulu Student Laboratory n Physcs Laboratory Exercses n Physcs 1 1 APPEDIX FITTIG A STRAIGHT LIE TO OBSERVATIOS In the physcal measurements we often make a seres of measurements of the dependent
More informationLeast squares cubic splines without B-splines S.K. Lucas
Least squares cubc splnes wthout B-splnes S.K. Lucas School of Mathematcs and Statstcs, Unversty of South Australa, Mawson Lakes SA 595 e-mal: stephen.lucas@unsa.edu.au Submtted to the Gazette of the Australan
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More information1. Statement of the problem
Volue 14, 010 15 ON THE ITERATIVE SOUTION OF A SYSTEM OF DISCRETE TIMOSHENKO EQUATIONS Peradze J. and Tsklaur Z. I. Javakhshvl Tbls State Uversty,, Uversty St., Tbls 0186, Georga Georgan Techcal Uversty,
More informationKernel Methods and SVMs Extension
Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general
More informationMATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS
MATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS These are nformal notes whch cover some of the materal whch s not n the course book. The man purpose s to gve a number of nontrval examples
More informationRELIABILITY ASSESSMENT
CHAPTER Rsk Analyss n Engneerng and Economcs RELIABILITY ASSESSMENT A. J. Clark School of Engneerng Department of Cvl and Envronmental Engneerng 4a CHAPMAN HALL/CRC Rsk Analyss for Engneerng Department
More informationSystem in Weibull Distribution
Internatonal Matheatcal Foru 4 9 no. 9 94-95 Relablty Equvalence Factors of a Seres-Parallel Syste n Webull Dstrbuton M. A. El-Dacese Matheatcs Departent Faculty of Scence Tanta Unversty Tanta Egypt eldacese@yahoo.co
More informationSpectral Graph Theory and its Applications September 16, Lecture 5
Spectral Graph Theory and ts Applcatons September 16, 2004 Lecturer: Danel A. Spelman Lecture 5 5.1 Introducton In ths lecture, we wll prove the followng theorem: Theorem 5.1.1. Let G be a planar graph
More informationC4B Machine Learning Answers II. = σ(z) (1 σ(z)) 1 1 e z. e z = σ(1 σ) (1 + e z )
C4B Machne Learnng Answers II.(a) Show that for the logstc sgmod functon dσ(z) dz = σ(z) ( σ(z)) A. Zsserman, Hlary Term 20 Start from the defnton of σ(z) Note that Then σ(z) = σ = dσ(z) dz = + e z e z
More informationMTH 263 Practice Test #1 Spring 1999
Pat Ross MTH 6 Practce Test # Sprng 999 Name. Fnd the area of the regon bounded by the graph r =acos (θ). Observe: Ths s a crcle of radus a, for r =acos (θ) r =a ³ x r r =ax x + y =ax x ax + y =0 x ax
More informationFinite Vector Space Representations Ross Bannister Data Assimilation Research Centre, Reading, UK Last updated: 2nd August 2003
Fnte Vector Space epresentatons oss Bannster Data Asslaton esearch Centre, eadng, UK ast updated: 2nd August 2003 Contents What s a lnear vector space?......... 1 About ths docuent............ 2 1. Orthogonal
More informationIntroduction. - The Second Lyapunov Method. - The First Lyapunov Method
Stablty Analyss A. Khak Sedgh Control Systems Group Faculty of Electrcal and Computer Engneerng K. N. Toos Unversty of Technology February 2009 1 Introducton Stablty s the most promnent characterstc of
More informationFixed-Point Iterations, Krylov Spaces, and Krylov Methods
Fxed-Pont Iteratons, Krylov Spaces, and Krylov Methods Fxed-Pont Iteratons Solve nonsngular lnear syste: Ax = b (soluton ˆx = A b) Solve an approxate, but spler syste: Mx = b x = M b Iprove the soluton
More information1 Review From Last Time
COS 5: Foundatons of Machne Learnng Rob Schapre Lecture #8 Scrbe: Monrul I Sharf Aprl 0, 2003 Revew Fro Last Te Last te, we were talkng about how to odel dstrbutons, and we had ths setup: Gven - exaples
More informationError Bars in both X and Y
Error Bars n both X and Y Wrong ways to ft a lne : 1. y(x) a x +b (σ x 0). x(y) c y + d (σ y 0) 3. splt dfference between 1 and. Example: Prmordal He abundance: Extrapolate ft lne to [ O / H ] 0. [ He
More informationElastic Collisions. Definition: two point masses on which no external forces act collide without losing any energy.
Elastc Collsons Defnton: to pont asses on hch no external forces act collde thout losng any energy v Prerequstes: θ θ collsons n one denson conservaton of oentu and energy occurs frequently n everyday
More informationCOMPLEX NUMBERS AND QUADRATIC EQUATIONS
COMPLEX NUMBERS AND QUADRATIC EQUATIONS INTRODUCTION We know that x 0 for all x R e the square of a real number (whether postve, negatve or ero) s non-negatve Hence the equatons x, x, x + 7 0 etc are not
More informationSection 8.3 Polar Form of Complex Numbers
80 Chapter 8 Secton 8 Polar Form of Complex Numbers From prevous classes, you may have encountered magnary numbers the square roots of negatve numbers and, more generally, complex numbers whch are the
More informationFormulas for the Determinant
page 224 224 CHAPTER 3 Determnants e t te t e 2t 38 A = e t 2te t e 2t e t te t 2e 2t 39 If 123 A = 345, 456 compute the matrx product A adj(a) What can you conclude about det(a)? For Problems 40 43, use
More informationADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING
1 ADVANCED ACHINE LEARNING ADVANCED ACHINE LEARNING Non-lnear regresson technques 2 ADVANCED ACHINE LEARNING Regresson: Prncple N ap N-dm. nput x to a contnuous output y. Learn a functon of the type: N
More informationWeek3, Chapter 4. Position and Displacement. Motion in Two Dimensions. Instantaneous Velocity. Average Velocity
Week3, Chapter 4 Moton n Two Dmensons Lecture Quz A partcle confned to moton along the x axs moves wth constant acceleraton from x =.0 m to x = 8.0 m durng a 1-s tme nterval. The velocty of the partcle
More informationMMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More informationIntegral Transforms and Dual Integral Equations to Solve Heat Equation with Mixed Conditions
Int J Open Probles Copt Math, Vol 7, No 4, Deceber 214 ISSN 1998-6262; Copyrght ICSS Publcaton, 214 www-csrsorg Integral Transfors and Dual Integral Equatons to Solve Heat Equaton wth Mxed Condtons Naser
More informationSTAT 3340 Assignment 1 solutions. 1. Find the equation of the line which passes through the points (1,1) and (4,5).
(out of 15 ponts) STAT 3340 Assgnment 1 solutons (10) (10) 1. Fnd the equaton of the lne whch passes through the ponts (1,1) and (4,5). β 1 = (5 1)/(4 1) = 4/3 equaton for the lne s y y 0 = β 1 (x x 0
More information8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 493 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces you have studed thus far n the text are real vector spaces because the scalars
More informationNUMERICAL DIFFERENTIATION
NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the
More informationSome Comments on Accelerating Convergence of Iterative Sequences Using Direct Inversion of the Iterative Subspace (DIIS)
Some Comments on Acceleratng Convergence of Iteratve Sequences Usng Drect Inverson of the Iteratve Subspace (DIIS) C. Davd Sherrll School of Chemstry and Bochemstry Georga Insttute of Technology May 1998
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More informationLecture 13 APPROXIMATION OF SECOMD ORDER DERIVATIVES
COMPUTATIONAL FLUID DYNAMICS: FDM: Appromaton of Second Order Dervatves Lecture APPROXIMATION OF SECOMD ORDER DERIVATIVES. APPROXIMATION OF SECOND ORDER DERIVATIVES Second order dervatves appear n dffusve
More informationρ some λ THE INVERSE POWER METHOD (or INVERSE ITERATION) , for , or (more usually) to
THE INVERSE POWER METHOD (or INVERSE ITERATION) -- applcaton of the Power method to A some fxed constant ρ (whch s called a shft), x λ ρ If the egenpars of A are { ( λ, x ) } ( ), or (more usually) to,
More informationCSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography
CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve
More informationTopic 5: Non-Linear Regression
Topc 5: Non-Lnear Regresson The models we ve worked wth so far have been lnear n the parameters. They ve been of the form: y = Xβ + ε Many models based on economc theory are actually non-lnear n the parameters.
More informationUniversity of Washington Department of Chemistry Chemistry 452/456 Summer Quarter 2013
Lecture 8/8/3 Unversty o Washngton Departent o Chestry Chestry 45/456 Suer Quarter 3 A. The Gbbs-Duhe Equaton Fro Lecture 7 and ro the dscusson n sectons A and B o ths lecture, t s clear that the actvty
More information1 Definition of Rademacher Complexity
COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture #9 Scrbe: Josh Chen March 5, 2013 We ve spent the past few classes provng bounds on the generalzaton error of PAClearnng algorths for the
More information4 Column generation (CG) 4.1 Basics of column generation. 4.2 Applying CG to the Cutting-Stock Problem. Basic Idea of column generation
4 Colun generaton (CG) here are a lot of probles n nteger prograng where even the proble defnton cannot be effcently bounded Specfcally, the nuber of coluns becoes very large herefore, these probles are
More informationDynamic Systems on Graphs
Prepared by F.L. Lews Updated: Saturday, February 06, 200 Dynamc Systems on Graphs Control Graphs and Consensus A network s a set of nodes that collaborates to acheve what each cannot acheve alone. A network,
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More information1 Derivation of Point-to-Plane Minimization
1 Dervaton of Pont-to-Plane Mnmzaton Consder the Chen-Medon (pont-to-plane) framework for ICP. Assume we have a collecton of ponts (p, q ) wth normals n. We want to determne the optmal rotaton and translaton
More informationFor all questions, answer choice E) NOTA" means none of the above answers is correct.
0 MA Natonal Conventon For all questons, answer choce " means none of the above answers s correct.. In calculus, one learns of functon representatons that are nfnte seres called power 3 4 5 seres. For
More informationIV. Performance Optimization
IV. Performance Optmzaton A. Steepest descent algorthm defnton how to set up bounds on learnng rate mnmzaton n a lne (varyng learnng rate) momentum learnng examples B. Newton s method defnton Gauss-Newton
More informationThis model contains two bonds per unit cell (one along the x-direction and the other along y). So we can rewrite the Hamiltonian as:
1 Problem set #1 1.1. A one-band model on a square lattce Fg. 1 Consder a square lattce wth only nearest-neghbor hoppngs (as shown n the fgure above): H t, j a a j (1.1) where,j stands for nearest neghbors
More informationLECTURE 9 CANONICAL CORRELATION ANALYSIS
LECURE 9 CANONICAL CORRELAION ANALYSIS Introducton he concept of canoncal correlaton arses when we want to quantfy the assocatons between two sets of varables. For example, suppose that the frst set of
More informationCIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M
CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute
More informationHomework Notes Week 7
Homework Notes Week 7 Math 4 Sprng 4 #4 (a Complete the proof n example 5 that s an nner product (the Frobenus nner product on M n n (F In the example propertes (a and (d have already been verfed so we
More informationDifference Equations
Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1
More informationOn a direct solver for linear least squares problems
ISSN 2066-6594 Ann. Acad. Rom. Sc. Ser. Math. Appl. Vol. 8, No. 2/2016 On a drect solver for lnear least squares problems Constantn Popa Abstract The Null Space (NS) algorthm s a drect solver for lnear
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationConvexity preserving interpolation by splines of arbitrary degree
Computer Scence Journal of Moldova, vol.18, no.1(52), 2010 Convexty preservng nterpolaton by splnes of arbtrary degree Igor Verlan Abstract In the present paper an algorthm of C 2 nterpolaton of dscrete
More informationSpectral Recomposition in Stratigraphic Interpretation*
Spectral Recoposton n Stratgraphc Interpretaton* Yhua Ca 1, Sergey Foel 2, and Honglu Zeng 2 Search and Dscovery Artcle #41376 (2014)** Posted June 30, 2014 *Adapted fro oral presentaton gven at Geoscence
More information