Data Fitting and Uncertainty

Transcription

1 TiloStrutz Data Fitting and Uncertainty A practical introduction to weighted least squares and beyond With 124 figures, 23 tables and 71 test questions and examples VIEWEG+ TEUBNER

2 IX Contents I Framework of Least-Squares Method 1 1 Introduction to Data-Fitting Problems What is data fitting? Notation Linear vs. nonlinear problems Example applications of linear data fitting Estimation of a constant value Parameter estimation for a straight line (linear regression) Polynomial function Multiple linear regression Selected nonlinear data-fitting problems Exponential functions Composite Gaussian bell functions Circle function Neural networks Test questions 24 2 Estimation of Model Parameters by the Method of Least Squares What are "Least Squares"? A general algorithm for minimisation problems Pitfalls Simplifications for linear model functions Curve approximation in case of unknown model function Example computations Constant value Straight line Polynomial approximation Plane approximation Linear prediction Cosine function Rotation and translation of coordinates 38

3 X Exponential model function Composite Gaussian bell functions Circle function Neural networks Test questions 46 3 Weights and Outliers What are the weights good for? Outliers Estimation of weights Estimation by binning Weights estimation using deviates Approaches towards outlier detection Standardised residuals Cluster criterion Application of weighted data fitting and outlier detection Constant value Straight line Plane approximation Coordinates transformation Linear prediction Cosine function Exponential model function Composite Gaussian bell functions Circle Comparison of binning and deviate-based weights estimation Conclusions Evaluation of weighting Comparison of outlier detectors Usefulness of weights Test questions Uncertainty of Results Goodness-of-fit, precision and accuracy Consistence of statistical model and data Sample variance 106

4 XI 4.2 Uncertainty of estimated parameters Uncertainty of data approximation Inspection of plots Example computations Constant value Straight line Cosine function Model mismatch Test questions 123 II Mathematics, Optimisation Methods, and Add ons Matrix Algebra Basics Determinants Numerical solutions for matrix inversion Cofactor-matrix method Inversion via Gauss-Jordan elimination Inversion via LU decomposition Inversion by singular value decomposition (SVD) Test questions The Idea behind Least Squares Normal distribution Maximum likelihood principle Fitting of linear model functions Standard approach Solution using singular value decomposition (SVD) Scaling of conditions Fitting of nonlinear model functions Error-surface approximation Gauss-Newton method Gradient-descent method Levenberg-Marquardt method Example of finding the minimum Test questions 166

5 XII 7 Supplemental Tools and Methods Alternative parameter estimation Recursive adaptation of parameters Iterative gradient descent Evolutionary approach Chauvenet's criterion for outlier detection Propagation of errors Manual calculation of linear least squares Combined treatment of different model functions Example 1: Coordinate transformation Example 2: Circular movement Total least squares Orthogonal fitting of a circle General approach Test questions 190 A Comparison of Approaches to Outlier Detection 191 A.I Normally distributed data 191 A.1.1 Data sets without outliers 191 A.1.2 Data sets containing outliers 193 A.2 Non-Gaussian distribution 196 A.2.1 Laplace distribution 196 A.2.2 Uniformly distributed data 196 A.3 Discussion 199 B Implementation 202 B.I Functionality 202 B.2 Manual 202 B.2.1 Input and output 204 B.2.2 Initialisation of model parameters 210 B.2.3 Processing control 213 B.2.4 Weights and outliers 213 B.3 General organisation of source code 219 B.4 Model functions 221 B.4.1 Numerical differentiation 221 B.4.2 Handling of multi-dimensional conditions 222

6 XIII B.4.3 Limitation of parameter space 223 B.4.4 Initialisation of parameters 223 B.5 Special algorithms 224 B.5.1 LU decomposition 224 B.5.2 Singular value decomposition 225 B.5.3 Sorting 225 B.6 Possible optimisations 225 B.7 Performance Test 226 B.7.1 Fitting linear systems 227 B.7.2 Fitting nonlinear systems 230 List of symbols 235 Bibliography 237 Index 241 S The Source Code (accessible via internet) Licence and Disclaimer Main functions Model functions Initialisation of nonlinear models Matrix processing Utils Allocation and matrix handling Command-line parsing Error Handling Other 49