Linear and Nonlinear Models

Transcription

1 Erik W. Grafarend Linear and Nonlinear Models Fixed Effects, Random Effects, and Mixed Models magic triangle 1 fixed effects 2 random effects 3 crror-in-variables model W DE G Walter de Gruyter Berlin New York

2 Contents The first problem of algebraic regression - consistent System of linear observational equations - underdetermined System of linear equations: {Ax = y AeK" xm,yeft(a)~rka = «,«= dimy} Introduction The front page example The front page example in matrix algebra Minimum norm Solution of the front page example by means of horizontal rank partitioning The ränge K(f) and the kernel Af(A) Interpretation of "MINOS" by three partitionings The minimum norm Solution: "MINOS" Adiscussionof the metric of the parameter space X Alternative choice of the metric of the parameter space X G x -MINOS and its generalized inverse Eigenvalue decomposition of G x -MINOS: canonical MINOS Case study: Orthogonal functions, Fourier series versus Fourier-Legendre series, circular harmonic versus spherical harmonic regression Fourier series Fourier-Legendre series Special nonlinear modeis Taylor polynomials, generalized Newton iteration Linearized modeis with darum defect Notes 82 The first problem of probabilistic regression - special Gauss- Markov model with datum defect - Setup of the linear uniformly minimum bias estimator of type LUMBE for fixed effects Setup of the linear uniformly minimum bias estimator of type LUMBE The Equivalence Theorem of G x -MINOS and S -LUMBE Examples 91 The second problem of algebraic regression - inconsistent system of linear observational equations - overdetermined System of linear equations: {Ax + i = y A e M" x ",y 7^(A) - rka = m,m = dimx} Introduction The front page example 97

3 XIV Contents 3-12 The front page example in matrix algebra Least Squares Solution of the front page example by means of vertical rank partitioning The ränge Tl(f) and the kernel Af(f), Interpretation of the least Squares Solution by three partitionings The least Squares Solution: "LESS" A discussionof the metric of the parameter space X Alternative choices of the metric of the Observation space Y Optimal choice of weight matrix: SOD The Taylor Karman criterion matrix Optimal choice of the weight matrix: 125 The space U(A) and TZ(Af Fuzzysets G x -LESS and its generalized inverse Eigenvalue decomposition of G y -LESS: canonical LESS Case study Partial redundancies, latent conditions, high leverage points versus break points, direct and inverse Grassmann coordinates, Plücker coordinates Canonical analysis of the hat matrix, partial redundancies, high leverage points Multilinear algebra, "join" and "meet", the Hodge star Operator From A to B: latent restrictions, Grassmann coordinates, Plücker coordinates From B to A: latent parametric equations, dual Grassmann coordinates, dual Plücker coordinates Break points Special linear and nonlinear modeis A family of means for direct observations A historical note on C. F. Gauss, A.-M. Legendre and the invention of Least Squares and its generalization 185 The second problem of probabilistic regression - special Gauss-Markov model without datum defect - Setup of BLUUE for the moments of first order and of BIQUUE for the central moment of second order Introduction The front page example Estimators of type BLUUE and BIQUUE of the front page example BLUUE and BIQUUE of the front page example, sample median, median absolute deviation 201

4 4-14 Alternative estimation Maximum Likelihood (MALE) Setup of the best linear uniformly unbiased estimators of type BLUUE for the moments of first order The best linear uniformly unbiased estimation % of \: y -BLUUE The Equivalence Theorem of G y -LESS and y -BLUUE Setup of the best invariant quadratic uniform by unbiased estimator of type BIQUUE for the central moments of second order Block partitioning of the dispersion matrix and linear space generated by variance-covariance components Invariant quadratic estimation of variance-covariance components of type IQE Invariant quadratic uniformly unbiased estimations of variance-covariance components of type IQUUE Invariant quadratic uniformly unbiased estimations of one variance component (IQUUE) from y -BLUUE: HIQUUE Invariant quadratic uniformly unbiased estimators of variance covariance components ofhelmert type: HIQUUE versus HIQE Best quadratic uniformly unbiased estimations of one variance component: BIQUUE 236 The third problem of algebraic regression - inconsistent System of linear observational equations with datum defect overdetermined- underdermined System of linear equations: {Ax + i = y A e W xm, y <$. U(A) ~ rk A < mm{m,n}} Introduction The front page example The front page example in matrix algebra Minimum norm - least Squares Solution of the front page example by means of additive rank partitioning Minimum norm - least Squares Solution of the front page example by means of multiplicative rank partitioning: The ränge 1Z(f) and the kernel N{f) Interpretation of "MINOLESS" by three partitionings MINOLESS and related Solutions like weighted minimum normweighted least Squares Solutions The minimum norm-least Squares Solution: "MINOLESS" (G x, G y ) -MINOS and its generalized inverse Eigenvalue decomposition of (G s, G y ) -MINOLESS Notes 282 XV

5 Contents 5-3 The hybrid approximation Solution: a-haps and Tykhonov- Phillips regularization 282 The third problem of probabilistic regression - special Gauss- Markov model with datum problem - Setup of BLUMBE and BLE for the moments of first order and of BIQUUE and BIQE for the central moment of second order Setup of the best linear minimum bias estimator of type BLUMBE Defmitions, lemmas and theorems The first example: BLUMBE versus BLE, BIQUUE versus BIQE, triangulär leveling network The first example: I 3, L-BLUMBE The first example: V, S-BLUMBE The first example: I 3,1 3 -BLE The first example: V, S-BLE Setup of the best linear estimators of type hom BLE, hom S-BLE and hom a-ble for fixed effects 312 A spherical problem of algebraic representation - Inconsistent System of directional observational equationsoverdetermined System of nonlinear equations on curved manifolds Introduction Minimal geodesic distance: MINGEODISC Special modeis: from the circular normal distribution to the oblique normal distribution A historical note of the von Mises distribution Oblique map projection A note on the angular metric Case study 341 The fourth problem of probabilistic regression - special Gauss-Markov model with random effects- Setup of BLIP and VIP for the moments of first order The random effect model Examples 362 The fifth problem of algebraic regression - the System of conditional equations: homogeneous and inhomogeneous equations - {By = Bi versus -c + By = Bi} G -LESS of System of inconsistent homogeneous conditional equations Solving a System of inconsistent inhomogeneous conditional equations 376

6 XVII 9-3 Examples 377 The fifth problem of probabilistic regression - general Gauss-Markov model with mixed effects- Setup of BLUUE for the moments of first order (Kolmogorov-Wiener prediction) Inhomogeneous general linear Gauss-Markov model (fixed effects and random effects) Explicit representations of errors in the general Gauss-Markov model with mixed effects An example for collocation Comments 397 The sixth problem of probabilistic regression - the random effect model - "errors-in-variables" Solving the nonlinear system of the model "errors-in-variables" Example: The straight line fit References 410 The sixth problem of generalized algebraic regression - the System of conditional equations with unknowns - (Gauss-Helmert model) Solving the system of homogeneous condition equations with unknowns W-LESS R,W-MINOLESS R,W-HAPS R, W-MINOLESS against R, W - HAPS Examples for the generalized algebraic regression problem: homogeneous conditional equations with unknowns The first case: I-LESS The second case: I, I-MINOLESS The third case: I, I-HAPS The fourth case: R, W-MINOLESS, Rpositive semidefinite, Wpositive semidefinite Solving the system of inhomogeneous condition equations with unknowns W-LESS R, W-MINOLESS RW-HAPS R, W-MINOLESS against R, W-HAPS Conditional equations with unknowns: from the algebraic approach to the stochastic one 429

7 xviii Contents Shift to the center The condition of unbiased estimators The first step: unbiased estimation of % and E{%) The second step: unbiased estimation K:, and K The nonlinear problem of the 3d datum transformation and the Procrustes Algorithm The 3d datum transformation and the Procrustes Algorithm The variance - covariance matrix of the error matrix E Case studies: The 3d datum transformation and the Procrustes Algorithm References The seventh problem of generalized algebraic regression revisited: The Grand Linear Model: The split level model of conditional equations with unknowns (general Gauss-Helmert model) Solutions of type W-LESS Solutions of type R, W-MINOLESS Solutions of type R, W-HAPS Review of the various modeis: the sixth problem Special problems of algebraic regression and stochastic estimation: multivariate Gauss-Markov model, the n-way Classification model, dynamical Systems The multivariate Gauss-Markov model - a special problem of probabilistic regression n-way Classification modeis A first example: 1-way Classification A second example: 2-way Classification without interaction A third example: 2-way Classification with interaction Higher classifications with interaction Dynamical Systems 476 Appendix A: Matrix Algebra 485 AI Matrix-Algebra 485 A2 Special Matrices 488 A3 Scalar Measures and Inverse Matrices 495 A4 Vectorvalued Matrix Forms 506 A5 Eigenvalues and Eigenvectors 509 A6 Generalized Inverses 513

8 Contents xix Appendix B: Matrix Analysis 522 Bl Derivations of Scalar-valued and Vector-valued Vector Functions 522 B2 Derivations of Trace Forms 523 B3 Derivations of Determinantal Forms 526 B4 Derivations of a Vector/Matrix Function of a Vector/Matrix 527 B5 Derivations of the Kronecker-Zehfuß product 528 B6 Matrix-valued Derivatives of Symmetrie or Antisymmetric Matrix Functions 528 B7 Higher order derivatives 530 Appendix C: Lagrange Multipliers 533 Cl A first way to solve the problem 533 Appendix D: Sampling distributions and their use: Confidence Intervals and Confidence Regions 543 Dl A first vehicle: Transformation of random variables 543 D2 A second vehicle: Transformation of random variables 547 D3 A first confidence interval of Gauss-Laplace normally distributed observations: ju,a 2 known, the Three Sigma Rule 553 D31 The forward computation of a first confidence interval of Gauss-Laplace normally distributed observations: /u,a 2 known 557 D32 The backward computation of a first confidence interval of Gauss-Laplace normally distributed observations: /u,<7 2 known 564 D4 Sampling from the Gauss-Laplace normal distribution: a second confidence interval for the mean, variance known 567 D41 Sampling distributions of the sample mean jj,, a 2 known, and of the sample variance d D42 The confidence interval for the sample mean, variance known 592 D5 Sampling from the Gauss-Laplace normal distribution: a third confidence interval for the mean, variance unknown 596 D51 Student's sampling distribution of the random variable {ß-^)l& 596 D52 The confidence interval for the sample mean, variance unknown 605 D53 The Uncertainty Principle 611 D6 Sampling from the Gauss-Laplace normal distribution: a fourth confidence interval for the variance 613 D61 The confidence interval for the variance D62 The Uncertainty Principle

9 XX Contents D7 Sampling from the multidimensional Gauss-Laplace normal distribution: the confidence region for the fixed Parameters in the linear Gauss-Markov model 621 Appendix E: Statistical Notions 163 El Moments of a probability distribution, the Gauss-Laplace normal distribution and the quasi-normal distribution 644 E2 Error propagation 648 E3 UsefuI identities 651 E4 The notions of identifiability and unbiasedness 652 Appendix F: Bibliographie Indexes 655 References 659 Index 745