# HENDERSON'S APPROACH TO VARIANCE COMPONENTS ESTIMATION FOR UNBALANCED DATA UDC Vera Djordjević, Vinko Lepojević

Save this PDF as:

Size: px
Start display at page:

Download "HENDERSON'S APPROACH TO VARIANCE COMPONENTS ESTIMATION FOR UNBALANCED DATA UDC Vera Djordjević, Vinko Lepojević"

## Transcription

1 FACTA UNIVERSITATIS Series: Economics and Organization Vol. 2, N o 1, 2003, pp HENDERSON'S APPROACH TO VARIANCE COMPONENTS ESTIMATION FOR UNBALANCED DATA UDC Vera Djordjević, Vinko Lepojević Faculty of Economics, University of Niš, Niš, Serbia and Montenegro Abstract. The analysis of variance is a frequently used technique of data analysing and probably the most used statistical method in 20th century. However, there are a lot of ambiguities in the interpretation of particular aspects. The aim of this text is an explanation of one of the fundamental parts of this method - the variance components estimation. Namely, by calculating variability in observations we can divide their variance into parts which correspond to the factors we examine, and that means that the subject of study are the variance components. This text is a presentation of Henderson's method for variance components estimation in models of analysis of variance. 1. INTRODUCTION Estimating variance components from unbalanced data is not as straightforward as that obtained from balanced data. This is so for two reasons. First, several methods of estimation are available (most of which are reduced to the analysis of variance method for balanced data), but not one of them has yet been clearly established as superior to the others. Second, all the methods involve relatively cumbersome algebra; discussion of unbalanced data can therefore easily deteriorate into a welter of symbols, a situation we do our best (perhaps not successfully) to minimize here. It is probably safe to describe the Henderson (1953) paper as the foundation paper dealing with variance component estimation from unbalanced data. The methods there described have, accordingly, often been referred to as Henderson's methods 1,2 and 3. As described in Searle (1968, 1971, 1992), Method 1 is simply an analogue of the analysis of variance method used with balanced data; Method 2 is designed to correct a deficiency of Method 1 that arises with mixed models; and Method 3 is based on the method of fitting constants so often used in fixed effects models. Prior to the publication of these methods Windsor and Clark (1940) had utilized the analysis of variance method in studying variation in the catch of plankton nets, Eisenhart (1947) had clearly specified distinctions between fixed, random, and mixed models, and Crump (1946, 1947, 1951) had established Received December 23, 2003

3 Henderson's Approach to Variance Components Estimation for Unbalanced Data 61 situations we assume that E(e) = 0, so that var (e) is " (ee') = σe 2 I. In addition, when β is a vector of fixed effects, E(βe') = βe(e') = 0; and when # includes elements that are random effects they are assumed to have zero mean and zero covariance with the elements in e; thus at all times we take E(βe') = E(eβ') = 0. In a fixed effects model β is a vector of fixed effects, E(y) = Xβ and V = var (y) = var (e) = σ e 2 IN, where are N observations, i.e., y is N X 1. Then (1) becomes: "(y'qy) = β'xqxβ + In a mixed model β' can be partitioned as 2 2σ etr (Q) ' ' ' ' β ' = ( β1β AβB... βk ), where β 1 contains all the fixed effects of the model (including the mean) and where the other β's each represent the set of random effects for the factors A, B, C,, K, these random effects having zero means and zero co variances with the effects of any other set. (Although, only single subscripts are used, interaction effects and/or nested-factor effects are not excluded by this notation. They are considered merely as factors, each identified by a single subscript rather than the letters of the appropriate main effects; for example, AB- interaction effects might be in the vector labeled # F ). 3. THE ANALYSIS OF VARIANCE METHOD (HENDERSON'S METHOD 1) The analysis of variance method of estimating variance components is essentially the only method in use for balanced data. It is also the most frequently used method with unbalanced data, although its application is not as straightforward and its deficiencies are more pronounced. Nevertheless, it is likely to continue as an oft-used method and so considerable attention is devoted to it here. With balanced data the method consists of equating mean squares to their expected values. Essentially the same procedure is used with unbalanced data, as is now shown in terms of an example, the two-way crossed classification with interaction. The model for a levels of an A-factor crossed with b levels of a B-factor is ijk = µ + αi + β j + γij eijk (2) y + where y ijk is the k-th observation (for k = 1,2,,n ij ) in the i-th level of the A-factor and the j-th level of the B-factor, where i = 1,2,., a and j = 1,2,.,b. Thus n ij is the number of observations in the (i,j) cell- the i-th level of A and the j-th level of B. Since not all of the cells may contain observations we let s represent the number that do; i.e., s is the number of n ij 's for which n ij > 0. Thus ab-s is the number of cells containing no data (n ij = 0). In (43), µ is a general mean, α i is the effect due to the i-th level of the A-factor, β j is the effect due to the j-th level of the B-factor, γ ij is the interaction effect and e ijk is the customary error term.

4 62 V. DJORDJEVIĆ, V. LEPOJEVIĆ 4. ADJUSTING FOR BIAS IN MIXED MODELS (HENDERSON METHOD 2) Mixed models involve dual estimation problems - estimating both fixed effects and variance components. For the moment attention is directed just to estimating the variance components. In some situations this is exactly what might be done in practice; with genetic data, for example, time or year effects might be considered fixed and of little interest compared to the genetic variance components. On the other hand, time trends may be of very real interest in some data, in which case their estimation together with that of the variance components would be considered simultaneously. This dual estimation problem is considered subsequently. The analysis of variance method for mixed model leads, with unbalanced data, to biased estimators of variance components. The method known as Method 2 in Henderson (1953) is designed to correct this deficiency. It uses the data first to estimate fixed effects of the model and then, using these estimators to adjust the data, variance components are estimated from the adjusted data, by the analysis of variance method. The whole procedure is designed so that the resulting variance component estimators are not biased by the presence of the fixed effects in the model, as they are with analysis of variance estimators derived from the basic data. So far as the criterion of unbiased ness is concerned, this is certainly achieved by Method 2. But the general method of analyzing data adjusted accordant to some estimator of the fixed effects is open to criticism on other grounds: it cannot be uniquely defined, and a simplified form of it, of which Henderson's Method 2 is a special case, cannot be used whenever the model includes interactions between the fixed effects and the random effects. The general approach of Method 2 can be considered in terms of the model: y = µ 1+ X f β f + X rβr + e where all fixed effects other than! are represented by # f, and all random effects by # r. As usual E(β r ) = 0 and so E(β r β' r ) = V(β r ), the variance-covariance matrix of the random effects. The general effect of correcting the data vector y accordant to an estimator of the fixed effects β f is to suppose that such an estimator is β ~ f = Ly for some matrix L so that ~ the vector of corrected data is z = yx f β f. It can then be shown (Searle 1992) that the model for z contains no terms in β f provided L is a generalized inverse of \$ f. Under this condition the analysis of variance method applied to y - X f β ~ f. will yield unbiased estimators of the variance components. However, the fact that L has only to be a generalized inverse of \$ f indicates the arbitrariness of the method. This lack of specificity means the method is not uniquely defined and hence is impractical. The model for z = y -X f β ~ f. just described contains no term in β f. An additional restriction would be for the model to have the same term \$ r β r as does the model for y, as wall as a mean term µ 1 1 where µ 1 is not necessarily equal to µ.

5 Henderson's Approach to Variance Components Estimation for Unbalanced Data THE FITTING CONSTANTS METHOD (HENDERSON'S METHOD 3) The third method described by Henderson is based on the method of fitting constants traditionally used in fixed effects models. It uses reductions in sums of squares due to fitting different subgroups of factors in the model, using them in exactly the same manner as the S's are used in the analysis of variance method, namely estimating the variance components by equating each computed reduction to its expected value. In an example of the two-way classification random model with interaction, the four quadratics equated to their expected values in the analysis of variance method are S A, S B, S AB and SSE shown as functions of T's in equations. The fitting constants method also uses four quadratics, derived from fitting the model: and three sub-models: y = µ + α + β + γ + e ijk i y ijk = µ + e ijk, y ijk = µ + αi + eijk, y ijk = µ + αi + β j + eijk In fitting these models the total sum of squares is T 0 = ΣΣΣ y 2 ijk. With each model there is a reduction in sum of squares, y'x(x'x) - X'y, due to fitting the model. These reductions can be donated by: R ( µ, A, B, AB), R (µ), R ( µ, A), R ( µ, A, B) respectively, where the letters in parentheses indicate the factors fitted in the respective models. (In this notation AB represents, as usual, A-by-B interaction). By way of example, the last of the above models fits µ, A-, and B- factors, and so the reduction is symbolized as R(µ,A,B). Writing the model as y = \$b + e, where b is the vector containing µ, the α's and β's we have R(µ,A,B) = y'x(x'x) - X'y for that \$. The fitting constants method of estimating variance components uses T 0 and these R( ) - reductions by equating certain differences among them to their expected values, so setting up equations in the σ 2 's whose solutions are the estimators. j ij ijk 6. CONCLUSION Henderson's approach to variance components estimation for unbalanced data was the foundation paper for other statisticians in attempt to solve this problem. The methods there described have, accordingly, often been referred to as Henderson's methods 1,2 and 3. Method 1 is simply an analogue of the analysis of variance method used with balanced data; Method 2 is designed to correct a deficiency of Method 1 that arises with mixed models; and Method 3 is based on the method of fitting constants so often used in fixed effects models.

6 64 V. DJORDJEVIĆ, V. LEPOJEVIĆ REFERENCES 1. Christensen, R Analysis of Variance, Design and Regression. London: Chapman&Hall. 2. Eisenhart, C "The assumptions underling the analysis of variance." Biometrics 3: Hartley, H.O "Expectations variances and covariance's of ANOVA mean squares by "synthesis." Biometrics 25: Hartley, H.O. and Rao, J.N.K "Maximum likelihood estimation for the mixed analysis of variance models." Biometrics 23: Hartley, H.O. and Searle, R.R "A discontinuity in mixed model analysis." Biometrics 25: Henderson, C.R "Estimation of variance and covariance components." Biometrics 9: Hoaglin, D.C., Mosteller, F., and Tukey, J.W Fundamentals of Exploratory Analysis of Variance. New York; Wiley. 8. Hockihg, R.R., Green, J.W. and Bremer, R.H "Variance component estimation with model-based diagnostics." Tehnomestrics 31: Hocking, R.R Methods and Applications of Linear Models: Regression and the Analysis of Variance. New York: Wiley. 10. Kempthorne, O "Fixed and mixed models in the analysis of variance." Biometrics 31: Khuri, A.I., and Sahai, H "Variance components analysis: Selective literature review." International Statistics 11: Miller, R.G Beyond ANOVA. Chapman&Hall 13. Sahai, H "A Bibliography on variance components." International Statistical Reviews 47: Sahai H., Ageel M., "The analysis of variance", Birkhauser, 15. Sheffe, H The analysis of Variance. New York: Wiley. 16. Searle, S.R "Another look at Hendersons methods of estimating variance components." Biometrics 24: Searle, S.R "Topics in variance component estimation." Biometrics 27: Searle, S.R., Casella, G., and McCulloch, C.E Variance Components. New York: Wiley. 19. Thompson, W.A.Jr., and Moore, J.R Non-negative estimates of variance components. Tehnometrics 5: HENDERSONOV PRISTUP OCENI VARIJANSNIH KOMPONENTI ZA NEURAVNOTEŽENE PODATKE Vera Djordjević, Vinko Lepojević Analiza varijanse je obilato korišćena tehnika analize podataka i verovatno najčešće korišćeni statistički metod u dvadesetom veku. No, i pored toga, još uvek postoje nejasnoće u tumačenju pojedinih njenih aspekata. Ovaj rad ima za cilj osvetljavanje jednog od fundamentalnih delova ovog metoda - ocenu varijansnih komponenti. Naime, u cilju izračunavanja varijabiliteta u posmatranjima možemo razložiti njihovu varijansu na delove koji odgovaraju faktorima koje proučavamo, a to u stvari znači da predmet proučavanja predstavljaju varijansne komponente. U radu je prezentovan kritički pristup Henderson-ov metode ocene varijansnih komponenti u modelima analize varijanse.

### Multiple Regression Analysis. Part III. Multiple Regression Analysis

Part III Multiple Regression Analysis As of Sep 26, 2017 1 Multiple Regression Analysis Estimation Matrix form Goodness-of-Fit R-square Adjusted R-square Expected values of the OLS estimators Irrelevant

### Inverse of a Square Matrix. For an N N square matrix A, the inverse of A, 1

Inverse of a Square Matrix For an N N square matrix A, the inverse of A, 1 A, exists if and only if A is of full rank, i.e., if and only if no column of A is a linear combination 1 of the others. A is

### Least Squares Estimation-Finite-Sample Properties

Least Squares Estimation-Finite-Sample Properties Ping Yu School of Economics and Finance The University of Hong Kong Ping Yu (HKU) Finite-Sample 1 / 29 Terminology and Assumptions 1 Terminology and Assumptions

### Linear regression with nested errors using probability-linked data

University of Wollongong Research Online Faculty of Engineering and Information Sciences - Papers: Part A Faculty of Engineering and Information Sciences 2014 Linear regression with nested errors using

### Practice Final Exam. December 14, 2009

Practice Final Exam December 14, 29 1 New Material 1.1 ANOVA 1. A purication process for a chemical involves passing it, in solution, through a resin on which impurities are adsorbed. A chemical engineer

### On Selecting Tests for Equality of Two Normal Mean Vectors

MULTIVARIATE BEHAVIORAL RESEARCH, 41(4), 533 548 Copyright 006, Lawrence Erlbaum Associates, Inc. On Selecting Tests for Equality of Two Normal Mean Vectors K. Krishnamoorthy and Yanping Xia Department

### MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7

MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7 1 Random Vectors Let a 0 and y be n 1 vectors, and let A be an n n matrix. Here, a 0 and A are non-random, whereas y is

### THE POWER OF THE MATRIX. Shayle R. Searle

THE POWER OF THE MATRIX BU-1563-M March 2001 Shayle R. Searle Keywords: matrix proofs, linear models. Abstract: The first linear models book by Graybill (1961) was one of the early major works to address

### Chapter 12 REML and ML Estimation

Chapter 12 REML and ML Estimation C. R. Henderson 1984 - Guelph 1 Iterative MIVQUE The restricted maximum likelihood estimator (REML) of Patterson and Thompson (1971) can be obtained by iterating on MIVQUE,

### Combining data from two independent surveys: model-assisted approach

Combining data from two independent surveys: model-assisted approach Jae Kwang Kim 1 Iowa State University January 20, 2012 1 Joint work with J.N.K. Rao, Carleton University Reference Kim, J.K. and Rao,

### STAT 705 Chapter 19: Two-way ANOVA

STAT 705 Chapter 19: Two-way ANOVA Timothy Hanson Department of Statistics, University of South Carolina Stat 705: Data Analysis II 1 / 38 Two-way ANOVA Material covered in Sections 19.2 19.4, but a bit

### Analysis of variance

Analysis of variance Andrew Gelman March 4, 2006 Abstract Analysis of variance (ANOVA) is a statistical procedure for summarizing a classical linear model a decomposition of sum of squares into a component

### Large Sample Properties of Estimators in the Classical Linear Regression Model

Large Sample Properties of Estimators in the Classical Linear Regression Model 7 October 004 A. Statement of the classical linear regression model The classical linear regression model can be written in

### [y i α βx i ] 2 (2) Q = i=1

Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation

### A note on the equality of the BLUPs for new observations under two linear models

ACTA ET COMMENTATIONES UNIVERSITATIS TARTUENSIS DE MATHEMATICA Volume 14, 2010 A note on the equality of the BLUPs for new observations under two linear models Stephen J Haslett and Simo Puntanen Abstract

### Using Estimating Equations for Spatially Correlated A

Using Estimating Equations for Spatially Correlated Areal Data December 8, 2009 Introduction GEEs Spatial Estimating Equations Implementation Simulation Conclusion Typical Problem Assess the relationship

### Statistics 203: Introduction to Regression and Analysis of Variance Penalized models

Statistics 203: Introduction to Regression and Analysis of Variance Penalized models Jonathan Taylor - p. 1/15 Today s class Bias-Variance tradeoff. Penalized regression. Cross-validation. - p. 2/15 Bias-variance

### Outline. Remedial Measures) Extra Sums of Squares Standardized Version of the Multiple Regression Model

Outline 1 Multiple Linear Regression (Estimation, Inference, Diagnostics and Remedial Measures) 2 Special Topics for Multiple Regression Extra Sums of Squares Standardized Version of the Multiple Regression

### WALD CONSISTENCY AND THE METHOD OF SIEVES IN REML ESTIMATION. By Jiming Jiang Case Western Reserve University

The Annals of Statistics 1997, Vol. 5, o. 4, 1781 1803 WALD COSISTECY AD THE METHOD OF SIEVES I REML ESTIMATIO By Jiming Jiang Case Western Reserve University We prove that for all unconfounded balanced

### Brief Suggested Solutions

DEPARTMENT OF ECONOMICS UNIVERSITY OF VICTORIA ECONOMICS 366: ECONOMETRICS II SPRING TERM 5: ASSIGNMENT TWO Brief Suggested Solutions Question One: Consider the classical T-observation, K-regressor linear

### Greene, Econometric Analysis (7th ed, 2012)

EC771: Econometrics, Spring 2012 Greene, Econometric Analysis (7th ed, 2012) Chapters 2 3: Classical Linear Regression The classical linear regression model is the single most useful tool in econometrics.

### The general linear regression with k explanatory variables is just an extension of the simple regression as follows

3. Multiple Regression Analysis The general linear regression with k explanatory variables is just an extension of the simple regression as follows (1) y i = β 0 + β 1 x i1 + + β k x ik + u i. Because

### Animal Models. Sheep are scanned at maturity by ultrasound(us) to determine the amount of fat surrounding the muscle. A model (equation) might be

Animal Models 1 Introduction An animal model is one in which there are one or more observations per animal, and all factors affecting those observations are described including an animal additive genetic

### All variances and covariances appearing in this formula are understood to be defined in the usual manner for finite populations; example

155 By: UBIASED COMPOET RATIO ESTIMATIO1 D. S. Robson and Chitra Vitbayasai, Cornell University ITRODUCTIO The precision of a ratio -type estimator such as can sometimes be substantially increased the

### Regression Estimation Least Squares and Maximum Likelihood

Regression Estimation Least Squares and Maximum Likelihood Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 3, Slide 1 Least Squares Max(min)imization Function to minimize

### STATISTICAL INFERENCE FOR SURVEY DATA ANALYSIS

STATISTICAL INFERENCE FOR SURVEY DATA ANALYSIS David A Binder and Georgia R Roberts Methodology Branch, Statistics Canada, Ottawa, ON, Canada K1A 0T6 KEY WORDS: Design-based properties, Informative sampling,

### Intraclass and Interclass Correlation Coefficients with Application

Developments in Statistics and Methodology A. Ferligoj and A. Kramberger (Editors) Metodološi zvezi, 9, Ljubljana : FDV 1993 Intraclass and Interclass Correlation Coefficients with Application Lada Smoljanović1

### Last updated: Oct 22, 2012 LINEAR CLASSIFIERS. J. Elder CSE 4404/5327 Introduction to Machine Learning and Pattern Recognition

Last updated: Oct 22, 2012 LINEAR CLASSIFIERS Problems 2 Please do Problem 8.3 in the textbook. We will discuss this in class. Classification: Problem Statement 3 In regression, we are modeling the relationship

### A process capability index for discrete processes

Journal of Statistical Computation and Simulation Vol. 75, No. 3, March 2005, 175 187 A process capability index for discrete processes MICHAEL PERAKIS and EVDOKIA XEKALAKI* Department of Statistics, Athens

### Linear Models 1. Isfahan University of Technology Fall Semester, 2014

Linear Models 1 Isfahan University of Technology Fall Semester, 2014 References: [1] G. A. F., Seber and A. J. Lee (2003). Linear Regression Analysis (2nd ed.). Hoboken, NJ: Wiley. [2] A. C. Rencher and

### Variable Selection and Model Building

LINEAR REGRESSION ANALYSIS MODULE XIII Lecture - 37 Variable Selection and Model Building Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur The complete regression

### THE ANOVA APPROACH TO THE ANALYSIS OF LINEAR MIXED EFFECTS MODELS

THE ANOVA APPROACH TO THE ANALYSIS OF LINEAR MIXED EFFECTS MODELS We begin with a relatively simple special case. Suppose y ijk = µ + τ i + u ij + e ijk, (i = 1,..., t; j = 1,..., n; k = 1,..., m) β =

### Estimating σ 2. We can do simple prediction of Y and estimation of the mean of Y at any value of X.

Estimating σ 2 We can do simple prediction of Y and estimation of the mean of Y at any value of X. To perform inferences about our regression line, we must estimate σ 2, the variance of the error term.

### Ma 3/103: Lecture 24 Linear Regression I: Estimation

Ma 3/103: Lecture 24 Linear Regression I: Estimation March 3, 2017 KC Border Linear Regression I March 3, 2017 1 / 32 Regression analysis Regression analysis Estimate and test E(Y X) = f (X). f is the

### X -1 -balance of some partially balanced experimental designs with particular emphasis on block and row-column designs

DOI:.55/bile-5- Biometrical Letters Vol. 5 (5), No., - X - -balance of some partially balanced experimental designs with particular emphasis on block and row-column designs Ryszard Walkowiak Department

### A Power Analysis of Variable Deletion Within the MEWMA Control Chart Statistic

A Power Analysis of Variable Deletion Within the MEWMA Control Chart Statistic Jay R. Schaffer & Shawn VandenHul University of Northern Colorado McKee Greeley, CO 869 jay.schaffer@unco.edu gathen9@hotmail.com

### Small Domain Estimation for a Brazilian Service Sector Survey

Proceedings 59th ISI World Statistics Congress, 5-30 August 013, Hong Kong (Session CPS003) p.334 Small Domain Estimation for a Brazilian Service Sector Survey André Neves 1, Denise Silva and Solange Correa

### A. Motivation To motivate the analysis of variance framework, we consider the following example.

9.07 ntroduction to Statistics for Brain and Cognitive Sciences Emery N. Brown Lecture 14: Analysis of Variance. Objectives Understand analysis of variance as a special case of the linear model. Understand

### Topic 17 - Single Factor Analysis of Variance. Outline. One-way ANOVA. The Data / Notation. One way ANOVA Cell means model Factor effects model

Topic 17 - Single Factor Analysis of Variance - Fall 2013 One way ANOVA Cell means model Factor effects model Outline Topic 17 2 One-way ANOVA Response variable Y is continuous Explanatory variable is

### On V-orthogonal projectors associated with a semi-norm

On V-orthogonal projectors associated with a semi-norm Short Title: V-orthogonal projectors Yongge Tian a, Yoshio Takane b a School of Economics, Shanghai University of Finance and Economics, Shanghai

### Economics 240A, Section 3: Short and Long Regression (Ch. 17) and the Multivariate Normal Distribution (Ch. 18)

Economics 240A, Section 3: Short and Long Regression (Ch. 17) and the Multivariate Normal Distribution (Ch. 18) MichaelR.Roberts Department of Economics and Department of Statistics University of California

### Heteroskedasticity ECONOMETRICS (ECON 360) BEN VAN KAMMEN, PHD

Heteroskedasticity ECONOMETRICS (ECON 360) BEN VAN KAMMEN, PHD Introduction For pedagogical reasons, OLS is presented initially under strong simplifying assumptions. One of these is homoskedastic errors,

### Empirical Market Microstructure Analysis (EMMA)

Empirical Market Microstructure Analysis (EMMA) Lecture 3: Statistical Building Blocks and Econometric Basics Prof. Dr. Michael Stein michael.stein@vwl.uni-freiburg.de Albert-Ludwigs-University of Freiburg

### Economic modelling and forecasting

Economic modelling and forecasting 2-6 February 2015 Bank of England he generalised method of moments Ole Rummel Adviser, CCBS at the Bank of England ole.rummel@bankofengland.co.uk Outline Classical estimation

### STATISTICS 174: APPLIED STATISTICS TAKE-HOME FINAL EXAM POSTED ON WEBPAGE: 6:00 pm, DECEMBER 6, 2004 HAND IN BY: 6:00 pm, DECEMBER 7, 2004 This is a

STATISTICS 174: APPLIED STATISTICS TAKE-HOME FINAL EXAM POSTED ON WEBPAGE: 6:00 pm, DECEMBER 6, 2004 HAND IN BY: 6:00 pm, DECEMBER 7, 2004 This is a take-home exam. You are expected to work on it by yourself

### Lecture 13: Simple Linear Regression in Matrix Format

See updates and corrections at http://www.stat.cmu.edu/~cshalizi/mreg/ Lecture 13: Simple Linear Regression in Matrix Format 36-401, Section B, Fall 2015 13 October 2015 Contents 1 Least Squares in Matrix

### Linear models. Linear models are computationally convenient and remain widely used in. applied econometric research

Linear models Linear models are computationally convenient and remain widely used in applied econometric research Our main focus in these lectures will be on single equation linear models of the form y

### Linear Models for Classification

Linear Models for Classification Oliver Schulte - CMPT 726 Bishop PRML Ch. 4 Classification: Hand-written Digit Recognition CHINE INTELLIGENCE, VOL. 24, NO. 24, APRIL 2002 x i = t i = (0, 0, 0, 1, 0, 0,

### CoDa-dendrogram: A new exploratory tool. 2 Dept. Informàtica i Matemàtica Aplicada, Universitat de Girona, Spain;

CoDa-dendrogram: A new exploratory tool J.J. Egozcue 1, and V. Pawlowsky-Glahn 2 1 Dept. Matemàtica Aplicada III, Universitat Politècnica de Catalunya, Barcelona, Spain; juan.jose.egozcue@upc.edu 2 Dept.

### Analysis of 2 n Factorial Experiments with Exponentially Distributed Response Variable

Applied Mathematical Sciences, Vol. 5, 2011, no. 10, 459-476 Analysis of 2 n Factorial Experiments with Exponentially Distributed Response Variable S. C. Patil (Birajdar) Department of Statistics, Padmashree

### Some properties of Likelihood Ratio Tests in Linear Mixed Models

Some properties of Likelihood Ratio Tests in Linear Mixed Models Ciprian M. Crainiceanu David Ruppert Timothy J. Vogelsang September 19, 2003 Abstract We calculate the finite sample probability mass-at-zero

### Ratio of Linear Function of Parameters and Testing Hypothesis of the Combination Two Split Plot Designs

Middle-East Journal of Scientific Research 13 (Mathematical Applications in Engineering): 109-115 2013 ISSN 1990-9233 IDOSI Publications 2013 DOI: 10.5829/idosi.mejsr.2013.13.mae.10002 Ratio of Linear

### On Projective Planes

C-UPPSATS 2002:02 TFM, Mid Sweden University 851 70 Sundsvall Tel: 060-14 86 00 On Projective Planes 1 2 7 4 3 6 5 The Fano plane, the smallest projective plane. By Johan Kåhrström ii iii Abstract It was

### Improved General Class of Ratio Type Estimators

[Volume 5 issue 8 August 2017] Page No.1790-1796 ISSN :2320-7167 INTERNATIONAL JOURNAL OF MATHEMATICS AND COMPUTER RESEARCH Improved General Class of Ratio Type Estimators 1 Banti Kumar, 2 Manish Sharma,

### Efficient Exponential Ratio Estimator for Estimating the Population Mean in Simple Random Sampling

Efficient Exponential Ratio Estimator for Estimating the Population Mean in Simple Random Sampling Ekpenyong, Emmanuel John and Enang, Ekaette Inyang Abstract This paper proposes, with justification, two

### Answers to Problem Set #4

Answers to Problem Set #4 Problems. Suppose that, from a sample of 63 observations, the least squares estimates and the corresponding estimated variance covariance matrix are given by: bβ bβ 2 bβ 3 = 2

### Mean squared error matrix comparison of least aquares and Stein-rule estimators for regression coefficients under non-normal disturbances

METRON - International Journal of Statistics 2008, vol. LXVI, n. 3, pp. 285-298 SHALABH HELGE TOUTENBURG CHRISTIAN HEUMANN Mean squared error matrix comparison of least aquares and Stein-rule estimators

### Application of Variance Homogeneity Tests Under Violation of Normality Assumption

Application of Variance Homogeneity Tests Under Violation of Normality Assumption Alisa A. Gorbunova, Boris Yu. Lemeshko Novosibirsk State Technical University Novosibirsk, Russia e-mail: gorbunova.alisa@gmail.com

### Chapter 11 Specification Error Analysis

Chapter Specification Error Analsis The specification of a linear regression model consists of a formulation of the regression relationships and of statements or assumptions concerning the explanator variables

### Improved ratio-type estimators using maximum and minimum values under simple random sampling scheme

Improved ratio-type estimators using maximum and minimum values under simple random sampling scheme Mursala Khan Saif Ullah Abdullah. Al-Hossain and Neelam Bashir Abstract This paper presents a class of

### GLS and FGLS. Econ 671. Purdue University. Justin L. Tobias (Purdue) GLS and FGLS 1 / 22

GLS and FGLS Econ 671 Purdue University Justin L. Tobias (Purdue) GLS and FGLS 1 / 22 In this lecture we continue to discuss properties associated with the GLS estimator. In addition we discuss the practical

### Marcia Gumpertz and Sastry G. Pantula Department of Statistics North Carolina State University Raleigh, NC

A Simple Approach to Inference in Random Coefficient Models March 8, 1988 Marcia Gumpertz and Sastry G. Pantula Department of Statistics North Carolina State University Raleigh, NC 27695-8203 Key Words

### A matrix-based method of moments for fitting the multivariate random effects model for meta-analysis and meta-regression

Biometrical Journal 55 (2013) 2, 231 245 DOI: 10.1002/bimj.201200152 231 A matrix-based method of moments for fitting the multivariate random effects model for meta-analysis and meta-regression Dan Jackson,1,

### Lecture 22: Variance and Covariance

EE5110 : Probability Foundations for Electrical Engineers July-November 2015 Lecture 22: Variance and Covariance Lecturer: Dr. Krishna Jagannathan Scribes: R.Ravi Kiran In this lecture we will introduce

### COS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION

COS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION SEAN GERRISH AND CHONG WANG 1. WAYS OF ORGANIZING MODELS In probabilistic modeling, there are several ways of organizing models:

### Homoskedasticity. Var (u X) = σ 2. (23)

Homoskedasticity How big is the difference between the OLS estimator and the true parameter? To answer this question, we make an additional assumption called homoskedasticity: Var (u X) = σ 2. (23) This

### 5.1 Model Specification and Data 5.2 Estimating the Parameters of the Multiple Regression Model 5.3 Sampling Properties of the Least Squares

5.1 Model Specification and Data 5. Estimating the Parameters of the Multiple Regression Model 5.3 Sampling Properties of the Least Squares Estimator 5.4 Interval Estimation 5.5 Hypothesis Testing for

### Association studies and regression

Association studies and regression CM226: Machine Learning for Bioinformatics. Fall 2016 Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar Association studies and regression 1 / 104 Administration

### ANALYSIS AND RATIO OF LINEAR FUNCTION OF PARAMETERS IN FIXED EFFECT THREE LEVEL NESTED DESIGN

ANALYSIS AND RATIO OF LINEAR FUNCTION OF PARAMETERS IN FIXED EFFECT THREE LEVEL NESTED DESIGN Mustofa Usman 1, Ibnu Malik 2, Warsono 1 and Faiz AM Elfaki 3 1 Department of Mathematics, Universitas Lampung,

### Sensitivity of GLS Estimators in Random Effects Models

Sensitivity of GLS Estimators in Random Effects Models Andrey L. Vasnev January 29 Faculty of Economics and Business, University of Sydney, NSW 26, Australia. E-mail: a.vasnev@econ.usyd.edu.au Summary

### Estimation of Mean Population in Small Area with Spatial Best Linear Unbiased Prediction Method

Journal of Physics: Conference Series PAPER OPEN ACCESS Estimation of Mean Population in Small Area with Spatial Best Linear Unbiased Prediction Method To cite this article: Syahril Ramadhan et al 2017

### Estadística II Chapter 4: Simple linear regression

Estadística II Chapter 4: Simple linear regression Chapter 4. Simple linear regression Contents Objectives of the analysis. Model specification. Least Square Estimators (LSE): construction and properties

### INFERENCE FOR MULTIPLE LINEAR REGRESSION MODEL WITH EXTENDED SKEW NORMAL ERRORS

Pak. J. Statist. 2016 Vol. 32(2), 81-96 INFERENCE FOR MULTIPLE LINEAR REGRESSION MODEL WITH EXTENDED SKEW NORMAL ERRORS A.A. Alhamide 1, K. Ibrahim 1 M.T. Alodat 2 1 Statistics Program, School of Mathematical

### Chapter 10. Supplemental Text Material

Chater 1. Sulemental Tet Material S1-1. The Covariance Matri of the Regression Coefficients In Section 1-3 of the tetbook, we show that the least squares estimator of β in the linear regression model y=

### Chapter 2: simple regression model

Chapter 2: simple regression model Goal: understand how to estimate and more importantly interpret the simple regression Reading: chapter 2 of the textbook Advice: this chapter is foundation of econometrics.

### Negative Multinomial Model and Cancer. Incidence

Generalized Linear Model under the Extended Negative Multinomial Model and Cancer Incidence S. Lahiri & Sunil K. Dhar Department of Mathematical Sciences, CAMS New Jersey Institute of Technology, Newar,

### Sampling Distributions

Merlise Clyde Duke University September 8, 2016 Outline Topics Normal Theory Chi-squared Distributions Student t Distributions Readings: Christensen Apendix C, Chapter 1-2 Prostate Example > library(lasso2);

### MODELLING REPEATED MEASURES DATA IN META ANALYSIS : AN ALTERNATIVE APPROACH

MODELLING REPEATED MEASURES DATA IN META ANALYSIS : AN ALTERNATIVE APPROACH Nik Ruzni Nik Idris Department Of Computational And Theoretical Sciences Kulliyyah Of Science, International Islamic University

### Propagation of Errors in Spatial Analysis

Stephen F. Austin State University SFA ScholarWorks Faculty Presentations Spatial Science 2001 Propagation of Errors in Spatial Analysis Peter P. Siska I-Kuai Hung Arthur Temple College of Forestry and

### Wooldridge, Introductory Econometrics, 4th ed. Chapter 15: Instrumental variables and two stage least squares

Wooldridge, Introductory Econometrics, 4th ed. Chapter 15: Instrumental variables and two stage least squares Many economic models involve endogeneity: that is, a theoretical relationship does not fit

### CHAPTER 6: SPECIFICATION VARIABLES

Recall, we had the following six assumptions required for the Gauss-Markov Theorem: 1. The regression model is linear, correctly specified, and has an additive error term. 2. The error term has a zero

### Ch4. Distribution of Quadratic Forms in y

ST4233, Linear Models, Semester 1 2008-2009 Ch4. Distribution of Quadratic Forms in y 1 Definition Definition 1.1 If A is a symmetric matrix and y is a vector, the product y Ay = i a ii y 2 i + i j a ij

### Efficient Analysis of Mixed Hierarchical and Cross-Classified Random Structures Using a Multilevel Model

Efficient Analysis of Mixed Hierarchical and Cross-Classified Random Structures Using a Multilevel Model Jon Rasbash; Harvey Goldstein Journal of Educational and Behavioral Statistics, Vol. 19, No. 4.

### MA 575 Linear Models: Cedric E. Ginestet, Boston University Regularization: Ridge Regression and Lasso Week 14, Lecture 2

MA 575 Linear Models: Cedric E. Ginestet, Boston University Regularization: Ridge Regression and Lasso Week 14, Lecture 2 1 Ridge Regression Ridge regression and the Lasso are two forms of regularized

### Chapter 5 Prediction of Random Variables

Chapter 5 Prediction of Random Variables C R Henderson 1984 - Guelph We have discussed estimation of β, regarded as fixed Now we shall consider a rather different problem, prediction of random variables,

### Dependence of the total -electron energy on a large number of non-bonding molecular orbitals

J. Serb. Chem. Soc. 69 (10) 777 782 (2004) UDC 54 724+537.872:519.17:54 12 JSCS 3204 Original scientific paper Dependence of the total -electron energy on a large number of non-bonding molecular orbitals

### Alternatives to Difference Scores: Polynomial Regression and Response Surface Methodology. Jeffrey R. Edwards University of North Carolina

Alternatives to Difference Scores: Polynomial Regression and Response Surface Methodology Jeffrey R. Edwards University of North Carolina 1 Outline I. Types of Difference Scores II. Questions Difference

### Multiple Regression Analysis: Estimation ECONOMETRICS (ECON 360) BEN VAN KAMMEN, PHD

Multiple Regression Analysis: Estimation ECONOMETRICS (ECON 360) BEN VAN KAMMEN, PHD Outline Motivation. Mechanics and Interpretation. Expected Values and Variances of the Estimators. Motivation for multiple

### Functional Analysis of Variance

Applied Mathematical Sciences, Vol. 2, 2008, no. 23, 1115-1129 Functional Analysis of Variance Abdelhak Zoglat Faculté des Sciences Département de Mathématiques et Informatique B.P. 1014 Rabat, Morocco

### Interaction effects for continuous predictors in regression modeling

Interaction effects for continuous predictors in regression modeling Testing for interactions The linear regression model is undoubtedly the most commonly-used statistical model, and has the advantage

### Use of Transformations and the Repeated Statement in PROC GLM in SAS Ed Stanek

Use of Transformations and the Reeated Statement in PROC GLM in SAS Ed Stanek Introduction We describe how the Reeated Statement in PROC GLM in SAS transforms the data to rovide tests of hyotheses of interest.

### Time Series Models and Inference. James L. Powell Department of Economics University of California, Berkeley

Time Series Models and Inference James L. Powell Department of Economics University of California, Berkeley Overview In contrast to the classical linear regression model, in which the components of the

### ABC random forest for parameter estimation. Jean-Michel Marin

ABC random forest for parameter estimation Jean-Michel Marin Université de Montpellier Institut Montpelliérain Alexander Grothendieck (IMAG) Institut de Biologie Computationnelle (IBC) Labex Numev! joint

### Chapter 5: Models used in conjunction with sampling. J. Kim, W. Fuller (ISU) Chapter 5: Models used in conjunction with sampling 1 / 70

Chapter 5: Models used in conjunction with sampling J. Kim, W. Fuller (ISU) Chapter 5: Models used in conjunction with sampling 1 / 70 Nonresponse Unit Nonresponse: weight adjustment Item Nonresponse:

### AN IMPROVEMENT TO THE ALIGNED RANK STATISTIC

Journal of Applied Statistical Science ISSN 1067-5817 Volume 14, Number 3/4, pp. 225-235 2005 Nova Science Publishers, Inc. AN IMPROVEMENT TO THE ALIGNED RANK STATISTIC FOR TWO-FACTOR ANALYSIS OF VARIANCE

### Topic 6. Two-way designs: Randomized Complete Block Design [ST&D Chapter 9 sections 9.1 to 9.7 (except 9.6) and section 15.8]

Topic 6. Two-way designs: Randomized Complete Block Design [ST&D Chapter 9 sections 9.1 to 9.7 (except 9.6) and section 15.8] The completely randomized design Treatments are randomly assigned to e.u. such