# INFERENCE FOR MULTIPLE LINEAR REGRESSION MODEL WITH EXTENDED SKEW NORMAL ERRORS

Save this PDF as:

Size: px
Start display at page:

Download "INFERENCE FOR MULTIPLE LINEAR REGRESSION MODEL WITH EXTENDED SKEW NORMAL ERRORS"

## Transcription

1 Pak. J. Statist Vol. 32(2), INFERENCE FOR MULTIPLE LINEAR REGRESSION MODEL WITH EXTENDED SKEW NORMAL ERRORS A.A. Alhamide 1, K. Ibrahim 1 M.T. Alodat 2 1 Statistics Program, School of Mathematical Sciences, Faculty of Science Technology, Universiti Kebangsaan Malaysia, Malaysia 2 Department of Mathematics, Statistics Physics, Qatar University, Qatar. ABSTRACT This paper presents the estimation of the parameters of the multiple linear regression model when errors are assumed to follow the independent extended skew normal distribution. The estimators of the regression parameters are determined using the maximum likelihood least squares methods. In addition, the asymptotic distributions of the estimators are studied. The properties of the estimators under both approaches are compared based on a simulation study a real data set is applied for illustration. KEYWORDS Extended skew normal distribution; Maximum likelihood estimates; Least squares estimator; Asymptotic distribution; Simulation; Multiple linear regression model; Berry- Esseen theorem. 1. INTRODUCTION The multiple linear regression (MLR) model is a common tool which is usually used for analyzing data, especially in the applied areas such as agriculture, environment, biometrics social science, see for example, Arellano-Valle et al. (2005). However, there are many cases the assumption of normality is not feasible, may be due to the presence of outliers or skewness in the data. If the observed data is skewed, it is difficult to apply the assumption of normality because it may lead to unrealistic results. Many researchers have used the transformation method to obtain normality by transforming the data to near normal. Instead of applying the transformation on the data, some symmetric distributions such as student-t, logistic, power exponential contaminated normal are adopted to deal with the lack of fit to the normal distribution, see Cordeiro et al. (2000), for example. However, in this study, the extended skew normal distribution is assumed when applying the multiple linear regression model on the data. Azzalini (1985) has introduced a new class of distribution known as the skew normal distribution by including the skewness parameter in addition to the scale location parameters to model the skewed data. One may refer to Liseo Loperfido (2003), Genton et al. (2001) Capitanio et al. (2003) for more detailed works on skew normal distribution its applications. Azzalini Capitanio (1999) have conducted a study on the applications of skew normal distributions some aspects of statistical inference Pakistan Journal of Statistics 81

2 82 Inference for multiple linear regression model Recently, Cancho et al. (2010) have provided the statistical inference for non-linear regression model based on the skew normal error model as suggested by Sahu et al. (2003). In addition to the skew normal distribution, the extended skew normal (ESN) distribution has also been considered in modeling the data with the presence of skewness. This ESN distribution has been introduced by Adcock (2010). As explained in his work, if a rom variable is said to follow an independent extended skew normal distribution with location parameter, scale parameter, skewness parameter extra parameter, then the probability density function of is given by ( ) represents the parameters, denote the probability density function (pdf) cumulative distribution function (CDF) of the normal distribution respectively. The notation can be used to describe the density of. The main purpose of this paper is to determine the estimators for the parameters of the multiple linear regression model the errors follow the independent extended skew normal distribution. In addition to determining the estimators, the properties of these estimators are also studied. In Section 2, we present the multiple linear regression model under ESN errors. In Section 3, we show two methods of parameter estimation, i.e., maximum likelihood least squares methods. In Section 4, we derive the asymptotic distributions for the estimators of the multiple linear regression model denoted as. In Section 5, we conduct a simulation study for computing the maximum likelihood the least square estimates of the parameters. In Section 6, we apply the findings to the Scottish hills races data. Finally, we conclude the article with a discussion of the results found. 2. MULTIPLE LINEAR REGRESSION MODEL WITH ESN ERRORS In this section, we consider the multiple linear regression model when the errors are independent identically distributed following the extended skew normal distribution given by ( ( ) ). Hence, the density function of is given by ( ( ) ) Let. Then, the density function of is given by ( ( ) ) ( ( ( ) ) ) (2.1)

3 Alhamide, Ibrahim Alodat 83 In order to simplify the mathematical derivation, we write the pdf of as follows: ( ( ) ) ( ( ) ) Then the joint pdf of is given by ( ) ( ) (2.2) ( ) ( ) ( ) We can easily show that the expectation variance-covariance of ( ) ( ) ( ( ) ) as follows: So, the log-likelihood function of is ( ) 3. METHODS OF ESTIMATION Maximum Likelihood Estimation We can find the maximum likelihood estimation (MLE) for each parameter in vector by taking the derivative of with respect to each parameter setting the derivatives to zero as follows: 1. Derivative with respect to : ( ( ) ) ( ) ( ) (3.1) 2. Derivative with respect to : ( ) ( ( ) ) ( ( ) ( ( )) ) ( ) ( ) (3.2) 3. Derivative with respect to : ( ( ) ) ( ( ) ) ( ) ( ) (3.3)

4 84 Inference for multiple linear regression model 4. Derivative with respect to : ( )( ( ) ) ( ) ( ) ( ( )) (3.4) We used the numerical technique to solve the equations above. Least Square Estimation of the Parameter vector We seek estimators that minimize the sum of squares of the deviation of the n observed from their predicted values It can be easily shown that the variance ( ) Then, the properties of the least squares estimator can be described by the following theorem: Theorem 1: i) ( ) ( ) i.e., is an unbiased estimator for. ii) ( ) ( ) ( ( ) ) iii) ( ) This theorem can easily be proven by referring to Rencher Schaalje (2008) 4. ASYMPTOTIC DISTRIBUTIONS FOR THE ESTIMATORS OF THE MULTIPLE LINEAR REGRESSION MODEL Consider the following multiple linear regression model: are unknown parameters is the error term. The least squares estimator for the parameters are: (4.1) { ( )( ) ( )( ) ( )( ) ( ) ( )

5 Alhamide, Ibrahim Alodat 85 Using equation (4.1) with some straightforward simplification, we have (4.2) ( ) ( ) Similarly, we can rewrite given in (4.1) to obtain (4.3) ( ) ( ) (4.4) ( ) ( ) It is easy to show that the estimators are unbiased for respectively the variances of these estimators as the following: ( ) ( ( ) ) { } ( ) ( ( ) ) ( ) ( ( ) ) In order to derive the asymptotic distributions of we need to use Berry-Esseen theorem Slutsky s theorem. For more details (see Chow Teicher, 1978; Alodat et al., 2010). Theorem 2. (Berry-Esseen) If * + are independent rom variables such that, for some

6 86 Inference for multiple linear regression model ( then there is exist a universal constant such that Theorem 3. By using Slutsky theorem Berry-Esseen theorem, we have ( ) ( ) ( ) ( ) { } ( ) ( ) ( ) ( ) For the proofs of Theorem 3, see the Appendix. 5. SIMULATION The Maximum Likelihood Estimation (MLE) Method In order to estimate the parameters of the multiple linear regression model under extended skew normal errors (ESN-MLR) normally distributed errors (N-MLR) using the maximum likelihood method, we conduct a simulation study for sample sizes iterations. Then, we compare the bias mean square errors (MSE) for the estimators in both cases. The simulation results are shown in Table (1).

7 Alhamide, Ibrahim Alodat 87 Table 1 Maximum Likelihood Estimates, Bias Mean Square Errors Assuming Certain Values for Parameters N-MLR ESN-MLR Parameters Estimate Bias Mse Estimate Bias Mse Based on the Table (1), we have parameter estimates for N-MLR ESN-MLR, in addition to the estimates of bias MSE for the respective estimators which have been found based on simulation. In general, we note from Table (1) that the estimates of bias MSE for the ESN-MLR are smaller than those in the N-MLR. The Least Squares Estimation (LSE) Method The LSE is a common method which can also be used for estimating the parameters for both MLR-N MLR-ESN. So, we conduct a simulation study to compare the stard errors for the estimators obtained. The results are shown in Table (2). Parameters Table 2 Least Square Estimates Stard Errors Assuming Certain Values of the Parameters N-MLR ESN-MLR Estimate S.E Estimate S.E From Table (2), we can notice that the stard errors of the parameter estimates are found to be smaller for ESN-MLR as compared to N-MLR, indicating that the estimation is more precise under ESN-MLR errors.

8 88 Inference for multiple linear regression model 6. AN APPLICATION: THE SCOTTISH HILLS RACES DATA The Scottish hills races data, which has also been applied by Chatterjee Hadi (2012), consist of observations the response variable (in seconds) is related to two other predictor variables, namely, (in mile) (in feet). This set of data is also available in the R package call (MASS). To investigate the presence of skewness in the data, as shown in Figure 1, normal skew normal probability density functions are fitted to the histogram of the residual found after fitting the skew normal error model. As given in Figure 2, the data are further plotted on the normal Q-Q plot. Several points fall away from the straight line on the normal Q-Q plot, indicating the presence of outliers. It will be further shown in the analysis that the extended skew normal model can nicely account for the presence of outliers in the data. Figure 1: The histogram of the residuals of the Scottish hills races data the fitted normal skew normal model

9 Alhamide, Ibrahim Alodat 89 Figure 2: The normal Q-Q plots of the Scottish hills races data Both N-MLR ESN-MLR are fitted by using the maximum likelihood least square methods to the Scottish hills races data the results found are given in Table (3) (4) respectively. Also, note that the Akaike Information Criterion (AIC) values shown in Table (3) indicate that ESN-MLR outperforms N-MLR since the smaller value is obtained when extended skew normal errors are assumed. Table 3 Results of Fitting N-MLR ESN-MLR to the Scottish Hills Races Data Involving Maximum Likelihood Estimates Stard Errors N-MLR ESN-MLR Parameters Estimate S.E Estimate S.E e-09 - Log-likelihood AIC

10 90 Inference for multiple linear regression model Table 4 Results of Fitting N-MLR ESN-MLR to the Scottish Hills Races Data Involving Least Squares Estimates Stard Errors N-MLR ESN-MLR Parameters Estimate S.E Estimate S.E CONCLUSION In this paper, we study the statistical inference estimation for the parameters using the maximum likelihood least squares methods for the multiple linear regression model under normal extended skew normal errors. Also, we have derived the asymptotic distributions of the estimators for the parameters of the multiple linear regression model under extended skew normal errors. From the comparison of the parameter estimates found based on the simulation study for the regression model under normal extended skew normal errors using the maximum likelihood method, the fitted model is better for the latter case. The results are further supported by the model fitting of real data since the response variable in the data exhibits some skewness properties due to the presence of outlying observation. There are more potential applications which can be investigated for this proposed regression model in addition to the given example. ACKNOWLEDGMENT The authors are thankful to the editor referees for their helpful comments suggestions on the earlier version of this paper. Also, we thank Faculty of Science Technology of Universiti Kebangsaan Malaysia for providing the partial support for this research under the grant DPP-2015-FST. REFERENCES 1. Adcock, C.J. (2010). Asset pricing portfolio selection based on the multivariate extended skew-student t distribution. Annals of Operations Research, 176(1), Alodat, M.T., Al-Rawwash, M.Y. Nawajah, I.M. (2010). Inference about the regression parameters using median-ranked set sampling. Communications in Statistics Theory Methods, 39(14), Arellano-Valle, R., Bolfarine, H. Lachos, V. (2005). Skew normal linear mixed models. Journal of Data Science, 3, Azzalini, A. (1985). A class of distributions which includes the normal ones. Scinavian Journal of Statistics, 12,

11 Alhamide, Ibrahim Alodat Azzalini, A., Capitanio, A. (1999). Statistical applications of the multivariate skew normal distribution. Journal of the Royal Statistical Society: Series B, 61(3), Bhattacharya, R.N. Rao, R. (1976). Normal Approximation Asymptotic Expansions. Wiley, New York. 7. Cancho, V.G., Ortega, E.M.M. Paula, G.A. (2010). On estimation influence diagnostics for log-birnbaum-saunders regression models. Journal of Statistical Planning Inference, 140, Capitanio, A., Azzalini, A. Stanghellini, E. (2003). Graphical models for skew normal variates. Scinavian Journal of Statistics, 30, Chatterjee, S. Hadi, A.S. (2012). Regression analysis by example. Edition, Wiley, New Jersey. 10. Chow, Y.S. Teicher, H. (1978). Probability Theory: Independence, Interchangeability, Martingales. Springer, New York. 11. Cordeiro, G.M., Ferrari, S.L., Uribe-Opazo, M.A. Vasconcellos, K.L. (2000). Corrected maximum likelihood estimation in a class of symmetric nonlinear regression models. Statistics Probability Letters, 46, Genton, M.G., He, L. Liu, X. (2001). Moments of skew normal rom vectors their quadratic forms. Statistics Probability Letters, 4, Liseo, B. Loperfido, N. (2003). A Bayesian interpretation of the multivariate skew normal distribution. Statistics Probability Letters, 61, Rencher, A. C. Schaalje, G. B. (2008). Linear Models in Statistics. Wiley, New Jersey. 15. Sahu, S.K., Dey, D.K. Branco, M. (2003). A new class of multivariate skew distributions with application to Bayesian regression models. Canadian Journal of Statistics, 31,

12 92 Inference for multiple linear regression model APPENDIX Proofs for the asymptotic distribution of the estimators Proof 1. Using the equation (4.3), we notice that then ( ( ) ( ) ) ( ( ) ) ( ) ( ) ( ( ) ) is independent of i. Also, Hence, we have ( ) ( ) ( ( ) ) By applying Berry-Esseen theorem, we obtain ( ( ( ) ) ) ( ) ( ) ( ( ) ) The following inequality, as given in Bhattacharya Rao (1976), is used for proving the theorem: ( ) By using this inequality assume that ( ) ( ), for, we have

13 Alhamide, Ibrahim Alodat 93 ( ( ) ( ) ) ( ) ( ) Then, based on a straightforward simplification setting, we get ( ) ( ) ( ( ) ( ) ) ( ( ) ( ) ) This conclude that for all. Consequently, we may conclude that { ( ( ( ) ) ) } Eventually, this implies that ( ) ( ( ) ) ( ) ( ) ( ) Then, by Slutsky s theorem, we conclude that ( ) Proof 2. Consider equation (4.4) we note that ( ( ) ) ( ( ) ) On the other h

14 94 Inference for multiple linear regression model ( ) ( ) is independent of Also, ( ( ) ) Hence, we have ( ( ) ) After applying Berry-Esseen theorem, we get ( ( ( ) ) ) ( ) ( ) ( ( ) ) Similarly, based on the inequality by Bhattacharya Rao (1976) as given earlier assuming for we have ( ( ) ( ) ) with more simplification, we obtain ( ) ( )

15 Alhamide, Ibrahim Alodat 95 ( ) ( ) ( ( ) ( ) ) ( ( ) ( ) ) This conclude that for all. Consequently, we may conclude that { ( ( ( ) ) ) } Thus, Berry-Esseen theorem is satisfied. Then, we have ( ) ( ( ) ) ( ) ( ) ( ) Then, by Slutsky s theorem, we conclude that ( ) ( ) Proof 3. Using equation (4.2) applying Berry-Esseen theorem, we can prove the following: for every Also, from Berry-Esseen theorem, we have ( ( ) ( ) ) ( ( ) ) ( ( ) )

16 96 Inference for multiple linear regression model Now, we compute the term ( ) as follows: ( ) ( ) ( ) By combining the two expressions of, we get ( ) ( ) Thus, we get ( ( ) ) ( ( ( ) ) ) which leads to the following conclusion: ( ) ( ) ( ) ( ) ( ( ) ) { ( ( ) ) Then, by Slutsky s theorem, we conclude that ( ) ( ) } ( )

### Using R in Undergraduate Probability and Mathematical Statistics Courses. Amy G. Froelich Department of Statistics Iowa State University

Using R in Undergraduate Probability and Mathematical Statistics Courses Amy G. Froelich Department of Statistics Iowa State University Undergraduate Probability and Mathematical Statistics at Iowa State

### Mean squared error matrix comparison of least aquares and Stein-rule estimators for regression coefficients under non-normal disturbances

METRON - International Journal of Statistics 2008, vol. LXVI, n. 3, pp. 285-298 SHALABH HELGE TOUTENBURG CHRISTIAN HEUMANN Mean squared error matrix comparison of least aquares and Stein-rule estimators

### Robustness of location estimators under t- distributions: a literature review

IOP Conference Series: Earth and Environmental Science PAPER OPEN ACCESS Robustness of location estimators under t- distributions: a literature review o cite this article: C Sumarni et al 07 IOP Conf.

### PARAMETER ESTIMATION FOR THE LOG-LOGISTIC DISTRIBUTION BASED ON ORDER STATISTICS

PARAMETER ESTIMATION FOR THE LOG-LOGISTIC DISTRIBUTION BASED ON ORDER STATISTICS Authors: Mohammad Ahsanullah Department of Management Sciences, Rider University, New Jersey, USA ahsan@rider.edu) Ayman

### Large Sample Properties of Estimators in the Classical Linear Regression Model

Large Sample Properties of Estimators in the Classical Linear Regression Model 7 October 004 A. Statement of the classical linear regression model The classical linear regression model can be written in

### On Comparison of Some Variation of Ranked Set Sampling (Tentang Perbandingan Beberapa Variasi Pensampelan Set Terpangkat)

Sains Malaysiana 40(4)(2011): 397 401 On Comparison of Some Variation of Ranked Set Sampling (Tentang Perbandingan Beberapa Variasi Pensampelan Set Terpangkat) KAMARULZAMAN IBRAHIM* ABSTRACT Many sampling

### Statistics - Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation

Statistics - Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence

### Illustrating the Implicit BIC Prior. Richard Startz * revised June Abstract

Illustrating the Implicit BIC Prior Richard Startz * revised June 2013 Abstract I show how to find the uniform prior implicit in using the Bayesian Information Criterion to consider a hypothesis about

### Statistics 203: Introduction to Regression and Analysis of Variance Penalized models

Statistics 203: Introduction to Regression and Analysis of Variance Penalized models Jonathan Taylor - p. 1/15 Today s class Bias-Variance tradeoff. Penalized regression. Cross-validation. - p. 2/15 Bias-variance

### Latent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent

Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary

### Multilevel Statistical Models: 3 rd edition, 2003 Contents

Multilevel Statistical Models: 3 rd edition, 2003 Contents Preface Acknowledgements Notation Two and three level models. A general classification notation and diagram Glossary Chapter 1 An introduction

### Testing Statistical Hypotheses

E.L. Lehmann Joseph P. Romano Testing Statistical Hypotheses Third Edition 4y Springer Preface vii I Small-Sample Theory 1 1 The General Decision Problem 3 1.1 Statistical Inference and Statistical Decisions

### Hypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3

Hypothesis Testing CB: chapter 8; section 0.3 Hypothesis: statement about an unknown population parameter Examples: The average age of males in Sweden is 7. (statement about population mean) The lowest

### Stein-Rule Estimation under an Extended Balanced Loss Function

Shalabh & Helge Toutenburg & Christian Heumann Stein-Rule Estimation under an Extended Balanced Loss Function Technical Report Number 7, 7 Department of Statistics University of Munich http://www.stat.uni-muenchen.de

### Using Estimating Equations for Spatially Correlated A

Using Estimating Equations for Spatially Correlated Areal Data December 8, 2009 Introduction GEEs Spatial Estimating Equations Implementation Simulation Conclusion Typical Problem Assess the relationship

### Generalized Linear Model under the Extended Negative Multinomial Model and Cancer Incidence

Generalized Linear Model under the Extended Negative Multinomial Model and Cancer Incidence Sunil Kumar Dhar Center for Applied Mathematics and Statistics, Department of Mathematical Sciences, New Jersey

### Testing Statistical Hypotheses

E.L. Lehmann Joseph P. Romano, 02LEu1 ttd ~Lt~S Testing Statistical Hypotheses Third Edition With 6 Illustrations ~Springer 2 The Probability Background 28 2.1 Probability and Measure 28 2.2 Integration.........

### How likely is Simpson s paradox in path models?

How likely is Simpson s paradox in path models? Ned Kock Full reference: Kock, N. (2015). How likely is Simpson s paradox in path models? International Journal of e- Collaboration, 11(1), 1-7. Abstract

### Model comparison and selection

BS2 Statistical Inference, Lectures 9 and 10, Hilary Term 2008 March 2, 2008 Hypothesis testing Consider two alternative models M 1 = {f (x; θ), θ Θ 1 } and M 2 = {f (x; θ), θ Θ 2 } for a sample (X = x)

### Generalized Method of Moments Estimation

Generalized Method of Moments Estimation Lars Peter Hansen March 0, 2007 Introduction Generalized methods of moments (GMM) refers to a class of estimators which are constructed from exploiting the sample

### Model Estimation Example

Ronald H. Heck 1 EDEP 606: Multivariate Methods (S2013) April 7, 2013 Model Estimation Example As we have moved through the course this semester, we have encountered the concept of model estimation. Discussions

### Ratio of Linear Function of Parameters and Testing Hypothesis of the Combination Two Split Plot Designs

Middle-East Journal of Scientific Research 13 (Mathematical Applications in Engineering): 109-115 2013 ISSN 1990-9233 IDOSI Publications 2013 DOI: 10.5829/idosi.mejsr.2013.13.mae.10002 Ratio of Linear

### TOLERANCE INTERVALS FOR DISCRETE DISTRIBUTIONS IN EXPONENTIAL FAMILIES

Statistica Sinica 19 (2009), 905-923 TOLERANCE INTERVALS FOR DISCRETE DISTRIBUTIONS IN EXPONENTIAL FAMILIES Tianwen Tony Cai and Hsiuying Wang University of Pennsylvania and National Chiao Tung University

### Geographically Weighted Regression as a Statistical Model

Geographically Weighted Regression as a Statistical Model Chris Brunsdon Stewart Fotheringham Martin Charlton October 6, 2000 Spatial Analysis Research Group Department of Geography University of Newcastle-upon-Tyne

### Multivariate Lineare Modelle

0-1 TALEB AHMAD CASE - Center for Applied Statistics and Economics Humboldt-Universität zu Berlin Motivation 1-1 Motivation Multivariate regression models can accommodate many explanatory which simultaneously

### Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics

Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics A short review of the principles of mathematical statistics (or, what you should have learned in EC 151).

### Regression I: Mean Squared Error and Measuring Quality of Fit

Regression I: Mean Squared Error and Measuring Quality of Fit -Applied Multivariate Analysis- Lecturer: Darren Homrighausen, PhD 1 The Setup Suppose there is a scientific problem we are interested in solving

### Keller: Stats for Mgmt & Econ, 7th Ed July 17, 2006

Chapter 17 Simple Linear Regression and Correlation 17.1 Regression Analysis Our problem objective is to analyze the relationship between interval variables; regression analysis is the first tool we will

### Time Series: Theory and Methods

Peter J. Brockwell Richard A. Davis Time Series: Theory and Methods Second Edition With 124 Illustrations Springer Contents Preface to the Second Edition Preface to the First Edition vn ix CHAPTER 1 Stationary

### Inverse of a Square Matrix. For an N N square matrix A, the inverse of A, 1

Inverse of a Square Matrix For an N N square matrix A, the inverse of A, 1 A, exists if and only if A is of full rank, i.e., if and only if no column of A is a linear combination 1 of the others. A is

### Estimation of direction of increase of gold mineralisation using pair-copulas

22nd International Congress on Modelling and Simulation, Hobart, Tasmania, Australia, 3 to 8 December 2017 mssanz.org.au/modsim2017 Estimation of direction of increase of gold mineralisation using pair-copulas

### Asymptotic Relative Efficiency in Estimation

Asymptotic Relative Efficiency in Estimation Robert Serfling University of Texas at Dallas October 2009 Prepared for forthcoming INTERNATIONAL ENCYCLOPEDIA OF STATISTICAL SCIENCES, to be published by Springer

### Negative Multinomial Model and Cancer. Incidence

Generalized Linear Model under the Extended Negative Multinomial Model and Cancer Incidence S. Lahiri & Sunil K. Dhar Department of Mathematical Sciences, CAMS New Jersey Institute of Technology, Newar,

### STA216: Generalized Linear Models. Lecture 1. Review and Introduction

STA216: Generalized Linear Models Lecture 1. Review and Introduction Let y 1,..., y n denote n independent observations on a response Treat y i as a realization of a random variable Y i In the general

### Applied Regression Modeling

Applied Regression Modeling A Business Approach Iain Pardoe University of Oregon Charles H. Lundquist College of Business Eugene, Oregon WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION CONTENTS

### The loss function and estimating equations

Chapter 6 he loss function and estimating equations 6 Loss functions Up until now our main focus has been on parameter estimating via the maximum likelihood However, the negative maximum likelihood is

### The Fundamentals of Heavy Tails Properties, Emergence, & Identification. Jayakrishnan Nair, Adam Wierman, Bert Zwart

The Fundamentals of Heavy Tails Properties, Emergence, & Identification Jayakrishnan Nair, Adam Wierman, Bert Zwart Why am I doing a tutorial on heavy tails? Because we re writing a book on the topic Why

### A nonparametric test for trend based on initial ranks

Journal of Statistical Computation and Simulation Vol. 76, No. 9, September 006, 89 837 A nonparametric test for trend based on initial ranks GLENN HOFMANN* and N. BALAKRISHNAN HSBC, USA ID Analytics,

### A Course in Applied Econometrics Lecture 18: Missing Data. Jeff Wooldridge IRP Lectures, UW Madison, August Linear model with IVs: y i x i u i,

A Course in Applied Econometrics Lecture 18: Missing Data Jeff Wooldridge IRP Lectures, UW Madison, August 2008 1. When Can Missing Data be Ignored? 2. Inverse Probability Weighting 3. Imputation 4. Heckman-Type

### IMPROVED PARAMETER ESTIMATION OF THE LOG-LOGISTIC DISTRIBUTION WITH APPLICATIONS

Michigan Technological University Digital Commons @ Michigan Tech Dissertations, Master's Theses and Master's Reports 2016 IMPROVED PARAMETER ESTIMATION OF THE LOG-LOGISTIC DISTRIBUTION WITH APPLICATIONS

### For Bonnie and Jesse (again)

SECOND EDITION A P P L I E D R E G R E S S I O N A N A L Y S I S a n d G E N E R A L I Z E D L I N E A R M O D E L S For Bonnie and Jesse (again) SECOND EDITION A P P L I E D R E G R E S S I O N A N A

### Empirical Comparison of ML and UMVU Estimators of the Generalized Variance for some Normal Stable Tweedie Models: a Simulation Study

Applied Mathematical Sciences, Vol. 10, 2016, no. 63, 3107-3118 HIKARI Ltd, www.m-hikari.com https://doi.org/10.12988/ams.2016.69238 Empirical Comparison of and Estimators of the Generalized Variance for

### FACTOR ANALYSIS AS MATRIX DECOMPOSITION 1. INTRODUCTION

FACTOR ANALYSIS AS MATRIX DECOMPOSITION JAN DE LEEUW ABSTRACT. Meet the abstract. This is the abstract. 1. INTRODUCTION Suppose we have n measurements on each of taking m variables. Collect these measurements

### On Autoregressive Order Selection Criteria

On Autoregressive Order Selection Criteria Venus Khim-Sen Liew Faculty of Economics and Management, Universiti Putra Malaysia, 43400 UPM, Serdang, Malaysia This version: 1 March 2004. Abstract This study

### Chapter 12 REML and ML Estimation

Chapter 12 REML and ML Estimation C. R. Henderson 1984 - Guelph 1 Iterative MIVQUE The restricted maximum likelihood estimator (REML) of Patterson and Thompson (1971) can be obtained by iterating on MIVQUE,

### Confidence Intervals for One-Way Repeated Measures Contrasts

Chapter 44 Confidence Intervals for One-Way Repeated easures Contrasts Introduction This module calculates the expected width of a confidence interval for a contrast (linear combination) of the means in

### Application and Use of Multivariate Control Charts In a BTA Deep Hole Drilling Process

Application and Use of Multivariate Control Charts In a BTA Deep Hole Drilling Process Amor Messaoud, Winfied Theis, Claus Weihs, and Franz Hering Fachbereich Statistik, Universität Dortmund, 44221 Dortmund,

### SEMIPARAMETRIC QUANTILE REGRESSION WITH HIGH-DIMENSIONAL COVARIATES. Liping Zhu, Mian Huang, & Runze Li. The Pennsylvania State University

SEMIPARAMETRIC QUANTILE REGRESSION WITH HIGH-DIMENSIONAL COVARIATES Liping Zhu, Mian Huang, & Runze Li The Pennsylvania State University Technical Report Series #10-104 College of Health and Human Development

### Behaviour of the UMDA c algorithm with truncation selection on monotone functions

Mannheim Business School Dept. of Logistics Technical Report 01/2005 Behaviour of the UMDA c algorithm with truncation selection on monotone functions Jörn Grahl, Stefan Minner, Franz Rothlauf Technical

### BLIND SEPARATION USING ABSOLUTE MOMENTS BASED ADAPTIVE ESTIMATING FUNCTION. Juha Karvanen and Visa Koivunen

BLIND SEPARATION USING ABSOLUTE MOMENTS BASED ADAPTIVE ESTIMATING UNCTION Juha Karvanen and Visa Koivunen Signal Processing Laboratory Helsinki University of Technology P.O. Box 3, IN-215 HUT, inland Tel.

### TEST POWER IN COMPARISON DIFFERENCE BETWEEN TWO INDEPENDENT PROPORTIONS

TEST POWER IN COMPARISON DIFFERENCE BETWEEN TWO INDEPENDENT PROPORTIONS Mehmet MENDES PhD, Associate Professor, Canakkale Onsekiz Mart University, Agriculture Faculty, Animal Science Department, Biometry

### On Least Squares Linear Regression Without Second Moment

On Least Squares Linear Regression Without Second Moment BY RAJESHWARI MAJUMDAR University of Connecticut If \ and ] are real valued random variables such that the first moments of \, ], and \] exist and

### After completing this chapter, you should be able to:

Chapter 2 Descriptive Statistics Chapter Goals After completing this chapter, you should be able to: Compute and interpret the mean, median, and mode for a set of data Find the range, variance, standard

### Frailty Models and Copulas: Similarities and Differences

Frailty Models and Copulas: Similarities and Differences KLARA GOETHALS, PAUL JANSSEN & LUC DUCHATEAU Department of Physiology and Biometrics, Ghent University, Belgium; Center for Statistics, Hasselt

### AN IMPROVEMENT TO THE ALIGNED RANK STATISTIC

Journal of Applied Statistical Science ISSN 1067-5817 Volume 14, Number 3/4, pp. 225-235 2005 Nova Science Publishers, Inc. AN IMPROVEMENT TO THE ALIGNED RANK STATISTIC FOR TWO-FACTOR ANALYSIS OF VARIANCE

### The Expectation Maximization Algorithm

The Expectation Maximization Algorithm Frank Dellaert College of Computing, Georgia Institute of Technology Technical Report number GIT-GVU-- February Abstract This note represents my attempt at explaining

### Forecasting Gold Price. A Comparative Study

Course of Financial Econometrics FIRM Forecasting Gold Price A Comparative Study Alessio Azzutti, University of Florence Abstract: This paper seeks to evaluate the appropriateness of a variety of existing

### M2S2 Statistical Modelling

UNIVERSITY OF LONDON BSc and MSci EXAMINATIONS (MATHEMATICS) May-June 2006 This paper is also taken for the relevant examination for the Associateship. M2S2 Statistical Modelling Date: Wednesday, 17th

### Sigmaplot di Systat Software

Sigmaplot di Systat Software SigmaPlot Has Extensive Statistical Analysis Features SigmaPlot is now bundled with SigmaStat as an easy-to-use package for complete graphing and data analysis. The statistical

### 1) A residual plot: A)

1) A residual plot: A) B) C) D) E) displays residuals of the response variable versus the independent variable. displays residuals of the independent variable versus the response variable. displays residuals

### Kernel Density Based Linear Regression Estimate

Kernel Density Based Linear Regression Estimate Weixin Yao and Zibiao Zao Abstract For linear regression models wit non-normally distributed errors, te least squares estimate (LSE will lose some efficiency

### Outlier Robust Nonlinear Mixed Model Estimation

Outlier Robust Nonlinear Mixed Model Estimation 1 James D. Williams, 2 Jeffrey B. Birch and 3 Abdel-Salam G. Abdel-Salam 1 Business Analytics, Dow AgroSciences, 9330 Zionsville Rd. Indianapolis, IN 46268,

### Bayesian Heteroskedasticity-Robust Regression. Richard Startz * revised February Abstract

Bayesian Heteroskedasticity-Robust Regression Richard Startz * revised February 2015 Abstract I offer here a method for Bayesian heteroskedasticity-robust regression. The Bayesian version is derived by

### Introduction to Probability Theory for Graduate Economics Fall 2008

Introduction to Probability Theory for Graduate Economics Fall 008 Yiğit Sağlam October 10, 008 CHAPTER - RANDOM VARIABLES AND EXPECTATION 1 1 Random Variables A random variable (RV) is a real-valued function

### Keywords: Estimator, Bias, Mean-squared error, normality, generalized Pareto distribution

Testing approxiate norality of an estiator using the estiated MSE and bias with an application to the shape paraeter of the generalized Pareto distribution J. Martin van Zyl Abstract In this work the norality

### Overview of Extreme Value Analysis (EVA)

Overview of Extreme Value Analysis (EVA) Brian Reich North Carolina State University July 26, 2016 Rossbypalooza Chicago, IL Brian Reich Overview of Extreme Value Analysis (EVA) 1 / 24 Importance of extremes

### Mixed effects models

Mixed effects models The basic theory and application in R Mitchel van Loon Research Paper Business Analytics Mixed effects models The basic theory and application in R Author: Mitchel van Loon Research

### Least Squares Estimation-Finite-Sample Properties

Least Squares Estimation-Finite-Sample Properties Ping Yu School of Economics and Finance The University of Hong Kong Ping Yu (HKU) Finite-Sample 1 / 29 Terminology and Assumptions 1 Terminology and Assumptions

### Shrinkage regression

Shrinkage regression Rolf Sundberg Volume 4, pp 1994 1998 in Encyclopedia of Environmetrics (ISBN 0471 899976) Edited by Abdel H. El-Shaarawi and Walter W. Piegorsch John Wiley & Sons, Ltd, Chichester,

### Problems with Penalised Maximum Likelihood and Jeffrey s Priors to Account For Separation in Large Datasets with Rare Events

Problems with Penalised Maximum Likelihood and Jeffrey s Priors to Account For Separation in Large Datasets with Rare Events Liam F. McGrath September 15, 215 Abstract When separation is a problem in binary

### Binary Models with Endogenous Explanatory Variables

Binary Models with Endogenous Explanatory Variables Class otes Manuel Arellano ovember 7, 2007 Revised: January 21, 2008 1 Introduction In Part I we considered linear and non-linear models with additive

### BAYESIAN KRIGING AND BAYESIAN NETWORK DESIGN

BAYESIAN KRIGING AND BAYESIAN NETWORK DESIGN Richard L. Smith Department of Statistics and Operations Research University of North Carolina Chapel Hill, N.C., U.S.A. J. Stuart Hunter Lecture TIES 2004

### An Analysis of Record Statistics based on an Exponentiated Gumbel Model

Communications for Statistical Applications and Methods 2013, Vol. 20, No. 5, 405 416 DOI: http://dx.doi.org/10.5351/csam.2013.20.5.405 An Analysis of Record Statistics based on an Exponentiated Gumbel

### Confidence Interval for the Ratio of Two Normal Variables (an Application to Value of Time)

Interdisciplinary Information Sciences Vol. 5, No. (009) 37 3 #Graduate School of Information Sciences, Tohoku University ISSN 30-9050 print/37-657 online DOI 0.036/iis.009.37 Confidence Interval for the

Load-Strength Interference Loads vary, strengths vary, and reliability usually declines for mechanical systems, electronic systems, and electrical systems. The cause of failures is a load-strength interference

### 4. Distributions of Functions of Random Variables

4. Distributions of Functions of Random Variables Setup: Consider as given the joint distribution of X 1,..., X n (i.e. consider as given f X1,...,X n and F X1,...,X n ) Consider k functions g 1 : R n

### Comparison of Stochastic Soybean Yield Response Functions to Phosphorus Fertilizer

"Science Stays True Here" Journal of Mathematics and Statistical Science, Volume 016, 111 Science Signpost Publishing Comparison of Stochastic Soybean Yield Response Functions to Phosphorus Fertilizer

### Measures of Fit from AR(p)

Measures of Fit from AR(p) Residual Sum of Squared Errors Residual Mean Squared Error Root MSE (Standard Error of Regression) R-squared R-bar-squared = = T t e t SSR 1 2 ˆ = = T t e t p T s 1 2 2 ˆ 1 1

### Prentice Hall Stats: Modeling the World 2004 (Bock) Correlated to: National Advanced Placement (AP) Statistics Course Outline (Grades 9-12)

National Advanced Placement (AP) Statistics Course Outline (Grades 9-12) Following is an outline of the major topics covered by the AP Statistics Examination. The ordering here is intended to define the

### Outline. Statistical inference for linear mixed models. One-way ANOVA in matrix-vector form

Outline Statistical inference for linear mixed models Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark general form of linear mixed models examples of analyses using linear mixed

### Discussion of Least Angle Regression

Discussion of Least Angle Regression David Madigan Rutgers University & Avaya Labs Research Piscataway, NJ 08855 madigan@stat.rutgers.edu Greg Ridgeway RAND Statistics Group Santa Monica, CA 90407-2138

### 1. Definition: Order Statistics of a sample.

AMS570 Order Statistics 1. Deinition: Order Statistics o a sample. Let X1, X2,, be a random sample rom a population with p.d.. (x). Then, 2. p.d.. s or W.L.O.G.(W thout Loss o Ge er l ty), let s ssu e

### SPATIAL ECONOMETRICS: METHODS AND MODELS

SPATIAL ECONOMETRICS: METHODS AND MODELS STUDIES IN OPERATIONAL REGIONAL SCIENCE Folmer, H., Regional Economic Policy. 1986. ISBN 90-247-3308-1. Brouwer, F., Integrated Environmental Modelling: Design

### Regression #4: Properties of OLS Estimator (Part 2)

Regression #4: Properties of OLS Estimator (Part 2) Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #4 1 / 24 Introduction In this lecture, we continue investigating properties associated

### On Monitoring Shift in the Mean Processes with. Vector Autoregressive Residual Control Charts of. Individual Observation

Applied Mathematical Sciences, Vol. 8, 14, no. 7, 3491-3499 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/.12988/ams.14.44298 On Monitoring Shift in the Mean Processes with Vector Autoregressive Residual

### Obtaining Uncertainty Measures on Slope and Intercept

Obtaining Uncertainty Measures on Slope and Intercept of a Least Squares Fit with Excel s LINEST Faith A. Morrison Professor of Chemical Engineering Michigan Technological University, Houghton, MI 39931

### Some Monte Carlo Evidence for Adaptive Estimation of Unit-Time Varying Heteroscedastic Panel Data Models

Some Monte Carlo Evidence for Adaptive Estimation of Unit-Time Varying Heteroscedastic Panel Data Models G. R. Pasha Department of Statistics, Bahauddin Zakariya University Multan, Pakistan E-mail: drpasha@bzu.edu.pk

### Semi Bayes Estimation of the Parameters of Binomial Distribution in the Presence of Outliers

Journal of Statistical Theory and Applications Volume 11, Number 2, 2012, pp. 143-163 ISSN 1538-7887 Semi Bayes Estimation of the Parameters of Binomial Distribution in the Presence of Outliers U. J. Dixit

### DEPARTMENT OF ECONOMICS

ISSN 089-64 ISBN 978 0 7340 405 THE UNIVERSITY OF MELBOURNE DEPARTMENT OF ECONOMICS RESEARCH PAPER NUMBER 06 January 009 Notes on the Construction of Geometric Representations of Confidence Intervals of

### Spatial Variation in Infant Mortality with Geographically Weighted Poisson Regression (GWPR) Approach

Spatial Variation in Infant Mortality with Geographically Weighted Poisson Regression (GWPR) Approach Kristina Pestaria Sinaga, Manuntun Hutahaean 2, Petrus Gea 3 1, 2, 3 University of Sumatera Utara,

### Bias-Correction in Vector Autoregressive Models: A Simulation Study

Econometrics 2014, 2, 45-71; doi:10.3390/econometrics2010045 OPEN ACCESS econometrics ISSN 2225-1146 www.mdpi.com/journal/econometrics Article Bias-Correction in Vector Autoregressive Models: A Simulation

### Quantile function expansion using regularly varying functions

Quantile function expansion using regularly varying functions arxiv:705.09494v [math.st] 9 Aug 07 Thomas Fung a, an Eugene Seneta b a Department of Statistics, Macquarie University, NSW 09, Australia b

### Problem Y is an exponential random variable with parameter λ = 0.2. Given the event A = {Y < 2},

ECE32 Spring 25 HW Solutions April 6, 25 Solutions to HW Note: Most of these solutions were generated by R. D. Yates and D. J. Goodman, the authors of our textbook. I have added comments in italics where

### Test Code: STA/STB (Short Answer Type) 2013 Junior Research Fellowship for Research Course in Statistics

Test Code: STA/STB (Short Answer Type) 2013 Junior Research Fellowship for Research Course in Statistics The candidates for the research course in Statistics will have to take two shortanswer type tests

### Bayesian Inference: Probit and Linear Probability Models

Utah State University DigitalCommons@USU All Graduate Plan B and other Reports Graduate Studies 5-1-2014 Bayesian Inference: Probit and Linear Probability Models Nate Rex Reasch Utah State University Follow

### Unbiased Estimation. Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others.

Unbiased Estimation Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others. To compare ˆθ and θ, two estimators of θ: Say ˆθ is better than θ if it

### Discussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon

Discussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon Jianqing Fan Department of Statistics Chinese University of Hong Kong AND Department of Statistics

### ECON3150/4150 Spring 2015

ECON3150/4150 Spring 2015 Lecture 3&4 - The linear regression model Siv-Elisabeth Skjelbred University of Oslo January 29, 2015 1 / 67 Chapter 4 in S&W Section 17.1 in S&W (extended OLS assumptions) 2