Chapter 8: Estimation 1
|
|
- Delilah McDaniel
- 6 years ago
- Views:
Transcription
1 Chapter 8: Estimation 1 Jae-Kwang Kim Iowa State University Fall, 2014 Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
2 Introduction 1 Introduction 2 Ratio estimation 3 Regression estimator Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
3 Introduction So far, we have discussed various sampling designs and its unbiased estimator. HT estimator is used for each sampling design (except for PPS sampling). No claim for optimality. Definition For parameter θ (y), y = (y 1, y 2,, y N ), an estimator ˆθ (A) is UMVUE (Uniformly unbiased minimum variance estimator) if } 1 Unbiased: E y {ˆθ (A) = θ (y), for all y } } 2 Minimum variance: V y {ˆθ (A) V y {ˆθ (A) for all unbiased estimator ˆθ (A) and for all y. Remark Uniformity is important: Suppose that my estimator is ˆθ 12. If θ = 12, then ˆθ (ˆθ) is unbiased and V = 0. That is, MVUE at θ = 12. But, it is not UMVUE. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
4 Introduction Proposition Consider a noncensus design with π k > 0, (k = 1, 2,, N), then there is no UMVUE of t = N i=1 y i exists. Proof Suppose that there exists ˆQ which is UMVUE of t. Fix any y = (y1,, y N ) R N. Now, consider Q (A) = k A y k y k π k + N yk. The new estimator Q (A) satisfies 1 Unbiased 2 The variance of Q (A) is zero at y = y. Because ˆQ is UMVUE, V y ( ˆQ) V y (Q ). Since V y (Q ) = 0 for y = y, we have V y ( ˆQ) = 0 for y = y. Since y can be arbitrary, we have V y ( ˆQ) = 0 for all y, which means that ˆQ = t for all y, which is impossible for any noncensus design. Therefore, UMVUE cannot exist. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33 i=1
5 Introduction Remark 1 In the proof of the proposition, Q (A) is called the difference estimator. The variance of the difference estimator is V {Q (A)} = k U The variance is small if y k = y k. l U kl y k y k π k y l y l π l. 2 The class of (design) unbiased estimator is too big. We cannot find the best one in this class. 3 If we define the class of the linear estimators as ˆt = w i y i, where w i are constants that are fixed in advance, the HT estimator is the only estimator among the class of linear unbiased estimators. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
6 Introduction Remark (Cont d) 4 We have the following alternative definition of the linear estimator: ˆt = w i (A) y i = w ia y i where w i (A) = w ia are constants that depends on the realized sample. That is, w i (A) = w ia are random variables. 5 One advantage of linear estimator is that it is internally consistent. An estimator is internally consistent if ˆt (y 1 + y 2 ) = ˆt (y 1 ) + ˆt (y 2 ), where ˆt (y) is an estimator of the total of item y. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
7 Ratio estimation 1 Introduction 2 Ratio estimation 3 Regression estimator Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
8 Ratio estimation Ratio estimator Basic Setup : Observe x (auxiliary variable) and y (study variable) in the sample We know X = N i=1 x i or X = N 1 N i=1 x i in advance. ˆX HT = π 1 i x i can be different from X. Ratio estimator : Ŷ r = X ŶHT ˆX HT = X ˆR Ȳ r = X ŶHT ˆX HT = X ˆR Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
9 Ratio estimation Algebraic properties Linear in y (thus it is internally consistent.) If ˆX HT < X, then ŶHT < Ŷr If ˆX HT > X, then ŶHT > Ŷr If y i = x i, then the ratio estimator equals to X. That is, w i x i = X for Ŷ r = w iy i. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
10 Ratio estimation Statistical properties - Bias ( ) It is biased because E ˆR R. Bias ( of ) ˆR = ( Ŷ HT ) / ˆX HT is called the ratio bias. That is, B ˆR = E ˆR R is called the ratio bias. Ratio bias ( ) ( ) Bias ˆR = Cov ˆR, ˆX HT /X. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
11 Ratio estimation Statistical properties - Bias (Cont d) Definition: Bias of ˆθ is negligible R.B.(ˆθ) = Bias(ˆθ) 0 as n. Var(ˆθ) Note: If the bias of ˆθ is negligible, then by CLT, and ˆθ θ Var(ˆθ) = ˆθ E(ˆθ) Var(ˆθ) { MSE(ˆθ) = V (ˆθ) + = V (ˆθ). = V (ˆθ). + Bias(ˆθ) N (0, 1), Var(ˆθ) { 1 + Bias(ˆθ) } 2 [ R.B.(ˆθ) ] 2 } Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
12 Ratio estimation Statistical properties - Bias (Cont d) Ratio bias is negligible. { } 2 V R.B.( ˆR) ( ˆX HT ) X 2 = { ( )} 2 CV ˆX HT 0. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
13 Ratio estimation Statistical properties - Variance Taylor expansion Ȳ r = Ȳ + ( Ȳ HT Ȳ ) R ( XHT X ) X 1 [ ( X HT X ) ( Ȳ HT Ȳ ) R ( X HT X ) 2 ] +o p ( n 1 ) where R = X 1 Y. Variance where E i = y i Rx i. ( ) ).= 1 V (Ŷr V E i π i Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
14 Ratio estimation Remark When the ratio estimator is better than the HT estimator? Note that ) V (Ŷr Thus, ) = V (ŶHT R ˆX HT ) ( ) ( ) = V (ŶHT 2RCov ˆXHT, ŶHT + R 2 V ˆX HT V (Ŷr ) V (ŶHT ) Cov( ˆX HT, ŶHT ) V ( ˆX 1 HT ) 2 R ( ) Corr ˆX HT, Ŷ HT 1 CV ( ˆX HT ) 2 CV (Ŷ HT ) where CV (Ŷ HT ) = {V (Ŷ HT )} 1/2 /E(Ŷ HT ). Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
15 Ratio estimation Variance estimation Variance estimation : Use Êi = y i ˆRx i in the HT (or SYG) variance estimator. Example: SRS V (Ŷr ) = N2 n For variance estimation, use ˆV (Ŷr ) = N2 n ( 1 n ) 1 N N 1 ( 1 n ) 1 N n 1 N (y i Rx i ) 2 i=1 ( y i ˆRx i ) 2. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
16 Ratio estimation Application of the ratio estimator Hajek estimator: Ratio estimator of the mean using x i = 1 Domain estimation : The parameter of interest can take the form of the ratio N i=1 Ȳ d = δ iy i N i=1 δ i where δ i = 1 if i D and δ i = 0 if i / D. Thus, Ȳ HTd = π 1 i π 1 i δ i δ i y i is an (approximately) unbiased estimator of Ȳ d. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
17 1 Introduction 2 Ratio estimation 3 Regression estimator Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
18 Basic Setup Observe x i = (x 1i,, x Ji ) (auxiliary variables) and y i (study variable) in the sample We know X = N i=1 x i or X = N 1 N i=1 x i in advance. Interested in estimating Y = N i=1 y i ˆX HT = π 1 i x i can be different from X. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
19 Motivation - Difference estimator Motivation: Use auxiliary information at estimation stage Use a regression approach: 1 Suppose we have y o k = J b j x jk = b x k, k = 1, 2,, N, j=1 for some known J-dimensional vector b. The y o k is a proxy for y k. 2 Difference estimator : Ŷ diff = N i=1 y o i + y i y o i π i Unbiased (regardless of choice of y o k ) The variance is small if y o k = y k. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
20 Regression estimator How to choose y o k = b x k? - Let s estimate b from the sample. Regression estimator: Ŷ reg = N ŷ i + i=1 y i ŷ i π i, where ŷ i = ˆb x i and ˆb is estimated from the sample. Motivated from the linear regression superpopulation model E ζ (y i ) = x iβ V ζ (y i ) = σ 2. Note that β and σ 2 are superpopulation parameters. (Thus, b is the finite population quantity corresponding to β.) Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
21 Regression estimator How to estimate b? 1 Note that, under census, b can be estimated by solving U (b) N (y i b x i ) x i = 0. i=1 2 Consider an unbiased estimator of U (b): Û (b) = 1 π i (y i b x i ) x i 3 Obtain a solution ˆb by solving Û (b) = 0 for b. The solution is ( ) 1 1 ˆb = x i x 1 i x i y i. π i π i Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
22 Remark We will study that, under some conditions, Ŷ reg is asymptotically equivalent to Ŷ diff. Thus, Ŷ reg is asymptotically unbiased and V (Ŷ reg ) = V (Ŷdiff ) { } 1 = V (y i x i b) π i where ( N ) 1 N b = x i x 1 i x i y i. π i i=1 i=1 Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
23 Regression estimator Regression estimator ( ) Ŷ reg = Ŷ HT + X ˆX HT ˆb where ˆb = ( π 1 i x i x i ) 1 π 1 i x i y i. Note that, if 1 is in the column space of x i, we can write Ŷ reg = N i=1 ŷ i where ŷ i = x i ˆb. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
24 Regression estimator Regression estimator (of the mean): Ȳ reg = Ȳ HT + ( X X HT ) ˆb where ( X HT, Ȳ ) 1 ) HT = (ˆX HT N, Ŷ HT = 1 1 ( ) x N π i, y i. i Note that we can express Ŷ reg = ˆb 0 + ˆb X where ˆb 0 = Ȳ HT ˆb X HT. Thus, it is the predicted value of y at x = X when the linear regression model is used. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
25 Algebraic properties Linear in y: where Also, Ŷ reg = 1 π i g ia y i ( ( g ia = 1 + X ˆX ) 1 HT π 1 i x i x i) x i. Ȳ reg = 1 N 1 π i g ia y i. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
26 Algebraic properties Calibration property 1 π i g ia x i = X. The property (*) is also called benchmarking property. ( ) Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
27 If x i = (1, x i1 ), then Ȳ reg = Ȳ π + ( X 1 X π1 ) ˆb 1 and Ŷ reg = N {Ȳπ + ( X ) } 1 X π1 ˆb 1, where Ȳ π and X π1 are the Hajek estimators of the form ˆb 1 = [ ( X π1, Ȳ π ) = ( π 1 i and X 1 = N 1 N i=1 x i1. π 1 i ) 1 ( xi1 X π1 ) ( xi1 X π1 ) ] 1 π 1 ( ) i x i1, y i, π 1 ( ) i xi1 X π1 yi, Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
28 Weights in Ŷ reg = w iy i can be derived by minimizing Q (w) = π i ( w i 1 π i ) 2 subject to 1 π i g ia x i = X. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
29 Statistical Properties Let s consider the regression estimator of the mean We can express Ȳ reg = Ȳ HT + ( X X HT ) ˆb Ȳ reg = Ȳ HT + ( X ) ) X HT b + ( X X HT (ˆb b). = Ȳ HT + ( X ) X HT b = Ŷ diff where b = ( N i=1 x ix i) 1 N i=1 x iy i. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
30 Taylor linearization Note that Ȳ reg = Ȳ HT + ( X X HT ) ˆb is a (nonlinear) function of ( X HT, Ȳ HT, ˆb). Taylor linearization of Ȳ reg : Ŷ reg = f ( X HT, Ȳ HT, ˆb) { }. = f ( X, Ȳ, b) + Ȳ f ( X, Ȳ, b) (ȲHT Ȳ ) { } + X f ( X, Ȳ, b) ( X HT X ) { } + ˆb f ( X, Ȳ, b) (ˆb b) = Ȳ + ( Ȳ HT Ȳ ) + b ( X ) XHT + 0 Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
31 Statistical Properties (Cont d) Bias : Negligible Variance : { ).= Var (Ŷreg Var π 1 ( i yi x ib )} Variance estimation ˆV (Ŷreg ) = j A ij π ij Ê i Ê j π i π j where Ê i = y i x i ˆb. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
32 Example: SRS Variance of the regression estimator ).= 1 ( V (Ŷreg 1 n ) Se 2 n N where Se 2 = 1 N ( yi x N 1 ib ) 2. i=1 If 1 C(X ), then SST = SSR + SSE. That is, N ( yi Ȳ ) 2 N ( = y o i i=1 i=1 Ȳ ) 2 + N i=1 (y i y o i ) 2, where y o i = x i b. Thus, V (Ŷ reg ). = V (Ŷ HT )(1 R 2 ) V (Ŷ HT ). where R 2 = SSR/SST is the coefficient of determination. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
33 Remark The regression estimator is derived using a regression model. The validity (i.e. asymptotic unbiasedness) of the regression estimator does not depend on whether the regression model holds or not. However, the variance of the regression estimator is small if the regression model is good. That is, it is model-assisted, not model-dependent. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33
Nonresponse weighting adjustment using estimated response probability
Nonresponse weighting adjustment using estimated response probability Jae-kwang Kim Yonsei University, Seoul, Korea December 26, 2006 Introduction Nonresponse Unit nonresponse Item nonresponse Basic strategy
More informationChapter 3: Element sampling design: Part 1
Chapter 3: Element sampling design: Part 1 Jae-Kwang Kim Fall, 2014 Simple random sampling 1 Simple random sampling 2 SRS with replacement 3 Systematic sampling Kim Ch. 3: Element sampling design: Part
More informationChapter 5: Models used in conjunction with sampling. J. Kim, W. Fuller (ISU) Chapter 5: Models used in conjunction with sampling 1 / 70
Chapter 5: Models used in conjunction with sampling J. Kim, W. Fuller (ISU) Chapter 5: Models used in conjunction with sampling 1 / 70 Nonresponse Unit Nonresponse: weight adjustment Item Nonresponse:
More informationA measurement error model approach to small area estimation
A measurement error model approach to small area estimation Jae-kwang Kim 1 Spring, 2015 1 Joint work with Seunghwan Park and Seoyoung Kim Ouline Introduction Basic Theory Application to Korean LFS Discussion
More informationCombining data from two independent surveys: model-assisted approach
Combining data from two independent surveys: model-assisted approach Jae Kwang Kim 1 Iowa State University January 20, 2012 1 Joint work with J.N.K. Rao, Carleton University Reference Kim, J.K. and Rao,
More informationChapter 4. Replication Variance Estimation. J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 1 / 28
Chapter 4 Replication Variance Estimation J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 1 / 28 Jackknife Variance Estimation Create a new sample by deleting one observation n 1 n n ( x (k) x) 2 = x (k) = n
More informationFractional Hot Deck Imputation for Robust Inference Under Item Nonresponse in Survey Sampling
Fractional Hot Deck Imputation for Robust Inference Under Item Nonresponse in Survey Sampling Jae-Kwang Kim 1 Iowa State University June 26, 2013 1 Joint work with Shu Yang Introduction 1 Introduction
More informationAn Efficient Estimation Method for Longitudinal Surveys with Monotone Missing Data
An Efficient Estimation Method for Longitudinal Surveys with Monotone Missing Data Jae-Kwang Kim 1 Iowa State University June 28, 2012 1 Joint work with Dr. Ming Zhou (when he was a PhD student at ISU)
More informationData Integration for Big Data Analysis for finite population inference
for Big Data Analysis for finite population inference Jae-kwang Kim ISU January 23, 2018 1 / 36 What is big data? 2 / 36 Data do not speak for themselves Knowledge Reproducibility Information Intepretation
More informationCombining Non-probability and Probability Survey Samples Through Mass Imputation
Combining Non-probability and Probability Survey Samples Through Mass Imputation Jae-Kwang Kim 1 Iowa State University & KAIST October 27, 2018 1 Joint work with Seho Park, Yilin Chen, and Changbao Wu
More informationCluster Sampling 2. Chapter Introduction
Chapter 7 Cluster Sampling 7.1 Introduction In this chapter, we consider two-stage cluster sampling where the sample clusters are selected in the first stage and the sample elements are selected in the
More informationAdvanced Topics in Survey Sampling
Advanced Topics in Survey Sampling Jae-Kwang Kim Wayne A Fuller Pushpal Mukhopadhyay Department of Statistics Iowa State University World Statistics Congress Short Course July 23-24, 2015 Kim & Fuller
More informationINSTRUMENTAL-VARIABLE CALIBRATION ESTIMATION IN SURVEY SAMPLING
Statistica Sinica 24 (2014), 1001-1015 doi:http://dx.doi.org/10.5705/ss.2013.038 INSTRUMENTAL-VARIABLE CALIBRATION ESTIMATION IN SURVEY SAMPLING Seunghwan Park and Jae Kwang Kim Seoul National Univeristy
More informationCalibration estimation using exponential tilting in sample surveys
Calibration estimation using exponential tilting in sample surveys Jae Kwang Kim February 23, 2010 Abstract We consider the problem of parameter estimation with auxiliary information, where the auxiliary
More informationIntroduction to Survey Data Integration
Introduction to Survey Data Integration Jae-Kwang Kim Iowa State University May 20, 2014 Outline 1 Introduction 2 Survey Integration Examples 3 Basic Theory for Survey Integration 4 NASS application 5
More informationRegression Estimation - Least Squares and Maximum Likelihood. Dr. Frank Wood
Regression Estimation - Least Squares and Maximum Likelihood Dr. Frank Wood Least Squares Max(min)imization Function to minimize w.r.t. β 0, β 1 Q = n (Y i (β 0 + β 1 X i )) 2 i=1 Minimize this by maximizing
More informationWeighting in survey analysis under informative sampling
Jae Kwang Kim and Chris J. Skinner Weighting in survey analysis under informative sampling Article (Accepted version) (Refereed) Original citation: Kim, Jae Kwang and Skinner, Chris J. (2013) Weighting
More information6. Fractional Imputation in Survey Sampling
6. Fractional Imputation in Survey Sampling 1 Introduction Consider a finite population of N units identified by a set of indices U = {1, 2,, N} with N known. Associated with each unit i in the population
More informationMaking sense of Econometrics: Basics
Making sense of Econometrics: Basics Lecture 2: Simple Regression Egypt Scholars Economic Society Happy Eid Eid present! enter classroom at http://b.socrative.com/login/student/ room name c28efb78 Outline
More informationImputation for Missing Data under PPSWR Sampling
July 5, 2010 Beijing Imputation for Missing Data under PPSWR Sampling Guohua Zou Academy of Mathematics and Systems Science Chinese Academy of Sciences 1 23 () Outline () Imputation method under PPSWR
More informationRegression #3: Properties of OLS Estimator
Regression #3: Properties of OLS Estimator Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #3 1 / 20 Introduction In this lecture, we establish some desirable properties associated with
More informationREPLICATION VARIANCE ESTIMATION FOR THE NATIONAL RESOURCES INVENTORY
REPLICATION VARIANCE ESTIMATION FOR THE NATIONAL RESOURCES INVENTORY J.D. Opsomer, W.A. Fuller and X. Li Iowa State University, Ames, IA 50011, USA 1. Introduction Replication methods are often used in
More informationEC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix)
1 EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix) Taisuke Otsu London School of Economics Summer 2018 A.1. Summation operator (Wooldridge, App. A.1) 2 3 Summation operator For
More informationFractional Imputation in Survey Sampling: A Comparative Review
Fractional Imputation in Survey Sampling: A Comparative Review Shu Yang Jae-Kwang Kim Iowa State University Joint Statistical Meetings, August 2015 Outline Introduction Fractional imputation Features Numerical
More informationEconometrics I Lecture 3: The Simple Linear Regression Model
Econometrics I Lecture 3: The Simple Linear Regression Model Mohammad Vesal Graduate School of Management and Economics Sharif University of Technology 44716 Fall 1397 1 / 32 Outline Introduction Estimating
More informationChapters 9. Properties of Point Estimators
Chapters 9. Properties of Point Estimators Recap Target parameter, or population parameter θ. Population distribution f(x; θ). { probability function, discrete case f(x; θ) = density, continuous case The
More informationECON The Simple Regression Model
ECON 351 - The Simple Regression Model Maggie Jones 1 / 41 The Simple Regression Model Our starting point will be the simple regression model where we look at the relationship between two variables In
More informationApplied Econometrics (QEM)
Applied Econometrics (QEM) The Simple Linear Regression Model based on Prinicples of Econometrics Jakub Mućk Department of Quantitative Economics Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple
More informationHomoskedasticity. Var (u X) = σ 2. (23)
Homoskedasticity How big is the difference between the OLS estimator and the true parameter? To answer this question, we make an additional assumption called homoskedasticity: Var (u X) = σ 2. (23) This
More informationBias Variance Trade-off
Bias Variance Trade-off The mean squared error of an estimator MSE(ˆθ) = E([ˆθ θ] 2 ) Can be re-expressed MSE(ˆθ) = Var(ˆθ) + (B(ˆθ) 2 ) MSE = VAR + BIAS 2 Proof MSE(ˆθ) = E((ˆθ θ) 2 ) = E(([ˆθ E(ˆθ)]
More informationEstimation MLE-Pandemic data MLE-Financial crisis data Evaluating estimators. Estimation. September 24, STAT 151 Class 6 Slide 1
Estimation September 24, 2018 STAT 151 Class 6 Slide 1 Pandemic data Treatment outcome, X, from n = 100 patients in a pandemic: 1 = recovered and 0 = not recovered 1 1 1 0 0 0 1 1 1 0 0 1 0 1 0 0 1 1 1
More informationUnbiased Estimation. Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others.
Unbiased Estimation Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others. To compare ˆθ and θ, two estimators of θ: Say ˆθ is better than θ if it
More informationEIE6207: Estimation Theory
EIE6207: Estimation Theory Man-Wai MAK Dept. of Electronic and Information Engineering, The Hong Kong Polytechnic University enmwmak@polyu.edu.hk http://www.eie.polyu.edu.hk/ mwmak References: Steven M.
More informationarxiv:math/ v1 [math.st] 23 Jun 2004
The Annals of Statistics 2004, Vol. 32, No. 2, 766 783 DOI: 10.1214/009053604000000175 c Institute of Mathematical Statistics, 2004 arxiv:math/0406453v1 [math.st] 23 Jun 2004 FINITE SAMPLE PROPERTIES OF
More informationELEG 5633 Detection and Estimation Minimum Variance Unbiased Estimators (MVUE)
1 ELEG 5633 Detection and Estimation Minimum Variance Unbiased Estimators (MVUE) Jingxian Wu Department of Electrical Engineering University of Arkansas Outline Minimum Variance Unbiased Estimators (MVUE)
More informationIntroduction to Simple Linear Regression
Introduction to Simple Linear Regression Yang Feng http://www.stat.columbia.edu/~yangfeng Yang Feng (Columbia University) Introduction to Simple Linear Regression 1 / 68 About me Faculty in the Department
More informationEconometrics I KS. Module 2: Multivariate Linear Regression. Alexander Ahammer. This version: April 16, 2018
Econometrics I KS Module 2: Multivariate Linear Regression Alexander Ahammer Department of Economics Johannes Kepler University of Linz This version: April 16, 2018 Alexander Ahammer (JKU) Module 2: Multivariate
More informationMotivation for multiple regression
Motivation for multiple regression 1. Simple regression puts all factors other than X in u, and treats them as unobserved. Effectively the simple regression does not account for other factors. 2. The slope
More informationRegression Estimation Least Squares and Maximum Likelihood
Regression Estimation Least Squares and Maximum Likelihood Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 3, Slide 1 Least Squares Max(min)imization Function to minimize
More informationChapter 2. Section Section 2.9. J. Kim (ISU) Chapter 2 1 / 26. Design-optimal estimator under stratified random sampling
Chapter 2 Section 2.4 - Section 2.9 J. Kim (ISU) Chapter 2 1 / 26 2.4 Regression and stratification Design-optimal estimator under stratified random sampling where (Ŝxxh, Ŝxyh) ˆβ opt = ( x st, ȳ st )
More informationarxiv: v2 [math.st] 20 Jun 2014
A solution in small area estimation problems Andrius Čiginas and Tomas Rudys Vilnius University Institute of Mathematics and Informatics, LT-08663 Vilnius, Lithuania arxiv:1306.2814v2 [math.st] 20 Jun
More informationChapter 4: Imputation
Chapter 4: Imputation Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Basic Theory for imputation 3 Variance estimation after imputation 4 Replication variance estimation
More informationShu Yang and Jae Kwang Kim. Harvard University and Iowa State University
Statistica Sinica 27 (2017), 000-000 doi:https://doi.org/10.5705/ss.202016.0155 DISCUSSION: DISSECTING MULTIPLE IMPUTATION FROM A MULTI-PHASE INFERENCE PERSPECTIVE: WHAT HAPPENS WHEN GOD S, IMPUTER S AND
More informationOn the bias of the multiple-imputation variance estimator in survey sampling
J. R. Statist. Soc. B (2006) 68, Part 3, pp. 509 521 On the bias of the multiple-imputation variance estimator in survey sampling Jae Kwang Kim, Yonsei University, Seoul, Korea J. Michael Brick, Westat,
More informationThe Simple Regression Model. Simple Regression Model 1
The Simple Regression Model Simple Regression Model 1 Simple regression model: Objectives Given the model: - where y is earnings and x years of education - Or y is sales and x is spending in advertising
More informationThe regression model with one fixed regressor cont d
The regression model with one fixed regressor cont d 3150/4150 Lecture 4 Ragnar Nymoen 27 January 2012 The model with transformed variables Regression with transformed variables I References HGL Ch 2.8
More informationA note on multiple imputation for general purpose estimation
A note on multiple imputation for general purpose estimation Shu Yang Jae Kwang Kim SSC meeting June 16, 2015 Shu Yang, Jae Kwang Kim Multiple Imputation June 16, 2015 1 / 32 Introduction Basic Setup Assume
More informationParametric fractional imputation for missing data analysis
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 Biometrika (????),??,?, pp. 1 15 C???? Biometrika Trust Printed in
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationBNAD 276 Lecture 10 Simple Linear Regression Model
1 / 27 BNAD 276 Lecture 10 Simple Linear Regression Model Phuong Ho May 30, 2017 2 / 27 Outline 1 Introduction 2 3 / 27 Outline 1 Introduction 2 4 / 27 Simple Linear Regression Model Managerial decisions
More informationCovariance function estimation in Gaussian process regression
Covariance function estimation in Gaussian process regression François Bachoc Department of Statistics and Operations Research, University of Vienna WU Research Seminar - May 2015 François Bachoc Gaussian
More informationof being selected and varying such probability across strata under optimal allocation leads to increased accuracy.
5 Sampling with Unequal Probabilities Simple random sampling and systematic sampling are schemes where every unit in the population has the same chance of being selected We will now consider unequal probability
More informationEmpirical Likelihood Methods
Handbook of Statistics, Volume 29 Sample Surveys: Theory, Methods and Inference Empirical Likelihood Methods J.N.K. Rao and Changbao Wu (February 14, 2008, Final Version) 1 Likelihood-based Approaches
More informationTwo-phase sampling approach to fractional hot deck imputation
Two-phase sampling approach to fractional hot deck imputation Jongho Im 1, Jae-Kwang Kim 1 and Wayne A. Fuller 1 Abstract Hot deck imputation is popular for handling item nonresponse in survey sampling.
More informationChapter 3: Maximum Likelihood Theory
Chapter 3: Maximum Likelihood Theory Florian Pelgrin HEC September-December, 2010 Florian Pelgrin (HEC) Maximum Likelihood Theory September-December, 2010 1 / 40 1 Introduction Example 2 Maximum likelihood
More informationCalibration estimation in survey sampling
Calibration estimation in survey sampling Jae Kwang Kim Mingue Park September 8, 2009 Abstract Calibration estimation, where the sampling weights are adjusted to make certain estimators match known population
More informationIntroduction to Estimation Methods for Time Series models. Lecture 1
Introduction to Estimation Methods for Time Series models Lecture 1 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 1 SNS Pisa 1 / 19 Estimation
More informationRegression #4: Properties of OLS Estimator (Part 2)
Regression #4: Properties of OLS Estimator (Part 2) Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #4 1 / 24 Introduction In this lecture, we continue investigating properties associated
More informationThe Simple Regression Model. Part II. The Simple Regression Model
Part II The Simple Regression Model As of Sep 22, 2015 Definition 1 The Simple Regression Model Definition Estimation of the model, OLS OLS Statistics Algebraic properties Goodness-of-Fit, the R-square
More informationRecent Advances in the analysis of missing data with non-ignorable missingness
Recent Advances in the analysis of missing data with non-ignorable missingness Jae-Kwang Kim Department of Statistics, Iowa State University July 4th, 2014 1 Introduction 2 Full likelihood-based ML estimation
More informationSensitivity of GLS estimators in random effects models
of GLS estimators in random effects models Andrey L. Vasnev (University of Sydney) Tokyo, August 4, 2009 1 / 19 Plan Plan Simulation studies and estimators 2 / 19 Simulation studies Plan Simulation studies
More informationReview of Econometrics
Review of Econometrics Zheng Tian June 5th, 2017 1 The Essence of the OLS Estimation Multiple regression model involves the models as follows Y i = β 0 + β 1 X 1i + β 2 X 2i + + β k X ki + u i, i = 1,...,
More informationEconometrics Multiple Regression Analysis: Heteroskedasticity
Econometrics Multiple Regression Analysis: João Valle e Azevedo Faculdade de Economia Universidade Nova de Lisboa Spring Semester João Valle e Azevedo (FEUNL) Econometrics Lisbon, April 2011 1 / 19 Properties
More informationBootstrap inference for the finite population total under complex sampling designs
Bootstrap inference for the finite population total under complex sampling designs Zhonglei Wang (Joint work with Dr. Jae Kwang Kim) Center for Survey Statistics and Methodology Iowa State University Jan.
More informationDomain estimation under design-based models
Domain estimation under design-based models Viviana B. Lencina Departamento de Investigación, FM Universidad Nacional de Tucumán, Argentina Julio M. Singer and Heleno Bolfarine Departamento de Estatística,
More informationA comparison of stratified simple random sampling and sampling with probability proportional to size
A comparison of stratified simple random sampling and sampling with probability proportional to size Edgar Bueno Dan Hedlin Per Gösta Andersson Department of Statistics Stockholm University Introduction
More informationLecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is
Lecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is Q = (Y i β 0 β 1 X i1 β 2 X i2 β p 1 X i.p 1 ) 2, which in matrix notation is Q = (Y Xβ) (Y
More informationEconometrics I KS. Module 1: Bivariate Linear Regression. Alexander Ahammer. This version: March 12, 2018
Econometrics I KS Module 1: Bivariate Linear Regression Alexander Ahammer Department of Economics Johannes Kepler University of Linz This version: March 12, 2018 Alexander Ahammer (JKU) Module 1: Bivariate
More informationLinear Regression. Junhui Qian. October 27, 2014
Linear Regression Junhui Qian October 27, 2014 Outline The Model Estimation Ordinary Least Square Method of Moments Maximum Likelihood Estimation Properties of OLS Estimator Unbiasedness Consistency Efficiency
More informationSimple Linear Regression Analysis
LINEAR REGRESSION ANALYSIS MODULE II Lecture - 6 Simple Linear Regression Analysis Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur Prediction of values of study
More informationCombining Non-probability and. Probability Survey Samples Through Mass Imputation
Combining Non-probability and arxiv:1812.10694v2 [stat.me] 31 Dec 2018 Probability Survey Samples Through Mass Imputation Jae Kwang Kim Seho Park Yilin Chen Changbao Wu January 1, 2019 Abstract. This paper
More informationStatistics II. Management Degree Management Statistics IIDegree. Statistics II. 2 nd Sem. 2013/2014. Management Degree. Simple Linear Regression
Model 1 2 Ordinary Least Squares 3 4 Non-linearities 5 of the coefficients and their to the model We saw that econometrics studies E (Y x). More generally, we shall study regression analysis. : The regression
More informationEconomics 582 Random Effects Estimation
Economics 582 Random Effects Estimation Eric Zivot May 29, 2013 Random Effects Model Hence, the model can be re-written as = x 0 β + + [x ] = 0 (no endogeneity) [ x ] = = + x 0 β + + [x ] = 0 [ x ] = 0
More informationA Short Course in Basic Statistics
A Short Course in Basic Statistics Ian Schindler November 5, 2017 Creative commons license share and share alike BY: C 1 Descriptive Statistics 1.1 Presenting statistical data Definition 1 A statistical
More informationNon-parametric Inference and Resampling
Non-parametric Inference and Resampling Exercises by David Wozabal (Last update 3. Juni 2013) 1 Basic Facts about Rank and Order Statistics 1.1 10 students were asked about the amount of time they spend
More informationInformation in a Two-Stage Adaptive Optimal Design
Information in a Two-Stage Adaptive Optimal Design Department of Statistics, University of Missouri Designed Experiments: Recent Advances in Methods and Applications DEMA 2011 Isaac Newton Institute for
More informationThe Multiple Regression Model Estimation
Lesson 5 The Multiple Regression Model Estimation Pilar González and Susan Orbe Dpt Applied Econometrics III (Econometrics and Statistics) Pilar González and Susan Orbe OCW 2014 Lesson 5 Regression model:
More informationUnbiased Estimation. Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others.
Unbiased Estimation Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others. To compare ˆθ and θ, two estimators of θ: Say ˆθ is better than θ if it
More informationStatistics II Lesson 1. Inference on one population. Year 2009/10
Statistics II Lesson 1. Inference on one population Year 2009/10 Lesson 1. Inference on one population Contents Introduction to inference Point estimators The estimation of the mean and variance Estimating
More informationCross-validation in model-assisted estimation
Graduate Theses and Dissertations Iowa State University Capstones, Theses and Dissertations 009 Cross-validation in model-assisted estimation Lifeng You Iowa State University Follow this and additional
More informationMultiple Regression Analysis: Heteroskedasticity
Multiple Regression Analysis: Heteroskedasticity y = β 0 + β 1 x 1 + β x +... β k x k + u Read chapter 8. EE45 -Chaiyuth Punyasavatsut 1 topics 8.1 Heteroskedasticity and OLS 8. Robust estimation 8.3 Testing
More informationVariance Estimation for Calibration to Estimated Control Totals
Variance Estimation for Calibration to Estimated Control Totals Siyu Qing Coauthor with Michael D. Larsen Associate Professor of Statistics Tuesday, 11/05/2013 2 Outline A. Background B. Calibration Technique
More informationCorrelation and Regression
Correlation and Regression October 25, 2017 STAT 151 Class 9 Slide 1 Outline of Topics 1 Associations 2 Scatter plot 3 Correlation 4 Regression 5 Testing and estimation 6 Goodness-of-fit STAT 151 Class
More informationIntroduction to Econometrics
Introduction to Econometrics Lecture 3 : Regression: CEF and Simple OLS Zhaopeng Qu Business School,Nanjing University Oct 9th, 2017 Zhaopeng Qu (Nanjing University) Introduction to Econometrics Oct 9th,
More informationMain sampling techniques
Main sampling techniques ELSTAT Training Course January 23-24 2017 Martin Chevalier Department of Statistical Methods Insee 1 / 187 Main sampling techniques Outline Sampling theory Simple random sampling
More informationStatistics and Econometrics I
Statistics and Econometrics I Point Estimation Shiu-Sheng Chen Department of Economics National Taiwan University September 13, 2016 Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I September 13,
More informationRegression and Statistical Inference
Regression and Statistical Inference Walid Mnif wmnif@uwo.ca Department of Applied Mathematics The University of Western Ontario, London, Canada 1 Elements of Probability 2 Elements of Probability CDF&PDF
More informationIt should be unbiased, or approximately unbiased. Variance of the variance estimator should be small. That is, the variance estimator is stable.
Chapter 10 Variace Estimatio 10.1 Itroductio Variace estimatio is a importat practical problem i survey samplig. Variace estimates are used i two purposes. Oe is the aalytic purpose such as costructig
More informationEstimation: Part 2. Chapter GREG estimation
Chapter 9 Estmaton: Part 2 9. GREG estmaton In Chapter 8, we have seen that the regresson estmator s an effcent estmator when there s a lnear relatonshp between y and x. In ths chapter, we generalzed the
More informationSimple Regression Model Setup Estimation Inference Prediction. Model Diagnostic. Multiple Regression. Model Setup and Estimation.
Statistical Computation Math 475 Jimin Ding Department of Mathematics Washington University in St. Louis www.math.wustl.edu/ jmding/math475/index.html October 10, 2013 Ridge Part IV October 10, 2013 1
More informationMultiple Linear Regression CIVL 7012/8012
Multiple Linear Regression CIVL 7012/8012 2 Multiple Regression Analysis (MLR) Allows us to explicitly control for many factors those simultaneously affect the dependent variable This is important for
More informationLECTURE 2 LINEAR REGRESSION MODEL AND OLS
SEPTEMBER 29, 2014 LECTURE 2 LINEAR REGRESSION MODEL AND OLS Definitions A common question in econometrics is to study the effect of one group of variables X i, usually called the regressors, on another
More informationData Analysis and Machine Learning Lecture 12: Multicollinearity, Bias-Variance Trade-off, Cross-validation and Shrinkage Methods.
TheThalesians Itiseasyforphilosopherstoberichiftheychoose Data Analysis and Machine Learning Lecture 12: Multicollinearity, Bias-Variance Trade-off, Cross-validation and Shrinkage Methods Ivan Zhdankin
More informationNonparametric Regression. Badr Missaoui
Badr Missaoui Outline Kernel and local polynomial regression. Penalized regression. We are given n pairs of observations (X 1, Y 1 ),...,(X n, Y n ) where Y i = r(x i ) + ε i, i = 1,..., n and r(x) = E(Y
More informationMaster s Written Examination
Master s Written Examination Option: Statistics and Probability Spring 05 Full points may be obtained for correct answers to eight questions Each numbered question (which may have several parts) is worth
More informationGraduate Econometrics I: Unbiased Estimation
Graduate Econometrics I: Unbiased Estimation Yves Dominicy Université libre de Bruxelles Solvay Brussels School of Economics and Management ECARES Yves Dominicy Graduate Econometrics I: Unbiased Estimation
More informationRegression Models - Introduction
Regression Models - Introduction In regression models, two types of variables that are studied: A dependent variable, Y, also called response variable. It is modeled as random. An independent variable,
More informationEstimation under cross-classified sampling with application to a childhood survey
Estimation under cross-classified sampling with application to a childhood survey arxiv:1511.00507v1 [math.st] 2 Nov 2015 Hélène Juillard Guillaume Chauvet Anne Ruiz-Gazen January 11, 2018 Abstract The
More informationStatement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.
MATHEMATICAL STATISTICS Homework assignment Instructions Please turn in the homework with this cover page. You do not need to edit the solutions. Just make sure the handwriting is legible. You may discuss
More informationMeasuring the fit of the model - SSR
Measuring the fit of the model - SSR Once we ve determined our estimated regression line, we d like to know how well the model fits. How far/close are the observations to the fitted line? One way to do
More information