# In this section we derive some finite-sample properties of the OLS estimator. b is an estimator of β. It is a function of the random sample data.

Size: px
Start display at page:

Download "In this section we derive some finite-sample properties of the OLS estimator. b is an estimator of β. It is a function of the random sample data."

## Transcription

1 17 3. OLS Part III I this sectio we derive some fiite-sample properties of the OLS estimator. 3.1 The Samplig Distributio of the OLS Estimator y = Xβ + ε ; ε ~ N[0, σ 2 I ] b = (X X) 1 X y = f(y) ε is radom y is radom b is radom b is a estimator of β. It is a fuctio of the radom sample data. b is a statistic. b has a probability distributio called its Samplig Distributio. Iterpretatio of samplig distributio Repeatedly draw all possible samples of size. Calculate values of b each time. Costruct relative frequecy distributio for the b values ad probability of occurrece. It is a hypothetical costruct. Why? Samplig distributio offers oe basis for aswerig the questio: How good is b as a estimator of β? Note: Quality of estimator is beig assessed i terms of performace i repeated samples. Tells us othig about quality of estimator for oe particular sample.

2 18 Let s explore some of the properties of the LS estimator, b, ad build up its samplig distributio. Itroduce some geeral results, ad apply them to our problem. Defiitio: A estimator, θ is a ubiased estimator of the parameter vector, θ, if E[θ ] = θ. That is, E[θ (y)] = θ. That is, θ (y)p(y θ)dy = θ. The quatity, B(θ, y) = E[θ (y) θ], is called the Bias of θ. Example: fiite variace, σ 2. {y 1, y 2,, y } is a radom sample from populatio with a fiite mea, μ, ad a Cosider the statistic y = 1 i=1 y i. i=1 ) The, E[y ] = E [ 1 y i=1 i] = 1 E(y i = 1 μ = i=1 ( 1 μ ) = μ. So, y is a ubiased estimator of the parameter, μ. Here, there are lots of possible ubiased estimators of μ. So, eed to cosider additioal characteristics of estimators to help choose. Retur to our LS problem b = (X X) 1 X y Recall either assume that X is o-radom, or coditio o X. We ll assume X is o-radom get same result if we coditio o X. The: E(b) = E[(X X) 1 X y] = (X X) 1 X E(y)

3 19 So, E(b) = (X X) 1 X E[Xβ + ε] = (X X) 1 X [Xβ + E(ε)] = (X X) 1 X [Xβ + 0] = (X X) 1 X Xβ = β. The LS estimator of β is Ubiased Defiitio: Ay estimator that is a liear fuctio of the radom sample data is called a Liear Estimator. Example: {y 1, y 2,, y } is a radom sample from populatio with a fiite mea, μ, ad a fiite variace, σ 2. Cosider the statistic y = 1 y i=1 i = 1 [y 1 + y y ]. This statistic is a liear estimator of μ. (Note that the weights are o-radom.) Retur to our LS problem b = (X X) 1 X y = Ay (k 1) (k )( 1) Note that, uder our assumptios, A is a o-radom matrix. So, b 1 a 11 a 1 y 1 ( ) = [ ] ( ). b k a k1 a k y For example, b 1 = [a 11 y 1 + a 12 y a 1 y ] ; etc.

4 20 The LS estimator, b, is a liear (& ubiased) estimator of β Now let s cosider the dispersio (variability) of b, as a estimator of β. Defiitio: Suppose we have a ( 1) radom vector, x. The the Covariace Matrix of x is defied as the ( ) matrix: V(x) = E[(x E(x))(x E(x)) ]. Diagoal elemets of V(x) are var. (x 1 ),., var. (x ). Off-diagoal elemets are covar. (x i, x j ) ; i, j = 1,, ; i j. Retur to our LS problem We have a (k 1) radom vector, b, ad we kow that E(b) = β. V(b) = E[(b E(b))(b E(b)) ] Now, b = (X X) 1 X y = (X X) 1 X (Xβ + ε) = (X X) 1 (X X)β + (X X) 1 X ε = Iβ + (X X) 1 X ε. So, (b β) = (X X) 1 X ε. [*] Usig the result, [*], i V(b), we have: V(b) = E{[(X X) 1 X ε][(x X) 1 X ε] } = (X X) 1 X E[εε ]X(X X) 1. We showed, earlier, that because E(ε) = 0, V(ε) = E(εε ) = σ 2 I. (What other assumptios did we use to get this result?) So, we have:

5 21 V(b) = (X X) 1 X E[εε ]X(X X) 1 = (X X) 1 X σ 2 IX(X X) 1 = σ 2 (X X) 1 (X X)(X X) 1 = σ 2 (X X) 1. V(b) = σ 2 (X X) 1 (k k) Iterpret diagoal ad off-diagoal elemets of this matrix. Fially, because the error term, ε is assumed to be Normally distributed, 1. y = Xβ + ε : this implies that y is also Normally distributed. (Why?) 2. b = (X X) 1 X y = Ay : this implies that b is also Normally distributed. So, we ow have the full Samplig Distributio of the LS estimator, b : b ~ N[β, σ 2 (X X) 1 ] Note: This result depeds o our various, rigid, assumptios about the various compoets of the regressio model. The Normal distributio here is a multivariate Normal distributio. (See hadout o Spherical Distributios.) As with estimatio of populatio mea, μ, i previous example, there are lots of other ubiased estimators of β i the model = Xβ + ε. How might we choose betwee these possibilities? Is liearity desirable? We eed to cosider other desirable properties that these ubiased estimators may have. Oe optio is to take accout of estimators' precisios.

6 The Efficiecy of OLS Defiitio: Suppose we have two ubiased estimators, θ 1 ad θ 2, of the (scalar) parameter, θ. The we say that θ 1 is at least as efficiet as θ 2 if var. ( θ 1 ) var. ( θ 2 ). Note: 1. The variace of a estimator is just the variace of its samplig distributio. 2. "Efficiecy" is a relative cocept. 3. What if there are 3 or more ubiased estimators beig compared? What if oe or more of the estimators beig compared is biased? I this case we ca take accout of both variace, ad ay bias, at the same time by usig "mea squared error" (MSE) of the estimators. Defiitio: Suppose we have two ubiased estimators, θ 1 ad θ 2, of the parameter vector, θ. The we say that θ 1 is at least as efficiet as θ 2 if Δ = V(θ 2 ) V( θ 1) is at least positive semi-defiite. Takig accout of its liearity, ubiasedess, ad its precisio, i what sese is the LS estimator, b, of β optimal? Theorem (Gauss-Markhov): I the "stadard" liear regressio model, y = Xβ + ε, the LS estimator, b, of β is Best Liear Ubiased (BLU). That is, it is Efficiet i the class of all liear ad ubiased estimators of β. 1. Is this a iterestig result? 2. What assumptios about the "stadard" model are we goig to exploit?

7 23 Proof Let b0 be ay other liear estimator of β: b 0 = Cy ; for some o-radom C. (k 1) (k )( 1) Now, V(b 0 ) = CV(y)C = C(σ 2 I )C = σ 2 CC (k k) Defie: D = C (X X) 1 X so that Dy = Cy (X X) 1 X y = b 0 b. Now restrict b0 to be ubiased, so that E(b 0 ) = E(Cy) = CXβ = β. This requires that CX = I, which i tur implies that DX = [C (X X) 1 X ]X = CX I = 0 (ad D X = 0) (What assumptios have we used so far?) Now, focus o covariace matrix of b0 : V(b 0 ) = σ 2 [D + (X X) 1 X ][D + (X X) 1 X ] = σ 2 [DD + (X X) 1 X X(X X) 1 ] ; DX = 0 = σ 2 DD + σ 2 (X X) 1 = σ 2 DD + V(b), or, [V(b 0 ) V(b)] = σ 2 DD ; σ 2 > 0 Now we just have to "sig" this (matrix) differece: η (DD )η = (D η) (D η) = v 2 v = i=1 v i 0. So, Δ = [V(b 0 ) V(b)] is a p.s.d. matrix, implyig that b0 is relatively less efficiet tha b.

8 24 Result: The LS estimator is the Best Liear Ubiased estimator of β. What assumptios did we use, ad where? Were there ay stadard assumptios that we did't use? What does this suggest?

### Random Variables, Sampling and Estimation

Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig

### Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d

Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y

### Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.

Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized

### Lecture 3. Properties of Summary Statistics: Sampling Distribution

Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary

### STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. Comments:

Recall: STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Commets:. So far we have estimates of the parameters! 0 ad!, but have o idea how good these estimates are. Assumptio: E(Y x)! 0 +! x (liear coditioal

### The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.

SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample

### Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator

Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters

### ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors

ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic

### Estimation for Complete Data

Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of

### Solution to Chapter 2 Analytical Exercises

Nov. 25, 23, Revised Dec. 27, 23 Hayashi Ecoometrics Solutio to Chapter 2 Aalytical Exercises. For ay ε >, So, plim z =. O the other had, which meas that lim E(z =. 2. As show i the hit, Prob( z > ε =

### 1.010 Uncertainty in Engineering Fall 2008

MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval

### Lecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett

Lecture Note 8 Poit Estimators ad Poit Estimatio Methods MIT 14.30 Sprig 2006 Herma Beett Give a parameter with ukow value, the goal of poit estimatio is to use a sample to compute a umber that represets

### Asymptotic Results for the Linear Regression Model

Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is

### Unbiased Estimation. February 7-12, 2008

Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom

### Properties and Hypothesis Testing

Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.

### MA Advanced Econometrics: Properties of Least Squares Estimators

MA Advaced Ecoometrics: Properties of Least Squares Estimators Karl Whela School of Ecoomics, UCD February 5, 20 Karl Whela UCD Least Squares Estimators February 5, 20 / 5 Part I Least Squares: Some Fiite-Sample

### Estimation of the Mean and the ACVF

Chapter 5 Estimatio of the Mea ad the ACVF A statioary process {X t } is characterized by its mea ad its autocovariace fuctio γ ), ad so by the autocorrelatio fuctio ρ ) I this chapter we preset the estimators

### Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.

Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator

### CEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering

CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio

### MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4

MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.

### Lecture 33: Bootstrap

Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece

### CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5

CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio

### MBACATÓLICA. Quantitative Methods. Faculdade de Ciências Económicas e Empresariais UNIVERSIDADE CATÓLICA PORTUGUESA 9. SAMPLING DISTRIBUTIONS

MBACATÓLICA Quatitative Methods Miguel Gouveia Mauel Leite Moteiro Faculdade de Ciêcias Ecoómicas e Empresariais UNIVERSIDADE CATÓLICA PORTUGUESA 9. SAMPLING DISTRIBUTIONS MBACatólica 006/07 Métodos Quatitativos

### Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator

Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday

Ecoomics 883 Sprig 205 Tauche Tradig Frictio Noise Setup Let Y be the usual cotiuous semi-martigale Y t = t 0 cs dw s We will cosider jump discotiuities later. The usual setup for modelig tradig frictio

### Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara

Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio

### Efficient GMM LECTURE 12 GMM II

DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet

### Linear Regression Models, OLS, Assumptions and Properties

Chapter 2 Liear Regressio Models, OLS, Assumptios ad Properties 2.1 The Liear Regressio Model The liear regressio model is the sigle most useful tool i the ecoometricia s kit. The multiple regressio model

### Element sampling: Part 2

Chapter 4 Elemet samplig: Part 2 4.1 Itroductio We ow cosider uequal probability samplig desigs which is very popular i practice. I the uequal probability samplig, we ca improve the efficiecy of the resultig

### Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.

Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio

### Understanding Samples

1 Will Moroe CS 109 Samplig ad Bootstrappig Lecture Notes #17 August 2, 2017 Based o a hadout by Chris Piech I this chapter we are goig to talk about statistics calculated o samples from a populatio. We

### 7.1 Convergence of sequences of random variables

Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

### 1 Inferential Methods for Correlation and Regression Analysis

1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet

### Maximum Likelihood Estimation

ECE 645: Estimatio Theory Sprig 2015 Istructor: Prof. Staley H. Cha Maximum Likelihood Estimatio (LaTeX prepared by Shaobo Fag) April 14, 2015 This lecture ote is based o ECE 645(Sprig 2015) by Prof. Staley

### Lecture 12: September 27

36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.

### Convergence of random variables. (telegram style notes) P.J.C. Spreij

Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

### Classical Linear Regression Model. Normality Assumption Hypothesis Testing Under Normality Maximum Likelihood Estimator Generalized Least Squares

Classical Liear Regressio Model Normality Assumptio Hypothesis Testig Uder Normality Maximum Likelihood Estimator Geeralized Least Squares Normality Assumptio Assumptio 5 e X ~ N(,s I ) Implicatios of

### 11 THE GMM ESTIMATION

Cotets THE GMM ESTIMATION 2. Cosistecy ad Asymptotic Normality..................... 3.2 Regularity Coditios ad Idetificatio..................... 4.3 The GMM Iterpretatio of the OLS Estimatio.................

### 17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15

17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig

### Lecture 19: Convergence

Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

### Hypothesis Testing. Evaluation of Performance of Learned h. Issues. Trade-off Between Bias and Variance

Hypothesis Testig Empirically evaluatig accuracy of hypotheses: importat activity i ML. Three questios: Give observed accuracy over a sample set, how well does this estimate apply over additioal samples?

### ARIMA Models. Dan Saunders. y t = φy t 1 + ɛ t

ARIMA Models Da Sauders I will discuss models with a depedet variable y t, a potetially edogeous error term ɛ t, ad a exogeous error term η t, each with a subscript t deotig time. With just these three

### Stat410 Probability and Statistics II (F16)

Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems

### Lecture 22: Review for Exam 2. 1 Basic Model Assumptions (without Gaussian Noise)

Lecture 22: Review for Exam 2 Basic Model Assumptios (without Gaussia Noise) We model oe cotiuous respose variable Y, as a liear fuctio of p umerical predictors, plus oise: Y = β 0 + β X +... β p X p +

### Linear Regression Demystified

Liear Regressio Demystified Liear regressio is a importat subject i statistics. I elemetary statistics courses, formulae related to liear regressio are ofte stated without derivatio. This ote iteds to

### Stat 421-SP2012 Interval Estimation Section

Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible

### Chapter 22. Comparing Two Proportions. Copyright 2010 Pearson Education, Inc.

Chapter 22 Comparig Two Proportios Copyright 2010 Pearso Educatio, Ic. Comparig Two Proportios Comparisos betwee two percetages are much more commo tha questios about isolated percetages. Ad they are more

### Chapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities

Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other

### First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise

First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >

### January 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS

Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we

### Parameter, Statistic and Random Samples

Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,

### Topic 9: Sampling Distributions of Estimators

Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

### Questions and Answers on Maximum Likelihood

Questios ad Aswers o Maximum Likelihood L. Magee Fall, 2008 1. Give: a observatio-specific log likelihood fuctio l i (θ) = l f(y i x i, θ) the log likelihood fuctio l(θ y, X) = l i(θ) a data set (x i,

### MCT242: Electronic Instrumentation Lecture 2: Instrumentation Definitions

Faculty of Egieerig MCT242: Electroic Istrumetatio Lecture 2: Istrumetatio Defiitios Overview Measuremet Error Accuracy Precisio ad Mea Resolutio Mea Variace ad Stadard deviatio Fiesse Sesitivity Rage

### Expectation and Variance of a random variable

Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio

### 7.1 Convergence of sequences of random variables

Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

### Notes 27 : Brownian motion: path properties

Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X

### Learning Theory: Lecture Notes

Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic

### Probability and Statistics

ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,

### Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.

Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,

### Random Signals and Noise Winter Semester 2017 Problem Set 12 Wiener Filter Continuation

Radom Sigals ad Noise Witer Semester 7 Problem Set Wieer Filter Cotiuatio Problem (Sprig, Exam A) Give is the sigal W t, which is a Gaussia white oise with expectatio zero ad power spectral desity fuctio

### ECE 901 Lecture 12: Complexity Regularization and the Squared Loss

ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality

### 1 Covariance Estimation

Eco 75 Lecture 5 Covariace Estimatio ad Optimal Weightig Matrices I this lecture, we cosider estimatio of the asymptotic covariace matrix B B of the extremum estimator b : Covariace Estimatio Lemma 4.

### Large Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution

Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,

### A General Family of Estimators for Estimating Population Variance Using Known Value of Some Population Parameter(s)

Rajesh Sigh, Pakaj Chauha, Nirmala Sawa School of Statistics, DAVV, Idore (M.P.), Idia Floreti Smaradache Uiversity of New Meico, USA A Geeral Family of Estimators for Estimatig Populatio Variace Usig

### Chapter 22. Comparing Two Proportions. Copyright 2010, 2007, 2004 Pearson Education, Inc.

Chapter 22 Comparig Two Proportios Copyright 2010, 2007, 2004 Pearso Educatio, Ic. Comparig Two Proportios Read the first two paragraphs of pg 504. Comparisos betwee two percetages are much more commo

### 18.S096: Homework Problem Set 1 (revised)

8.S096: Homework Problem Set (revised) Topics i Mathematics of Data Sciece (Fall 05) Afoso S. Badeira Due o October 6, 05 Exteded to: October 8, 05 This homework problem set is due o October 6, at the

### IE 230 Seat # Name < KEY > Please read these directions. Closed book and notes. 60 minutes.

IE 230 Seat # Name < KEY > Please read these directios. Closed book ad otes. 60 miutes. Covers through the ormal distributio, Sectio 4.7 of Motgomery ad Ruger, fourth editio. Cover page ad four pages of

### The central limit theorem for Student s distribution. Problem Karim M. Abadir and Jan R. Magnus. Econometric Theory, 19, 1195 (2003)

The cetral limit theorem for Studet s distributio Problem 03.6.1 Karim M. Abadir ad Ja R. Magus Ecoometric Theory, 19, 1195 (003) Z Ecoometric Theory, 19, 003, 1195 1198+ Prited i the Uited States of America+

### Statisticians use the word population to refer the total number of (potential) observations under consideration

6 Samplig Distributios Statisticias use the word populatio to refer the total umber of (potetial) observatios uder cosideratio The populatio is just the set of all possible outcomes i our sample space

### Review Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn

Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom

### Maximum Likelihood Methods (Hogg Chapter Six)

Maximum Likelihood Methods Hogg Chapter ix TAT 406-0: Mathematical tatistics II prig emester 06 Cotets 0 Admiistrata Maximum Likelihood Estimatio. Maximum Likelihood Estimates............ Motivatio....................

### Chapter 6 Principles of Data Reduction

Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a

### Statistical inference: example 1. Inferential Statistics

Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either

### Solutions: Homework 3

Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe

### Big Picture. 5. Data, Estimates, and Models: quantifying the accuracy of estimates.

5. Data, Estimates, ad Models: quatifyig the accuracy of estimates. 5. Estimatig a Normal Mea 5.2 The Distributio of the Normal Sample Mea 5.3 Normal data, cofidece iterval for, kow 5.4 Normal data, cofidece

### CLRM estimation Pietro Coretto Econometrics

Slide Set 4 CLRM estimatio Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Thursday 24 th Jauary, 2019 (h08:41) P. Coretto

### EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

### Machine Learning Brett Bernstein

Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio

### Simple Random Sampling!

Simple Radom Samplig! Professor Ro Fricker! Naval Postgraduate School! Moterey, Califoria! Readig:! 3/26/13 Scheaffer et al. chapter 4! 1 Goals for this Lecture! Defie simple radom samplig (SRS) ad discuss

### Lecture 7: Properties of Random Samples

Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ

### NANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS

NANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS STRUCTURE OF EXAMINATION PAPER. There will be oe 2-hour paper cosistig of 4 questios.

### Chapter 13: Tests of Hypothesis Section 13.1 Introduction

Chapter 13: Tests of Hypothesis Sectio 13.1 Itroductio RECAP: Chapter 1 discussed the Likelihood Ratio Method as a geeral approach to fid good test procedures. Testig for the Normal Mea Example, discussed

### Double Stage Shrinkage Estimator of Two Parameters. Generalized Exponential Distribution

Iteratioal Mathematical Forum, Vol., 3, o. 3, 3-53 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/.9/imf.3.335 Double Stage Shrikage Estimator of Two Parameters Geeralized Expoetial Distributio Alaa M.

### Lecture 8: Non-parametric Comparison of Location. GENOME 560, Spring 2016 Doug Fowler, GS

Lecture 8: No-parametric Compariso of Locatio GENOME 560, Sprig 2016 Doug Fowler, GS (dfowler@uw.edu) 1 Review What do we mea by oparametric? What is a desirable locatio statistic for ordial data? What

### This is an introductory course in Analysis of Variance and Design of Experiments.

1 Notes for M 384E, Wedesday, Jauary 21, 2009 (Please ote: I will ot pass out hard-copy class otes i future classes. If there are writte class otes, they will be posted o the web by the ight before class

### LINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity

LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 9 Multicolliearity Dr Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Multicolliearity diagostics A importat questio that

### Correlation Regression

Correlatio Regressio While correlatio methods measure the stregth of a liear relatioship betwee two variables, we might wish to go a little further: How much does oe variable chage for a give chage i aother

### 32 estimating the cumulative distribution function

32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio

### Lecture 2: Monte Carlo Simulation

STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?

### Section 14. Simple linear regression.

Sectio 14 Simple liear regressio. Let us look at the cigarette dataset from [1] (available to dowload from joural s website) ad []. The cigarette dataset cotais measuremets of tar, icotie, weight ad carbo

### CEU Department of Economics Econometrics 1, Problem Set 1 - Solutions

CEU Departmet of Ecoomics Ecoometrics, Problem Set - Solutios Part A. Exogeeity - edogeeity The liear coditioal expectatio (CE) model has the followig form: We would like to estimate the effect of some

### Lecture 20: Multivariate convergence and the Central Limit Theorem

Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties

### FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures

FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals

### It should be unbiased, or approximately unbiased. Variance of the variance estimator should be small. That is, the variance estimator is stable.

Chapter 10 Variace Estimatio 10.1 Itroductio Variace estimatio is a importat practical problem i survey samplig. Variace estimates are used i two purposes. Oe is the aalytic purpose such as costructig

### Since X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain

Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the

### Direction: This test is worth 250 points. You are required to complete this test within 50 minutes.

Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely