Linear Regression Models, OLS, Assumptions and Properties

Size: px
Start display at page:

Download "Linear Regression Models, OLS, Assumptions and Properties"

Transcription

1 Chapter 2 Liear Regressio Models, OLS, Assumptios ad Properties 2.1 The Liear Regressio Model The liear regressio model is the sigle most useful tool i the ecoometricia s kit. The multiple regressio model is the study if the relatioship betwee a depedet variable ad oe or more idepedet variables. I geeral it ca be writte as: y = f(x 1,x 2,...,x K )+ε (2.1) = x 1 β 1 + x 2 β 2 + +x K β K + ε. The radom disturbace ε arises because we caot capture every ifluece o a ecoomic variable i a model. The et effect of the omitted variables is captured by the disturbace. Notice that y is the summatio of two parts, the determiistic part ad the radom part, ε. The objective is to estimate the ukow parameters (β 1, β 2,..., β K ) i this model. 2.2 Assumptios The classical liear regressio model cosist of a set of assumptios how a data set will be produced by the uderlyig data-geeratig process. The assumptios are: A1. Liearity A2. Full rak A3. Exogeeity of the idepedet variables A4. Homoscedasticity ad oautocorrelatio A5. Data geeratio A6. Normal distributio 5

2 6 2 Liear Regressio Models, OLS, Assumptios ad Properties Liearity The model specifies a liear relatioship betwee y ad x 1,x 2,...,x K. The colum vector x k cotais the observatios of the variable x k, k = 1,2,...K, which ca be writte i a sigle K data matrix X. Whe the model is estimated with a costat term, the first colum of X is assumed to be a colums oe oes, makig β 1 the coefficiet associated with the costat of the model. The vector y cotais the observatios, y 1,y 2,...,y, ad the vector ε cotais all the disturbaces. The model ca be represeted as: y 1 1 x 11 x x K1 β 1 ε 1 y 2 1 x 12 x x K2 β 2 ε 2. y 1 = x 1 x 2... x K I matrix form the model ca be writte as: K. β K K 1 +. ε K 1 ASSUMPTION 1 : y=xβ + ε (2.2) Notice that the assumptio meas that Equatio 2.2 ca be either i terms of the origial variables or after some trasformatio. For example, cosider the followig two equatios: y = Ax β e ε y = Ax β + ε. While the first equatio ca be made liear by takig logs, the secod equatio is ot liear. Typical examples iclude the costat elasticity model: ly = β 1 + β 2 lx 2 + β 3 x 3 + +β K x K + ε where the elasticity betwee y ad x k is give by ly/ lx k = β k. Aother commo model is the semilog Full rak Full rak or idetificatio coditio meas that there are o exact liear relatioships betwee the variables. ASSUMPTION 2 : X is a K matrix with rak K. (2.3) X is a full colum rak meas that the colums of X are liearly idepedet ad that there are at least K observatios.

3 2.2 Assumptios Exogeeity of the idepedet variables Exogeeity of the idepedet variables meas that each of the disturbace terms is assumed to have zero expected value. This ca be writte as: ASSUMPTION 3 : E[ε X] = 0. (2.4) This assumptio meas that the disturbaces are purely radom draws from some populatio ad that o observatio o x covey iformatio about the expected value of the disturbace (ε ad X are ucorrelated). Give Assumptio 3, Xβ is the coditioal mea fuctio because Assumptio 3 implies that: E[y X]=Xβ. (2.5) Homoscedasticity ad oautocorrelatio The combiatio of homoscedasticity ad oautocorrelatio is also kow as spherical disturbaces ad it refers to the variaces ad covariaces of the disturbaces. Homoscedasticity meas costat variace ad ca be writte as: Var[ε i X]=σ 2 for all,2,,, Cov[ε i,ε j X]=0 for all i j. Typical examples of regressio models with heteroscedastic errors are household expeditures ad firms profits. Autocorrelatio (or serial correlatio) o the other side, meas just correlatio betwee the errors i differet time periods. Hece, autocorrelatio is usually a problem with time series or pael data. σ E[εε 0 σ X]= σ 2 which ca be writte as: ASSUMPTION 4 : E[εε X]=σ 2 I. (2.6)

4 8 2 Liear Regressio Models, OLS, Assumptios ad Properties Data geeratio It is mathematically coveiet to assume x i is ostochastic, like i a agricultural experimet where y i is yield ad x i is the fertilizer ad water applied. However, social scietist are very likely to fid stochastic x i. The assumptio we will use is that X ca be a mixture of costat ad radom variables, ad the mea ad variace of ε i are both idepedet of all elemets of X. ASSUMPTION 5 : X may be fixed or radom. (2.7) Normal distributio The last assumptio, which is coveiet, but ot ecessary to obtai may of the results of the liear regressio model is that the residuals follow a ormal distributio with zero mea ad costat variace. That is addig ormality to Assumptios 3 ad 4. ASSUMPTION 6 : ε X N[0,σ 2 I]. (2.8) The assumptios of the liear regressio model are summarized i Figure 2.1. Fig. 2.1 Classical Regressio Model, from [Greee (2008)].

5 2.3 Ordiary Least Squares Regressio Ordiary Least Squares Regressio The first distictio eeded at this poit is betwee populatio parameters ad sample estimates. From the previous discussio we have β ad ε i as populatio parameters, hece we use b ad e i as their sample estimates. For the populatio regressio we have E[y i x i ]=x iβ, however β is ukow ad we use its estimate b. Therefore we have: E[y i x i ]=ŷ i = x ib. (2.9) For observatio i, the (populatio) disturbace term is give by: ε i = y i x iβ. (2.10) Oce we estimate b the estimate of the disturbace term ε i is its sample couterpart, the residual: 1 e i = y i x ib. (2.11) It follows that y i = x iβ + ε i = x ib+e i. (2.12) A graphical summary of this discussio is preseted i Figure 2.2. This figure shows the simple example of a sigle regressor Least Squares Coefficiets The problem i had is to obtai a estimate of the ukow populatio vector β based o the sample data (y i, x i ) for,2,,. I this sectio we will derive the least squares estimator vector for β, deoted by b. By defiitio, the least squares coefficiet vector miimizes the sum of squared residuals: e 2 i0 = (y i x ib 0 ) 2. (2.13) The idea is to pick the vector b 0 that makes the summatio i Equatio 2.13 the smallest. I matrix otatio: mi b 0 S(b 0 ) = e 0e 0 =(y Xb 0 ) (y Xb 0 ). (2.14) mi b 0 S(b 0 ) = y y b 0X y y Xb 0 + b 0X Xb 0 mi b 0 S(b 0 ) = y 2y Xb 0 + b 0X Xb 0. 1 [Dougherty (2007)] follows a similar otatio, but most textbooks, e.g. [Wooldridge (2009)], use ˆβ as the sample estimate of β.

6 10 2 Liear Regressio Models, OLS, Assumptios ad Properties Fig. 2.2 Populatio ad Sample Regressio, from [Greee (2008)]. The first order ecessary coditio is: S(b 0 ) b 0 = 2X y+2xxb 0 = 0. (2.15) Let b be the solutio. The, give that X is full rak, (X X) 1 exists ad the solutio is: b=(x X) 1 X y. (2.16) The secod order coditio is: 2 S(b 0 ) b 0 b = 2X y+2xxb 0 = 0. (2.17) 0 That is satisfied if it yields a positive defiite matrix. This will be the case if X is full rak, the the least squares solutio b is uique ad miimizes the sum of squared residuals. Example 1 Derivatio of the least squares coefficiet estimators for the simple case of a sigle regressor ad a costat. y i = b 0 + b 1 x i + e i (2.18)

7 2.3 Ordiary Least Squares Regressio 11 ŷ i = b 0 + b 1 x i For observatio i we obtai the residual, the square it ad fially sum across all observatios to obtai the sum of squared residuals: e i = y i ŷ i (2.19) e 2 i = (y i ŷ i ) 2 e 2 i = (y i ŷ i ) 2 Agai, the coefficiets b 0 ad b 1 are chose to miimize the sum of squared residuals: mi b 0,b 1 (y i ŷ i ) 2 (2.20) mi b 0,b (y i b 0 b 1 x i ) 2 1 The first order ecessary coditio are: 2 2 (y i b 0 b 1 x i ) = 0 w.r.t. b 0 (2.21) x i (y i b 0 b 1 x i ) = 0 w.r.t. b 1 (2.22) Dividig Equatio 2.22 by ad workig through some math we obtai the OLS estimators for the costat: Pluggig this result ito Equatio 2.22 we obtai: b 0 = ȳ b 1 x. (2.23) b 1 = i=0 (x i x)(y i ȳ) i=0 (x i x) 2. (2.24) Normal equatios From the first order coditios i Equatio 2.15 we ca obtai the ormal equatios: X Xb X y= X (y Xb)= X e=0. (2.25)

8 12 2 Liear Regressio Models, OLS, Assumptios ad Properties Therefore, followig X e = 0 we ca derive a umber of properties: 1. The observed values of X are ucorrelated with the residuals. For every colum of X, x k e = 0. I additio, if the regressio icludes a costat: 2. The sum of the residuals is zero. x 1 e= i e = i e i = The sample mea of the residuals is zero. ē= i e i = The regressio hyperplae passes through the meas of the data. This follows from ē = 0. Recall that e = y Xb. Dividig by, we have ē = ȳ xb. This implies that ȳ = x b. 5. The predicted values of y are ucorrelated with the residuals. ŷ e = (Xb) e = b X e = The mea of the fitted values is equal to the mea of the actual values. Because y = ŷ + e. We have i e i = 0, the ŷ = ȳ Projectio matrix The matrix M (residual maker) is fudametal i regressio aalysis. It is give by: M=I X(X X) 1 X. (2.26) It geerates the vector of least square residuals i a regressio of y o X whe it premultiplies ay vector y. It ca be easily derived from the least square residuals: e = y Xb (2.27) = y X(X X) 1 X y = (I X(X X) 1 X )y = My. M is a symmetric (M = M ) ad idempotet (M = M 2 ) matrix. For example, it is useful to see that if we regress X o X we have a perfect fit ad the residuals should be zero: MX=0. (2.28) The projectio matrix P is also a symmetric ad idempotet matrix formed from X. Whe y is premultiplied by P, it results o the the fitted values ŷ i the regressio of y o X. I is give by: P=X(X X) 1 X. (2.29) It ca be obtaied by startig from the equatio y=xb+e. We kow that ŷ=xb, the y=ŷ+ethat gives: ŷ = y e (2.30) = y My

9 2.3 Ordiary Least Squares Regressio 13 = (I M)y = X(X X) 1 X y = Py. Notice that M = I P. I additio, M ad P are orthogoal ad i a regressio of X o X, the fitted values are also X, that is, PX = X. Followig from the results of these M ad P matrices, we ca see that the least squares regressio partitios the vector y ito two orthogoal parts, the projectio ad the residual, y=py+my. (2.31) Goodess of fit ad aalysis of variace The variatio of the depedet variable is captured i terms of deviatios from its mea, y i ȳ. The the total variatio i y is the sum of square deviatios: SST= (y i ȳ) 2. (2.32) To decompose this sum of square deviatios ito the part the regressio model explai ad the part the model does ot explai, we first look at a sigle observatio to get some ituitio. For observatio i we have: Subtractig ȳ we obtai: y i = ŷ i + e i = x ib+e i. (2.33) y i ȳ=ŷ i ȳ+e i =(x i x)b+e i. (2.34) Figure 2.3 illustrates the ituitio for the case of a sigle regressor. Let the symmetric ad idempotet matrix M 0 have (1 1/) i all its diagoal elemets ad 1/ i all its off-diagoal elemets: [ M 0 = I 1 ii ] = M 0 trasforms observatios ito deviatios from sample meas. The, M 0 is useful i computig sum of square deviatios. For example, the sum of squared deviatios about the mea for x i is: (x i x) 2 = x M 0 x. (2.35)

10 14 2 Liear Regressio Models, OLS, Assumptios ad Properties Fig. 2.3 Decompositio of y i, from [Greee (2008)]. Now, if we start with y = Xb + e ad premultiply it by M 0 we obtai: The, we traspose this equatio to obtai: Premultiply Equatio 2.36 by Equatio 2.37: M 0 y=m 0 Xb+M 0 e (2.36) y M 0 = b X M 0 + e M 0 (2.37) y M 0 y = (b X M 0 + e M 0 )(M 0 Xb+M 0 e) (2.38) = b X M 0 Xb+b X M 0 e+e M 0 Xb+e M 0 e = b X M 0 Xb+e e. The secod term o the right-had-side i the last lie is zero because M 0 e=ead X e = 0, while the third term is zero because e M 0 X = e X = 0 (the regressors are orthogoal to the residuals). Equatio 2.38 show the decompositio of the total sum of squares ito regressio sum of squares plus the error sum of squares: SST=SSR+SSE (2.39)

11 2.4 Properties of OLS 15 If we calculate the fractio of the total variatio i y that is explaied by the model, we are talkig about the coefficiet of determiatio, R 2 : R 2 = SSR SST = b X M 0 Xb y M 0 = 1 e e y y M 0 y (2.40) As we iclude variables ito the model the R 2 will ever decrease. Hece, for small samples, a better measure of fit is the adjusted R 2 or R 2 : R 2 = 1 e e/( K) y M 0 y/( 1) (2.41) 2.4 Properties of OLS Ubiasedess The least square estimator b is ubiased for every value of. b = (X X) 1 X y (2.42) = (X X) 1 X (Xβ + ε) = β +(X X) 1 X ε E[b X] = β + E[(X X) 1 X ε] = β. The secod term after takig expectatios is zero because the errors are assumed to be orthogoal to the regressio residuals Variace ad the Gauss-Markov Theorem It is relatively simple to derive the samplig variace of the OLS estimators. However, the key assumptio i the derivatio is that the matrix X is costat. If X is ot costat, the the expectatios should be take coditioal o the observed X. From the derivatio of the ubiasedess of OLS we have that b β =(X X) 1 X ε. Usig this i the variace-covariace matrix of the OLS we have: Var[b X] = E[(b β)(b β) X] = E[((X X) 1 X ε)((x X) 1 X ε) X] = E[((X X) 1 X ε)(ε X(X X) 1 ) X] = (X X) 1 X E[εε X]X(X X) 1

12 16 2 Liear Regressio Models, OLS, Assumptios ad Properties = (X X) 1 X (σ 2 I)X(X X) 1 = σ 2 (X X) 1 (2.43) Gauss-Markov Theorem. I a liear regressio model (with spherical disturbaces), the Best Liear Ubiased Estimator (BLUE) of the coefficiets is the ordiary least squares estimator. I the Gauss-Markov Theorem, best refers to miimum variace. I additio, errors do ot eed to have a ormal distributio ad the X could be either stochastic or ostochastic Estimatig the Variace I order to obtai a sample estimate of the variace-covariace matrix preseted i Equatio2.43, we eed a estimate of the populatio parameter σ 2. We ca use: ˆσ 2 = 1 e 2 i, (2.44) which makes sese because e i is the sample estimate of ε i, ad σ 2 is the expected value of εi 2. However, this oe is biased because β is ot observed directly. To obtai a ubiased estimator of σ 2 we ca start with the expected value of the sum of squared residuals. Recall that e = My = M[Xβ + ε] = M ε. The, the sum of squared residuals is: e e = ε Mε (2.45) E[e e X] = E[ε Mε X] = E[tr(ε Mε) X] = E[tr(Mεε ) X] = tr(me[εε X]) = tr(mσ 2 I) = σ 2 tr(m) = σ 2 tr(i X(X X) 1 X ) = σ 2 [tr(i ) tr(x(x X) 1 X )] = σ 2 [tr(i ) tr(i K )] = σ 2 ( K), where ε M ε is a scalar (1 1 matrix), so it it equal to its trace, ad the operatio from the third to the fourth lie follows from the results o cyclic permutatios. From Equatio 2.45 we obtai the ubiased estimator of σ 2 :

13 2.4 Properties of OLS 17 s 2 = e e K. (2.46) Hece, the stadard errors of the estimators b ca be obtaied by first obtaiig a estimate of σ 2 usig Equatio 2.46 ad the pluggig s 2 ito Equatio Statistical Iferece Give that b is a liear fuctio of ε, if ε has a multivariate ormal distributio we have that: b X N[β,σ 2 (X X) 1 ]. (2.47) Hypothesis Testig Assumig ormality coditioal o X ad with S kk beig the kth diagoal elemet of X X 1 we have that: z k = b k β k σ 2 S kk (2.48) has a stadard ormal distributio. However, σ 2 is a ukow populatio parameter. Hece, we use: t k = b k β k (2.49) s 2 S kk that has a t distributio with ( K) degrees of freedom. We use Equatio 2.49 for hypotheses testig about the elemets of β Cofidece Itervals Based o Equatio 2.49 we ca obtai the (1 α) cofidece iterval for the populatio parameter β k usig: P(b k t α/2, K s bk β k b k +t α/2, K s bk )=1 α. (2.50) What this equatio is sayig is that the true populatio parameter β k will lie betwee the lower cofidece iterval b k t α/2, K s bk ad the upper cofidece iterval b k + t α/2, K s bk i (1 α)% of the times. Moreover, t α/2, K is the critical value from the t distributio with ( K) degrees of freedom. This is illustrated i Figure 2.4.

14 18 2 Liear Regressio Models, OLS, Assumptios ad Properties Fig. 2.4 Cofidece Itervals. Refereces [Dougherty (2007)] Dougherty, C., Itroductio to Ecoometrics. 3rd ed. New York: Oxford Uiversity Press. [Greee (2008)] Greee, W.H Ecoometric Aalysis. 6th ed. New Jersey: Pearso Pretice Hall. [Wooldridge (2009)] Wooldridge, J.M., Itroductory Ecoometrics: A Models Approach. 4th ed. New York: South-Wester Publishers.

Properties and Hypothesis Testing

Properties and Hypothesis Testing Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.

More information

Simple Linear Regression

Simple Linear Regression Chapter 2 Simple Liear Regressio 2.1 Simple liear model The simple liear regressio model shows how oe kow depedet variable is determied by a sigle explaatory variable (regressor). Is is writte as: Y i

More information

Algebra of Least Squares

Algebra of Least Squares October 19, 2018 Algebra of Least Squares Geometry of Least Squares Recall that out data is like a table [Y X] where Y collects observatios o the depedet variable Y ad X collects observatios o the k-dimesioal

More information

CLRM estimation Pietro Coretto Econometrics

CLRM estimation Pietro Coretto Econometrics Slide Set 4 CLRM estimatio Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Thursday 24 th Jauary, 2019 (h08:41) P. Coretto

More information

Geometry of LS. LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT

Geometry of LS. LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT OCTOBER 7, 2016 LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT Geometry of LS We ca thik of y ad the colums of X as members of the -dimesioal Euclidea space R Oe ca

More information

First, note that the LS residuals are orthogonal to the regressors. X Xb X y = 0 ( normal equations ; (k 1) ) So,

First, note that the LS residuals are orthogonal to the regressors. X Xb X y = 0 ( normal equations ; (k 1) ) So, 0 2. OLS Part II The OLS residuals are orthogoal to the regressors. If the model icludes a itercept, the orthogoality of the residuals ad regressors gives rise to three results, which have limited practical

More information

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y

More information

ECON 3150/4150, Spring term Lecture 3

ECON 3150/4150, Spring term Lecture 3 Itroductio Fidig the best fit by regressio Residuals ad R-sq Regressio ad causality Summary ad ext step ECON 3150/4150, Sprig term 2014. Lecture 3 Ragar Nymoe Uiversity of Oslo 21 Jauary 2014 1 / 30 Itroductio

More information

Random Variables, Sampling and Estimation

Random Variables, Sampling and Estimation Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig

More information

(all terms are scalars).the minimization is clearer in sum notation:

(all terms are scalars).the minimization is clearer in sum notation: 7 Multiple liear regressio: with predictors) Depedet data set: y i i = 1, oe predictad, predictors x i,k i = 1,, k = 1, ' The forecast equatio is ŷ i = b + Use matrix otatio: k =1 b k x ik Y = y 1 y 1

More information

Statistical Properties of OLS estimators

Statistical Properties of OLS estimators 1 Statistical Properties of OLS estimators Liear Model: Y i = β 0 + β 1 X i + u i OLS estimators: β 0 = Y β 1X β 1 = Best Liear Ubiased Estimator (BLUE) Liear Estimator: β 0 ad β 1 are liear fuctio of

More information

Chapter 1 Simple Linear Regression (part 6: matrix version)

Chapter 1 Simple Linear Regression (part 6: matrix version) Chapter Simple Liear Regressio (part 6: matrix versio) Overview Simple liear regressio model: respose variable Y, a sigle idepedet variable X Y β 0 + β X + ε Multiple liear regressio model: respose Y,

More information

Linear Regression Models

Linear Regression Models Liear Regressio Models Dr. Joh Mellor-Crummey Departmet of Computer Sciece Rice Uiversity johmc@cs.rice.edu COMP 528 Lecture 9 15 February 2005 Goals for Today Uderstad how to Use scatter diagrams to ispect

More information

1 Inferential Methods for Correlation and Regression Analysis

1 Inferential Methods for Correlation and Regression Analysis 1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet

More information

Lecture 22: Review for Exam 2. 1 Basic Model Assumptions (without Gaussian Noise)

Lecture 22: Review for Exam 2. 1 Basic Model Assumptions (without Gaussian Noise) Lecture 22: Review for Exam 2 Basic Model Assumptios (without Gaussia Noise) We model oe cotiuous respose variable Y, as a liear fuctio of p umerical predictors, plus oise: Y = β 0 + β X +... β p X p +

More information

In this section we derive some finite-sample properties of the OLS estimator. b is an estimator of β. It is a function of the random sample data.

In this section we derive some finite-sample properties of the OLS estimator. b is an estimator of β. It is a function of the random sample data. 17 3. OLS Part III I this sectio we derive some fiite-sample properties of the OLS estimator. 3.1 The Samplig Distributio of the OLS Estimator y = Xβ + ε ; ε ~ N[0, σ 2 I ] b = (X X) 1 X y = f(y) ε is

More information

ECON 3150/4150, Spring term Lecture 1

ECON 3150/4150, Spring term Lecture 1 ECON 3150/4150, Sprig term 2013. Lecture 1 Ragar Nymoe Uiversity of Oslo 15 Jauary 2013 1 / 42 Refereces to Lecture 1 ad 2 Hill, Griffiths ad Lim, 4 ed (HGL) Ch 1-1.5; Ch 2.8-2.9,4.3-4.3.1.3 Bårdse ad

More information

Simple Regression Model

Simple Regression Model Simple Regressio Model 1. The Model y i 0 1 x i u i where y i depedet variable x i idepedet variable u i disturbace/error term i 1,..., Eg: y wage (measured i 1976 dollars per hr) x educatio (measured

More information

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday

More information

S Y Y = ΣY 2 n. Using the above expressions, the correlation coefficient is. r = SXX S Y Y

S Y Y = ΣY 2 n. Using the above expressions, the correlation coefficient is. r = SXX S Y Y 1 Sociology 405/805 Revised February 4, 004 Summary of Formulae for Bivariate Regressio ad Correlatio Let X be a idepedet variable ad Y a depedet variable, with observatios for each of the values of these

More information

Linear Regression Demystified

Linear Regression Demystified Liear Regressio Demystified Liear regressio is a importat subject i statistics. I elemetary statistics courses, formulae related to liear regressio are ofte stated without derivatio. This ote iteds to

More information

of the matrix is =-85, so it is not positive definite. Thus, the first

of the matrix is =-85, so it is not positive definite. Thus, the first BOSTON COLLEGE Departmet of Ecoomics EC771: Ecoometrics Sprig 4 Prof. Baum, Ms. Uysal Solutio Key for Problem Set 1 1. Are the followig quadratic forms positive for all values of x? (a) y = x 1 8x 1 x

More information

3/3/2014. CDS M Phil Econometrics. Types of Relationships. Types of Relationships. Types of Relationships. Vijayamohanan Pillai N.

3/3/2014. CDS M Phil Econometrics. Types of Relationships. Types of Relationships. Types of Relationships. Vijayamohanan Pillai N. 3/3/04 CDS M Phil Old Least Squares (OLS) Vijayamohaa Pillai N CDS M Phil Vijayamoha CDS M Phil Vijayamoha Types of Relatioships Oly oe idepedet variable, Relatioship betwee ad is Liear relatioships Curviliear

More information

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters

More information

Simple Linear Regression

Simple Linear Regression Simple Liear Regressio 1. Model ad Parameter Estimatio (a) Suppose our data cosist of a collectio of pairs (x i, y i ), where x i is a observed value of variable X ad y i is the correspodig observatio

More information

11 Correlation and Regression

11 Correlation and Regression 11 Correlatio Regressio 11.1 Multivariate Data Ofte we look at data where several variables are recorded for the same idividuals or samplig uits. For example, at a coastal weather statio, we might record

More information

MA Advanced Econometrics: Properties of Least Squares Estimators

MA Advanced Econometrics: Properties of Least Squares Estimators MA Advaced Ecoometrics: Properties of Least Squares Estimators Karl Whela School of Ecoomics, UCD February 5, 20 Karl Whela UCD Least Squares Estimators February 5, 20 / 5 Part I Least Squares: Some Fiite-Sample

More information

Efficient GMM LECTURE 12 GMM II

Efficient GMM LECTURE 12 GMM II DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet

More information

Lesson 11: Simple Linear Regression

Lesson 11: Simple Linear Regression Lesso 11: Simple Liear Regressio Ka-fu WONG December 2, 2004 I previous lessos, we have covered maily about the estimatio of populatio mea (or expected value) ad its iferece. Sometimes we are iterested

More information

Full file at

Full file at Chapter Ecoometrics There are o exercises or applicatios i Chapter. 0 Pearso Educatio, Ic. Publishig as Pretice Hall Chapter The Liear Regressio Model There are o exercises or applicatios i Chapter. 0

More information

Matrix Representation of Data in Experiment

Matrix Representation of Data in Experiment Matrix Represetatio of Data i Experimet Cosider a very simple model for resposes y ij : y ij i ij, i 1,; j 1,,..., (ote that for simplicity we are assumig the two () groups are of equal sample size ) Y

More information

Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.

Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample. Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized

More information

FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures

FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Correlation Regression

Correlation Regression Correlatio Regressio While correlatio methods measure the stregth of a liear relatioship betwee two variables, we might wish to go a little further: How much does oe variable chage for a give chage i aother

More information

Section 14. Simple linear regression.

Section 14. Simple linear regression. Sectio 14 Simple liear regressio. Let us look at the cigarette dataset from [1] (available to dowload from joural s website) ad []. The cigarette dataset cotais measuremets of tar, icotie, weight ad carbo

More information

STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. Comments:

STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. Comments: Recall: STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Commets:. So far we have estimates of the parameters! 0 ad!, but have o idea how good these estimates are. Assumptio: E(Y x)! 0 +! x (liear coditioal

More information

The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.

The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2. SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample

More information

CEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering

CEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio

More information

Chapters 5 and 13: REGRESSION AND CORRELATION. Univariate data: x, Bivariate data (x,y).

Chapters 5 and 13: REGRESSION AND CORRELATION. Univariate data: x, Bivariate data (x,y). Chapters 5 ad 13: REGREION AND CORRELATION (ectios 5.5 ad 13.5 are omitted) Uivariate data: x, Bivariate data (x,y). Example: x: umber of years studets studied paish y: score o a proficiecy test For each

More information

Lecture 11 Simple Linear Regression

Lecture 11 Simple Linear Regression Lecture 11 Simple Liear Regressio Fall 2013 Prof. Yao Xie, yao.xie@isye.gatech.edu H. Milto Stewart School of Idustrial Systems & Egieerig Georgia Tech Midterm 2 mea: 91.2 media: 93.75 std: 6.5 2 Meddicorp

More information

t distribution [34] : used to test a mean against an hypothesized value (H 0 : µ = µ 0 ) or the difference

t distribution [34] : used to test a mean against an hypothesized value (H 0 : µ = µ 0 ) or the difference EXST30 Backgroud material Page From the textbook The Statistical Sleuth Mea [0]: I your text the word mea deotes a populatio mea (µ) while the work average deotes a sample average ( ). Variace [0]: The

More information

Simple Regression. Acknowledgement. These slides are based on presentations created and copyrighted by Prof. Daniel Menasce (GMU) CS 700

Simple Regression. Acknowledgement. These slides are based on presentations created and copyrighted by Prof. Daniel Menasce (GMU) CS 700 Simple Regressio CS 7 Ackowledgemet These slides are based o presetatios created ad copyrighted by Prof. Daiel Measce (GMU) Basics Purpose of regressio aalysis: predict the value of a depedet or respose

More information

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic

More information

Asymptotic Results for the Linear Regression Model

Asymptotic Results for the Linear Regression Model Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is

More information

TAMS24: Notations and Formulas

TAMS24: Notations and Formulas TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =

More information

Expectation and Variance of a random variable

Expectation and Variance of a random variable Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Regression, Inference, and Model Building

Regression, Inference, and Model Building Regressio, Iferece, ad Model Buildig Scatter Plots ad Correlatio Correlatio coefficiet, r -1 r 1 If r is positive, the the scatter plot has a positive slope ad variables are said to have a positive relatioship

More information

Open book and notes. 120 minutes. Cover page and six pages of exam. No calculators.

Open book and notes. 120 minutes. Cover page and six pages of exam. No calculators. IE 330 Seat # Ope book ad otes 120 miutes Cover page ad six pages of exam No calculators Score Fial Exam (example) Schmeiser Ope book ad otes No calculator 120 miutes 1 True or false (for each, 2 poits

More information

Estimation for Complete Data

Estimation for Complete Data Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of

More information

The standard deviation of the mean

The standard deviation of the mean Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider

More information

Goodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen)

Goodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen) Goodess-of-Fit Tests ad Categorical Data Aalysis (Devore Chapter Fourtee) MATH-252-01: Probability ad Statistics II Sprig 2019 Cotets 1 Chi-Squared Tests with Kow Probabilities 1 1.1 Chi-Squared Testig................

More information

MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4

MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4 MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.

More information

A statistical method to determine sample size to estimate characteristic value of soil parameters

A statistical method to determine sample size to estimate characteristic value of soil parameters A statistical method to determie sample size to estimate characteristic value of soil parameters Y. Hojo, B. Setiawa 2 ad M. Suzuki 3 Abstract Sample size is a importat factor to be cosidered i determiig

More information

POLS, GLS, FGLS, GMM. Outline of Linear Systems of Equations. Common Coefficients, Panel Data Model. Preliminaries

POLS, GLS, FGLS, GMM. Outline of Linear Systems of Equations. Common Coefficients, Panel Data Model. Preliminaries Outlie of Liear Systems of Equatios POLS, GLS, FGLS, GMM Commo Coefficiets, Pael Data Model Prelimiaries he liear pael data model is a static model because all explaatory variables are dated cotemporaeously

More information

1 General linear Model Continued..

1 General linear Model Continued.. Geeral liear Model Cotiued.. We have We kow y = X + u X o radom u v N(0; I ) b = (X 0 X) X 0 y E( b ) = V ar( b ) = (X 0 X) We saw that b = (X 0 X) X 0 u so b is a liear fuctio of a ormally distributed

More information

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n. Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator

More information

A Relationship Between the One-Way MANOVA Test Statistic and the Hotelling Lawley Trace Test Statistic

A Relationship Between the One-Way MANOVA Test Statistic and the Hotelling Lawley Trace Test Statistic http://ijspccseetorg Iteratioal Joural of Statistics ad Probability Vol 7, No 6; 2018 A Relatioship Betwee the Oe-Way MANOVA Test Statistic ad the Hotellig Lawley Trace Test Statistic Hasthika S Rupasighe

More information

University of California, Los Angeles Department of Statistics. Practice problems - simple regression 2 - solutions

University of California, Los Angeles Department of Statistics. Practice problems - simple regression 2 - solutions Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 00C Istructor: Nicolas Christou EXERCISE Aswer the followig questios: Practice problems - simple regressio - solutios a Suppose y,

More information

Stat 319 Theory of Statistics (2) Exercises

Stat 319 Theory of Statistics (2) Exercises Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.

More information

Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].

Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)]. Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x

More information

This is an introductory course in Analysis of Variance and Design of Experiments.

This is an introductory course in Analysis of Variance and Design of Experiments. 1 Notes for M 384E, Wedesday, Jauary 21, 2009 (Please ote: I will ot pass out hard-copy class otes i future classes. If there are writte class otes, they will be posted o the web by the ight before class

More information

CEU Department of Economics Econometrics 1, Problem Set 1 - Solutions

CEU Department of Economics Econometrics 1, Problem Set 1 - Solutions CEU Departmet of Ecoomics Ecoometrics, Problem Set - Solutios Part A. Exogeeity - edogeeity The liear coditioal expectatio (CE) model has the followig form: We would like to estimate the effect of some

More information

Statistical Inference Based on Extremum Estimators

Statistical Inference Based on Extremum Estimators T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0

More information

Session 5. (1) Principal component analysis and Karhunen-Loève transformation

Session 5. (1) Principal component analysis and Karhunen-Loève transformation 200 Autum semester Patter Iformatio Processig Topic 2 Image compressio by orthogoal trasformatio Sessio 5 () Pricipal compoet aalysis ad Karhue-Loève trasformatio Topic 2 of this course explais the image

More information

Describing the Relation between Two Variables

Describing the Relation between Two Variables Copyright 010 Pearso Educatio, Ic. Tables ad Formulas for Sulliva, Statistics: Iformed Decisios Usig Data 010 Pearso Educatio, Ic Chapter Orgaizig ad Summarizig Data Relative frequecy = frequecy sum of

More information

6. Kalman filter implementation for linear algebraic equations. Karhunen-Loeve decomposition

6. Kalman filter implementation for linear algebraic equations. Karhunen-Loeve decomposition 6. Kalma filter implemetatio for liear algebraic equatios. Karhue-Loeve decompositio 6.1. Solvable liear algebraic systems. Probabilistic iterpretatio. Let A be a quadratic matrix (ot obligatory osigular.

More information

Lecture 2: Monte Carlo Simulation

Lecture 2: Monte Carlo Simulation STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?

More information

7-1. Chapter 4. Part I. Sampling Distributions and Confidence Intervals

7-1. Chapter 4. Part I. Sampling Distributions and Confidence Intervals 7-1 Chapter 4 Part I. Samplig Distributios ad Cofidece Itervals 1 7- Sectio 1. Samplig Distributio 7-3 Usig Statistics Statistical Iferece: Predict ad forecast values of populatio parameters... Test hypotheses

More information

University of California, Los Angeles Department of Statistics. Simple regression analysis

University of California, Los Angeles Department of Statistics. Simple regression analysis Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 100C Istructor: Nicolas Christou Simple regressio aalysis Itroductio: Regressio aalysis is a statistical method aimig at discoverig

More information

Important Formulas. Expectation: E (X) = Σ [X P(X)] = n p q σ = n p q. P(X) = n! X1! X 2! X 3! X k! p X. Chapter 6 The Normal Distribution.

Important Formulas. Expectation: E (X) = Σ [X P(X)] = n p q σ = n p q. P(X) = n! X1! X 2! X 3! X k! p X. Chapter 6 The Normal Distribution. Importat Formulas Chapter 3 Data Descriptio Mea for idividual data: X = _ ΣX Mea for grouped data: X= _ Σf X m Stadard deviatio for a sample: _ s = Σ(X _ X ) or s = 1 (Σ X ) (Σ X ) ( 1) Stadard deviatio

More information

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio

More information

SIMPLE LINEAR REGRESSION AND CORRELATION ANALYSIS

SIMPLE LINEAR REGRESSION AND CORRELATION ANALYSIS SIMPLE LINEAR REGRESSION AND CORRELATION ANALSIS INTRODUCTION There are lot of statistical ivestigatio to kow whether there is a relatioship amog variables Two aalyses: (1) regressio aalysis; () correlatio

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Investigating the Significance of a Correlation Coefficient using Jackknife Estimates

Investigating the Significance of a Correlation Coefficient using Jackknife Estimates Iteratioal Joural of Scieces: Basic ad Applied Research (IJSBAR) ISSN 2307-4531 (Prit & Olie) http://gssrr.org/idex.php?joural=jouralofbasicadapplied ---------------------------------------------------------------------------------------------------------------------------

More information

Apply change-of-basis formula to rewrite x as a linear combination of eigenvectors v j.

Apply change-of-basis formula to rewrite x as a linear combination of eigenvectors v j. Eigevalue-Eigevector Istructor: Nam Su Wag eigemcd Ay vector i real Euclidea space of dimesio ca be uiquely epressed as a liear combiatio of liearly idepedet vectors (ie, basis) g j, j,,, α g α g α g α

More information

Randomized Algorithms I, Spring 2018, Department of Computer Science, University of Helsinki Homework 1: Solutions (Discussed January 25, 2018)

Randomized Algorithms I, Spring 2018, Department of Computer Science, University of Helsinki Homework 1: Solutions (Discussed January 25, 2018) Radomized Algorithms I, Sprig 08, Departmet of Computer Sciece, Uiversity of Helsiki Homework : Solutios Discussed Jauary 5, 08). Exercise.: Cosider the followig balls-ad-bi game. We start with oe black

More information

Lecture 1, Jan 19. i=1 p i = 1.

Lecture 1, Jan 19. i=1 p i = 1. Lecture 1, Ja 19 Review of the expected value, covariace, correlatio coefficiet, mea, ad variace. Radom variable. A variable that takes o alterative values accordig to chace. More specifically, a radom

More information

The Method of Least Squares. To understand least squares fitting of data.

The Method of Least Squares. To understand least squares fitting of data. The Method of Least Squares KEY WORDS Curve fittig, least square GOAL To uderstad least squares fittig of data To uderstad the least squares solutio of icosistet systems of liear equatios 1 Motivatio Curve

More information

Chapter 6 Principles of Data Reduction

Chapter 6 Principles of Data Reduction Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a

More information

Worksheet 23 ( ) Introduction to Simple Linear Regression (continued)

Worksheet 23 ( ) Introduction to Simple Linear Regression (continued) Worksheet 3 ( 11.5-11.8) Itroductio to Simple Liear Regressio (cotiued) This worksheet is a cotiuatio of Discussio Sheet 3; please complete that discussio sheet first if you have ot already doe so. This

More information

DS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10

DS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10 DS 00: Priciples ad Techiques of Data Sciece Date: April 3, 208 Name: Hypothesis Testig Discussio #0. Defie these terms below as they relate to hypothesis testig. a) Data Geeratio Model: Solutio: A set

More information

Regression. Correlation vs. regression. The parameters of linear regression. Regression assumes... Random sample. Y = α + β X.

Regression. Correlation vs. regression. The parameters of linear regression. Regression assumes... Random sample. Y = α + β X. Regressio Correlatio vs. regressio Predicts Y from X Liear regressio assumes that the relatioship betwee X ad Y ca be described by a lie Regressio assumes... Radom sample Y is ormally distributed with

More information

Chapter Vectors

Chapter Vectors Chapter 4. Vectors fter readig this chapter you should be able to:. defie a vector. add ad subtract vectors. fid liear combiatios of vectors ad their relatioship to a set of equatios 4. explai what it

More information

Mathematical Statistics - MS

Mathematical Statistics - MS Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios

More information

Quick Review of Probability

Quick Review of Probability Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter & Teachig Material.

More information

CHAPTER 5. Theory and Solution Using Matrix Techniques

CHAPTER 5. Theory and Solution Using Matrix Techniques A SERIES OF CLASS NOTES FOR 2005-2006 TO INTRODUCE LINEAR AND NONLINEAR PROBLEMS TO ENGINEERS, SCIENTISTS, AND APPLIED MATHEMATICIANS DE CLASS NOTES 3 A COLLECTION OF HANDOUTS ON SYSTEMS OF ORDINARY DIFFERENTIAL

More information

Problem Set 4 Due Oct, 12

Problem Set 4 Due Oct, 12 EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios

More information

Estimation of the Mean and the ACVF

Estimation of the Mean and the ACVF Chapter 5 Estimatio of the Mea ad the ACVF A statioary process {X t } is characterized by its mea ad its autocovariace fuctio γ ), ad so by the autocorrelatio fuctio ρ ) I this chapter we preset the estimators

More information

Mathematical Notation Math Introduction to Applied Statistics

Mathematical Notation Math Introduction to Applied Statistics Mathematical Notatio Math 113 - Itroductio to Applied Statistics Name : Use Word or WordPerfect to recreate the followig documets. Each article is worth 10 poits ad ca be prited ad give to the istructor

More information

Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting

Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would

More information

An Introduction to Randomized Algorithms

An Introduction to Randomized Algorithms A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis

More information

ECE 8527: Introduction to Machine Learning and Pattern Recognition Midterm # 1. Vaishali Amin Fall, 2015

ECE 8527: Introduction to Machine Learning and Pattern Recognition Midterm # 1. Vaishali Amin Fall, 2015 ECE 8527: Itroductio to Machie Learig ad Patter Recogitio Midterm # 1 Vaishali Ami Fall, 2015 tue39624@temple.edu Problem No. 1: Cosider a two-class discrete distributio problem: ω 1 :{[0,0], [2,0], [2,2],

More information

Statistics 511 Additional Materials

Statistics 511 Additional Materials Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability

More information

Statistics 203 Introduction to Regression and Analysis of Variance Assignment #1 Solutions January 20, 2005

Statistics 203 Introduction to Regression and Analysis of Variance Assignment #1 Solutions January 20, 2005 Statistics 203 Itroductio to Regressio ad Aalysis of Variace Assigmet #1 Solutios Jauary 20, 2005 Q. 1) (MP 2.7) (a) Let x deote the hydrocarbo percetage, ad let y deote the oxyge purity. The simple liear

More information

1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable

More information

Discrete Mathematics for CS Spring 2008 David Wagner Note 22

Discrete Mathematics for CS Spring 2008 David Wagner Note 22 CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig

More information

MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND.

MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND. XI-1 (1074) MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND. R. E. D. WOOLSEY AND H. S. SWANSON XI-2 (1075) STATISTICAL DECISION MAKING Advaced

More information

Cov(aX, cy ) Var(X) Var(Y ) It is completely invariant to affine transformations: for any a, b, c, d R, ρ(ax + b, cy + d) = a.s. X i. as n.

Cov(aX, cy ) Var(X) Var(Y ) It is completely invariant to affine transformations: for any a, b, c, d R, ρ(ax + b, cy + d) = a.s. X i. as n. CS 189 Itroductio to Machie Learig Sprig 218 Note 11 1 Caoical Correlatio Aalysis The Pearso Correlatio Coefficiet ρ(x, Y ) is a way to measure how liearly related (i other words, how well a liear model

More information