Training Sample Model: Given n observations, [[( Yi, x i the sample model can be expressed as (1) where, zero and variance σ
|
|
- Clara Evans
- 5 years ago
- Views:
Transcription
1 Stat 74 Estmato for Geeral Lear Model Prof. Goel Broad Outle Geeral Lear Model (GLM): Trag Samle Model: Gve observatos, [[( Y, x ), x = ( x,, xr )], =,,, the samle model ca be exressed as Y = µ ( x, x,, x ) + ε, =,,,, () r where,,,,, zero ad varace σ. ε =, deote the ose (radom errors), each wth mea From ow o, we deote the features f, =,,,, themselves as coded redctor varables x, x,, x. I the smlest settg, the radom errors are assumed to be ucorrelated wth equal varace. Thus the samle GLM ca be exressed as = β + ε, [ ε ] =, [ ε ] = σ, ( ε, εk ) =,. = Y x E Var Cov k () EY [ ] = µ = β x. = x (3) Vector/matrx otato for the resose, redctor varables, error terms ad the ukow coeffcets: Y β ε x. x Y β ε x x. Y =, β =, ε =,ad X =. Also, let x. =, Y β ε x x. deote the th colum of X,.e., X = [ x., x.,, x. ]. Gve the resose vector Y, ad the desg matrx X, the samle GLM ca be wrtte as Y = Xβ + ε E ε = Cov ε = ε ε = E ε ε = E εε = σ I (4), [ ], [ ] ((cov(, ))) (( ( ))) [ ]. Thus, E[ Y] = μ= Xβ, Cov( Y) = E[( Y-Xβ)( Y-Xβ) ] = E[ εε ] = σ I. (5)
2 Ordary Least Square (OLS): For a estmate β of β, corresodg Resdual Sum of Squares: l ( β) = ( Y β x ) = e ( β) = ( β) ( β). e e = = = Problem: Fd a estmated coeffcet vector ˆ β = arg m l( β) I matrx otato, β R m e ( β) e( β) = m( YX β)( Y X β) = e ( ˆ β) e( ˆ β). β R β R (6) Exad S( β) = ( YXβ)( Y Xβ) = YY YX β β XY + β XX β. O settg the artal dervatves of S( β ) wth resect to β equal to zero, we get the Normal Equatos XX β = XY. (7) Ay soluto of (7) s a otmal soluto to the OLS roblem. Full rak case: Examle Smle Lear Regresso, Multle Regresso wth learly deedet features. If the matrx XXs o-sgular (the desg matrx X s of full colum rak ), verse of XXexsts, ad the uque otmal least square soluto s ˆ = XX XY (8) βˆ ( ). Note that E[ β] = ( ) E[ ] = ( ) β = β XX X Y XX XX. Sce, Cov( TY) = TCov ( Y) T, therefore, ˆ Cov( β) = Cov( TY) = σ [ TIT ], where T = ( X X) X. Therefore, Cov( β) σ ( ). ˆ = XX
3 Not full-rak case: Examle - ANOVA for Desged Exermets The ormal equatos (7) are cosstet, but the system has ftely may solutos. Each soluto ca be exressed as β = ( XX ) XY, where ( XX ) s a geeralzed verse of XX. I fact, E β = ( XX ) X E[ Y] = ( XX ) ( XX ) β = H β β, so some comoets of β do ot ossess ubased estmators. The bases β may deed o the geeralzed verse used obtag the artcular OLS soluto. All these estmators ca t be regarded as otmal estmator of the vector β. Why? Otmal wth resect to what crtera? May eed to add a addtoal crtero to OLS to get a uque soluto, e.g., Mmum orm OLS estmator: β + = ( XX ) + XY, where ( XX ) + s the Moore-Perose geeralzed verse (Pseudo-verse) of XX. Coordate Free (Vector Sace) Aroach: Iterret the model μ= X β as - μ C[ X ]. However, ote that ˆµ = X β = PY = X(X X) X Y, the roecto of Y oto C[ X ]. The symmetrc matrx P= X( XX ) X s the orthogoal roecto matrx oto C[ X ], the sace saed by the colums of X. Eve though o-full rak there are ftely may solutos ( β ) to the ormal equatos case, the roecto ˆ µ = PY (also called Y ˆ ) s uque,.e., the matrx P does ot chage wth the choce of a geeralzed verse of XX. For a vector u C[ X ],.e., u = Xb, for some vector b, Pu=X( XX ) Xu = X( XX ) XXb = Xb= u. Thus the roecto of a vector u C[ X ] oto C[ X] s u tself.
4 Furthermore, for a arbtrary vector Y V, PY C[ X], therefore, P(PY) = PY holds true for all Y R. Hece, (P - P) = P(I - P) =. Thus, P = P, (.e., P s a demotet matrx). Is P a symmetrc matrx?. Fact: Every symmetrc demotet matrx s a orthogoal roecto matrx oto the sace saed by ts colums. Sce P (I-P) =, rows (colums) of P are orthogoal to the colums (rows) of (I-P),.e., Py ad ( I - P) y are orthogoal. Note that, Xβ = ˆ µ = PY = Yˆ, Y Y ˆ = (I - P)Y = e, the vector of resduals. Therefore, the vectors Yˆ ad e are orthogoal,.e., Examle: Detals of full-rak lear regresso model. The Key Questo: How to characterze the class of lear fuctos Ye ˆ = ye ˆ =. be estmated uquely through the least squares solutos? c β = c β that ca Estmable fuctos: A lear arametrc fucto c β s sad to be estmable, f there exst at least oe ubased estmator. If there does ot exst ay ubased estmator of the lear fucto c β, t s sad to be o-estmable. Why cosder estmable fuctos? We wll dscuss ts coecto wth the cocet of Idetfablty. Note that Y s a ubased estmator of x. β. (Why?) Thus x. β s estmable for each row of the matrx X. Hecec β, where the vector c s some lear combato of rows of X, s also estmable. Fact: A lear arametrc fucto c β of β ' c C ( X ) Row sace of X. (Prove t.) c C ( X ) c= Xl for some l. s = s estmable f ad oly f
5 Therefore, for ay OLS β, c β = lx ( XX ) XY = lpy = l ˆ µ. Thus c β s varat to the choce of geeralzed verse [Uque OLS soluto for ubased estmator ofc β. Gauss-Markov Theorem - c β Best (Mmum Varace) Lear Ubased Estmator (B.L.U.E.) of a estmable lear fuctoc β. Geeralzed Least Squares: Var( ε) = σ V, where V s a kow.d. matrx. Reduce ths roblem to a OLS roblem by a o-sgular trasformato Sce V s a ostve defte matrx, there exsts a o-sgular matrx T such that V = TT. Now, cosder the lear trasformato Z = TY. Note E( Z ) = TE( Y) = TXβ, Cov( Z) = σ TVT = σ T( T T) T = σ I. Ca cosder OLS roblem for Z.
6 Backgroud - Vector dfferetato: Vector of Partal dervatves of a lear form l u = lu,ad a quadratc form u Au = a uu, = = = for a symmetrc matrx A : ( lu) ( u Au) au + a u l au ( ) l ( ) = lu lu ( ) a u a u + = ; ( ) u Au u au = l u Au =. = = = Au l a u + au au ( lu ) ( ) u Au Whe A s ot symmetrc, uau = u {( A+ A ) / } u, wth {( A+ A ) / } symmetrc. Therefore, ( u Au) = ( A + A ) u.
Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Aalyss of Varace ad Desg of Exermets-I MODULE II LECTURE - GENERAL LINEAR HYPOTHESIS AND ANALYSIS OF VARIANCE Dr Shalabh Deartmet of Mathematcs ad Statstcs Ida Isttute of Techology Kaur Tukey s rocedure
More informationMultivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
More informationLecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
More information2SLS Estimates ECON In this case, begin with the assumption that E[ i
SLS Estmates ECON 3033 Bll Evas Fall 05 Two-Stage Least Squares (SLS Cosder a stadard lear bvarate regresso model y 0 x. I ths case, beg wth the assumto that E[ x] 0 whch meas that OLS estmates of wll
More informationDr. Shalabh. Indian Institute of Technology Kanpur
Aalyss of Varace ad Desg of Expermets-I MODULE -I LECTURE - SOME RESULTS ON LINEAR ALGEBRA, MATRIX THEORY AND DISTRIBUTIONS Dr. Shalabh Departmet t of Mathematcs t ad Statstcs t t Ida Isttute of Techology
More informationSpecial Instructions / Useful Data
JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth
More informationLinear Regression with One Regressor
Lear Regresso wth Oe Regressor AIM QA.7. Expla how regresso aalyss ecoometrcs measures the relatoshp betwee depedet ad depedet varables. A regresso aalyss has the goal of measurg how chages oe varable,
More informationChapter 3 Experimental Design Models
Chater 3 Exermetal Desg Models We cosder the models whch are used desgg a exermet. The exermetal codtos, exermetal setu ad the obectve of the study essetally determe that what tye of desg s to be used
More informationQualifying Exam Statistical Theory Problem Solutions August 2005
Qualfyg Exam Statstcal Theory Problem Solutos August 5. Let X, X,..., X be d uform U(,),
More informationIntroduction to Matrices and Matrix Approach to Simple Linear Regression
Itroducto to Matrces ad Matrx Approach to Smple Lear Regresso Matrces Defto: A matrx s a rectagular array of umbers or symbolc elemets I may applcatos, the rows of a matrx wll represet dvduals cases (people,
More informationSTK4011 and STK9011 Autumn 2016
STK4 ad STK9 Autum 6 Pot estmato Covers (most of the followg materal from chapter 7: Secto 7.: pages 3-3 Secto 7..: pages 3-33 Secto 7..: pages 35-3 Secto 7..3: pages 34-35 Secto 7.3.: pages 33-33 Secto
More informationSingular Value Decomposition. Linear Algebra (3) Singular Value Decomposition. SVD and Eigenvectors. Solving LEs with SVD
Sgular Value Decomosto Lear Algera (3) m Cootes Ay m x matrx wth m ca e decomosed as follows Dagoal matrx A UWV m x x Orthogoal colums U U I w1 0 0 w W M M 0 0 x Orthoormal (Pure rotato) VV V V L 0 L 0
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationLecture Note to Rice Chapter 8
ECON 430 HG revsed Nov 06 Lecture Note to Rce Chapter 8 Radom matrces Let Y, =,,, m, =,,, be radom varables (r.v. s). The matrx Y Y Y Y Y Y Y Y Y Y = m m m s called a radom matrx ( wth a ot m-dmesoal dstrbuto,
More informationENGI 3423 Simple Linear Regression Page 12-01
ENGI 343 mple Lear Regresso Page - mple Lear Regresso ometmes a expermet s set up where the expermeter has cotrol over the values of oe or more varables X ad measures the resultg values of aother varable
More informationMultiple Choice Test. Chapter Adequacy of Models for Regression
Multple Choce Test Chapter 06.0 Adequac of Models for Regresso. For a lear regresso model to be cosdered adequate, the percetage of scaled resduals that eed to be the rage [-,] s greater tha or equal to
More informationMidterm Exam 1, section 1 (Solution) Thursday, February hour, 15 minutes
coometrcs, CON Sa Fracsco State Uversty Mchael Bar Sprg 5 Mdterm am, secto Soluto Thursday, February 6 hour, 5 mutes Name: Istructos. Ths s closed book, closed otes eam.. No calculators of ay kd are allowed..
More informationCOV. Violation of constant variance of ε i s but they are still independent. The error term (ε) is said to be heteroscedastic.
c Pogsa Porchawseskul, Faculty of Ecoomcs, Chulalogkor Uversty olato of costat varace of s but they are stll depedet. C,, he error term s sad to be heteroscedastc. c Pogsa Porchawseskul, Faculty of Ecoomcs,
More informationMaximum Likelihood Estimation
Marquette Uverst Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 08 b Marquette Uverst Maxmum Lkelhood Estmato We have bee sag that ~
More informationMidterm Exam 1, section 2 (Solution) Thursday, February hour, 15 minutes
coometrcs, CON Sa Fracsco State Uverst Mchael Bar Sprg 5 Mdterm xam, secto Soluto Thursda, Februar 6 hour, 5 mutes Name: Istructos. Ths s closed book, closed otes exam.. No calculators of a kd are allowed..
More informationX X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then
Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
More information( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model
Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch
More information4. Standard Regression Model and Spatial Dependence Tests
4. Stadard Regresso Model ad Spatal Depedece Tests Stadard regresso aalss fals the presece of spatal effects. I case of spatal depedeces ad/or spatal heterogeet a stadard regresso model wll be msspecfed.
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted
More informationSimple Linear Regression and Correlation.
Smple Lear Regresso ad Correlato. Correspods to Chapter 0 Tamhae ad Dulop Sldes prepared b Elzabeth Newto (MIT) wth some sldes b Jacquele Telford (Johs Hopks Uverst) Smple lear regresso aalss estmates
More informationSimple Linear Regression
Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato
More informationLINEAR REGRESSION ANALYSIS
LINEAR REGRESSION ANALYSIS MODULE V Lecture - Correctg Model Iadequaces Through Trasformato ad Weghtg Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Aalytcal methods for
More informationLinear Regression Linear Regression with Shrinkage. Some slides are due to Tommi Jaakkola, MIT AI Lab
Lear Regresso Lear Regresso th Shrkage Some sldes are due to Tomm Jaakkola, MIT AI Lab Itroducto The goal of regresso s to make quattatve real valued predctos o the bass of a vector of features or attrbutes.
More informationECON 5360 Class Notes GMM
ECON 560 Class Notes GMM Geeralzed Method of Momets (GMM) I beg by outlg the classcal method of momets techque (Fsher, 95) ad the proceed to geeralzed method of momets (Hase, 98).. radtoal Method of Momets
More informationLecture Notes 2. The ability to manipulate matrices is critical in economics.
Lecture Notes. Revew of Matrces he ablt to mapulate matrces s crtcal ecoomcs.. Matr a rectagular arra of umbers, parameters, or varables placed rows ad colums. Matrces are assocated wth lear equatos. lemets
More informationε. Therefore, the estimate
Suggested Aswers, Problem Set 3 ECON 333 Da Hugerma. Ths s ot a very good dea. We kow from the secod FOC problem b) that ( ) SSE / = y x x = ( ) Whch ca be reduced to read y x x = ε x = ( ) The OLS model
More informationECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity
ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data
More informationBASIC PRINCIPLES OF STATISTICS
BASIC PRINCIPLES OF STATISTICS PROBABILITY DENSITY DISTRIBUTIONS DISCRETE VARIABLES BINOMIAL DISTRIBUTION ~ B 0 0 umber of successes trals Pr E [ ] Var[ ] ; BINOMIAL DISTRIBUTION B7 0. B30 0.3 B50 0.5
More informationStats & Summary
Stts 443.3 & 85.3 Summr The Woodbur Theorem BCD B C D B D where the verses C C D B, d est. Block Mtrces Let the m mtr m q q m be rttoed to sub-mtrces,,,, Smlrl rtto the m k mtr B B B mk m B B l kl Product
More informationChapter 2 General Linear Hypothesis and Analysis of Variance
Chater Geeral Lear Hyothess ad Aalyss of Varace Regresso model for the geeral lear hyothess Let Y, Y,..., Y be a seuece of deedet radom varables assocated wth resoses. The we ca wrte t as EY ( ) = β x,
More informationLecture 3 Probability review (cont d)
STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto
More informationObjectives of Multiple Regression
Obectves of Multple Regresso Establsh the lear equato that best predcts values of a depedet varable Y usg more tha oe eplaator varable from a large set of potetal predctors {,,... k }. Fd that subset of
More informationDISTURBANCE TERMS. is a scalar and x i
DISTURBANCE TERMS I a feld of research desg, we ofte have the qesto abot whether there s a relatoshp betwee a observed varable (sa, ) ad the other observed varables (sa, x ). To aswer the qesto, we ma
More informationChapter 5 Properties of a Random Sample
Lecture 3 o BST 63: Statstcal Theory I Ku Zhag, /6/006 Revew for the revous lecture Cocets: radom samle, samle mea, samle varace Theorems: roertes of a radom samle, samle mea, samle varace Examles: how
More informationUnit 9. The Tangent Bundle
Ut 9. The Taget Budle ========================================================================================== ---------- The taget sace of a submafold of R, detfcato of taget vectors wth dervatos at
More informationWu-Hausman Test: But if X and ε are independent, βˆ. ECON 324 Page 1
Wu-Hausma Test: Detectg Falure of E( ε X ) Caot drectly test ths assumpto because lack ubased estmator of ε ad the OLS resduals wll be orthogoal to X, by costructo as ca be see from the momet codto X'
More informationECON 482 / WH Hong The Simple Regression Model 1. Definition of the Simple Regression Model
ECON 48 / WH Hog The Smple Regresso Model. Defto of the Smple Regresso Model Smple Regresso Model Expla varable y terms of varable x y = β + β x+ u y : depedet varable, explaed varable, respose varable,
More informationMultiple Linear Regression Analysis
LINEA EGESSION ANALYSIS MODULE III Lecture - 4 Multple Lear egresso Aalyss Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Cofdece terval estmato The cofdece tervals multple
More informationESS Line Fitting
ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here
More informationSTA 108 Applied Linear Models: Regression Analysis Spring Solution for Homework #1
STA 08 Appled Lear Models: Regresso Aalyss Sprg 0 Soluto for Homework #. Let Y the dollar cost per year, X the umber of vsts per year. The the mathematcal relato betwee X ad Y s: Y 300 + X. Ths s a fuctoal
More information4 Inner Product Spaces
11.MH1 LINEAR ALGEBRA Summary Notes 4 Ier Product Spaces Ier product s the abstracto to geeral vector spaces of the famlar dea of the scalar product of two vectors or 3. I what follows, keep these key
More informationSTK3100 and STK4100 Autumn 2017
SK3 ad SK4 Autum 7 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Sectos 4..5, 4.3.5, 4.3.6, 4.4., 4.4., ad 4.4.3 Sectos 5.., 5.., ad 5.5. Ørulf Borga Deartmet of Mathematcs
More information6.867 Machine Learning
6.867 Mache Learg Problem set Due Frday, September 9, rectato Please address all questos ad commets about ths problem set to 6.867-staff@a.mt.edu. You do ot eed to use MATLAB for ths problem set though
More informationChapter 3 Multiple Linear Regression Model
Chapter 3 Multple Lear Regresso Model We cosder the problem of regresso whe study varable depeds o more tha oe explaatory or depedet varables, called as multple lear regresso model. Ths model geeralzes
More information{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:
Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed
More information9 U-STATISTICS. Eh =(m!) 1 Eh(X (1),..., X (m ) ) i.i.d
9 U-STATISTICS Suppose,,..., are P P..d. wth CDF F. Our goal s to estmate the expectato t (P)=Eh(,,..., m ). Note that ths expectato requres more tha oe cotrast to E, E, or Eh( ). Oe example s E or P((,
More informationChapter 4 Multiple Random Variables
Revew for the prevous lecture: Theorems ad Examples: How to obta the pmf (pdf) of U = g (, Y) ad V = g (, Y) Chapter 4 Multple Radom Varables Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples:
More informationSTA302/1001-Fall 2008 Midterm Test October 21, 2008
STA3/-Fall 8 Mdterm Test October, 8 Last Name: Frst Name: Studet Number: Erolled (Crcle oe) STA3 STA INSTRUCTIONS Tme allowed: hour 45 mutes Ads allowed: A o-programmable calculator A table of values from
More informationSTK3100 and STK4100 Autumn 2018
SK3 ad SK4 Autum 8 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Cofdece tervals by vertg tests Cosder a model wth a sgle arameter β We may obta a ( α% cofdece terval for
More informationRecall MLR 5 Homskedasticity error u has the same variance given any values of the explanatory variables Var(u x1,...,xk) = 2 or E(UU ) = 2 I
Chapter 8 Heterosedastcty Recall MLR 5 Homsedastcty error u has the same varace gve ay values of the eplaatory varables Varu,..., = or EUU = I Suppose other GM assumptos hold but have heterosedastcty.
More informationTHE ROYAL STATISTICAL SOCIETY 2016 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5
THE ROYAL STATISTICAL SOCIETY 06 EAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5 The Socety s provdg these solutos to assst cadtes preparg for the examatos 07. The solutos are teded as learg ads ad should
More informationOverview. Basic concepts of Bayesian learning. Most probable model given data Coin tosses Linear regression Logistic regression
Overvew Basc cocepts of Bayesa learg Most probable model gve data Co tosses Lear regresso Logstc regresso Bayesa predctos Co tosses Lear regresso 30 Recap: regresso problems Iput to learg problem: trag
More informationDimensionality Reduction and Learning
CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that
More information12.2 Estimating Model parameters Assumptions: ox and y are related according to the simple linear regression model
1. Estmatg Model parameters Assumptos: ox ad y are related accordg to the smple lear regresso model (The lear regresso model s the model that says that x ad y are related a lear fasho, but the observed
More informationInvestigation of Partially Conditional RP Model with Response Error. Ed Stanek
Partally Codtoal Radom Permutato Model 7- vestgato of Partally Codtoal RP Model wth Respose Error TRODUCTO Ed Staek We explore the predctor that wll result a smple radom sample wth respose error whe a
More information1 Solution to Problem 6.40
1 Soluto to Problem 6.40 (a We wll wrte T τ (X 1,...,X where the X s are..d. wth PDF f(x µ, σ 1 ( x µ σ g, σ where the locato parameter µ s ay real umber ad the scale parameter σ s > 0. Lettg Z X µ σ we
More informationLine Fitting and Regression
Marquette Uverst MSCS6 Le Fttg ad Regresso Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 8 b Marquette Uverst Least Squares Regresso MSCS6 For LSR we have pots
More informationChannel Models with Memory. Channel Models with Memory. Channel Models with Memory. Channel Models with Memory
Chael Models wth Memory Chael Models wth Memory Hayder radha Electrcal ad Comuter Egeerg Mchga State Uversty I may ractcal etworkg scearos (cludg the Iteret ad wreless etworks), the uderlyg chaels are
More informationρ < 1 be five real numbers. The
Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:
More informationENGI 4421 Propagation of Error Page 8-01
ENGI 441 Propagato of Error Page 8-01 Propagato of Error [Navd Chapter 3; ot Devore] Ay realstc measuremet procedure cotas error. Ay calculatos based o that measuremet wll therefore also cota a error.
More informationEcon 388 R. Butler 2016 rev Lecture 5 Multivariate 2 I. Partitioned Regression and Partial Regression Table 1: Projections everywhere
Eco 388 R. Butler 06 rev Lecture 5 Multvarate I. Parttoed Regresso ad Partal Regresso Table : Projectos everywhere P = ( ) ad M = I ( ) ad s a vector of oes assocated wth the costat term Sample Model Regresso
More informationCS 2750 Machine Learning Lecture 5. Density estimation. Density estimation
CS 750 Mache Learg Lecture 5 esty estmato Mlos Hausrecht mlos@tt.edu 539 Seott Square esty estmato esty estmato: s a usuervsed learg roblem Goal: Lear a model that rereset the relatos amog attrbutes the
More information. The set of these sums. be a partition of [ ab, ]. Consider the sum f( x) f( x 1)
Chapter 7 Fuctos o Bouded Varato. Subject: Real Aalyss Level: M.Sc. Source: Syed Gul Shah (Charma, Departmet o Mathematcs, US Sargodha Collected & Composed by: Atq ur Rehma (atq@mathcty.org, http://www.mathcty.org
More informationFactorization of Finite Abelian Groups
Iteratoal Joural of Algebra, Vol 6, 0, o 3, 0-07 Factorzato of Fte Abela Grous Khald Am Uversty of Bahra Deartmet of Mathematcs PO Box 3038 Sakhr, Bahra kamee@uobedubh Abstract If G s a fte abela grou
More informationSome Different Perspectives on Linear Least Squares
Soe Dfferet Perspectves o Lear Least Squares A stadard proble statstcs s to easure a respose or depedet varable, y, at fed values of oe or ore depedet varables. Soetes there ests a deterstc odel y f (,,
More informationTHE ROYAL STATISTICAL SOCIETY HIGHER CERTIFICATE
THE ROYAL STATISTICAL SOCIETY 00 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE PAPER I STATISTICAL THEORY The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for the
More informationSTRONG CONSISTENCY FOR SIMPLE LINEAR EV MODEL WITH v/ -MIXING
Joural of tatstcs: Advaces Theory ad Alcatos Volume 5, Number, 6, Pages 3- Avalable at htt://scetfcadvaces.co. DOI: htt://d.do.org/.864/jsata_7678 TRONG CONITENCY FOR IMPLE LINEAR EV MODEL WITH v/ -MIXING
More informationThe equation is sometimes presented in form Y = a + b x. This is reasonable, but it s not the notation we use.
INTRODUCTORY NOTE ON LINEAR REGREION We have data of the form (x y ) (x y ) (x y ) These wll most ofte be preseted to us as two colum of a spreadsheet As the topc develops we wll see both upper case ad
More informationENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections
ENGI 441 Jot Probablty Dstrbutos Page 7-01 Jot Probablty Dstrbutos [Navd sectos.5 ad.6; Devore sectos 5.1-5.] The jot probablty mass fucto of two dscrete radom quattes, s, P ad p x y x y The margal probablty
More informationAssignment 5/MATH 247/Winter Due: Friday, February 19 in class (!) (answers will be posted right after class)
Assgmet 5/MATH 7/Wter 00 Due: Frday, February 9 class (!) (aswers wll be posted rght after class) As usual, there are peces of text, before the questos [], [], themselves. Recall: For the quadratc form
More informationChapter Business Statistics: A First Course Fifth Edition. Learning Objectives. Correlation vs. Regression. In this chapter, you learn:
Chapter 3 3- Busess Statstcs: A Frst Course Ffth Edto Chapter 2 Correlato ad Smple Lear Regresso Busess Statstcs: A Frst Course, 5e 29 Pretce-Hall, Ic. Chap 2- Learg Objectves I ths chapter, you lear:
More informationMOLECULAR VIBRATIONS
MOLECULAR VIBRATIONS Here we wsh to vestgate molecular vbratos ad draw a smlarty betwee the theory of molecular vbratos ad Hückel theory. 1. Smple Harmoc Oscllator Recall that the eergy of a oe-dmesoal
More informationProbability and. Lecture 13: and Correlation
933 Probablty ad Statstcs for Software ad Kowledge Egeers Lecture 3: Smple Lear Regresso ad Correlato Mocha Soptkamo, Ph.D. Outle The Smple Lear Regresso Model (.) Fttg the Regresso Le (.) The Aalyss of
More informationLecture Notes Types of economic variables
Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte
More informationPoint Estimation: definition of estimators
Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.
More informationChapter 2 Supplemental Text Material
-. Models for the Data ad the t-test Chapter upplemetal Text Materal The model preseted the text, equato (-3) s more properl called a meas model. ce the mea s a locato parameter, ths tpe of model s also
More informationBayes (Naïve or not) Classifiers: Generative Approach
Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg
More informationå 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018
Chrs Pech Fal Practce CS09 Dec 5, 08 Practce Fal Examato Solutos. Aswer: 4/5 8/7. There are multle ways to obta ths aswer; here are two: The frst commo method s to sum over all ossbltes for the rak of
More informationQuiz 1- Linear Regression Analysis (Based on Lectures 1-14)
Quz - Lear Regreo Aaly (Baed o Lecture -4). I the mple lear regreo model y = β + βx + ε, wth Tme: Hour Ε ε = Ε ε = ( ) 3, ( ), =,,...,, the ubaed drect leat quare etmator ˆβ ad ˆβ of β ad β repectvely,
More informationEconometrics. 3) Statistical properties of the OLS estimator
30C0000 Ecoometrcs 3) Statstcal propertes of the OLS estmator Tmo Kuosmae Professor, Ph.D. http://omepre.et/dex.php/tmokuosmae Today s topcs Whch assumptos are eeded for OLS to work? Statstcal propertes
More informationLecture Notes Forecasting the process of estimating or predicting unknown situations
Lecture Notes. Ecoomc Forecastg. Forecastg the process of estmatg or predctg ukow stuatos Eample usuall ecoomsts predct future ecoomc varables Forecastg apples to a varet of data () tme seres data predctg
More informationDepartment of Mathematics UNIVERSITY OF OSLO. FORMULAS FOR STK4040 (version 1, September 12th, 2011) A - Vectors and matrices
Deartet of Matheatcs UNIVERSITY OF OSLO FORMULAS FOR STK4040 (verso Seteber th 0) A - Vectors ad atrces A) For a x atrx A ad a x atrx B we have ( AB) BA A) For osgular square atrces A ad B we have ( )
More informationA class of Liu-type estimators based on ridge regression under multicollinearity with an application to mixture experiments
A class of Lu-te estmators based o rdge regresso uder multcolleart wth a alcato to mture eermets Preseter: A-Chu Che 陳愛群 Advsor: aesh Emura Jue 6, 5 Graduate Isttute of Statstcs, NCU Outle Itroducto Methodolog
More information2. Independence and Bernoulli Trials
. Ideedece ad Beroull Trals Ideedece: Evets ad B are deedet f B B. - It s easy to show that, B deedet mles, B;, B are all deedet ars. For examle, ad so that B or B B B B B φ,.e., ad B are deedet evets.,
More informationCHAPTER 6. d. With success = observation greater than 10, x = # of successes = 4, and
CHAPTR 6 Secto 6.. a. We use the samle mea, to estmate the oulato mea µ. Σ 9.80 µ 8.407 7 ~ 7. b. We use the samle meda, 7 (the mddle observato whe arraged ascedg order. c. We use the samle stadard devato,
More informationAn Introduction to. Support Vector Machine
A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork
More informationChapter 9 Jordan Block Matrices
Chapter 9 Jorda Block atrces I ths chapter we wll solve the followg problem. Gve a lear operator T fd a bass R of F such that the matrx R (T) s as smple as possble. f course smple s a matter of taste.
More information: At least two means differ SST
Formula Card for Eam 3 STA33 ANOVA F-Test: Completely Radomzed Desg ( total umber of observatos, k = Number of treatmets,& T = total for treatmet ) Step : Epress the Clam Step : The ypotheses: :... 0 A
More informationLecture 8: Linear Regression
Lecture 8: Lear egresso May 4, GENOME 56, Sprg Goals Develop basc cocepts of lear regresso from a probablstc framework Estmatg parameters ad hypothess testg wth lear models Lear regresso Su I Lee, CSE
More informationAnswer key to problem set # 2 ECON 342 J. Marcelo Ochoa Spring, 2009
Aswer key to problem set # ECON 34 J. Marcelo Ochoa Sprg, 009 Problem. For T cosder the stadard pael data model: y t x t β + α + ǫ t a Numercally compare the fxed effect ad frst dfferece estmates. b Compare
More information