Lecture Note to Rice Chapter 8
|
|
- Andra Young
- 5 years ago
- Views:
Transcription
1 ECON 430 HG revsed Nov 06 Lecture Note to Rce Chapter 8 Radom matrces Let Y, =,,, m, =,,, be radom varables (r.v. s). The matrx Y Y Y Y Y Y Y Y Y Y = m m m s called a radom matrx ( wth a ot m-dmesoal dstrbuto, f ( y, y,, ym )). The expected value of Y s defed as () E( Y ) E( Y) E( Y ) E( Y ) E( Y ) E( Y ) E( Y ) def = E( Ym ) E( Ym) E( Ym) The expectato satsfes the followg rules (whch follows drectly from the defto () combed wth the correspodg lear propertes for the expectato the scalar case):. E( AY + C) = A E( Y) + C where A, C, are ay matrces of costats wth dmesos compatble wth Y (.e. A~ k m, ad C ~ k, where k s arbtrary).. E( AYB + C) = A E( Y ) B + C where A, B, C are ay costat matrces compatble wth Y dmeso so that the product ad sum s well defed... Y [ Y ] E( ') = E( ) ' where A ' deotes the trasposed matrx
2 Y If Y = s a -dmesoal radom vector, t s expectato, μ = E( Y ) (sometmes Y wrtte, μ Y ), s therefore the vector of dvdual expectatos, E( Y) μ μ = E( Y ) = = E( Y) μ Let σ = E ( Y μ )( Y μ ) = σ be the covarace betwee Y ad Y. I partcular we have σ = E ( Y μ ) = var( Y ). The covarace matrx of Y (deoted as cov( Y ) ) s defed as the matrx σ σ var( Y ) cov( Y, Y) Σ= = σ σ cov( Y, Y) var( Y) whch ca be expressed as Y μ cov( Y) = E [( Y μ)( Y μ)' ] = E ( Y μ,, Y μ) = Y μ ( Y μ) ( Y μ)( Y μ ) σ σ () = E = =Σ ( Y μ)( Y μ) ( Y μ) σ σ Example Suppose that vector Y, Y,, Y are d wth expectato E( Y ) Y' = ( Y,, Y ) has expectato = η ad var( Y ) σ =. The the η E( Y ) = = η η
3 3 ad covarace matrx (sce σ = cov( Y, Y ) = 0 for ): cov( ) σ σ Y σ = = σ = σ I where I s the -dmesoal detty matrx. (Ed of example) If Y' = ( Y,, Y ) s a radom vector, ad A a p costat matrx, we obta from.-. (ad the fact that ( BC)' = C' B' for matrces B ad C): () E( AY) = A E( Y) = Aμ ad (3) cov( AY) = A cov( Y) A' = AΣ A' (.e. a p p matrx) whch follows from [ μ μ ] [ μ μ ] [ μ μ ] cov( AY) = E ( AY A )( AY A )' = E A( Y )( Y )' A' = A E ( Y )( Y )' A' = AΣA' I partcular, f Z s a lear combato of Y,, Y,.e. Z = ay + + ay, the (4) var( Z ) = var( ay ' ) = a' Σ a where a' = ( a,, a ) ad Σ = cov( Y ). Y Z = ( a,, a ) = a' Y ca be cosdered a Y Z ' = Z, ad, therefore, cov( Z ) = var( Z) (.e., [Proof: Sce that [ ] matrx, we must have cov( Z) = E ( Z E( Z))( Z E( Z)) ' = E ( Z E( Z)) = var( Z) ). We the see that (4) s a specal case of (3) wth A= a' ]
4 4 Example Ordary least squares (OLS) Cosder the stadard multple regresso model wth oe respose, Y, ad p explaatory varables (5) β0 β βp p Y = + x + + x + u for =,,, where, for smplcty, all x are cosdered fxed, o radom quattes, ad the errors, u, u,, u are assumed to be d ad ormally dstrbuted wth expectato, E( u ) = 0 ad var( u ) = σ. We ca wrte (5) matrx form as follows Y Y β0 + βx + βx + + βpx p u x x p β0 u Y β 0 βx βx βpx p u x x p β = = + = u + Y β0 + βx + βx + + βpx p x u x p β p u The three matrces o the rght we deote by X, β,ad u respectvely. The model ca ow be wrtte as (6) Y = Xβ + u where X s the p (so called) desg matrx, β the ( p + ) vector of regresso coeffcets, ad u the vector of errors. Sce E( u) 0 E( u ) 0 = = = E( u) 0 E( u) 0 (where 0 deotes a vector of zeroes), we get from. (otg that X β s o radom) (7) E( Y) = Xβ + E( u) = Xβ The covarace matrx for Y becomes, sce Y Xβ = u, ad usg example, Σ = cov( Y) = E ( Y Xβ)( Y Xβ)' = E uu' = cov( u) = σ I (8) [ ] [ ] Y The OLS estmator, β, for β s obtaed by mmzg the sum of squares
5 5 = ( ) β0 β βp Q= Y x xp wth respect to β. Dfferetatg Q wth respect to all the β s, ad settg the dervatves equal to 0, leads to the followg system of equatos that the β s must satsfy β0 + ( ) ( ) x x β + + p βp = Y ( ) ( ) ( ) x β + x β + + x x β p p = x Y 0 x β + x x β + + x β = x Y ( ) 0 ( ) ( ) p p p p p Notg that the coeffcets of the left sde are exactly the elemets the ( p+ ) ( p+ ) matrx X ' X, ad that the rght sde, wrtte as a vector, smply s X ' Y, we ca wrte the system more compactly as X ' X β = X ' Y Assumg that X ' X s o sgular (whch ca be show to be the case f o sgle x- varable ca be wrtte exactly as a lear combato of the other x-varables,.e., there s o exact collearty betwee the explaatory varables), we obta the soluto (the OLS estmator) (9) ( ' ) ' β = X X X Y It s ow easy to prove that β s ubased sce, from. ad (7) (0). (7) E( β ) = E ( X ' X) X ' Y = ( X ' X) X 'E( Y) = ( X ' X) X ' Xβ = Ipβ = β Wrtg C = ( X ' X) X ', we have β = CY, ad obta the covarace matrx from (3) ad (8) [ad also usg the rule that the trasposed of a verse square matrx s the verse of the trasposed, A ' = ( A'), whch s see by takg the trasposed of the equato, A A = I. Remember also the AI = A for ay p -matrx A, ad that, f c s a scalar, the c as factor ca be take outsde a matrx product, A ( cb) = cab. ]. ( ) (3) (8) cov( β) = cov( CY ) = CΣ YC ' = C σ I C ' = σ CC ' = σ ( X ' X ) XX '( X ' X )
6 6 Hece () β = σ X X (Ed of example.) cov( ) ( ' ) Multormal dstrbutos We say that the vector X ' = ( X,, X ) s (mult)ormally dstrbuted wth expectato μ = E( X ), ad covarace matrx, Σ = cov( X ) (wrtte shortly X ~ N( μ, Σ )), f the ot pdf s gve by () ( )' x μ Σ ( x μ ) f( x,, x μ, Σ ) = e where ( π ) det( Σ) meas the determat of Σ. x x = x ad det( Σ ) Ths dstrbuto has a lot of coveet mathematcal propertes (see e.g. Greee, Ecoometrc Aalyss, chapter 3, for a summary), but here we oly eed the followg: (3). If X ~ N( μ, Σ ) ad A s a p costat matrx ( p ) ad b a p costat vector, the Y = A X + b~ N(E( Y), cov( Y)) = N( Aμ + b, AΣ A') [For proof see e.g. Greee chapter 3. ] I partcular, ths shows that all margal dstrbutos are also ormal. For example, the margal dstrbuto of X, X s ormal sce X X = AX where 0 0 A = 0 0 whch gves (check!) (4) X X X μ σ σ ~ N(E, cov ) = N(, ), X X X μ σ σ.e. a bvarate ormal dstrbuto Exercse. Show that the pdf (4) as defed (), reduces to the bvarate ormal desty as defed Example F Rce secto 3.3 (both edtos). [Ht:
7 7 Itroduce the correlato, ρ, betwee X ad X, ρ = σ σσ, mplyg X σ = σσρ, ad the determat, det(cov ) = σσ σ = σσ( ρ ) X etc. ] Example 3 (Cotuato of example ) The error vector, u, (6) has expectato 0 ad covarace, Σ =cov( u u ) = σ I. We see from () that sayg that u ~ N(0, σ I ) s the same as sayg that u, u,, u are d ad ormally dstrbuted wth expectato, E( u ) = 0 ad var( u ) = σ. I fact, we have the determat σ σ 0 det( Σ u) = det( σ ) = det = 0 0 σ ad the expoet () reduces to I σ. ( ) ( u E( u))' Σ ( u E( u)) = u' σ I u = u' I u = u' u = u σ σ σ u Substtutg (), shows that the ot dstrbuto () reduces to the product of oedmesoal N(0, σ ) -dstrbutos as the d statemet would mply. By (3), (7), ad (8) we obta that Y s ormally dstrbuted, Y ~ N(E( Y), cov( Y)) = N( Xβ, σ I ), ad, by (3) aga, that ormally dstrbuted ( ' ) ' β = X X X Y s β β β = β σ X X ~ N(E( ),cov( )) N(, ( ' ) ) (Ed of example.)
8 8 3 O the asymptotc dstrbuto for mle estmators (the mult parameter case) I ths secto we wll oly descrbe how to determe the asymptotc dstrbuto for the mle estmator case there are several ukow parameters the model, wthout gog to detals of dervatos ad proofs. A good summary of the theory ca be foud chapter 4 of Greee s book, Ecoometrc Aalyss. See also Rce at the ed of secto 8.5. (both edtos). Suppose that X, X,, X are d wth X ~ f( x θ ) (pdf), where θ ' = ( θ, θ,, θ r ) s a r-dmesoal vector of ukow parameters. The the ot pdf s f( x θ ) ad the log lkelhood s = l( θ ) = l f( x θ ) = The mle estmator, θ, solves r equatos = l f ( x θ ) = 0, =,,, r θ I order to defe the r r Fsher formato matrx that s eeded the asymptotc dstrbuto of θ, we troduce l f( X θ ) m ( θ ) = E, =,,, r θ θ The the Fsher formato matrx for oe observato s defed as m( θ ) m r ( θ ) I( θ ) = mr( θ ) mr r( θ ) Uder regularty codtos smlar to the oe-parameter case (see Greee for detals), we have that the mle satsfes D ) ( θ θ) N(0, I( θ) The defto of covergece dstrbuto for radom vectors s smlar but slghtly more techcal tha the defto for the oe-dmesoal case, ad we skp the detals
9 9 here (see Greee for a precse defto). However, the terpretato of the result s the same as the oe-dmesoal case,.e., that for large, approxmately ~, θ Nθ I ( θ ) Hece we ca say that θ s asymptotcally ubased wth asymptotc covarace matrx, ( Iθ ) ( ). Ths matrx s ukow sce θ s ukow, but ca be cosstetly estmated by replacg θ by θ (or ay other cosstet estmator of θ ). [That θ s cosstet meas smply that θ P θ for all =,,, r]. A geeralzato of Slutsk s lemma to the multvarate case (detals omtted), ow allow us to coclude that, for large approxmately (5) θ ~ N θ, I( θ ) whch s the mportat result that you should kow. Usg (3) we also have (6) approxmately Aθ ~ N Aθ, A I( θ) A' for ay costat, p r matrx A. From ths we get the followg: Let k ( θ ) deote elemet, I( θ ). The the estmated asymptotc varace of θ s the -th elemet o the ma dagoal the estmated covarace matrx,.e. k ( )/. θ [Follows from (6). I fact, let a ' = (0,,,,0) where the s posto ad zeroes elsewhere. The from (6) approx. k ( ) θ θ = a' θ ~ Na' θ, a' I( θ) a= N θ, ] Hece, we obta a approxmate α CI for θ : θ ± zα k( θ) where z α s the upper α -pot N(0,).
10 0 Example 4. Assume we wat a CI for the trasformed parameter, η = θ θ. Ths we obta from (6): Let b ' = (,,0,,0). The, by (6), approx. ' ( ) ' ~ ( ', bi θ b η = θ θ = b θ N b θ = Nθ θ, ( k( θ) + k( θ) k( θ)) whch leads to the approxmate α CI for θ θ: θ θ ± z α k( θ) + k( θ) k( θ ) [Note that all covarace matrces are symmetrc. Hece k ( θ ) = k( θ ). ] (Ed of example.) Example 5 (O example C Rce secto 8.5 (both edtos) precptato data) Let X be the amout of precptato for rastorm o., =,,, ( = 7 observatos). Model: X, X,, X are d wth X ~ Γ ( α, λ). The ot dstrbuto s α α λ X, X,, X ~ f( x αλ, ) = ( xx x) e = Γ ( α) λ x The log lkelhood s (7) l ( α, λ) = αl λ+ ( α ) l x λ x l Γ( α) The frst dervatves of l are l = lλ + l x l Γ( α ) α α l α = x λ λ Settg the dervatves equal to zero ad solvg wth respect to α ad λ, gves the mle estmators α ad λ. [Note. There are o explct formulas for the soluto, they must be foud by umercal teratos. For example, Excel works well ths case by the Solver
11 module: Choose two cells for the argumets α ad λ, wth start values e.g. at the momet estmates, ad the a thrd cell for the fucto (7). The use Solver to maxmze (7). Ths ca also be doe STATA by the ml-commad, but slghtly more volved.] Usg hs program, Rce obtaed the mle estmates. α = 0, 44 ad λ =, 96 We wat approxmate 90% CI s for α ad λ based o the asymptotc ormal dstrbuto of α ad λ. I order to calculate the asymptotc stadard errors we eed the so called d- ad trgamma fuctos: Dgamma fucto: ψ ( α) = l Γ( α) α Trgamma fucto: ψ '( α) = l Γ( α) α Both fuctos ca be calculated STATA (uder the ames dgamma ad trgamma). We eed the Fsher formato matrx: gvg l f ( X α, λ) = αlλ l Γ ( α) + ( α )l X λx l f α = l λ ψ( α) + l X ad l f λ α = X λ Hece l f α = ψ '( α) (trgamma) l f l f = = α λ λ α λ l f λ α = λ Hece the Fsher formato matrx for oe observato
12 ψ '( α) ψ '( α) λ λ I( αλ, ) = E = α α λ λ λ λ The verse of a symmetrc matrx s a c b c = c b ab c c a Hece I( αλ, ) α λ λ α λ = = αψ '( α ) αψ '( α) λ λψ' ( α) ψ '( α) λ λ λ We obta a estmate of ths by substtutg the mle, α = 0,44 ad λ =, 96, for α ad λ I( αλ, ) α λ 0, 5903,53 = = αψ '( α) λ λ ψ '( α),53 3,8770 Here we foud ψ '( α ) = 6,869 from STATA by the commad: d trgamma(0.44) From the theory we have that α approx. α ~ N,, where the asymptotc covarace s C λ λ 0,004 0, C I(, ) αλ = = 0, , Hece the asymptotc stadard errors se( α ) = 0,004 = 0,03378 ad se( λ ) = 0, = 0,468
13 3 Accordg to the theory we the obta approxmate 90% CI for α ad λ α ±, 64 se( α) = 0, 44 ± (, 64)(0, 03378) = [0,386, 0, 496] λ±,64 se( λ) =,96 ± (,64)(0,47) = [,55,,37] Rce (example E, secto 8.5.3)) obtas approxmate 90% CI s by the parametrc bootstrap method: α : [0,404, 0,53] λ : [,46,,3] The dfferece betwee the asymptotc tervals ad the bootstrap tervals does ot appear to be substatal. Wth as much as 7 observatos t s to be expected that the asymptotc theory should work well.
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted
More informationX X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then
Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationLecture 3 Probability review (cont d)
STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationρ < 1 be five real numbers. The
Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace
More informationIntroduction to Matrices and Matrix Approach to Simple Linear Regression
Itroducto to Matrces ad Matrx Approach to Smple Lear Regresso Matrces Defto: A matrx s a rectagular array of umbers or symbolc elemets I may applcatos, the rows of a matrx wll represet dvduals cases (people,
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
More informationAnswer key to problem set # 2 ECON 342 J. Marcelo Ochoa Spring, 2009
Aswer key to problem set # ECON 34 J. Marcelo Ochoa Sprg, 009 Problem. For T cosder the stadard pael data model: y t x t β + α + ǫ t a Numercally compare the fxed effect ad frst dfferece estmates. b Compare
More informationChapter 4 Multiple Random Variables
Revew for the prevous lecture: Theorems ad Examples: How to obta the pmf (pdf) of U = g (, Y) ad V = g (, Y) Chapter 4 Multple Radom Varables Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples:
More informationSTK4011 and STK9011 Autumn 2016
STK4 ad STK9 Autum 6 Pot estmato Covers (most of the followg materal from chapter 7: Secto 7.: pages 3-3 Secto 7..: pages 3-33 Secto 7..: pages 35-3 Secto 7..3: pages 34-35 Secto 7.3.: pages 33-33 Secto
More informationMultivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
More informationFunctions of Random Variables
Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationQualifying Exam Statistical Theory Problem Solutions August 2005
Qualfyg Exam Statstcal Theory Problem Solutos August 5. Let X, X,..., X be d uform U(,),
More informationLecture Notes 2. The ability to manipulate matrices is critical in economics.
Lecture Notes. Revew of Matrces he ablt to mapulate matrces s crtcal ecoomcs.. Matr a rectagular arra of umbers, parameters, or varables placed rows ad colums. Matrces are assocated wth lear equatos. lemets
More informationLecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
More information( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model
Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationMultiple Linear Regression Analysis
LINEA EGESSION ANALYSIS MODULE III Lecture - 4 Multple Lear egresso Aalyss Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Cofdece terval estmato The cofdece tervals multple
More informationX ε ) = 0, or equivalently, lim
Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece
More informationPoint Estimation: definition of estimators
Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.
More informationChapter 5 Properties of a Random Sample
Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample
More informationDimensionality Reduction and Learning
CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that
More information1 Solution to Problem 6.40
1 Soluto to Problem 6.40 (a We wll wrte T τ (X 1,...,X where the X s are..d. wth PDF f(x µ, σ 1 ( x µ σ g, σ where the locato parameter µ s ay real umber ad the scale parameter σ s > 0. Lettg Z X µ σ we
More informationENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections
ENGI 441 Jot Probablty Dstrbutos Page 7-01 Jot Probablty Dstrbutos [Navd sectos.5 ad.6; Devore sectos 5.1-5.] The jot probablty mass fucto of two dscrete radom quattes, s, P ad p x y x y The margal probablty
More informationChapter 9 Jordan Block Matrices
Chapter 9 Jorda Block atrces I ths chapter we wll solve the followg problem. Gve a lear operator T fd a bass R of F such that the matrx R (T) s as smple as possble. f course smple s a matter of taste.
More informationDr. Shalabh. Indian Institute of Technology Kanpur
Aalyss of Varace ad Desg of Expermets-I MODULE -I LECTURE - SOME RESULTS ON LINEAR ALGEBRA, MATRIX THEORY AND DISTRIBUTIONS Dr. Shalabh Departmet t of Mathematcs t ad Statstcs t t Ida Isttute of Techology
More informationECON 5360 Class Notes GMM
ECON 560 Class Notes GMM Geeralzed Method of Momets (GMM) I beg by outlg the classcal method of momets techque (Fsher, 95) ad the proceed to geeralzed method of momets (Hase, 98).. radtoal Method of Momets
More informationSimulation Output Analysis
Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5
More information{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:
Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed
More informationChapter 4 Multiple Random Variables
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for Chapter 4-5 Notes: Although all deftos ad theorems troduced our lectures ad ths ote are mportat ad you should be famlar wth, but I put those
More informationLecture 9: Tolerant Testing
Lecture 9: Tolerat Testg Dael Kae Scrbe: Sakeerth Rao Aprl 4, 07 Abstract I ths lecture we prove a quas lear lower boud o the umber of samples eeded to do tolerat testg for L dstace. Tolerat Testg We have
More informationGENERALIZED METHOD OF MOMENTS CHARACTERISTICS AND ITS APPLICATION ON PANELDATA
Sc.It.(Lahore),26(3),985-990,2014 ISSN 1013-5316; CODEN: SINTE 8 GENERALIZED METHOD OF MOMENTS CHARACTERISTICS AND ITS APPLICATION ON PANELDATA Beradhta H. S. Utam 1, Warsoo 1, Da Kurasar 1, Mustofa Usma
More information9 U-STATISTICS. Eh =(m!) 1 Eh(X (1),..., X (m ) ) i.i.d
9 U-STATISTICS Suppose,,..., are P P..d. wth CDF F. Our goal s to estmate the expectato t (P)=Eh(,,..., m ). Note that ths expectato requres more tha oe cotrast to E, E, or Eh( ). Oe example s E or P((,
More informationLecture Notes to Rice Chapter 5
ECON 430 Revsed Sept. 06 Lecture Notes to Rce Chapter 5 By H. Goldste. Chapter 5 gves a troducto to probablstc approxmato methods, but s suffcet for the eeds of a adequate study of ecoometrcs. The commo
More informationSpecial Instructions / Useful Data
JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth
More informationMOLECULAR VIBRATIONS
MOLECULAR VIBRATIONS Here we wsh to vestgate molecular vbratos ad draw a smlarty betwee the theory of molecular vbratos ad Hückel theory. 1. Smple Harmoc Oscllator Recall that the eergy of a oe-dmesoal
More informationMATH 247/Winter Notes on the adjoint and on normal operators.
MATH 47/Wter 00 Notes o the adjot ad o ormal operators I these otes, V s a fte dmesoal er product space over, wth gve er * product uv, T, S, T, are lear operators o V U, W are subspaces of V Whe we say
More informationLecture 3. Sampling, sampling distributions, and parameter estimation
Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called
More informationMidterm Exam 1, section 1 (Solution) Thursday, February hour, 15 minutes
coometrcs, CON Sa Fracsco State Uversty Mchael Bar Sprg 5 Mdterm am, secto Soluto Thursday, February 6 hour, 5 mutes Name: Istructos. Ths s closed book, closed otes eam.. No calculators of ay kd are allowed..
More informationCHAPTER 3 POSTERIOR DISTRIBUTIONS
CHAPTER 3 POSTERIOR DISTRIBUTIONS If scece caot measure the degree of probablt volved, so much the worse for scece. The practcal ma wll stck to hs apprecatve methods utl t does, or wll accept the results
More informationSTK3100 and STK4100 Autumn 2017
SK3 ad SK4 Autum 7 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Sectos 4..5, 4.3.5, 4.3.6, 4.4., 4.4., ad 4.4.3 Sectos 5.., 5.., ad 5.5. Ørulf Borga Deartmet of Mathematcs
More informationSTK3100 and STK4100 Autumn 2018
SK3 ad SK4 Autum 8 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Cofdece tervals by vertg tests Cosder a model wth a sgle arameter β We may obta a ( α% cofdece terval for
More informationMaximum Likelihood Estimation
Marquette Uverst Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 08 b Marquette Uverst Maxmum Lkelhood Estmato We have bee sag that ~
More informationESS Line Fitting
ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here
More informationCHAPTER VI Statistical Analysis of Experimental Data
Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca
More informationENGI 3423 Simple Linear Regression Page 12-01
ENGI 343 mple Lear Regresso Page - mple Lear Regresso ometmes a expermet s set up where the expermeter has cotrol over the values of oe or more varables X ad measures the resultg values of aother varable
More information6.867 Machine Learning
6.867 Mache Learg Problem set Due Frday, September 9, rectato Please address all questos ad commets about ths problem set to 6.867-staff@a.mt.edu. You do ot eed to use MATLAB for ths problem set though
More informationPart 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971))
art 4b Asymptotc Results for MRR usg RESS Recall that the RESS statstc s a specal type of cross valdato procedure (see Alle (97)) partcular to the regresso problem ad volves fdg Y $,, the estmate at the
More informationMidterm Exam 1, section 2 (Solution) Thursday, February hour, 15 minutes
coometrcs, CON Sa Fracsco State Uverst Mchael Bar Sprg 5 Mdterm xam, secto Soluto Thursda, Februar 6 hour, 5 mutes Name: Istructos. Ths s closed book, closed otes exam.. No calculators of a kd are allowed..
More information4. Standard Regression Model and Spatial Dependence Tests
4. Stadard Regresso Model ad Spatal Depedece Tests Stadard regresso aalss fals the presece of spatal effects. I case of spatal depedeces ad/or spatal heterogeet a stadard regresso model wll be msspecfed.
More informationM2S1 - EXERCISES 8: SOLUTIONS
MS - EXERCISES 8: SOLUTIONS. As X,..., X P ossoλ, a gve that T ˉX, the usg elemetary propertes of expectatos, we have E ft [T E fx [X λ λ, so that T s a ubase estmator of λ. T X X X Furthermore X X X From
More informationProbability and. Lecture 13: and Correlation
933 Probablty ad Statstcs for Software ad Kowledge Egeers Lecture 3: Smple Lear Regresso ad Correlato Mocha Soptkamo, Ph.D. Outle The Smple Lear Regresso Model (.) Fttg the Regresso Le (.) The Aalyss of
More informationENGI 4421 Propagation of Error Page 8-01
ENGI 441 Propagato of Error Page 8-01 Propagato of Error [Navd Chapter 3; ot Devore] Ay realstc measuremet procedure cotas error. Ay calculatos based o that measuremet wll therefore also cota a error.
More informationChapter 2 - Free Vibration of Multi-Degree-of-Freedom Systems - II
CEE49b Chapter - Free Vbrato of Mult-Degree-of-Freedom Systems - II We ca obta a approxmate soluto to the fudametal atural frequecy through a approxmate formula developed usg eergy prcples by Lord Raylegh
More informationSimple Linear Regression
Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato
More informationWu-Hausman Test: But if X and ε are independent, βˆ. ECON 324 Page 1
Wu-Hausma Test: Detectg Falure of E( ε X ) Caot drectly test ths assumpto because lack ubased estmator of ε ad the OLS resduals wll be orthogoal to X, by costructo as ca be see from the momet codto X'
More informationLINEAR REGRESSION ANALYSIS
LINEAR REGRESSION ANALYSIS MODULE V Lecture - Correctg Model Iadequaces Through Trasformato ad Weghtg Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Aalytcal methods for
More informationSummary of the lecture in Biostatistics
Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the
More informationLogistic regression (continued)
STAT562 page 138 Logstc regresso (cotued) Suppose we ow cosder more complex models to descrbe the relatoshp betwee a categorcal respose varable (Y) that takes o two (2) possble outcomes ad a set of p explaatory
More information2SLS Estimates ECON In this case, begin with the assumption that E[ i
SLS Estmates ECON 3033 Bll Evas Fall 05 Two-Stage Least Squares (SLS Cosder a stadard lear bvarate regresso model y 0 x. I ths case, beg wth the assumto that E[ x] 0 whch meas that OLS estmates of wll
More informationThird handout: On the Gini Index
Thrd hadout: O the dex Corrado, a tala statstca, proposed (, 9, 96) to measure absolute equalt va the mea dfferece whch s defed as ( / ) where refers to the total umber of dvduals socet. Assume that. The
More informationSTA 108 Applied Linear Models: Regression Analysis Spring Solution for Homework #1
STA 08 Appled Lear Models: Regresso Aalyss Sprg 0 Soluto for Homework #. Let Y the dollar cost per year, X the umber of vsts per year. The the mathematcal relato betwee X ad Y s: Y 300 + X. Ths s a fuctoal
More informationThe number of observed cases The number of parameters. ith case of the dichotomous dependent variable. the ith case of the jth parameter
LOGISTIC REGRESSION Notato Model Logstc regresso regresses a dchotomous depedet varable o a set of depedet varables. Several methods are mplemeted for selectg the depedet varables. The followg otato s
More informationCHAPTER 4 RADICAL EXPRESSIONS
6 CHAPTER RADICAL EXPRESSIONS. The th Root of a Real Number A real umber a s called the th root of a real umber b f Thus, for example: s a square root of sce. s also a square root of sce ( ). s a cube
More information12.2 Estimating Model parameters Assumptions: ox and y are related according to the simple linear regression model
1. Estmatg Model parameters Assumptos: ox ad y are related accordg to the smple lear regresso model (The lear regresso model s the model that says that x ad y are related a lear fasho, but the observed
More informationTraining Sample Model: Given n observations, [[( Yi, x i the sample model can be expressed as (1) where, zero and variance σ
Stat 74 Estmato for Geeral Lear Model Prof. Goel Broad Outle Geeral Lear Model (GLM): Trag Samle Model: Gve observatos, [[( Y, x ), x = ( x,, xr )], =,,, the samle model ca be exressed as Y = µ ( x, x,,
More informationModule 7. Lecture 7: Statistical parameter estimation
Lecture 7: Statstcal parameter estmato Parameter Estmato Methods of Parameter Estmato 1) Method of Matchg Pots ) Method of Momets 3) Mamum Lkelhood method Populato Parameter Sample Parameter Ubased estmato
More informationε. Therefore, the estimate
Suggested Aswers, Problem Set 3 ECON 333 Da Hugerma. Ths s ot a very good dea. We kow from the secod FOC problem b) that ( ) SSE / = y x x = ( ) Whch ca be reduced to read y x x = ε x = ( ) The OLS model
More informationSection 2 Notes. Elizabeth Stone and Charles Wang. January 15, Expectation and Conditional Expectation of a Random Variable.
Secto Notes Elzabeth Stoe ad Charles Wag Jauar 5, 9 Jot, Margal, ad Codtoal Probablt Useful Rules/Propertes. P ( x) P P ( x; ) or R f (x; ) d. P ( xj ) P (x; ) P ( ) 3. P ( x; ) P ( xj ) P ( ) 4. Baes
More informationObjectives of Multiple Regression
Obectves of Multple Regresso Establsh the lear equato that best predcts values of a depedet varable Y usg more tha oe eplaator varable from a large set of potetal predctors {,,... k }. Fd that subset of
More informationECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity
ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data
More informationChapter 3 Sampling For Proportions and Percentages
Chapter 3 Samplg For Proportos ad Percetages I may stuatos, the characterstc uder study o whch the observatos are collected are qualtatve ature For example, the resposes of customers may marketg surveys
More informationLecture 2: The Simple Regression Model
Lectre Notes o Advaced coometrcs Lectre : The Smple Regresso Model Takash Yamao Fall Semester 5 I ths lectre we revew the smple bvarate lear regresso model. We focs o statstcal assmptos to obta based estmators.
More informationIntroduction to F-testing in linear regression models
ECON 43 Harald Goldste, revsed Nov. 4 Itroducto to F-testg lear regso s (Lecture ote to lecture Frday 4..4) Itroducto A F-test usually s a test where several parameters are volved at oce the ull hypothess
More informationBayesian Inferences for Two Parameter Weibull Distribution Kipkoech W. Cheruiyot 1, Abel Ouko 2, Emily Kirimi 3
IOSR Joural of Mathematcs IOSR-JM e-issn: 78-578, p-issn: 9-765X. Volume, Issue Ver. II Ja - Feb. 05, PP 4- www.osrjourals.org Bayesa Ifereces for Two Parameter Webull Dstrbuto Kpkoech W. Cheruyot, Abel
More informationBAYESIAN INFERENCES FOR TWO PARAMETER WEIBULL DISTRIBUTION
Iteratoal Joural of Mathematcs ad Statstcs Studes Vol.4, No.3, pp.5-39, Jue 06 Publshed by Europea Cetre for Research Trag ad Developmet UK (www.eajourals.org BAYESIAN INFERENCES FOR TWO PARAMETER WEIBULL
More informationClass 13,14 June 17, 19, 2015
Class 3,4 Jue 7, 9, 05 Pla for Class3,4:. Samplg dstrbuto of sample mea. The Cetral Lmt Theorem (CLT). Cofdece terval for ukow mea.. Samplg Dstrbuto for Sample mea. Methods used are based o CLT ( Cetral
More informationBounds on the expected entropy and KL-divergence of sampled multinomial distributions. Brandon C. Roy
Bouds o the expected etropy ad KL-dvergece of sampled multomal dstrbutos Brado C. Roy bcroy@meda.mt.edu Orgal: May 18, 2011 Revsed: Jue 6, 2011 Abstract Iformato theoretc quattes calculated from a sampled
More informationSTA302/1001-Fall 2008 Midterm Test October 21, 2008
STA3/-Fall 8 Mdterm Test October, 8 Last Name: Frst Name: Studet Number: Erolled (Crcle oe) STA3 STA INSTRUCTIONS Tme allowed: hour 45 mutes Ads allowed: A o-programmable calculator A table of values from
More informationbest estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best
Error Aalyss Preamble Wheever a measuremet s made, the result followg from that measuremet s always subject to ucertaty The ucertaty ca be reduced by makg several measuremets of the same quatty or by mprovg
More information4 Inner Product Spaces
11.MH1 LINEAR ALGEBRA Summary Notes 4 Ier Product Spaces Ier product s the abstracto to geeral vector spaces of the famlar dea of the scalar product of two vectors or 3. I what follows, keep these key
More informationSTATISTICAL INFERENCE
(STATISTICS) STATISTICAL INFERENCE COMPLEMENTARY COURSE B.Sc. MATHEMATICS III SEMESTER ( Admsso) UNIVERSITY OF CALICUT SCHOOL OF DISTANCE EDUCATION CALICUT UNIVERSITY P.O., MALAPPURAM, KERALA, INDIA -
More informationECON 482 / WH Hong The Simple Regression Model 1. Definition of the Simple Regression Model
ECON 48 / WH Hog The Smple Regresso Model. Defto of the Smple Regresso Model Smple Regresso Model Expla varable y terms of varable x y = β + β x+ u y : depedet varable, explaed varable, respose varable,
More informationDerivation of 3-Point Block Method Formula for Solving First Order Stiff Ordinary Differential Equations
Dervato of -Pot Block Method Formula for Solvg Frst Order Stff Ordary Dfferetal Equatos Kharul Hamd Kharul Auar, Kharl Iskadar Othma, Zara Bb Ibrahm Abstract Dervato of pot block method formula wth costat
More informationLecture Notes Types of economic variables
Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Aalyss of Varace ad Desg of Exermets-I MODULE II LECTURE - GENERAL LINEAR HYPOTHESIS AND ANALYSIS OF VARIANCE Dr Shalabh Deartmet of Mathematcs ad Statstcs Ida Isttute of Techology Kaur Tukey s rocedure
More informationFeature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)
CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.
More informationPoint Estimation: definition of estimators
Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.
More informationHandout #8. X\Y f(x) 0 1/16 1/ / /16 3/ / /16 3/16 0 3/ /16 1/16 1/8 g(y) 1/16 1/4 3/8 1/4 1/16 1
Hadout #8 Ttle: Foudatos of Ecoometrcs Course: Eco 367 Fall/05 Istructor: Dr. I-Mg Chu Lear Regresso Model So far we have focused mostly o the study of a sgle radom varable, ts correspodg theoretcal dstrbuto,
More informationThe Mathematical Appendix
The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.
More information18.413: Error Correcting Codes Lab March 2, Lecture 8
18.413: Error Correctg Codes Lab March 2, 2004 Lecturer: Dael A. Spelma Lecture 8 8.1 Vector Spaces A set C {0, 1} s a vector space f for x all C ad y C, x + y C, where we take addto to be compoet wse
More informationMatrix Algebra Tutorial With Examples in Matlab
Matr Algebra Tutoral Wth Eamples Matlab by Klaus Moelter Departmet of Agrcultural ad Appled Ecoomcs Vrga Tech emal: moelter@vt.edu web: http://faculty.ageco.vt.edu/moelter/ Specfcally desged as a / day
More informationTHE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA
THE ROYAL STATISTICAL SOCIETY EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA PAPER II STATISTICAL THEORY & METHODS The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for
More informationLecture Notes Forecasting the process of estimating or predicting unknown situations
Lecture Notes. Ecoomc Forecastg. Forecastg the process of estmatg or predctg ukow stuatos Eample usuall ecoomsts predct future ecoomc varables Forecastg apples to a varet of data () tme seres data predctg
More informationå 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018
Chrs Pech Fal Practce CS09 Dec 5, 08 Practce Fal Examato Solutos. Aswer: 4/5 8/7. There are multle ways to obta ths aswer; here are two: The frst commo method s to sum over all ossbltes for the rak of
More informationLecture 07: Poles and Zeros
Lecture 07: Poles ad Zeros Defto of poles ad zeros The trasfer fucto provdes a bass for determg mportat system respose characterstcs wthout solvg the complete dfferetal equato. As defed, the trasfer fucto
More informationExtreme Value Theory: An Introduction
(correcto d Extreme Value Theory: A Itroducto by Laures de Haa ad Aa Ferrera Wth ths webpage the authors ted to form the readers of errors or mstakes foud the book after publcato. We also gve extesos for
More information