Solutions: Homework 3
|
|
- Emery McDaniel
- 6 years ago
- Views:
Transcription
1 Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe that Y i Normal( xi, ):i =,...,. With this iformatio, the log-likelihood fuctio relative to a observed sample y =(y,...,y ) R is `( ; y) = l( ) (y i x i ). The fuctio `( ; y) : R is cotiuously di eretiable i. I particular `0( ; y) = x i (y i x i )= h X x i y i i : = x i. Notice that Also This meas that `( ; y) : `0( ; y) > 0, < `0( ; y) < 0, > x i y i x i y i. lim `( ; y) =.!± R has a global maximum (which depeds o y) at MAX(y) = x i y i Fially, the MLE of will be the fuctio MAX : R! R evaluated at the sample Y =(Y,...,Y ), i.e ˆ = MAX (Y )= x i Y i. Fid the distributio of ˆ. Solutio: Observe that ˆ ca be expressed as ˆ = c i Y i : c i = x i. The last relatio says that ˆ is a liear combiatio of idepedet radom variables with Normal distributio. The ˆ Normal(µ, )where µ =E[ˆ] = c i E[Y i ]= = Var[ ˆ] = c i Var[Y i ]= S xx x i ( x i )= x i ( )=.
2 Fid the CRLB for estimatig with ˆ. Solutio: Because Y,...,Y are radom variables whose distributio belogs to the Expoetial Family CRLB( )= E[`00 ( ; Y )] : where `00 ( ; y) = for ay observed sample y R. This shows that `00 ( ; Y ) is costat as a fuctio of Y.The E[`00 ( ; Y )] = S xx ) CRLB( )=. Show that ˆ = UMVUE( ). Solutio: We proved earlier that Show that ˆ Normal,. From this it follows that ˆ is a ubiased estimator for belogs to the Expoetial Family ad This proves that ˆ = UMVUE( ). Var[ ˆ] = = CRLB( ):8 R. T = X i. Also ˆ has a distributio that is the UMVUE for R + if X,...,X IID Normal(0, ). Solutio: Give that the distributios for X,...,X belog to the Expoetial Family CRLB( )= E[`00 ( ; X)] where X = (X,...,X ). The associated log-likelihood fuctio relative to a observed sample x =(x,...,x ) R is Notice that With this iformatio E[`00 ( ; X)] = O the other had `( ; x) = l( ) l( ) `00 ( ; x) = ( ) 3 Var[X ] T = ( ) ( ) 3 x i. x i. ( ) = ( ) ) CRLB( )= ( ). U : U = with W i =( ) X i. Because W,...,W IID (), U (). The T has a distributio that belogs to the Expoetial Family (Chage of Variables) ad W i E[T ]= E[U ]=
3 i.e, a ubiased estimator for. Also Var[T ]= ( ) Var[U ]= ( ) = CRLB( ):8 R +. From this calculatios it follows that T = UMVUE( ). Let X,...,X IID Beroulli( ) with (0, ). Fid the Bayes Estimator ˆ B of with respect to the Uiform(0, ) prior uder the Loss Fuctio L(t, ) = (t ) : t R, (0, ). ( ) Solutio: The first step is to deduce the posterior distributio for give a observed sample x =(x,...,x ) {0, }. With the provided iformatio h Y ( x) / xi ( ) ii( xi (0, )) = s ( ) s I( (0, )) : s = x i. Notice that this expressio is the fuctioal part of a Beta(, ) desity with parameters = s + ad = s +. The x Beta(, ). The ext step is to calculate the posterior expected loss for our observed sample. I this case E[L(t, ) x] = B(, ) Z (0,) (t ) s ( ) s d : t R Observe that f( ) = s ( ) s I( (0, )) : R is proportioal to a Beta(, ) desity with parameters = s ad = s. Because of this observatio E[L(t, ) x] = B(, ) B(, ) E[(t? ) x] with? x Beta(, ). As a cosequece of the above relatio, the value t MIN (x) R (which depeds o x) that miimizes E[L(t, ) x] is the same that miimizes E[(t? ) x]. This last problem has a explicit solutio (Squared Error Loss) t MIN (x) =E[? x] = + = s = x. Fially, the Bayes Estimator will be the fuctio t MIN : {0, }! R evaluated at the sample X =(X,...,X ), i.e ˆ B = t MIN (X) =X. Solutios: Homework 4 Let X,...,X be a radom sample from a Normal(, ) populatio ( kow). Cosider estimatig usig Squared Error Loss ad a Normal(µ, ) prior distributio for. Let be the Bayes Estimator for. Show that the posterior distributio of is Normal(m,v ) with parameters m = x + + µ + v = +. 3
4 Solutio: Our task it s reduce to idetify the fuctioal part of the posterior desity for give a observed sample x =(x,...,x ) R. Igorig all terms that do t deped o Notice that exp Fially h Y ( x) / exp =exp h X (x i ) + (x i ) i exp h X (x i ) + ( i µ) / exp ( x) / exp which meas that x Normal(m,v ). =exp =exp / exp ( m ) v ( µ) I( R) ( i µ) I( R). h + h + h m i v ( m ) v. v I( R) x + µ i x + µ Fid the Bayes Estimator ˆ B of uder Squared Error Loss. Solutio: I geeral, whe the posterior distributio admits at least secod-order momets, the Bayes Estimator uder Squared Error Loss is the Posterior Mea. I our case, the Normal distributio has fiite momets for all orders. The with (X) =E[ X] = X + + µ + a = + µ b = +. = a + b Complete parts (a)-(c) of Problem 7.6. Solutio: For ay costats a R ad b R\{0}, the estimator (X) =ax +b has a Normal(a + b, a ) distributio give (ivariace uder liear trasformatios). The associated Risk Fuctio (Squared Error Loss) is R(, ) =(E[ (X) ] ) + Var[ (X) ] =((a + b) ) + a =( (a ) + b) + a =(b ( a) ) + a. I particular, for (X) =a X + b the coe ciets a R + ad b R satisfy the relatio b =( a )µ. The R(, ) =(b ( a ) ) + a =( a ) (µ ) + a = c (µ ) +( c ) : c = a = +. i 4
5 Fially, the associated Bayes Risk for the Bayes Estimator is B(, ) =E[R(, )] = c E[(µ ) ]+( c ) = ( ) ( + ) + ( ) ( + ) = h = + = c. i Solutios: Homework 5 = c +( c ) Cosider a radom sample of size from a distributio with discrete pdf f(x p) =p( for x =0,,..., ad 0 otherwise. p) x Before proceedig otice that E(X )=( p)/p ad Var(X )=( p)/p.. The MLE of p is the value that solves the followig d dp log f(x p) =0) d dp log p ( p) P i xi =0 P i ) x i =0 p ( p) ) ( p) =p X i x i ) p = P i x i + Now sice d dp log f(x p) = P i x i p < 0, ˆp is ideed a maximum. Further, the ( p) likelihood at p = ad p = 0 is zero. Thus, the MLE of p is ˆp = P i x i +. By the ivariace properties of MLEs, the MLE of =( p)/p is ˆ = ˆp = ˆp X. 3. The CRLB associated with ubiased estimators of is apple d dp I (p). Now d dp = d dp ( p)/p = /p ad I (p) =E [ ddp log f(x p)] apple d = E dp apple = E log f(x p) P i x i ( p) p = p + p p ( p) = p ( p) Thus the CRLB associated with ubiased estimators of is apple d dp I (p) = [ /p ] p ( p) = p p 5
6 4. Sice the variace of X attais the CRLB (i.e., Var( X) = (/)Var(X ) = ( p)/(p )), the ˆ is UMVUE for 5. First otice that ˆ is ubiased for. That is E( X) =( p)/p. Thesice lim E[(ˆ ) ]= lim Var( X) p = lim!!! p =0 ˆ is MSEC 6. From the asymptotic results of MLEs we have p (ˆp p) d! N(0, /I (p)) ) p (ˆp p) d! N(0,p ( p)) The from the Delta Method results we have p (ˆ ) d! N(0, (d /dp) /I (p)) ) p (ˆ ) d! N(0, (/p 4 )p ( p)) ) p (ˆ ) d! N(0, ( p)/p ) Thus, we ca say (somewhat iformally) that ˆ. N(, ( p)/(p )). 7. First the risk fuctio associated with ˆ = X. (Notice that i what follows that the risk fuctios are fuctios of p. If so desired, they could be made fuctios of.) " # Rˆ (p) (ˆ ) =E[L(ˆ, )] = E + = h + E (ˆ ) i = + Var( X) sice ˆ is ubiased for =( p)/p p p = + = /(p) + = /(p) + = 6
7 Now for the risk fuctio associated with = X/( + ) " # R (p) ( ) =E[L(, )] = E + = h + E ( ) i = + [Var( X/( + )) + bias( X/( + )) ] " = # p + + p + p p + p p " = # + + p + + " = # + + p + + = + p ( + ) R(p) R θ ~ (p) R θ^(p) p 7
8 Solutios: Homework 6. (C& B 7.58 pg. 366) Let X be a observatio from the pdf f(x ) = x ( ) x, x =, 0, [0, ] (a) Fid the MLE of. To fid the MLE of for oe observatio of X it is eough to determie the value of [0, ] that maximizes f(x ) for each of the 3 possible X values. Notice that f(x = ) =f(x = ) = /. Ad = is the value of that maximizes f(x ). Similarly f(x =0 ) = ad = 0 is the value of that maximizes f(x ). Therefore the MLE is ˆ = x = 0 x =0 ) ˆ = x (b) Defie a estimator T (X) = x = 0 otherwise Show that T (X) is a ubiased estimator of theta. Notice that E [T (X)] = P (X = ) = ( /) = (c) Fid a estimator that is better tha T (X) ad prove that it is better. From cotext I am workig uder the assumptio that we are lookig for a better ubiased estimator. First otice that E(ˆ ) =P ( X = ) = P (X = or X = ) = P (X = ) + P (X = ) = /+ / = Thus, ˆ is ubiased. Note further that ad Var(T (X)) = E(T (X) ) [E(T (X)) ]=4P (X = ) = 4( /) = Var(ˆ ) =P ( X = ) = Fially for ay [0, ] we have that Var(ˆ ) apple Var(T (X)) ad therefore the MLE ˆ is a better estimator tha T (X) Alteratively, oe ca show that ˆ is a complete ad su ciet statistic for ad the by Method I coclude that ˆ is UMVUE. First to show that ˆ is use the factorizatio theorem. Let g(,s)=( /) s ( ) s for s = x ad h(x) =I[x {, 0, }]. The f(x ) = g(s, )h(x) ad by the factorizatio theorem x is su ciet for theta. To see that x is also complete, suppose that m( x ) is a fuctio such that E[m( x )] = 0 for all [0, ]. The 0=E[m( x )] = m( )( /) + m( 0 )( ) +m( )( /) = m() + m(0)( ) for all [0, ]. I particular for =0wehave0=m(0) ad for =wehave 0=m(). Therefore if 0 = E [m( x )] for all the m( x ) = 0 for x {, 0, } ) P (m( X ) = 0) = for all [0, ]. By defiitio x is complete. Sice ˆ is a fuctio of a complete ad su ciet statistic it is UMVUE ad therefore better tha T (X) 8
9 . Let X,...,X be iid N(, ), R. (a) By Method I ad sice we are ca assume that P X i is complete if we ca fid a ubiased estimator of ( ) = that is a fuctio of P X i the we are doe. As a first guess, cosider X. Note that X N(, /) ad thus E( X )=Var( X)+E( X) =/ +. This is obviously ot a ubiased estimator of ( ), but T (X) = X / is. Sice T P (X) is ubiased ad a fuctio of a complete ad su ciet statistic (we have show X i to be su ciet for uder N(, )) the T (X) is UMVUE. (b) To fid the Var(T (X)) otice that Z = p ( X ) N(0, ) ad Z () with E(Z ) = ad Var(Z ) =. Additioally, we will use the fact that skewess (as defied as the third cetral momet) is zero for symmetric distributios. That is, for X N(µ, )wehavee(x µ) 3 = 0. Thus for Z N(0, ), E(Z 3 ) = 0. Now we ca rewrite Hece the variace of T (X) is T (X) = X / =[ p ( X )+ p ] (/) / =[Z p ] (/) /. Var(T (X)) = Var( p ( X )+ p ] (/) /) =/ Var(Z Z p + ) =/ [Var(Z )+4 Var(Z) 4 p Cov(Z,Z)] =/ [ p (E(Z 3 ) E(Z )E(Z))] =/ [ p E(Z 3 )] sice E(Z) =0 =/ [ + 4 ] sice E(Z 3 ) = 0 see above =/ +4 / (c) Sice X i are iid, the CRLB of ( ) = is [ 0 ( )] /I ( ) with[ 0 ( )] =4 ad I ( ) = log f(x ) = (/) log( ) (/)(X ) = E (/)(X )( = E [ ] = Thus the CLRB is 4 / (d) Notice that Var(T (X)) = / +4 / =/ + CRLB > CRLB 9
10 3. (7.60 of C & B pg. 366). Fid the best ubiased estimator of / whe X,...,X are iid Gamma(, ) ad is kow. We will show that T = P X i is a complete ad su ciet statistic ad use Method. Notice that f(x, )= ( ) x exp{ x/ }I[x >0] c( )h(x)exp{q( )t(x)}i[x >0] where c( )=, h(x) =x / ( ), q( )=/, ad t(x) =x. Notice further that the support of the distributio A = {x : x>0} does ot deped o ad {q( ): > 0} = { / : > 0} =(, 0) is ope i R. Thusthegamma(, ) distributio with kow is a member of the expoetial family ad the ecessary requiremets are fulfilled to coclude that T = P X i is complete ad su ciet for. A logical first choice for a ubaised estimator of / is /T Note that T gamma(, E(/T )= = = = Z ) ad also that (/T ) 0 ( ) T exp{ T/ }dt Z ( ) T exp{ T/ }dt 0 Z ( ) ( ) 0 ( ) T exp{ T/ }dt itegral of desity is ad Gamma(a) =(a )! ( ) So defie T? =( )/T. Goig through the same procedure it is easy to show that T? is a ubaised estimate of / ad sice it is a fuctio of T (which is complete ad su ciet) the it is UMVUE. 0
11
12
13
14
15
16
Statistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
More informationStatistical Theory MT 2008 Problems 1: Solution sketches
Statistical Theory MT 008 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. a) Let 0 < θ < ad put fx, θ) = θ)θ x ; x = 0,,,... b) c) where α
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationUnbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationReview Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn
Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationLecture 3. Properties of Summary Statistics: Sampling Distribution
Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationEXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY
EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA, 016 MODULE : Statistical Iferece Time allowed: Three hours Cadidates should aswer FIVE questios. All questios carry equal marks. The umber
More informationExponential Families and Bayesian Inference
Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More informationAMS570 Lecture Notes #2
AMS570 Lecture Notes # Review of Probability (cotiued) Probability distributios. () Biomial distributio Biomial Experimet: ) It cosists of trials ) Each trial results i of possible outcomes, S or F 3)
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationStat410 Probability and Statistics II (F16)
Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. Comments:
Recall: STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Commets:. So far we have estimates of the parameters! 0 ad!, but have o idea how good these estimates are. Assumptio: E(Y x)! 0 +! x (liear coditioal
More informationAsymptotic distribution of products of sums of independent random variables
Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationSTATISTICAL INFERENCE
STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More informationLECTURE NOTES 9. 1 Point Estimation. 1.1 The Method of Moments
LECTURE NOTES 9 Poit Estimatio Uder the hypothesis that the sample was geerated from some parametric statistical model, a atural way to uderstad the uderlyig populatio is by estimatig the parameters of
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More information6. Sufficient, Complete, and Ancillary Statistics
Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary
More information2. The volume of the solid of revolution generated by revolving the area bounded by the
IIT JAM Mathematical Statistics (MS) Solved Paper. A eigevector of the matrix M= ( ) is (a) ( ) (b) ( ) (c) ( ) (d) ( ) Solutio: (a) Eigevalue of M = ( ) is. x So, let x = ( y) be the eigevector. z (M
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More informationLecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationLecture 18: Sampling distributions
Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from
More informationSOLUTION FOR HOMEWORK 7, STAT np(1 p) (α + β + n) + ( np + α
SOLUTION FOR HOMEWORK 7, STAT 6331 1 Exerc733 Here we just recall that MSE(ˆp B ) = p(1 p) (α + β + ) + ( p + α 2 α + β + p) 2 The you plug i α = β = (/4) 1/2 After simplificatios MSE(ˆp B ) = 4( 1/2 +
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationBayesian Methods: Introduction to Multi-parameter Models
Bayesia Methods: Itroductio to Multi-parameter Models Parameter: θ = ( θ, θ) Give Likelihood p(y θ) ad prior p(θ ), the posterior p proportioal to p(y θ) x p(θ ) Margial posterior ( θ, θ y) is Iterested
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationHomework for 2/3. 1. Determine the values of the following quantities: a. t 0.1,15 b. t 0.05,15 c. t 0.1,25 d. t 0.05,40 e. t 0.
Name: ID: Homework for /3. Determie the values of the followig quatities: a. t 0.5 b. t 0.055 c. t 0.5 d. t 0.0540 e. t 0.00540 f. χ 0.0 g. χ 0.0 h. χ 0.00 i. χ 0.0050 j. χ 0.990 a. t 0.5.34 b. t 0.055.753
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationEconomics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator
Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters
More informationFirst Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise
First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationSummary. Recap. Last Lecture. Let W n = W n (X 1,, X n ) = W n (X) be a sequence of estimators for
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 17 Asymptotic Evaluatio of oit Estimators Hyu Mi Kag March 19th, 2013 What is a Bayes Risk? What is the Bayes rule Estimator miimizig square
More informationConfidence Level We want to estimate the true mean of a random variable X economically and with confidence.
Cofidece Iterval 700 Samples Sample Mea 03 Cofidece Level 095 Margi of Error 0037 We wat to estimate the true mea of a radom variable X ecoomically ad with cofidece True Mea μ from the Etire Populatio
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationLecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett
Lecture Note 8 Poit Estimators ad Poit Estimatio Methods MIT 14.30 Sprig 2006 Herma Beett Give a parameter with ukow value, the goal of poit estimatio is to use a sample to compute a umber that represets
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationMaximum Likelihood Estimation
ECE 645: Estimatio Theory Sprig 2015 Istructor: Prof. Staley H. Cha Maximum Likelihood Estimatio (LaTeX prepared by Shaobo Fag) April 14, 2015 This lecture ote is based o ECE 645(Sprig 2015) by Prof. Staley
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationLecture 10 October Minimaxity and least favorable prior sequences
STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least
More informationLecture 23: Minimal sufficiency
Lecture 23: Miimal sufficiecy Maximal reductio without loss of iformatio There are may sufficiet statistics for a give problem. I fact, X (the whole data set) is sufficiet. If T is a sufficiet statistic
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationQuestions and Answers on Maximum Likelihood
Questios ad Aswers o Maximum Likelihood L. Magee Fall, 2008 1. Give: a observatio-specific log likelihood fuctio l i (θ) = l f(y i x i, θ) the log likelihood fuctio l(θ y, X) = l i(θ) a data set (x i,
More informationSTAT431 Review. X = n. n )
STAT43 Review I. Results related to ormal distributio Expected value ad variace. (a) E(aXbY) = aex bey, Var(aXbY) = a VarX b VarY provided X ad Y are idepedet. Normal distributios: (a) Z N(, ) (b) X N(µ,
More informationLecture 9: September 19
36-700: Probability ad Mathematical Statistics I Fall 206 Lecturer: Siva Balakrisha Lecture 9: September 9 9. Review ad Outlie Last class we discussed: Statistical estimatio broadly Pot estimatio Bias-Variace
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationAsymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values
of the secod half Biostatistics 6 - Statistical Iferece Lecture 6 Fial Exam & Practice Problems for the Fial Hyu Mi Kag Apil 3rd, 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Rao-Blackwell
More informationIntroduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT
Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory
More informationSTAT Homework 2 - Solutions
STAT-36700 Homework - Solutios Fall 08 September 4, 08 This cotais solutios for Homework. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better isight.
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationn n i=1 Often we also need to estimate the variance. Below are three estimators each of which is optimal in some sense: n 1 i=1 k=1 i=1 k=1 i=1 k=1
MATH88T Maria Camero Cotets Basic cocepts of statistics Estimators, estimates ad samplig distributios 2 Ordiary least squares estimate 3 3 Maximum lielihood estimator 3 4 Bayesia estimatio Refereces 9
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationECE 8527: Introduction to Machine Learning and Pattern Recognition Midterm # 1. Vaishali Amin Fall, 2015
ECE 8527: Itroductio to Machie Learig ad Patter Recogitio Midterm # 1 Vaishali Ami Fall, 2015 tue39624@temple.edu Problem No. 1: Cosider a two-class discrete distributio problem: ω 1 :{[0,0], [2,0], [2,2],
More informationf(x i ; ) L(x; p) = i=1 To estimate the value of that maximizes L or equivalently ln L we will set =0, for i =1, 2,...,m p x i (1 p) 1 x i i=1
Parameter Estimatio Samples from a probability distributio F () are: [,,..., ] T.Theprobabilitydistributio has a parameter vector [,,..., m ] T. Estimator: Statistic used to estimate ukow. Estimate: Observed
More informationProblems from 9th edition of Probability and Statistical Inference by Hogg, Tanis and Zimmerman:
Math 224 Fall 2017 Homework 4 Drew Armstrog Problems from 9th editio of Probability ad Statistical Iferece by Hogg, Tais ad Zimmerma: Sectio 2.3, Exercises 16(a,d),18. Sectio 2.4, Exercises 13, 14. Sectio
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationChapter 8: Estimating with Confidence
Chapter 8: Estimatig with Cofidece Sectio 8.2 The Practice of Statistics, 4 th editio For AP* STARNES, YATES, MOORE Chapter 8 Estimatig with Cofidece 8.1 Cofidece Itervals: The Basics 8.2 8.3 Estimatig
More informationLast Lecture. Biostatistics Statistical Inference Lecture 16 Evaluation of Bayes Estimator. Recap - Example. Recap - Bayes Estimator
Last Lecture Biostatistics 60 - Statistical Iferece Lecture 16 Evaluatio of Bayes Estimator Hyu Mi Kag March 14th, 013 What is a Bayes Estimator? Is a Bayes Estimator the best ubiased estimator? Compared
More informationECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization
ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where
More informationStat 421-SP2012 Interval Estimation Section
Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible
More informationLecture 3: MLE and Regression
STAT/Q SCI 403: Itroductio to Resamplig Methods Sprig 207 Istructor: Ye-Chi Che Lecture 3: MLE ad Regressio 3. Parameters ad Distributios Some distributios are idexed by their uderlyig parameters. Thus,
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationCEU Department of Economics Econometrics 1, Problem Set 1 - Solutions
CEU Departmet of Ecoomics Ecoometrics, Problem Set - Solutios Part A. Exogeeity - edogeeity The liear coditioal expectatio (CE) model has the followig form: We would like to estimate the effect of some
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More informationLecture 16: UMVUE: conditioning on sufficient and complete statistics
Lecture 16: UMVUE: coditioig o sufficiet ad complete statistics The 2d method of derivig a UMVUE whe a sufficiet ad complete statistic is available Fid a ubiased estimator of ϑ, say U(X. Coditioig o a
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationBHW #13 1/ Cooper. ENGR 323 Probabilistic Analysis Beautiful Homework # 13
BHW # /5 ENGR Probabilistic Aalysis Beautiful Homework # Three differet roads feed ito a particular freeway etrace. Suppose that durig a fixed time period, the umber of cars comig from each road oto the
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationTHE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA STATISTICAL THEORY AND METHODS PAPER I
THE ROYAL STATISTICAL SOCIETY 5 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA STATISTICAL THEORY AND METHODS PAPER I The Society provides these solutios to assist cadidates preparig for the examiatios i future
More informationLecture Stat Maximum Likelihood Estimation
Lecture Stat 461-561 Maximum Likelihood Estimatio A.D. Jauary 2008 A.D. () Jauary 2008 1 / 63 Maximum Likelihood Estimatio Ivariace Cosistecy E ciecy Nuisace Parameters A.D. () Jauary 2008 2 / 63 Parametric
More informationDirection: This test is worth 150 points. You are required to complete this test within 55 minutes.
Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More information