Lecture 16: UMVUE: conditioning on sufficient and complete statistics
|
|
|
- Kelly Conley
- 5 years ago
- Views:
Transcription
1 Lecture 16: UMVUE: coditioig o sufficiet ad complete statistics The 2d method of derivig a UMVUE whe a sufficiet ad complete statistic is available Fid a ubiased estimator of ϑ, say U(X. Coditioig o a sufficiet ad complete statistic T (X: E[U(X T ] is the UMVUE of ϑ. We eed to derive a explicit form of E[U(X T ] From the uiqueess of the UMVUE, it does ot matter which U(X is used. Thus, we should choose U(X so as to make the calculatio of E[U(X T ] as easy as possible. We do ot eed the distributio of T. But we eed to work out the coditioal expectatio E[U(X T ]. Usig the idepedece of some statistics (Basu s theorem, we may avoid to work o coditioal distributios. UW-Madiso (Statistics Stat 709 Lecture / 15
2 Example (biomial family Let X 1,...,X be iid from biomial(k,θ with kow k ad ukow θ (0,1. We wat to estimate g(θ = P θ (X 1 = 1 = kθ(1 θ k 1. Note that T = i=1 X i biomial(k,θ is the sufficiet ad complete statistic for θ. But o ubiased estimator based o it is immediately evidet. To apply coditioig, we take the simple ubiased estimator of P θ (X 1 = 1, the idicator fuctio I(X 1 = 1. By Theorem , the UMVUE of g(θ is ψ(t = E[I(X 1 = 1 T = P(X 1 = 1 T We eed to simply ψ(t ad obtai a explicit form. Whe T = 0, P(X 1 = 1 T = 0 = 0. For t = 1,...,k, UW-Madiso (Statistics Stat 709 Lecture / 15
3 ψ(t = P (X 1 = 1 T = t = P θ (X 1 = 1, i=1 X i = t P θ ( i=1 X i = t = P θ (X 1 = 1, i=2 X i = t 1 P θ ( i=1 X i = t = P θ (X 1 = 1P θ ( i=2 X i = t 1 P θ ( i=1 X i = t [ (k( 1 kθ(1 θ k 1 ] t 1 θ t 1 (1 θ k( 1 (t 1 = θ t (1 θ k t = k( k( 1 t 1 ( k t ( k t Hece, the UMVUE of g(θ = kθ(1 θ k 1 is k( k( 1 T 1 T = 1,...,k ψ(t = ( k T 0 T = 0 UW-Madiso (Statistics Stat 709 Lecture / 15
4 Example 3.3 Let X 1,...,X be i.i.d. from the expoetial distributio E(0,θ. F θ (x = (1 e x/θ I (0, (x. Cosider the estimatio of ϑ = 1 F θ (t. X is sufficiet ad complete for θ > 0. I (t, (X 1 is ubiased for ϑ, E[I (t, (X 1 ] = P(X 1 > t = ϑ. Hece T (X = E[I (t, (X 1 X] = P(X 1 > t X is the UMVUE of ϑ. If the coditioal distributio of X 1 give X is available, the we ca calculate P(X 1 > t X directly. By Basu s theorem (Theorem 2.4, X 1 / X ad X are idepedet. By Propositio 1.10(vii, P(X 1 > t X = x = P(X 1 / X > t/ X X = x = P(X 1 / X > t/ x UW-Madiso (Statistics Stat 709 Lecture / 15
5 To compute this ucoditioal probability, we eed the distributio of X 1 / i=1 X i = X 1 / ( X 1 + i=2 X i. Usig the trasformatio techique discussed i ad the fact that i=2 X i is idepedet of X 1 ad has a gamma distributio, we obtai that X 1 / i=1 X i has the Lebesgue p.d.f. ( 1(1 x 2 I (0,1 (x. Hece P(X 1 > t X 1 = x = ( 1 (1 x 2 dx ( = 1 t x t/( x 1 ad the UMVUE of ϑ is ( T (X = 1 t 1 X. UW-Madiso (Statistics Stat 709 Lecture / 15
6 Example 3.4 Let X 1,...,X be i.i.d. from N(µ,σ 2 with ukow µ R ad σ 2 > 0. From Example 2.18, T = ( X,S 2 is sufficiet ad complete for θ = (µ,σ 2 X ad ( 1S 2 /σ 2 are idepedet X has the N(µ,σ 2 / distributio S 2 has the chi-square distributio χ 2 1. Usig the method of solvig for h directly, we fid that the UMVUE for µ is X; the UMVUE of µ 2 is X 2 S 2 /; the UMVUE for σ r with r > 1 is k 1,r S r, where k,r = r/2 Γ ( 2 2 r/2 Γ ( +r 2 the UMVUE of µ/σ is k 1, 1 X/S, if > 2. UW-Madiso (Statistics Stat 709 Lecture / 15
7 Example 3.4 (cotiued Suppose that ϑ satisfies P(X 1 ϑ = p with a fixed p (0,1. Let Φ be the c.d.f. of the stadard ormal distributio. The ϑ = µ + σφ 1 (p ad its UMVUE is X + k 1,1 SΦ 1 (p. Let c be a fixed costat ad ( c µ ϑ = P(X 1 c = Φ σ We ca fid the UMVUE of ϑ usig the method of coditioig. Sice I (,c (X 1 is a ubiased estimator of ϑ, the UMVUE of ϑ is. E[I (,c (X 1 T ] = P(X 1 c T. By Basu s theorem, the acillary statistic Z (X = (X 1 X/S is idepedet of T = ( X,S 2. UW-Madiso (Statistics Stat 709 Lecture / 15
8 Example 3.4 (cotiued The, by Propositio 1.10(vii, ( P X 1 c T = ( x,s 2 ( = P = P It ca be show that Z has the Lebesgue p.d.f. f (z = ( Γ 1 2 π( 1Γ ( 2 2 Hece the UMVUE of ϑ is Z c X S T = ( x,s2 ( Z c x. s ] (/2 2 [1 z2 ( 1 2 I (0,( 1/ ( z (c X/S P(X 1 c T = ( 1/ f (zdz UW-Madiso (Statistics Stat 709 Lecture / 15
9 Example 3.4 (cotiued Suppose that we would like to estimate ϑ = 1 σ Φ ( c µ σ the Lebesgue p.d.f. of X 1 evaluated at a fixed c, where Φ is the first-order derivative of Φ. By the previous result, the coditioal p.d.f. of X 1 give X = x ad S 2 = s 2 is s 1 f ( x x s. Let f T be the joit p.d.f. of T = ( X,S 2. The ϑ = Hece the UMVUE of ϑ is, ( [ ( ] 1 c x 1 c s f f T (tdt = E s S f X. S ( 1 c S f X. S UW-Madiso (Statistics Stat 709 Lecture / 15
10 Example Let X 1,...,X be i.i.d. with Lebesgue p.d.f. f θ (x = θx 2 I (θ, (x, where θ > 0 is ukow. Suppose that ϑ = P(X 1 > t for a costat t > 0. The smallest order statistic X (1 is sufficiet ad complete for θ. Hece, the UMVUE of ϑ is P(X 1 > t X (1 = P(X 1 > t X (1 = x (1 ( X1 = P > t X (1 X X (1 = x (1 (1 ( X1 = P > t X (1 x X (1 = x (1 (1 ( X1 = P > s X (1 (Basu s theorem, where s = t/x (1. If s 1, this probability is 1. UW-Madiso (Statistics Stat 709 Lecture / 15
11 Example (cotiued Cosider s > 1 ad assume θ = 1 i the calculatio: ( ( X1 X1 P > s = X (1 P > s,x X (1 = X i i=1 (1 ( X1 = P > s,x X (1 = X i i=2 (1 ( X1 = ( 1P > s,x X (1 = X (1 = ( 1P (X 1 > sx,x 2 > X,...,X 1 > X 1 = ( 1 dx 1 dx = ( 1 x 1 >sx,x 2 >x,...,x 1 >x i=1 [ 1 = ( 1 1 sx 1 sx +1 1 i=2 ( x 1 xi 2 x 2 i dx = ( 1x (1 t ] 1 1 dx i dx 1 dx x 2 1 x 2 UW-Madiso (Statistics Stat 709 Lecture / 15
12 Example (cotiued This shows that the UMVUE of P(X 1 > t is Aother solutio h(x (1 = The UMVUE must be h(x (1 The Lebesgue p.d.f. of X (1 is { ( 1X(1 t X (1 < t 1 X (1 t θ x +1 I (θ, (x. Use the method of fidig h If θ t, the P(X 1 > t = 1 ad P(t > X (1 = 0. Hece, if X (1 t, h(x (1 must be 1 a.s. P θ The value of h(x (1 for X (1 < t is ot specified. UW-Madiso (Statistics Stat 709 Lecture / 15
13 If θ < t, t = h(x θ θ x +1 dx + t Sice P(X 1 > t = θ/t, we have E[h(X (1 ] = h(x θ dx θ x +1 θ t x +1 dx = θ t t = h(x θ θ dx + θ x +1 t i.e., 1 t tθ 1 = h(x θ x +1 dx + 1 t Differetiatig both sizes w.r.t. θ leads to Hece, for ay X (1 < t, 1 tθ = h(θ θ +1 θ h(x θ θ dx + x +1 t h(x (1 = ( 1X (1. t UW-Madiso (Statistics Stat 709 Lecture / 15
14 Ubiased estimators of 0 If a sufficiet ad complete statistic is ot available, the what should we do? If W is ubiased for ϑ ad T is sufficiet, the by Theorem 2.5 (Rao-Blackwell, E(W T is better tha W. If we have aother sufficiet statistic S, should we cosider E[E(W T S]? If there is a fuctio h such that S = h(t, the by the properties of coditioal expectatio, E[E(W T S] = E(W S = E[E(W S T ] That is, we should always coditioig o a simpler sufficiet statistic, such as a miimal sufficiet statistic. To see whe a ubiased estimator is best ubiased, we might ask how could we improve upo a give ubiased estimator? Suppose that T (X is ubiased for g(θ ad U(X is a statistic satisfyig E θ (U = 0 for all θ, i.e., U is ubiased for 0. The, for ay costat a, UW-Madiso (Statistics Stat 709 Lecture / 15
15 is ubiased for g(θ. Ca it be better tha T (X? T (X + au(x Var θ (T + au = Var θ (T + 2aCov θ (T,U + a 2 Var θ (U If for some θ 0, Cov θ0 (T,U < 0, the we ca make 2aCov θ0 (T,U + a 2 Var θ0 (U < 0 by choosig 0 < a 2Cov θ0 (T,U/Var θ0 (U. Hece, T (X + au(x is better tha T (X at least whe θ = θ 0 ad T (X caot be UMVUE. Similarly, if Cov θ0 (T,U > 0 for some θ 0, the T (X caot be UMVUE either. Thus, Cov θ (T,U = 0 is ecessary for T (X to be a UMVUE, for all ubiased estimators of 0. It turs out that Cov θ (T,U = 0 for all U(X ubiased for 0 is also sufficiet for T (X beig a UMVUE. UW-Madiso (Statistics Stat 709 Lecture / 15
Lecture 18: Sampling distributions
Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from
Lecture 23: Minimal sufficiency
Lecture 23: Miimal sufficiecy Maximal reductio without loss of iformatio There are may sufficiet statistics for a give problem. I fact, X (the whole data set) is sufficiet. If T is a sufficiet statistic
Direction: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
Lecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
Chapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
Lecture 4: UMVUE and unbiased estimators of 0
Lecture 4: UMVUE and unbiased estimators of Problems of approach 1. Approach 1 has the following two main shortcomings. Even if Theorem 7.3.9 or its extension is applicable, there is no guarantee that
Lecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
6. Sufficient, Complete, and Ancillary Statistics
Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary
Lecture 10 October Minimaxity and least favorable prior sequences
STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least
Asymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values
of the secod half Biostatistics 6 - Statistical Iferece Lecture 6 Fial Exam & Practice Problems for the Fial Hyu Mi Kag Apil 3rd, 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Rao-Blackwell
APPM 5720 Solutions to Problem Set Five
APPM 5720 Solutios to Problem Set Five 1. The pdf is f(x; α, β) 1 Γ(α) βα x α 1 e βx I (0, ) (x). The joit pdf is f( x; α, β) 1 [Γ(α)] β α ( i1 x i ) α 1 e β i1 x i i1 I (0, ) (x i ) 1 [Γ(α)] βα } {{ }
Lecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
SOLUTION FOR HOMEWORK 7, STAT np(1 p) (α + β + n) + ( np + α
SOLUTION FOR HOMEWORK 7, STAT 6331 1 Exerc733 Here we just recall that MSE(ˆp B ) = p(1 p) (α + β + ) + ( p + α 2 α + β + p) 2 The you plug i α = β = (/4) 1/2 After simplificatios MSE(ˆp B ) = 4( 1/2 +
Lecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
Lecture 15: Density estimation
Lecture 15: Desity estimatio Why do we estimate a desity? Suppose that X 1,...,X are i.i.d. radom variables from F ad that F is ukow but has a Lebesgue p.d.f. f. Estimatio of F ca be doe by estimatig f.
Statistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
Lecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
Lecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
Statistical Theory MT 2008 Problems 1: Solution sketches
Statistical Theory MT 008 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. a) Let 0 < θ < ad put fx, θ) = θ)θ x ; x = 0,,,... b) c) where α
MATH 472 / SPRING 2013 ASSIGNMENT 2: DUE FEBRUARY 4 FINALIZED
MATH 47 / SPRING 013 ASSIGNMENT : DUE FEBRUARY 4 FINALIZED Please iclude a cover sheet that provides a complete setece aswer to each the followig three questios: (a) I your opiio, what were the mai ideas
Lecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
Direction: This test is worth 150 points. You are required to complete this test within 55 minutes.
Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem
STAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
Lecture 6 Ecient estimators. Rao-Cramer bound.
Lecture 6 Eciet estimators. Rao-Cramer boud. 1 MSE ad Suciecy Let X (X 1,..., X) be a radom sample from distributio f θ. Let θ ˆ δ(x) be a estimator of θ. Let T (X) be a suciet statistic for θ. As we have
Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
Last Lecture. Unbiased Test
Last Lecture Biostatistics 6 - Statistical Iferece Lecture Uiformly Most Powerful Test Hyu Mi Kag March 8th, 3 What are the typical steps for costructig a likelihood ratio test? Is LRT statistic based
Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
Lecture 3. Properties of Summary Statistics: Sampling Distribution
Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary
SDS 321: Introduction to Probability and Statistics
SDS 321: Itroductio to Probability ad Statistics Lecture 23: Cotiuous radom variables- Iequalities, CLT Puramrita Sarkar Departmet of Statistics ad Data Sciece The Uiversity of Texas at Austi www.cs.cmu.edu/
ST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
![Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].](/thumbs/90/101606445.jpg "Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].")
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. Comments:
Recall: STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Commets:. So far we have estimates of the parameters! 0 ad!, but have o idea how good these estimates are. Assumptio: E(Y x)! 0 +! x (liear coditioal
Last Lecture. Wald Test
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig
Estimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
IIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
Exponential Families and Bayesian Inference
Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where
Solutions: Homework 3
Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe
Unbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA STATISTICAL THEORY AND METHODS PAPER I
THE ROYAL STATISTICAL SOCIETY 5 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA STATISTICAL THEORY AND METHODS PAPER I The Society provides these solutios to assist cadidates preparig for the examiatios i future
MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
2. The volume of the solid of revolution generated by revolving the area bounded by the
IIT JAM Mathematical Statistics (MS) Solved Paper. A eigevector of the matrix M= ( ) is (a) ( ) (b) ( ) (c) ( ) (d) ( ) Solutio: (a) Eigevalue of M = ( ) is. x So, let x = ( y) be the eigevector. z (M
Confidence Level We want to estimate the true mean of a random variable X economically and with confidence.
Cofidece Iterval 700 Samples Sample Mea 03 Cofidece Level 095 Margi of Error 0037 We wat to estimate the true mea of a radom variable X ecoomically ad with cofidece True Mea μ from the Etire Populatio
Mathmatical Statisticals
Mathmatical Statisticals Che, L.-A. Chapter 4. Distributio of Fuctio of Radom variables Sample space S : set of possible outcome i a experimet. Probability set fuctio P: ()P (A) 0, A S. ()P (S) =. (3)P
( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2
82 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Defiitio: et X be a radom sample with joit p.m/d.f. f X x θ. The geeralised likelihood ratio test g.l.r.t. of the NH : θ H 0 agaist the alterative AH : θ H 1,
HOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
January 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
Introduction to Econometrics (4th Edition) Solutions to Odd-Numbered End-of-Chapter Exercises: Chapter 2*
Itroductio to Ecoometrics (4th Editio by James H. Stock ad Mark W. Watso Solutios to Odd-Numbered Ed-of-Chapter Exercises: Chapter 2* (This versio September 14, 2018 1 2.1. (a Probability distributio fuctio
Problem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
Please do NOT write in this box. Multiple Choice. Total
Istructor: Math 0560, Worksheet Alteratig Series Jauary, 3000 For realistic exam practice solve these problems without lookig at your book ad without usig a calculator. Multiple choice questios should
Lecture 12: November 13, 2018
Mathematical Toolkit Autum 2018 Lecturer: Madhur Tulsiai Lecture 12: November 13, 2018 1 Radomized polyomial idetity testig We will use our kowledge of coditioal probability to prove the followig lemma,
Fundamentals of Statistics
Chapter 2 Fudametals of Statistics Exercise 1 (#2.9). Cosider the family of double expoetial distributios: P = { 2 1 e t µ :
Stat410 Probability and Statistics II (F16)
Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems
This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
Bayesian Methods: Introduction to Multi-parameter Models
Bayesia Methods: Itroductio to Multi-parameter Models Parameter: θ = ( θ, θ) Give Likelihood p(y θ) ad prior p(θ ), the posterior p proportioal to p(y θ) x p(θ ) Margial posterior ( θ, θ y) is Iterested
Stat 319 Theory of Statistics (2) Exercises
Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.
STAT Homework 2 - Solutions
STAT-36700 Homework - Solutios Fall 08 September 4, 08 This cotais solutios for Homework. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better isight.
Review Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn
Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom
The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
Exercise 4.3 Use the Continuity Theorem to prove the Cramér-Wold Theorem, Theorem. (1) φ a X(1).
Assigmet 7 Exercise 4.3 Use the Cotiuity Theorem to prove the Cramér-Wold Theorem, Theorem 4.12. Hit: a X d a X implies that φ a X (1) φ a X(1). Sketch of solutio: As we poited out i class, the oly tricky
Topic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
Topic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
Math 152. Rumbos Fall Solutions to Review Problems for Exam #2. Number of Heads Frequency
Math 152. Rumbos Fall 2009 1 Solutios to Review Problems for Exam #2 1. I the book Experimetatio ad Measuremet, by W. J. Youde ad published by the by the Natioal Sciece Teachers Associatio i 1962, the
Since X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
Stat 421-SP2012 Interval Estimation Section
Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible
APPLIED MULTIVARIATE ANALYSIS
ALIED MULTIVARIATE ANALYSIS FREQUENTLY ASKED QUESTIONS AMIT MITRA & SHARMISHTHA MITRA DEARTMENT OF MATHEMATICS & STATISTICS INDIAN INSTITUTE OF TECHNOLOGY KANUR X = X X X [] The variace covariace atrix
This section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
Output Analysis and Run-Length Control
IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%
Economics 326 Methods of Empirical Research in Economics. Lecture 18: The asymptotic variance of OLS and heteroskedasticity
Ecoomics 326 Methods of Empirical Research i Ecoomics Lecture 8: The asymptotic variace of OLS ad heteroskedasticity Hiro Kasahara Uiversity of British Columbia December 24, 204 Asymptotic ormality I I
Math 132, Fall 2009 Exam 2: Solutions
Math 3, Fall 009 Exam : Solutios () a) ( poits) Determie for which positive real umbers p, is the followig improper itegral coverget, ad for which it is diverget. Evaluate the itegral for each value of
Last Lecture. Biostatistics Statistical Inference Lecture 16 Evaluation of Bayes Estimator. Recap - Example. Recap - Bayes Estimator
Last Lecture Biostatistics 60 - Statistical Iferece Lecture 16 Evaluatio of Bayes Estimator Hyu Mi Kag March 14th, 013 What is a Bayes Estimator? Is a Bayes Estimator the best ubiased estimator? Compared
EE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
Randomized Algorithms I, Spring 2018, Department of Computer Science, University of Helsinki Homework 1: Solutions (Discussed January 25, 2018)
Radomized Algorithms I, Sprig 08, Departmet of Computer Sciece, Uiversity of Helsiki Homework : Solutios Discussed Jauary 5, 08). Exercise.: Cosider the followig balls-ad-bi game. We start with oe black
Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
Machine Learning Brett Bernstein
Machie Learig Brett Berstei Week Lecture: Cocept Check Exercises Starred problems are optioal. Statistical Learig Theory. Suppose A = Y = R ad X is some other set. Furthermore, assume P X Y is a discrete
7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
AMS 216 Stochastic Differential Equations Lecture 02 Copyright by Hongyun Wang, UCSC ( ( )) 2 = E X 2 ( ( )) 2
AMS 216 Stochastic Differetial Equatios Lecture 02 Copyright by Hogyu Wag, UCSC Review of probability theory (Cotiued) Variace: var X We obtai: = E X E( X ) 2 = E( X 2 ) 2E ( X )E X var( X ) = E X 2 Stadard
Mathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
Distribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
Lecture 7: Density Estimation: k-nearest Neighbor and Basis Approach
STAT 425: Itroductio to Noparametric Statistics Witer 28 Lecture 7: Desity Estimatio: k-nearest Neighbor ad Basis Approach Istructor: Ye-Chi Che Referece: Sectio 8.4 of All of Noparametric Statistics.
Lecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett
Lecture Note 8 Poit Estimators ad Poit Estimatio Methods MIT 14.30 Sprig 2006 Herma Beett Give a parameter with ukow value, the goal of poit estimatio is to use a sample to compute a umber that represets
Discrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
Quick Review of Probability
Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter 2 & Teachig
Topic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
Quick Review of Probability
Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter & Teachig Material.
An Introduction to Asymptotic Theory
A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu
Chapter 3: Unbiased Estimation Lecture 22: UMVUE and the method of using a sufficient and complete statistic
Chapter 3: Unbiased Estimation Lecture 22: UMVUE and the method of using a sufficient and complete statistic Unbiased estimation Unbiased or asymptotically unbiased estimation plays an important role in
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
Goodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen)
Goodess-of-Fit Tests ad Categorical Data Aalysis (Devore Chapter Fourtee) MATH-252-01: Probability ad Statistics II Sprig 2019 Cotets 1 Chi-Squared Tests with Kow Probabilities 1 1.1 Chi-Squared Testig................
Expectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
STATISTICAL INFERENCE
STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample
Notes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
Parameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,