Last Lecture. Biostatistics Statistical Inference Lecture 14 Obtaining Best Unbiased Estimator. Related Theorems. Rao-Blackwell Theorem
|
|
|
- Sophie Gilmore
- 5 years ago
- Views:
Transcription
1 Last Lecture Biostatistics 62 - Statistical Inference Lecture 14 Obtaining Best Unbiased Estimator Hyun Min Kang February 28th, 213 For single-parameter exponential family, is Cramer-Rao bound always attainable? How about exponential family with two or more parameters? For any statistic T(X, does ϕ(t always result in a better unbiased estimator than W? Why? What is the Rao-Blackwell Theorem? Is the best unbiased estimator ( for τ(θ unique? What is the relationship between the and the unbiased estimators of zero? Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Rao-Blackwell Theorem Related Theorems Theorem 7317 Let W(X be any unbiased estimator of τ(θ, and T be a sufficient statistic for θ Define ϕ(t = E[W T] Then the followings hold 1 E[ϕ(T θ] = τ(θ 2 Var[ϕ(T θ] Var(W θ for all θ That is, ϕ(t is a uniformly better unbiased estimator of τ(θ Theorem Uniqueness of Ịf W is a best unbiased estimator of τ(θ, then W is unique Theorem and unbiased estimators of zero If E[W(X] = τ(θ W is the best unbiased estimator of τ(θ if an only if W is uncorrelated with all unbiased estimator of Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23
2 The power of complete sufficient statistics Remarks from previous Theorems - #1 Theorem 7323 Let T be a complete sufficient statistic for parameter θ Let ϕ(t be any estimator based on T Then ϕ(t is the unique best unbiased estimator of its expected value From Rao-Blackwell Theorem, we can always improve an unbiased estimator by conditioning it on a sufficient statistics W(X : unbiased for τ(θ T (X : sufficient statistic for θ ϕ(t = E[W(X T(X] is a better unbiased estimator of τ(θ Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Remarks from previous Theorems - #2 Remarks from previous Theorems - #3 In fact, we only need to consider functions of minimal sufficient statistics to find the best unbiased estimator Let T(X be a minimal sufficient, and T (X be a sufficient statistic Then by definition, there exists a function h that satisfies T = h(t E[ϕ(T T ] = E [ϕ {h(t } T ] = ϕ {h(t } = ϕ(t Therefore ϕ(t remains the same after conditioning on any sufficient statistic T Complete sufficient statistics is a very useful ingredient to obtain a ϕ(t is an unbiased estimator for E[ϕ(T] = τ(θ By Theorem 732, ϕ(t is the best unbiased estimator if and only if ϕ(t if and only of ϕ(t is uncorrelated with U(T, which is any unbiased esimator of By definition, T is complete is E[U(T] = for all θ implies U(T = almost surely Suppose that T is a complete statistic, then U(T can only be zero almost surely Therefore, Cov(ϕ(T, U(T = Cov(ϕ(T, =, and ϕ(t is the best unbiased estimator of its expected value (Theorem 7323 Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23
3 of Method 2 for obtaining Example - Normal Distribution Use complete sufficient statistic to find the best unbiased estimator for τ(θ 1 Find complete sufficient statistic T for θ 2 Obtain ϕ(t, an unbiased estimator of τ(θ using either of the following two ways Guess a functon ϕ(t such that E[ϕ(T] = τ(θ Guess an unbiased estimator h(x of τ(θ Construct ϕ(t = E[h(X T], then E[ϕ(T] = E[h(X] = τ(θ Problem iid X 1,, X n N (µ, σ 2 Find the best unbiased estimator for (1 µ, (2 σ 2, (3 µ 2 Solution First, we need to find a complete and sufficient statistic for (µ, σ 2 We know that T(X = (X, s 2 X is complete, sufficient statistic for (µ, σ 2 Because E[X] = µ, X is an unbiased estimator for µ, X is also a function of T(X Therefore, X is the best unbiased estimator for µ Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Example - Normal Distribution (cont d Example - Normal Distribution (cont d To obtain for µ 2, we need a ϕ(t = ϕ(x, s 2 X such that E[ϕ(T] = µ 2 E(X = µ E(s 2 X = σ2 s 2 X is a function of T Therefore s 2 X is the best unbiased estimator of σ2 E((X 2 = Var(X + E(X 2 = σ2 n + µ2 E (X 2 σ2 = µ 2 n ( E X 2 s2 X = µ 2 n X 2 s 2 X /n is unbiased estimator for µ2 And it is a function of (X, s 2 X Hence, X 2 s 2 X /n is the best unbiased estimator for µ2 Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23
4 Example - Normal Distribution - Alternative method Example - Uniform Distribution X 1 X 2 is unbiased for µ 2 because E[X 1 X 2 ] = E(X 1 E(X 2 = µ 2 i j ϕ(t = E[X 1 X 2 T] = E[X ix j T] n(n 1 n = E[X2 i T] + i j E[X ix j T] n E[X2 i T] n(n 1 = E[( n X i 2 T] E[ n X2 i T] n(n 1 = E[( nx 2 (n 1s 2 X nx 2 T] n(n 1 = X 2 s 2 X /n = n(n 1X2 (n 1s 2 X n(n 1 Problem iid X 1,, X n Uniform(, θ Find the best unbiased estimator for (1 θ, (2 g(θ differentiable on (, θ (3 θ 2, (4 1/θ Solution - MVUE of θ T(X = X (n is a complete and sufficient statistic for θ f T (t = nθ n t n 1 I( < t < θ E[T] = E[X (n ] = tnθ n t n 1 dt = n n+1 θ (biased E[ϕ(T] = E [ n+1 n X (n] = θ n+1 n X (n is the best unbiased estimator of θ Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Example - Uniform Distribution - for g(θ Example - Uniform Distribution - for g(θ (cont d We need to find a function of ϕ(t = X (n such that E[ϕ(T] = g(θ g(θ = E[ϕ(T] = ϕ(tnθ n t n 1 dt Taking derivative with respect to θ, and applying Leibnitz s rule g (θ = d dθ ϕ(tnθ n t n 1 dt = ϕ(θnθ n θ n 1 + = ϕ(θnθ 1 + ϕ(tt n 1 n d dθ θ n dt ϕ(tt n 1 n( nθ n 1 dt = ϕ(θnθ 1 nθ 1 ϕ(tnt n 1 θ n dt g (θ = ϕ(θnθ 1 nθ 1 ϕ(tnt n 1 θ n dt = ϕ(θnθ 1 nθ 1 E[ϕ(T] = ϕ(θnθ 1 nθ 1 g(θ ϕ(θ = g (θ + nθ 1 g(θ nθ 1 Therefore, the best unbiased estimator of g(θ is ϕ(t = g (T + nt 1 g(t nt 1 ϕ(x (n = g (X (n + nx 1 (n g(x (n nx 1 (n = 1 n X (ng (X (n + g(x (n Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23
5 Example - Uniform Distribution - for θ 2 Example - Uniform Distribution - for 1/θ g(θ = θ 2, and g (θ = 2θ ϕ(x (n = 1 n X (n 2X (n + X 2 (n = n + 2 n X2 (n g(θ = 1/θ, and g (θ = 1/θ 2 ϕ(x (n = 1 n X (n = n 1 nx (n ( 1 X 2 (n + 1 X (n Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Example - Binomial best unbiased estimator Problem iid X 1,, X n Binomial(k, θ Estimate the probability of exactly one success Solution The quantity we need to estimate is τ(θ = Pr(X = 1 θ = kθ(1 θ k 1 We know that T(X = n X i Binomial(kn, θ and it is a complete sufficient statistic So we need to find a ϕ(t that satisfies E[ϕ(T] = τ(θ There is no imeediately evident unbiased estimator of τ(θ as a function of T Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Solution - Binomial best unbiased estimator Start with a simple-minded estimator { 1 X1 = 1 W(X = otherwise The expectation of W is E[W] = k x 1 = = kθ(1 θ k 1 ( k W(x 1 θ x 1 (1 θ k x 1 x 1 and hence is an unbiased estimator of τ(θ = kθ(1 θ k 1 The best unbiased estimator of τ(θ is ϕ(t = E[W T] = E [W(X T(X] Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23
6 Solution - Binomial best unbiased estimator (cont d Solution - Binomial best unbiased estimator (cont d [ ] [ ] n n ϕ(t = E W(X X i = t = Pr X 1 = 1 X i = t = Pr(X 1 = 1, n X i = t Pr( n X i = t = Pr(X 1 = 1, n i=2 X i = t 1 Pr( n X i = t = Pr(X 1 = 1 Pr( n i=2 X i = t 1 Pr( n X i = t [ (k(n 1 [kθ(1 θ k 1 ] ] t 1 θ t 1 (1 θ k(n 1 t 1 = ( kn = k n θ t (1 θ kn t ( k(n 1 t 1 ( kn t Therefore, the unbiased estimator of kθ(1 θ k 1 is ( n ϕ X i = k ( k(n 1 Xi 1 ( kn Xi Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23 Today Rao-Blackwell Theorem Methods for obtaining Next Lecture Bayesian Estimators Hyun Min Kang Biostatistics 62 - Lecture 14 February 28th, / 23
Methods of evaluating estimators and best unbiased estimators Hamid R. Rabiee
Stochastic Processes Methods of evaluating estimators and best unbiased estimators Hamid R. Rabiee 1 Outline Methods of Mean Squared Error Bias and Unbiasedness Best Unbiased Estimators CR-Bound for variance
Summary. Ancillary Statistics What is an ancillary statistic for θ? .2 Can an ancillary statistic be a sufficient statistic?
Biostatistics 62 - Statistical Inference Lecture 5 Hyun Min Kang 1 What is an ancillary statistic for θ? 2 Can an ancillary statistic be a sufficient statistic? 3 What are the location parameter and the
ST5215: Advanced Statistical Theory
Department of Statistics & Applied Probability Wednesday, October 19, 2011 Lecture 17: UMVUE and the first method of derivation Estimable parameters Let ϑ be a parameter in the family P. If there exists
Summary. Complete Statistics ... Last Lecture. .1 What is a complete statistic? .2 Why it is called as complete statistic?
Last Lecture Biostatistics 602 - Statistical Inference Lecture 06 Hyun Min Kang January 29th, 2013 1 What is a complete statistic? 2 Why it is called as complete statistic? 3 Can the same statistic be
Mathematical Statistics
Mathematical Statistics Chapter Three. Point Estimation 3.4 Uniformly Minimum Variance Unbiased Estimator(UMVUE) Criteria for Best Estimators MSE Criterion Let F = {p(x; θ) : θ Θ} be a parametric distribution
Last Lecture - Key Questions. Biostatistics Statistical Inference Lecture 03. Minimal Sufficient Statistics
Last Lecture - Key Questions Biostatistics 602 - Statistical Inference Lecture 03 Hyun Min Kang January 17th, 2013 1 How do we show that a statistic is sufficient for θ? 2 What is a necessary and sufficient
Chapter 3: Unbiased Estimation Lecture 22: UMVUE and the method of using a sufficient and complete statistic
Chapter 3: Unbiased Estimation Lecture 22: UMVUE and the method of using a sufficient and complete statistic Unbiased estimation Unbiased or asymptotically unbiased estimation plays an important role in
February 26, 2017 COMPLETENESS AND THE LEHMANN-SCHEFFE THEOREM
February 26, 2017 COMPLETENESS AND THE LEHMANN-SCHEFFE THEOREM Abstract. The Rao-Blacwell theorem told us how to improve an estimator. We will discuss conditions on when the Rao-Blacwellization of an estimator
Lecture 4: UMVUE and unbiased estimators of 0
Lecture 4: UMVUE and unbiased estimators of Problems of approach 1. Approach 1 has the following two main shortcomings. Even if Theorem 7.3.9 or its extension is applicable, there is no guarantee that
Unbiased Estimation. Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others.
Unbiased Estimation Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others. To compare ˆθ and θ, two estimators of θ: Say ˆθ is better than θ if it
1. (Regular) Exponential Family
1. (Regular) Exponential Family The density function of a regular exponential family is: [ ] Example. Poisson(θ) [ ] Example. Normal. (both unknown). ) [ ] [ ] [ ] [ ] 2. Theorem (Exponential family &
557: MATHEMATICAL STATISTICS II BIAS AND VARIANCE
557: MATHEMATICAL STATISTICS II BIAS AND VARIANCE An estimator, T (X), of θ can be evaluated via its statistical properties. Typically, two aspects are considered: Expectation Variance either in terms
Today s Outline. Biostatistics Statistical Inference Lecture 01 Introduction to BIOSTAT602 Principles of Data Reduction
Today s Outline Biostatistics 602 - Statistical Inference Lecture 01 Introduction to Principles of Hyun Min Kang Course Overview of January 10th, 2013 Hyun Min Kang Biostatistics 602 - Lecture 01 January
Lecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
Asymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values
of the secod half Biostatistics 6 - Statistical Iferece Lecture 6 Fial Exam & Practice Problems for the Fial Hyu Mi Kag Apil 3rd, 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Rao-Blackwell
Unbiased Estimation. Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others.
Unbiased Estimation Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others. To compare ˆθ and θ, two estimators of θ: Say ˆθ is better than θ if it
Elements of statistics (MATH0487-1)
Elements of statistics (MATH0487-1) Prof. Dr. Dr. K. Van Steen University of Liège, Belgium November 12, 2012 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Analysis -
Proof In the CR proof. and
Question Under what conditions will we be able to attain the Cramér-Rao bound and find a MVUE? Lecture 4 - Consequences of the Cramér-Rao Lower Bound. Searching for a MVUE. Rao-Blackwell Theorem, Lehmann-Scheffé
A Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2008 Prof. Gesine Reinert 1 Data x = x 1, x 2,..., x n, realisations of random variables X 1, X 2,..., X n with distribution (model)
Evaluating the Performance of Estimators (Section 7.3)
Evaluating the Performance of Estimators (Section 7.3) Example: Suppose we observe X 1,..., X n iid N(θ, σ 2 0 ), with σ2 0 known, and wish to estimate θ. Two possible estimators are: ˆθ = X sample mean
Chapters 9. Properties of Point Estimators
Chapters 9. Properties of Point Estimators Recap Target parameter, or population parameter θ. Population distribution f(x; θ). { probability function, discrete case f(x; θ) = density, continuous case The
Theory of Statistics.
Theory of Statistics. Homework V February 5, 00. MT 8.7.c When σ is known, ˆµ = X is an unbiased estimator for µ. If you can show that its variance attains the Cramer-Rao lower bound, then no other unbiased
Chapter 8.8.1: A factorization theorem
LECTURE 14 Chapter 8.8.1: A factorization theorem The characterization of a sufficient statistic in terms of the conditional distribution of the data given the statistic can be difficult to work with.
March 10, 2017 THE EXPONENTIAL CLASS OF DISTRIBUTIONS
March 10, 2017 THE EXPONENTIAL CLASS OF DISTRIBUTIONS Abstract. We will introduce a class of distributions that will contain many of the discrete and continuous we are familiar with. This class will help
STA 260: Statistics and Probability II
Al Nosedal. University of Toronto. Winter 2017 1 Properties of Point Estimators and Methods of Estimation 2 3 If you can t explain it simply, you don t understand it well enough Albert Einstein. Definition
A Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random
Brief Review on Estimation Theory
Brief Review on Estimation Theory K. Abed-Meraim ENST PARIS, Signal and Image Processing Dept. abed@tsi.enst.fr This presentation is essentially based on the course BASTA by E. Moulines Brief review on
Microeconometrics: Clustering. Ethan Kaplan
Microeconometrics: Clustering Ethan Kaplan Gauss Markov ssumptions OLS is minimum variance unbiased (MVUE) if Linear Model: Y i = X i + i E ( i jx i ) = V ( i jx i ) = 2 < cov i ; j = Normally distributed
STAT 512 sp 2018 Summary Sheet
STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}
Completeness. On the other hand, the distribution of an ancillary statistic doesn t depend on θ at all.
Completeness A minimal sufficient statistic achieves the maximum amount of data reduction while retaining all the information the sample has concerning θ. On the other hand, the distribution of an ancillary
ECE534, Spring 2018: Solutions for Problem Set #3
ECE534, Spring 08: Solutions for Problem Set #3 Jointly Gaussian Random Variables and MMSE Estimation Suppose that X, Y are jointly Gaussian random variables with µ X = µ Y = 0 and σ X = σ Y = Let their
Limiting Distributions
Limiting Distributions We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the
5.6 The Normal Distributions
STAT 41 Lecture Notes 13 5.6 The Normal Distributions Definition 5.6.1. A (continuous) random variable X has a normal distribution with mean µ R and variance < R if the p.d.f. of X is f(x µ, ) ( π ) 1/
ECE 275B Homework # 1 Solutions Version Winter 2015
ECE 275B Homework # 1 Solutions Version Winter 2015 1. (a) Because x i are assumed to be independent realizations of a continuous random variable, it is almost surely (a.s.) 1 the case that x 1 < x 2
Moments. Raw moment: February 25, 2014 Normalized / Standardized moment:
Moments Lecture 10: Central Limit Theorem and CDFs Sta230 / Mth 230 Colin Rundel Raw moment: Central moment: µ n = EX n ) µ n = E[X µ) 2 ] February 25, 2014 Normalized / Standardized moment: µ n σ n Sta230
Limiting Distributions
We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the two fundamental results
Example: An experiment can either result in success or failure with probability θ and (1 θ) respectively. The experiment is performed independently
Chapter 3 Sufficient statistics and variance reduction Let X 1,X 2,...,X n be a random sample from a certain distribution with p.m/d.f fx θ. A function T X 1,X 2,...,X n = T X of these observations is
Chapter 4 HOMEWORK ASSIGNMENTS. 4.1 Homework #1
Chapter 4 HOMEWORK ASSIGNMENTS These homeworks may be modified as the semester progresses. It is your responsibility to keep up to date with the correctly assigned homeworks. There may be some errors in
STAT 730 Chapter 4: Estimation
STAT 730 Chapter 4: Estimation Timothy Hanson Department of Statistics, University of South Carolina Stat 730: Multivariate Analysis 1 / 23 The likelihood We have iid data, at least initially. Each datum
ECE 275B Homework # 1 Solutions Winter 2018
ECE 275B Homework # 1 Solutions Winter 2018 1. (a) Because x i are assumed to be independent realizations of a continuous random variable, it is almost surely (a.s.) 1 the case that x 1 < x 2 < < x n Thus,
Mathematical statistics
October 4 th, 2018 Lecture 12: Information Where are we? Week 1 Week 2 Week 4 Week 7 Week 10 Week 14 Probability reviews Chapter 6: Statistics and Sampling Distributions Chapter 7: Point Estimation Chapter
18.175: Lecture 15 Characteristic functions and central limit theorem
18.175: Lecture 15 Characteristic functions and central limit theorem Scott Sheffield MIT Outline Characteristic functions Outline Characteristic functions Characteristic functions Let X be a random variable.
Chapter 3: Maximum Likelihood Theory
Chapter 3: Maximum Likelihood Theory Florian Pelgrin HEC September-December, 2010 Florian Pelgrin (HEC) Maximum Likelihood Theory September-December, 2010 1 / 40 1 Introduction Example 2 Maximum likelihood
ECE531 Lecture 10b: Maximum Likelihood Estimation
ECE531 Lecture 10b: Maximum Likelihood Estimation D. Richard Brown III Worcester Polytechnic Institute 05-Apr-2011 Worcester Polytechnic Institute D. Richard Brown III 05-Apr-2011 1 / 23 Introduction So
Example continued. Math 425 Intro to Probability Lecture 37. Example continued. Example
continued : Coin tossing Math 425 Intro to Probability Lecture 37 Kenneth Harris kaharri@umich.edu Department of Mathematics University of Michigan April 8, 2009 Consider a Bernoulli trials process with
PROBABILITY THEORY LECTURE 3
PROBABILITY THEORY LECTURE 3 Per Sidén Division of Statistics Dept. of Computer and Information Science Linköping University PER SIDÉN (STATISTICS, LIU) PROBABILITY THEORY - L3 1 / 15 OVERVIEW LECTURE
STA 732: Inference. Notes 10. Parameter Estimation from a Decision Theoretic Angle. Other resources
STA 732: Inference Notes 10. Parameter Estimation from a Decision Theoretic Angle Other resources 1 Statistical rules, loss and risk We saw that a major focus of classical statistics is comparing various
Introduction to Self-normalized Limit Theory
Introduction to Self-normalized Limit Theory Qi-Man Shao The Chinese University of Hong Kong E-mail: qmshao@cuhk.edu.hk Outline What is the self-normalization? Why? Classical limit theorems Self-normalized
Stat 451 Lecture Notes Monte Carlo Integration
Stat 451 Lecture Notes 06 12 Monte Carlo Integration Ryan Martin UIC www.math.uic.edu/~rgmartin 1 Based on Chapter 6 in Givens & Hoeting, Chapter 23 in Lange, and Chapters 3 4 in Robert & Casella 2 Updated:
Asymptotic Statistics-III. Changliang Zou
Asymptotic Statistics-III Changliang Zou The multivariate central limit theorem Theorem (Multivariate CLT for iid case) Let X i be iid random p-vectors with mean µ and and covariance matrix Σ. Then n (
Estimation theory. Parametric estimation. Properties of estimators. Minimum variance estimator. Cramer-Rao bound. Maximum likelihood estimators
Estimation theory Parametric estimation Properties of estimators Minimum variance estimator Cramer-Rao bound Maximum likelihood estimators Confidence intervals Bayesian estimation 1 Random Variables Let
Lecture 16: UMVUE: conditioning on sufficient and complete statistics
Lecture 16: UMVUE: coditioig o sufficiet ad complete statistics The 2d method of derivig a UMVUE whe a sufficiet ad complete statistic is available Fid a ubiased estimator of ϑ, say U(X. Coditioig o a
p y (1 p) 1 y, y = 0, 1 p Y (y p) = 0, otherwise.
1. Suppose Y 1, Y 2,..., Y n is an iid sample from a Bernoulli(p) population distribution, where 0 < p < 1 is unknown. The population pmf is p y (1 p) 1 y, y = 0, 1 p Y (y p) = (a) Prove that Y is the
Practice Problem - Skewness of Bernoulli Random Variable. Lecture 7: Joint Distributions and the Law of Large Numbers. Joint Distributions - Example
A little more E(X Practice Problem - Skewness of Bernoulli Random Variable Lecture 7: and the Law of Large Numbers Sta30/Mth30 Colin Rundel February 7, 014 Let X Bern(p We have shown that E(X = p Var(X
5.2 Fisher information and the Cramer-Rao bound
Stat 200: Introduction to Statistical Inference Autumn 208/9 Lecture 5: Maximum likelihood theory Lecturer: Art B. Owen October 9 Disclaimer: These notes have not been subjected to the usual scrutiny reserved
Module 1 - Signal estimation
, Arraial do Cabo, 2009 Module 1 - Signal estimation Sérgio M. Jesus (sjesus@ualg.pt) Universidade do Algarve, PT-8005-139 Faro, Portugal www.siplab.fct.ualg.pt February 2009 Outline of Module 1 Parameter
Lecture 11: Probability, Order Statistics and Sampling
5-75: Graduate Algorithms February, 7 Lecture : Probability, Order tatistics and ampling Lecturer: David Whitmer cribes: Ilai Deutel, C.J. Argue Exponential Distributions Definition.. Given sample space
System Identification, Lecture 4
System Identification, Lecture 4 Kristiaan Pelckmans (IT/UU, 2338) Course code: 1RT880, Report code: 61800 - Spring 2012 F, FRI Uppsala University, Information Technology 30 Januari 2012 SI-2012 K. Pelckmans
System Identification, Lecture 4
System Identification, Lecture 4 Kristiaan Pelckmans (IT/UU, 2338) Course code: 1RT880, Report code: 61800 - Spring 2016 F, FRI Uppsala University, Information Technology 13 April 2016 SI-2016 K. Pelckmans
Stochastic Convergence, Delta Method & Moment Estimators
Stochastic Convergence, Delta Method & Moment Estimators Seminar on Asymptotic Statistics Daniel Hoffmann University of Kaiserslautern Department of Mathematics February 13, 2015 Daniel Hoffmann (TU KL)
Markov Chain Monte Carlo Lecture 1
What are Monte Carlo Methods? The subject of Monte Carlo methods can be viewed as a branch of experimental mathematics in which one uses random numbers to conduct experiments. Typically the experiments
Section 27. The Central Limit Theorem. Po-Ning Chen, Professor. Institute of Communications Engineering. National Chiao Tung University
Section 27 The Central Limit Theorem Po-Ning Chen, Professor Institute of Communications Engineering National Chiao Tung University Hsin Chu, Taiwan 3000, R.O.C. Identically distributed summands 27- Central
Summary. Recap. Last Lecture. Let W n = W n (X 1,, X n ) = W n (X) be a sequence of estimators for
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 17 Asymptotic Evaluatio of oit Estimators Hyu Mi Kag March 19th, 2013 What is a Bayes Risk? What is the Bayes rule Estimator miimizig square
Spring 2012 Math 541A Exam 1. X i, S 2 = 1 n. n 1. X i I(X i < c), T n =
Spring 2012 Math 541A Exam 1 1. (a) Let Z i be independent N(0, 1), i = 1, 2,, n. Are Z = 1 n n Z i and S 2 Z = 1 n 1 n (Z i Z) 2 independent? Prove your claim. (b) Let X 1, X 2,, X n be independent identically
f(y θ) = g(t (y) θ)h(y)
EXAM3, FINAL REVIEW (and a review for some of the QUAL problems): No notes will be allowed, but you may bring a calculator. Memorize the pmf or pdf f, E(Y ) and V(Y ) for the following RVs: 1) beta(δ,
Graphical Models 101. Biostatistics 615/815 Lecture 12: Hidden Markov Models. Example probability distribution. An example graphical model
101 Biostatistics 615/815 Lecture 12: Hidden Markov Models Hyun Min Kang February 15th, 2011 Marriage between probability theory and graph theory Each random variable is represented as vertex Dependency
ECE531 Lecture 8: Non-Random Parameter Estimation
ECE531 Lecture 8: Non-Random Parameter Estimation D. Richard Brown III Worcester Polytechnic Institute 19-March-2009 Worcester Polytechnic Institute D. Richard Brown III 19-March-2009 1 / 25 Introduction
Midterm #1. Lecture 10: Joint Distributions and the Law of Large Numbers. Joint Distributions - Example, cont. Joint Distributions - Example
Midterm #1 Midterm 1 Lecture 10: and the Law of Large Numbers Statistics 104 Colin Rundel February 0, 01 Exam will be passed back at the end of class Exam was hard, on the whole the class did well: Mean:
Parameter estimation! and! forecasting! Cristiano Porciani! AIfA, Uni-Bonn!
Parameter estimation! and! forecasting! Cristiano Porciani! AIfA, Uni-Bonn! Questions?! C. Porciani! Estimation & forecasting! 2! Cosmological parameters! A branch of modern cosmological research focuses
Lecture Topic 4: Chapter 7 Sampling and Sampling Distributions
Lecture Topic 4: Chapter 7 Sampling and Sampling Distributions Statistical Inference: The aim is to obtain information about a population from information contained in a sample. A population is the set
A Gentle Introduction to Stein s Method for Normal Approximation I
A Gentle Introduction to Stein s Method for Normal Approximation I Larry Goldstein University of Southern California Introduction to Stein s Method for Normal Approximation 1. Much activity since Stein
Statistical Inference
Statistical Inference Classical and Bayesian Methods Class 7 AMS-UCSC Tue 31, 2012 Winter 2012. Session 1 (Class 7) AMS-132/206 Tue 31, 2012 1 / 13 Topics Topics We will talk about... 1 Hypothesis testing
1. Fisher Information
1. Fisher Information Let f(x θ) be a density function with the property that log f(x θ) is differentiable in θ throughout the open p-dimensional parameter set Θ R p ; then the score statistic (or score
Introduction to Machine Learning. Lecture 2
Introduction to Machine Learning Lecturer: Eran Halperin Lecture 2 Fall Semester Scribe: Yishay Mansour Some of the material was not presented in class (and is marked with a side line) and is given for
EIE6207: Estimation Theory
EIE6207: Estimation Theory Man-Wai MAK Dept. of Electronic and Information Engineering, The Hong Kong Polytechnic University enmwmak@polyu.edu.hk http://www.eie.polyu.edu.hk/ mwmak References: Steven M.
Theory and Applications of Stochastic Systems Lecture Exponential Martingale for Random Walk
Instructor: Victor F. Araman December 4, 2003 Theory and Applications of Stochastic Systems Lecture 0 B60.432.0 Exponential Martingale for Random Walk Let (S n : n 0) be a random walk with i.i.d. increments
Stochastic Gradient Descent in Continuous Time
Stochastic Gradient Descent in Continuous Time Justin Sirignano University of Illinois at Urbana Champaign with Konstantinos Spiliopoulos (Boston University) 1 / 27 We consider a diffusion X t X = R m
Statistics and Econometrics I
Statistics and Econometrics I Point Estimation Shiu-Sheng Chen Department of Economics National Taiwan University September 13, 2016 Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I September 13,
Notes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed
18.466 Notes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed 1. MLEs in exponential families Let f(x,θ) for x X and θ Θ be a likelihood function, that is, for present purposes,
Estimation of parametric functions in Downton s bivariate exponential distribution
Estimation of parametric functions in Downton s bivariate exponential distribution George Iliopoulos Department of Mathematics University of the Aegean 83200 Karlovasi, Samos, Greece e-mail: geh@aegean.gr
ELEG 5633 Detection and Estimation Minimum Variance Unbiased Estimators (MVUE)
1 ELEG 5633 Detection and Estimation Minimum Variance Unbiased Estimators (MVUE) Jingxian Wu Department of Electrical Engineering University of Arkansas Outline Minimum Variance Unbiased Estimators (MVUE)
Estimation and Detection
stimation and Detection Lecture 2: Cramér-Rao Lower Bound Dr. ir. Richard C. Hendriks & Dr. Sundeep P. Chepuri 7//207 Remember: Introductory xample Given a process (DC in noise): x[n]=a + w[n], n=0,,n,
ENEE 621 SPRING 2016 DETECTION AND ESTIMATION THEORY THE PARAMETER ESTIMATION PROBLEM
c 2007-2016 by Armand M. Makowski 1 ENEE 621 SPRING 2016 DETECTION AND ESTIMATION THEORY THE PARAMETER ESTIMATION PROBLEM 1 The basic setting Throughout, p, q and k are positive integers. The setup With
1 Complete Statistics
Complete Statistics February 4, 2016 Debdeep Pati 1 Complete Statistics Suppose X P θ, θ Θ. Let (X (1),..., X (n) ) denote the order statistics. Definition 1. A statistic T = T (X) is complete if E θ g(t
Probability and Statistics qualifying exam, May 2015
Probability and Statistics qualifying exam, May 2015 Name: Instructions: 1. The exam is divided into 3 sections: Linear Models, Mathematical Statistics and Probability. You must pass each section to pass
Massachusetts Institute of Technology
6.04/6.43: Probabilistic Systems Analysis (Fall 00) Problem Set 7: Solutions. (a) The event of the ith success occuring before the jth failure is equivalent to the ith success occurring within the first
Bayesian Nonparametrics
Bayesian Nonparametrics Lorenzo Rosasco 9.520 Class 18 April 11, 2011 About this class Goal To give an overview of some of the basic concepts in Bayesian Nonparametrics. In particular, to discuss Dirichelet
Spring 2012 Math 541B Exam 1
Spring 2012 Math 541B Exam 1 1. A sample of size n is drawn without replacement from an urn containing N balls, m of which are red and N m are black; the balls are otherwise indistinguishable. Let X denote
Midterm Examination. STA 215: Statistical Inference. Due Wednesday, 2006 Mar 8, 1:15 pm
Midterm Examination STA 215: Statistical Inference Due Wednesday, 2006 Mar 8, 1:15 pm This is an open-book take-home examination. You may work on it during any consecutive 24-hour period you like; please
Mathematics Ph.D. Qualifying Examination Stat Probability, January 2018
Mathematics Ph.D. Qualifying Examination Stat 52800 Probability, January 2018 NOTE: Answers all questions completely. Justify every step. Time allowed: 3 hours. 1. Let X 1,..., X n be a random sample from
Math Spring Practice for the final Exam.
Math 4 - Spring 8 - Practice for the final Exam.. Let X, Y, Z be three independnet random variables uniformly distributed on [, ]. Let W := X + Y. Compute P(W t) for t. Honors: Compute the CDF function
Chapter 3. Point Estimation. 3.1 Introduction
Chapter 3 Point Estimation Let (Ω, A, P θ ), P θ P = {P θ θ Θ}be probability space, X 1, X 2,..., X n : (Ω, A) (IR k, B k ) random variables (X, B X ) sample space γ : Θ IR k measurable function, i.e.
t x 1 e t dt, and simplify the answer when possible (for example, when r is a positive even number). In particular, confirm that EX 4 = 3.
Mathematical Statistics: Homewor problems General guideline. While woring outside the classroom, use any help you want, including people, computer algebra systems, Internet, and solution manuals, but mae
All other items including (and especially) CELL PHONES must be left at the front of the room.
TEST #2 / STA 5327 (Inference) / Spring 2017 (April 24, 2017) Name: Directions This exam is closed book and closed notes. You will be supplied with scratch paper, and a copy of the Table of Common Distributions
Properties of Random Variables
Properties of Random Variables 1 Definitions A discrete random variable is defined by a probability distribution that lists each possible outcome and the probability of obtaining that outcome If the random
Course arrangements. Foundations of Statistical Inference
Course arrangements Foundations of Statistical Inference Jonathan Marchini Department of Statistics University of Oxford MT 013 Lectures M. and Th.10 SPR1 weeks 1-8 Classes Weeks 3-8 Friday 1-1, 4-5, 5-6
Parametric Inference
Parametric Inference Moulinath Banerjee University of Michigan April 14, 2004 1 General Discussion The object of statistical inference is to glean information about an underlying population based on a
Qualifying Exam in Probability and Statistics. https://www.soa.org/files/edu/edu-exam-p-sample-quest.pdf
Part 1: Sample Problems for the Elementary Section of Qualifying Exam in Probability and Statistics https://www.soa.org/files/edu/edu-exam-p-sample-quest.pdf Part 2: Sample Problems for the Advanced Section
Chapter 6: Large Random Samples Sections
Chapter 6: Large Random Samples Sections 6.1: Introduction 6.2: The Law of Large Numbers Skip p. 356-358 Skip p. 366-368 Skip 6.4: The correction for continuity Remember: The Midterm is October 25th in
Continuous Distributions
Continuous Distributions 1.8-1.9: Continuous Random Variables 1.10.1: Uniform Distribution (Continuous) 1.10.4-5 Exponential and Gamma Distributions: Distance between crossovers Prof. Tesler Math 283 Fall