Lecture 23: Minimal sufficiency
|
|
- Scarlett Dickerson
- 5 years ago
- Views:
Transcription
1 Lecture 23: Miimal sufficiecy Maximal reductio without loss of iformatio There are may sufficiet statistics for a give problem. I fact, X (the whole data set) is sufficiet. If T is a sufficiet statistic ad T = ψ(s), where ψ is a fuctio ad S is aother statistic, the S is sufficiet. For istace, if X 1,...,X are iid with P(X i = 1) = θ ad P(X i = 0) = 1 θ, the ( m X i, i=m+1 X i) is sufficiet for θ, where m is ay fixed iteger betwee 1 ad. If T is sufficiet ad T = ψ(s) with a measurable fuctio ψ that is ot oe-to-oe, the T is more useful tha S, sice T provides a further reductio of the data without loss of iformatio. Is there a sufficiet statistics that provides maximal" reductio of the data? Defiitio (miimal sufficiecy) A sufficiet statistic T (X) is a miimal sufficiet statistic if, for ay other sufficiet statistic U(X), T (x) is a fuctio of U(x). UW-Madiso (Statistics) Stat 609 Lecture / 15
2 T (x) is a fuctio of U(x) iff U(x) = U(y) implies that T (x) = T (y) for ay pair (x,y). I terms of the partitios of the rage of the data set, if T (x) is a fuctio of U(x), the {x : U(x) = t} {x : T (x) = t} Thus, a miimal sufficiet statistic achieves the greatest possible data reductio as a sufficiet statistic. If both T ad S are miimal sufficiet statistics, the by defiitio there is a oe-to-oe fuctio ψ such that T = ψ(s); hece, the miimal sufficiet statistic is uique i the sese that two statistics that are fuctios of each other ca be treated as oe statistic. For example, if T is miimal sufficiet, the so is (T,e T ), but o oe is goig to use (T,e T ). If the rage of X is R k, the there exists a miimal sufficiet statistic. Example Let X 1,...,X be iid form the uiform(θ,θ + 1) distributio, where θ R is a ukow parameter. UW-Madiso (Statistics) Stat 609 Lecture / 15
3 The joit pdf of (X 1,...,X ) is I({θ < x i < θ + 1}) = I({x () 1 < θ < x (1) }), (x 1,...,x ) R, where x (i) deotes the ith ordered value of x 1,...,x. By the factorizatio theorem, T = (X (1),X () ) is sufficiet for θ. Note that x (1) = sup{θ : f θ (x) > 0} ad x () = 1 + if{θ : f θ (x) > 0}. If S(X) is a statistic sufficiet for θ, the by the factorizatio theorem, there are fuctios h ad g θ such that f θ (x) = g θ (S(x))h(x). For x with h(x) > 0, x (1) = sup{θ : g θ (S(x)) > 0} ad x () = 1 + if{θ : g θ (S(x)) > 0}. Hece, there is a fuctio ψ such that T (x) = ψ(s(x)) whe h(x) > 0. Sice P(h(X) > 0) = 1, we coclude that T is miimal sufficiet. I this example, the dimesio of a miimal sufficiet statistic is larger tha the dimesio of θ. UW-Madiso (Statistics) Stat 609 Lecture / 15
4 Fidig a miimal sufficiet statistic by defiitio is ot coveiet. The ext theorem is a useful tool. Theorem Let f θ (x) be the pmf or pdf of X. Suppose that T (x) is sufficiet for θ ad that, for every pair x ad y with at least oe of f θ (x) ad f θ (y) is ot 0, f θ (x)/f θ (y) does ot deped o θ implies T (x) = T (y). (Here, we defie a/0 = for ay a > 0.) The T (X) is miimal sufficiet for θ. Proof. Let U(X) be aother sufficiet statistic. By the factorizatio theorem, there are fuctios h ad g θ such that f θ (x) = g θ (U(x))h(x) for all x ad θ. For x ad y such that at least oe of f θ (x) ad f θ (y) is ot 0, i.e., at least oe of h(x) ad h(y) is ot 0, if U(x) = U(y), the which does ot deped o θ. f θ (x) f θ (y) = g θ (U(x))h(x) g θ (U(y))h(y) = h(x) h(y) UW-Madiso (Statistics) Stat 609 Lecture / 15
5 By the assumptio of the theorem, T (x) = T (y). This shows that there is a fuctio ψ such that T (x) = ψ(s(x)) Hece, T is miimal sufficiet for θ Example We re-visit this example ad show how to apply Theorem If X 1,...,X are iid form the uiform(θ,θ + 1) distributio, the the joit pdf is f θ (x) = I({x () 1 < θ < x (1) }), x = (x 1,...,x ) R, Let x ad y be two data poits. Each of f θ (x) ad f θ (y) oly takes two possible values, 0 ad 1. Let A x = (x () 1,x (1) ) ad A y = (y () 1,y (1) ). The f θ (x) 0 θ A x,θ A y f θ (y) = 1 θ A x,θ A y θ A x,θ A y This depeds o θ uless A x = A y. UW-Madiso (Statistics) Stat 609 Lecture / 15
6 Thus, if the ratio does ot deped o θ, we must have x (1) = y (1) ad x () = y (). Therefore, by Theorem , (X (1),X () ) is miimal sufficiet for θ. Example Let X 1,...,X be iid from N(µ,σ 2 ) with ukow µ R ad σ > 0. Earlier, we showed that T = ( X,S 2 ) is sufficiet for θ = (µ,σ 2 ). Is T miimal sufficiet? Let f θ (x) be the joit pdf of the sample, x ad y be two sample poits. The f θ (x) f θ (y) = (2πσ 2 ) /2 exp( [(( x µ) 2 + ( 1)s 2 x]/(2σ 2 )) (2πσ 2 ) /2 exp( [((ȳ µ) 2 + ( 1)s 2 y]/(2σ 2 )) = exp([ ( x 2 ȳ 2 ) + 2µ( x ȳ) ( 1)(s 2 x s 2 y)]/(2σ 2 )) where x ad s 2 x are the sample mea ad variace based o the sample poit x, ad ȳ ad s 2 y are the sample mea ad variace based o the sample poit y. UW-Madiso (Statistics) Stat 609 Lecture / 15
7 This ratio does ot deped o θ = (µ,σ 2 ) iff x = ȳ ad s 2 x = s 2 y. By Theorem , T = ( X,S 2 ) is miimal sufficiet for θ = (µ,σ 2 ). Like the cocept of sufficiecy, miimal sufficiecy also depeds o the parameters we are iterested i. I Example , let us cosider two sub-families. Suppose that it is kow that σ = 1. The T = ( X,S 2 ) is still sufficiet but ot miimal sufficiet for µ. I fact, usig Theorem we ca show that X is miimal sufficiet for µ whe σ = 1 ad, T = ( X,S 2 ) is ot a fuctio of X (why?). Suppose that it is kow that µ 2 = σ 2 so that we oly have oe ukow parameter µ R, µ 0. (This is called a curved expoetial family.) Is T = ( X,S 2 ) miimal sufficiet for θ = µ? Note that T is certaily sufficiet for θ. The aswer ca be foud as a special case of the followig result. UW-Madiso (Statistics) Stat 609 Lecture / 15
8 Miimal sufficiet statistics i expoetial families Let X 1,...,X be iid from a expoetial family with pdf f θ (x) = h(x)c(θ)exp ( η(θ) T (x) ), θ Θ where η(θ) = (w 1 (θ),...,w k (θ)) ad T (x) = (t 1 (x),...,t k (x)). Suppose that there exists {θ 0,θ 1,...,θ k } Θ such that the vectors η i = η(θ i ) η(θ 0 ), i = 1,...,k, are liearly idepedet i R k. (This is true if the family is of full rak). We have show that T (X) is sufficiet for θ. We ow show that T is i fact miimal sufficiet for θ. Note that we ca focus o the pits x at which h(x) > 0. For ay θ, f θ (x) f θ (y) = exp( η(θ) [T (x) T (y)] ) h(x) h(y) If this ratio equals φ(x,y) ot depedig o θ, the which implies η(θ) [T (x) T (y)] = log(φ(x,y)) + log(h(y)/h(x)) [η(θ i ) η(θ 0 )] [T (x) T (y)] = 0 i = 1,...,k. UW-Madiso (Statistics) Stat 609 Lecture / 15
9 Sice η(θ i ) η(θ 0 ), i = 1,...,k, are liearly idepedet, we must have T (x) = T (y). By Theorem , T is miimal sufficiet for θ. Applicatio to curved ormal family We ca ow show that T = ( X,S 2 ) miimal sufficiet for θ = µ whe the sample is a radom sample from N(µ, µ 2 ), µ R, µ 0. It ca be show that ( µ η(θ) = σ 2, 1 ) ( 2σ 2 = µ, 1 ) 2µ 2 Poits θ 0 = (1,1), θ 1 = ( 1,1), ad θ 2 = (1/2,1/2) are i the parameter space, ad ( ) ( ) ( 2 η(θ 1 ) η(θ 0 ) = = ( 1)/2 ( 1)/2 0 ( η(θ 2 ) η(θ 0 ) = /2 2( 1) ) ( ( 1)/2 ) ( = ) ) /2 3( 1)/2 UW-Madiso (Statistics) Stat 609 Lecture / 15
10 Are these two vectors liearly idepedet? If there are c ad d such that i.e., c[η(θ 1 ) η(θ 0 )] + d[η(θ 2 ) η(θ 0 )] = 0 ( 2 c 0 ) ( + d /2 3( 1)/2 ) = 0 The, we must have d = 0 ad the c = 0. That meas vectors η(θ 1 ) η(θ 0 ) ad η(θ 2 ) η(θ 0 ) are liearly idepedet ad the coditio for the result of expoetial family is satisfied. Therefore, T = ( X,S 2 ) is miimal sufficiet for θ = µ. Example. Let (X 1,...,X ) be a radom sample from Cauchy(µ,σ), where µ R ad σ > 0 are ukow parameters. The joit pdf of (X 1,...,X ) is f µ,σ (x) = σ 1 π σ 2 + (x i µ) 2, x = (x 1,...,x ) R. UW-Madiso (Statistics) Stat 609 Lecture / 15
11 For ay x = (x 1,...,x ) ad y = (y 1,...,y ), if f µ,σ (x) = ψ(x,y)f µ,σ (y) holds for ay µ ad σ, where ψ does ot deped o (µ,σ), the [ 1 + (y i µ) 2] [ = ψ(x,y) 1 + (x i µ) 2] µ R Both sides of the above idetity are polyomials of degree 2 i µ. Compariso of the coefficiets to the highest terms gives ψ(x,y) = 1 ad hece [ 1 + (y i µ) 2] = [ 1 + (x i µ) 2] µ R As a polyomial of µ, the left-had side of the above idetity has 2 complex roots x i ± ı, i = 1,...,, while the right-had side of the above idetity has 2 complex roots y i ± ı, i = 1,...,. Sice the two sets of roots must agree, the ordered values of x i s are the same as the ordered values of y i s. By Theorem , the order statistics of X 1,...,X is miimal sufficiet for (µ,σ). UW-Madiso (Statistics) Stat 609 Lecture / 15
12 The followig result may be useful i establishig miimal sufficiecy outside of the expoetial family. Theorem Let f 0,f 1,f 2,... be a sequece of pdf s or pmf s o the rage of X, all have the same support. a. The statistic ( f1 (X) T (X) = f 0 (X), f ) 2(X) f 0 (X),... is miimal sufficiet for the family {f 0,f 1,f 2,...} of populatios for X. b. If P is a family of pdf s or pmf s for X with commo support, ad (i) f i P, i = 0,1,2,... (ii) T (X) is sufficiet for P, the T is miimal sufficiet for P. Proof of part a. Let g i (T ) = T i, the ith compoet of T, i = 0,1,2,..., ad let g 0 (T ) = 1. The f i (x) = g i (T (x))f 0 (x), i = 0,1,2,... By the factorizatio theorem, T is sufficiet for P 0 = {f 0,f 1,f 2,...}. UW-Madiso (Statistics) Stat 609 Lecture / 15
13 Suppose that S(X) is aother sufficiet statistic. By the factorizatio theorem, there are Borel fuctios h ad g i such that f i (x) = g i (S(x))h(x), i = 0,1,2,... The T i (x) = f i(x) f 0 (x) = g i(s(x)) g 0 (S(x)), i = 0,1,2,... Thus, T is a fuctio of S. By defiitio, T is miimal sufficiet for P 0. Proof of part b. The two additioal coditios are P 0 P ad T is sufficiet for P. Let S be ay sufficiet statistic for P. The S is also sufficiet for P 0. From part a, T is miimal sufficiet for P 0 so there is a fuctio ψ such that T = ψ(s). By defiitio, T is miimal sufficiet for P sice T is sufficiet for P. UW-Madiso (Statistics) Stat 609 Lecture / 15
14 I fact, the result i Theorem still holds if the commo support coditio is replaced by that the support of f 0 cotais the support of ay pdf or pmf i P. I most applicatios, we ca choose P 0 cotaiig fiitely may elemets. Example Let X 1,...,X be a radom sample from double-expoetial(µ,1), µ R, i.e., the joit pdf of X is ) f µ (x) = 1 2 exp ( x i µ Cosider P 0 = {f 0,f 1,...,f }, where each f j is the pdf with µ = j, j = 0,1,...,. By Theorem 6.6.5a, a miimal sufficiet statistic for P 0 is ( ) ) T = exp( X i X i j, j = 1,..., which is equivalet to UW-Madiso (Statistics) Stat 609 Lecture / 15
15 ( U = X i X i j, j = 1,..., We ca further show that U is equivalet to S, the set of order statistics. Sice S is sufficiet, by Theorem 6.6.5b, S, U, ad T are all miimal sufficiet. Defiitio (Necessity). A statistic is said to be ecessary if it ca be writte as a fuctio of every sufficiet statistic. T is ecessary if, for every sufficiet S, there is a fuctio ψ such that T = ψ(s). From the defiitios of the ecessity ad miimal sufficiecy, we ca reach the followig coclusio. Theorem A statistic is miimal sufficiet if ad oly if it is ecessary ad sufficiet. ) UW-Madiso (Statistics) Stat 609 Lecture / 15
Chapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More information6. Sufficient, Complete, and Ancillary Statistics
Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationLecture 18: Sampling distributions
Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationLecture 16: UMVUE: conditioning on sufficient and complete statistics
Lecture 16: UMVUE: coditioig o sufficiet ad complete statistics The 2d method of derivig a UMVUE whe a sufficiet ad complete statistic is available Fid a ubiased estimator of ϑ, say U(X. Coditioig o a
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More informationLast Lecture. Unbiased Test
Last Lecture Biostatistics 6 - Statistical Iferece Lecture Uiformly Most Powerful Test Hyu Mi Kag March 8th, 3 What are the typical steps for costructig a likelihood ratio test? Is LRT statistic based
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationMath 155 (Lecture 3)
Math 55 (Lecture 3) September 8, I this lecture, we ll cosider the aswer to oe of the most basic coutig problems i combiatorics Questio How may ways are there to choose a -elemet subset of the set {,,,
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationLecture 17: Minimal sufficiency
Lecture 17: Minimal sufficiency Maximal reduction without loss of information There are many sufficient statistics for a given family P. In fact, X (the whole data set) is sufficient. If T is a sufficient
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationStatistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
More informationStatistical Theory MT 2008 Problems 1: Solution sketches
Statistical Theory MT 008 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. a) Let 0 < θ < ad put fx, θ) = θ)θ x ; x = 0,,,... b) c) where α
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationWeek 5-6: The Binomial Coefficients
Wee 5-6: The Biomial Coefficiets March 6, 2018 1 Pascal Formula Theorem 11 (Pascal s Formula For itegers ad such that 1, ( ( ( 1 1 + 1 The umbers ( 2 ( 1 2 ( 2 are triagle umbers, that is, The petago umbers
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationLecture 6 Ecient estimators. Rao-Cramer bound.
Lecture 6 Eciet estimators. Rao-Cramer boud. 1 MSE ad Suciecy Let X (X 1,..., X) be a radom sample from distributio f θ. Let θ ˆ δ(x) be a estimator of θ. Let T (X) be a suciet statistic for θ. As we have
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationFirst Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise
First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More information5. Likelihood Ratio Tests
1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationQuestions and Answers on Maximum Likelihood
Questios ad Aswers o Maximum Likelihood L. Magee Fall, 2008 1. Give: a observatio-specific log likelihood fuctio l i (θ) = l f(y i x i, θ) the log likelihood fuctio l(θ y, X) = l i(θ) a data set (x i,
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationLecture 24: Variable selection in linear models
Lecture 24: Variable selectio i liear models Cosider liear model X = Z β + ε, β R p ad Varε = σ 2 I. Like the LSE, the ridge regressio estimator does ot give 0 estimate to a compoet of β eve if that compoet
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationStat 421-SP2012 Interval Estimation Section
Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationLesson 10: Limits and Continuity
www.scimsacademy.com Lesso 10: Limits ad Cotiuity SCIMS Academy 1 Limit of a fuctio The cocept of limit of a fuctio is cetral to all other cocepts i calculus (like cotiuity, derivative, defiite itegrals
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationCHAPTER 5. Theory and Solution Using Matrix Techniques
A SERIES OF CLASS NOTES FOR 2005-2006 TO INTRODUCE LINEAR AND NONLINEAR PROBLEMS TO ENGINEERS, SCIENTISTS, AND APPLIED MATHEMATICIANS DE CLASS NOTES 3 A COLLECTION OF HANDOUTS ON SYSTEMS OF ORDINARY DIFFERENTIAL
More informationThe multiplicative structure of finite field and a construction of LRC
IERG6120 Codig for Distributed Storage Systems Lecture 8-06/10/2016 The multiplicative structure of fiite field ad a costructio of LRC Lecturer: Keeth Shum Scribe: Zhouyi Hu Notatios: We use the otatio
More informationSummary. Recap ... Last Lecture. Summary. Theorem
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationLecture 3. Properties of Summary Statistics: Sampling Distribution
Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationIntroduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT
Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory
More informationSolutions: Homework 3
Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe
More informationLecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
More informationStat410 Probability and Statistics II (F16)
Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems
More informationMATHEMATICAL SCIENCES PAPER-II
MATHEMATICAL SCIENCES PAPER-II. Let {x } ad {y } be two sequeces of real umbers. Prove or disprove each of the statemets :. If {x y } coverges, ad if {y } is coverget, the {x } is coverget.. {x + y } coverges
More informationProperties of Point Estimators and Methods of Estimation
CHAPTER 9 Properties of Poit Estimators ad Methods of Estimatio 9.1 Itroductio 9. Relative Efficiecy 9.3 Cosistecy 9.4 Sufficiecy 9.5 The Rao Blackwell Theorem ad Miimum-Variace Ubiased Estimatio 9.6 The
More informationMATH 472 / SPRING 2013 ASSIGNMENT 2: DUE FEBRUARY 4 FINALIZED
MATH 47 / SPRING 013 ASSIGNMENT : DUE FEBRUARY 4 FINALIZED Please iclude a cover sheet that provides a complete setece aswer to each the followig three questios: (a) I your opiio, what were the mai ideas
More informationStat 200 -Testing Summary Page 1
Stat 00 -Testig Summary Page 1 Mathematicias are like Frechme; whatever you say to them, they traslate it ito their ow laguage ad forthwith it is somethig etirely differet Goethe 1 Large Sample Cofidece
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More information1. ARITHMETIC OPERATIONS IN OBSERVER'S MATHEMATICS
1. ARITHMETIC OPERATIONS IN OBSERVER'S MATHEMATICS We cosider a ite well-ordered system of observers, where each observer sees the real umbers as the set of all iite decimal fractios. The observers are
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationKurskod: TAMS11 Provkod: TENB 21 March 2015, 14:00-18:00. English Version (no Swedish Version)
Kurskod: TAMS Provkod: TENB 2 March 205, 4:00-8:00 Examier: Xiagfeg Yag (Tel: 070 2234765). Please aswer i ENGLISH if you ca. a. You are allowed to use: a calculator; formel -och tabellsamlig i matematisk
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationBinomial Distribution
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible
More informationJoint Probability Distributions and Random Samples. Jointly Distributed Random Variables. Chapter { }
UCLA STAT A Applied Probability & Statistics for Egieers Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistat: Neda Farziia, UCLA Statistics Uiversity of Califoria, Los Ageles, Sprig
More informationIt is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.
MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied
More informationApproximations and more PMFs and PDFs
Approximatios ad more PMFs ad PDFs Saad Meimeh 1 Approximatio of biomial with Poisso Cosider the biomial distributio ( b(k,,p = p k (1 p k, k λ: k Assume that is large, ad p is small, but p λ at the limit.
More informationLecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett
Lecture Note 8 Poit Estimators ad Poit Estimatio Methods MIT 14.30 Sprig 2006 Herma Beett Give a parameter with ukow value, the goal of poit estimatio is to use a sample to compute a umber that represets
More informationTests of Hypotheses Based on a Single Sample (Devore Chapter Eight)
Tests of Hypotheses Based o a Sigle Sample Devore Chapter Eight MATH-252-01: Probability ad Statistics II Sprig 2018 Cotets 1 Hypothesis Tests illustrated with z-tests 1 1.1 Overview of Hypothesis Testig..........
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationGoodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen)
Goodess-of-Fit Tests ad Categorical Data Aalysis (Devore Chapter Fourtee) MATH-252-01: Probability ad Statistics II Sprig 2019 Cotets 1 Chi-Squared Tests with Kow Probabilities 1 1.1 Chi-Squared Testig................
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationBeurling Integers: Part 2
Beurlig Itegers: Part 2 Isomorphisms Devi Platt July 11, 2015 1 Prime Factorizatio Sequeces I the last article we itroduced the Beurlig geeralized itegers, which ca be represeted as a sequece of real umbers
More informationUnderstanding Samples
1 Will Moroe CS 109 Samplig ad Bootstrappig Lecture Notes #17 August 2, 2017 Based o a hadout by Chris Piech I this chapter we are goig to talk about statistics calculated o samples from a populatio. We
More informationDIVISIBILITY PROPERTIES OF GENERALIZED FIBONACCI POLYNOMIALS
DIVISIBILITY PROPERTIES OF GENERALIZED FIBONACCI POLYNOMIALS VERNER E. HOGGATT, JR. Sa Jose State Uiversity, Sa Jose, Califoria 95192 ad CALVIN T. LONG Washigto State Uiversity, Pullma, Washigto 99163
More information5 : Exponential Family and Generalized Linear Models
0-708: Probabilistic Graphical Models 0-708, Sprig 206 5 : Expoetial Family ad Geeralized Liear Models Lecturer: Matthew Gormley Scribes: Yua Li, Yichog Xu, Silu Wag Expoetial Family Probability desity
More informationChapter 6 Sampling Distributions
Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to
More information1. By using truth tables prove that, for all statements P and Q, the statement
Author: Satiago Salazar Problems I: Mathematical Statemets ad Proofs. By usig truth tables prove that, for all statemets P ad Q, the statemet P Q ad its cotrapositive ot Q (ot P) are equivalet. I example.2.3
More informationMeasure and Measurable Functions
3 Measure ad Measurable Fuctios 3.1 Measure o a Arbitrary σ-algebra Recall from Chapter 2 that the set M of all Lebesgue measurable sets has the followig properties: R M, E M implies E c M, E M for N implies
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More information10-701/ Machine Learning Mid-term Exam Solution
0-70/5-78 Machie Learig Mid-term Exam Solutio Your Name: Your Adrew ID: True or False (Give oe setece explaatio) (20%). (F) For a cotiuous radom variable x ad its probability distributio fuctio p(x), it
More informationMath 61CM - Solutions to homework 3
Math 6CM - Solutios to homework 3 Cédric De Groote October 2 th, 208 Problem : Let F be a field, m 0 a fixed oegative iteger ad let V = {a 0 + a x + + a m x m a 0,, a m F} be the vector space cosistig
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationBayesian Methods: Introduction to Multi-parameter Models
Bayesia Methods: Itroductio to Multi-parameter Models Parameter: θ = ( θ, θ) Give Likelihood p(y θ) ad prior p(θ ), the posterior p proportioal to p(y θ) x p(θ ) Margial posterior ( θ, θ y) is Iterested
More informationAnalytic Continuation
Aalytic Cotiuatio The stadard example of this is give by Example Let h (z) = 1 + z + z 2 + z 3 +... kow to coverge oly for z < 1. I fact h (z) = 1/ (1 z) for such z. Yet H (z) = 1/ (1 z) is defied for
More informationReview Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn
Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom
More informationIntroductory statistics
CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key
More information( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2
82 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Defiitio: et X be a radom sample with joit p.m/d.f. f X x θ. The geeralised likelihood ratio test g.l.r.t. of the NH : θ H 0 agaist the alterative AH : θ H 1,
More information