Chi-Squared Tests Math 6070, Spring 2006
|
|
- Pearl Shepherd
- 5 years ago
- Views:
Transcription
1 Chi-Squared Tests Math 6070, Sprig 2006 Davar Khoshevisa Uiversity of Utah February XXX, 2006 Cotets MLE for Goodess-of Fit 2 2 The Multiomial Distributio 3 3 Applicatio to Goodess-of-Fit 6 3 Testig for a Fiite Distributio 6 32 Testig for a Desity 8 Chi-squared tests refers to a family of statistical test whose large-sample asymptotics are related i oe way or aother to the χ 2 family of distributios These are our first examples of o-parametric procedures Let me metio oe particularly useful chi-squared test here before we get started properly Cosider a populatio whose distributio is believed to follow a kow desity fuctio f say, expoetial(2) The there is a chi-squared test that ca be used to check our hypothesis This chi-squared test is easy to implemet, ad works well for large-sized samples First, we study a related parametric estimatio problem The we fid a cosistet MLE Ad fially, we use our estimate to devise a test for our hypothesis This will be doe i several stages It is likely that you already kow somethig of the resultig χ 2 -test (page 7) But here you will fid a rigorous proof, i lie with the spirit of this course, of the fact that the said χ 2 -test actually works A [covicig but] o-rigorous justificatio of the same fact ca be foud i pages of the excellet moograph of P Bickel ad K Doksum [Mathematical Statistics, Holde Day, First editio, 977] There, a likelihood priciple argumet is devised, which usually works
2 MLE for Goodess-of Fit Suppose X,, X is a idepedet sample from a fiite populatio More precisely, let t,, t m be m distict umbers (or types ), ad θ := (θ,, θ m ) a m-dimesioal probability vector Cosider iid radom variables X,, X such that P{X = t i } = θ i for all i m That is, each X is of type t i with probability θ i We wish to fid a (actually, the ) MLE for the mass fuctio of the X i s That is, the MLE for the vector θ := (θ,, θ m ) Note that, here, Θ is the collectio of all θ such that 0 θ,, θ m ad θ + + θ m = This is a kid of parametric problem, of course, ad we proceed by computig the MLE Defie N := (N,, N m ) to be the cout statistics for the various types Usig colum-vector otatio, this meas that j= I {X j = t } N = () j= I {X j = t m } I words, N k deotes the umber of occuraces of type k i the sample X := (X,, X ) Note that m j= N j =, so the N j s are obviously depedet rv s Let us calculate their joit mass fuctio Let = (,, m ) be a m-vector of positive itegers such that m j= j = Such s are the possible values of N Moreover, P θ {N = } = P θ {X = s,, X = s }, (2) where the sum is take over all s = (s,, s ) such that of them are of type t, 2 of type t 2, etc For all such s, P θ {X = s,, X = s } = θ θm m (3) Moreover, the umber of all such s s is exactly ( )! :=,, m! m! (4) Therefore, we have derived the followig Lemma The radom -vector N has the multiomial distributio with parameters (, θ) That is, for all itegers,, m such that + + m =, ( ) P θ {N = } = θ,, θm m (5) m Cosider the special case = 2 The, N biomial(, θ), ad N 2 = N So the multiomial distributio is a extesio of the biomial i this 2
3 sese We write N Multiomial(, θ) i place of the statemet that N is multiomial with parameters (, θ) It turs out that we ca compute the MLE for θ by usig N as the basic data To do so, we use Lemma ad first fid the log-likelihood as follows: L(θ) = A + m N i l θ i, (6) where A does ot deped o θ The case where some θ i {0, } is trivial ad ca be ruled out easily Therefore, we may assume that 0 < θ i < for all i =,, m Thus, for all l =,, m, θ l L(θ) = m ( Ni θ i θ ) i (7) θ l As θ varies i Θ, θ,, θ m roam freely subject to the sigle coditio that θ m = m θ i I other words, ( θ i / θ l ) = 0 for all i =,, m differet from l, but ( θ m / θ l ) = This proves that ( L(θ)/ θ l ) = (N m /θ m ) (N l /θ l ) for all l =,, m Set it equal to zero (for MLE) to see that there must exist γ such that N j = γθ j for all j =,, m But m j= N j =, whereas m j= θ j = Therefore, γ = We have proved the followig Lemma 2 The MLE for θ based o N,, N m is give by N / ˆθ := (8) N m / [To prove that L/ θ l = 0 ideed yields a maximum, you take secod derivatives ad apply similar argumets] 2 The Multiomial Distributio Choose ad fix a probability vector θ, ad assume it is the true oe so that we write P, E, etc i place of the more cumbersome P θ, E θ, etc We ca use () to say a few more thigs about the distributio of N Note first that () is equivalet to the followig: N = I {X j = t } I {X j = t m } j= We ca compute directly to fid that := Y j (9) j= EY = θ, (0) 3
4 ad θ ( θ ) θ θ 2 θ θ m θ θ 2 θ 2 ( θ 2 ) θ 2 θ m CovY = θ m θ θ m θ 2 θ m ( θ m ) := Q 0 () Because Y,, Y are iid radom vectors, the followig is a cosequece of the precedig, cosidered i cojuctio with the multidimesioal CLT Theorem 3 As, N θ d Nm (0, Q 0 ) (2) There is a variat of Theorem 3 which is more useful to us Defie (N θ )/ θ Ñ := (N m θ m )/ (3) θ m This too is a sum of iid radom vectors Ỹ,, Ỹ, ad has EỸ := 0 ad CovỸ = Q, (4) where Q is the (m m)-dimesioal covariace matrix give below: θ θ θ 2 θ θ m θ θ 2 θ 2 θ 2 θ m Q = θ m θ (5) θ m θ 2 θ m The we obtai, Theorem 4 As, Ñ d N m (0, Q) (6) Remark 5 Oe should be a little careful at this stage because Q is sigular [For example, cosider the case m = 2] However, N m (0, Q) makes sese eve i this case The reaso is this: If C is a o sigular covariace matrix, the we kow what it meas for X N m (0, C), ad a direct computatio reveals that for all real umbers t ad all m-vectors ξ, E [exp (itξ X)] = E [ exp ( t2 2 ξ Cξ )] (7) 4
5 This defies the characteristic fuctio of the radom variable ξ X uiquely There is a theorem (Ma Wold device) that says that a uique descriptio of the characteristic fuctio of ξ X for all ξ also describes the distributio of X uiquely This meas that ay radom vector X that satisfies (7) for all t R ad ξ R m is N m (0, C) Moreover, this approach does ot require C to be ivertible To lear more about multivariate ormals, you should atted Math 600 As a cosequece of Theorem 4 we have the followig: As, Ñ 2 d Nm (0, Q) 2 (8) [Recall that x 2 = x x 2 m] Next, we idetify the distributio of N m (0, Q) 2 A direct (ad somewhat messy) calculatio shows that Q is a projectio matrix That is, Q 2 := QQ = Q It follows fairly readily from this that its eigevalues are all zeros ad oes Here is the proof: Let λ be a eigevalue ad x a eigevector Without loss of geerality, we may assume that x = ; else, cosider the eigevector y := x/ x istead This meas that Qx = λx Therefore, Q 2 x = λ 2 x Because Q 2 = Q, we have prove that λ = λ 2, whece it follows that λ {0, }, as promised Write Q i its spectral form That is, λ 0 0 Q = P 0 λ 2 0 P, (9) 0 0 λ m where P is a orthogoal (m m) matrix [This is because Q is positive semidefiite You ca lear more o this i Math 600] Defie λ λ2 0 Λ := P (20) 0 0 λm The, Q = Λ Λ, which is why Λ is called the (Perose) square root of Q Let Z be a m-vector of iid stadard ormals ad cosider ΛZ This is multivariate ormal (check characteristic fuctios), E[ΛZ] = 0, ad Cov(ΛZ) = Λ Cov(Z)Λ = Λ Λ = Q I other words, we ca view N m (0, Q) as ΛZ Therefore, N m (0, Q) 2 d = ΛZ 2 = m λ j Zi 2 (2) Because λ i {0, }, we have proved that N m (0, Q) χ 2 r, where r = rak(q) Aother messy but elemetary computatio shows that the rak of 5
6 Q is m [Roughly speakig, this is because the θ i s have oly oe source of liear depedece Namely, that θ + + θ m = ] This proves that Ñ 2 d χ 2 m (22) Write out the left-had side explicitly to fid the followig: Theorem 6 As, m (N i θ i ) 2 θ i d χ 2 m (23) It took a while to get here, but we will soo be rewarded for our effort 3 Applicatio to Goodess-of-Fit 3 Testig for a Fiite Distributio I light of the developmet of Sectio, ca paraphrase Theorem 6 as sayig that uder P θ, as, X (θ) d χ 2 m, where X (θ) := ) 2 m (ˆθi θ i (24) θ i Therefore, we are i a positio to derive a asymptotic ( α)-level cofidece set for θ Defie for all a > 0, { C (a) := θ Θ : X (θ) a } (25) Equatio (24) shows that for all θ Θ, lim P θ {θ C (a)} = P { χ 2 m a } (26) We ca fid (eg, from χ 2 -tables) a umber χ 2 (α) such that It follows that P { χ 2 m > χ 2 m (α) } = α (27) C ( χ 2 m (α) ) is a asymptotic level ( α) cofidece iterval for θ (28) The statistic X (θ) was discovered by K Pearso i 900, ad is therefore called the Pearso χ 2 statistic Icidetally, the term stadard deviatio is also due to Karl Pearso (893) 6
7 The statistics X (θ) is a little awkward to work with, ad this makes the computatio of C (χ 2 m (α)) difficult Nevertheless, it is easy eough to use C (χ 2 m (α)) to test simple hypotheses for θ Suppose we are iterested i the test H 0 : θ = θ 0 versus H : θ θ 0, (29) where θ 0 = (θ 0,, θ 0m ) is a (kow) probability vector i Θ The we ca reject H 0 (at asymptotic level ( α)) if θ 0 C (χ 2 m (α)) I other words, we reject H 0 if X (θ 0 ) > χ2 m (α) (30) Because X (θ 0 ) is easy to compute, (29) ca be carried out with relative ease Moreover, X (θ 0 ) has the followig iterpretatio: Uder H 0, X (θ 0 ) = m (Obs i Exp i ) 2, (3) Exp i where Obs i deotes the umber of observatios i the sample that are of type i, ad Exp i := θ 0i is the expected umber uder H 0 of sample poits that are of type i Next we study two examples Example 7 A certai populatio cotais oly the digits 0,, ad 2 We wish to kow if every digit is equally likely to occur i this populatio That is, we wish to test H 0 : θ = ( 3, 3, 3 ) versus H : θ ( 3, 3, 3 ) (32) To this ed, suppose we make, 000 idepedet draws from this populatio, ad fid that we have draw 300 zeros (ˆθ = 03), 340 oes (ˆθ 2 = 034), ad 360 twos (ˆθ 3 = 036) That is, Types Obs Exp (Obs Exp) 2 /Exp / / / Total (approx) Suppose we wish to test (32) at the traditioal (asymptotic) level 95% Here, m = 3 ad α = 005, ad we fid from chi-square tables that χ 2 2(005) 5999 Therefore, accordig to (30), we reject if ad oly if X (θ 0 ) is greater tha χ 2 2(005)/ Thus, we do ot reject the ull hypothesis at asymptotic level 95% O the other had, χ 2 2(0) 4605, ad X (θ 0 ) is greater tha χ 2 2(0)/ Thus, had we performed the test at 7
8 asymptotic level 90% we would ideed reject H 0 I fact, this discussio has show that the asymptotic P -value of our test is somewhere betwee 005 ad 0 It is more importat to report/estimate P -values tha to make blaket declaratios such as accept H 0 or reject H 0 Cosult your Math text(s) To illustrate this example further, suppose the table of observed vs expected remais the same but = 2000 The, X (θ 0 ) is a good deal greater tha χ 2 2(005)/ ad we reject H 0 soudly at the asymptotic level 95% I fact, i this case the P -value is somewhere betwee 0005 ad (check this!) You ca fid may olie χ 2 -tables Oe such table ca be foud at [The ufortuate misspellig usefull is ot due to me] 32 Testig for a Desity Suppose we wish to kow if a certai data-set comes from the expoetial(2) distributio (say!) So let X,, X be a idepedet sample We have H 0 : X expoetial(2) versus H : X expoetial(2) (33) Now take a fie mesh 0 < z < z 2 < < z m < of real umbers The, P H0 {z j X z j+ } = 2 zj+ z j e 2r dr = e 2zj e 2zj+ (34) These are θ 0j s, ad we ca test to see whether or ot θ = θ 0, where θ j is the true probability P{z j X z j+ } for all j =,, m The χ 2 -test of the previous sectio hadles just such a situatio That is, let ˆθ j deote the proportio of the data X,, X that lies i [z j, z j+ ] The (30) provides us with a asymptotic ( α)-level test for H 0 that the data is expoetial(2) Other distributios are hadled similarly, but θ 0j is i geeral z j+ z j f 0 (x) dx, where f 0 is the desity fuctio uder H 0 Example 8 Suppose we wish to test (33) Let m = 5, ad defie z j = (j )/0 for j =,, 5 The, { j θ 0j = P H0 0 X j } ( ) ( 6(j ) = exp exp 6j ) (35) That is, j θ 0j
9 Cosider the followig (hypothetical) data: = 200, ad j θ 0j ˆθj The, we form the χ 2 -table, as before, ad obtai Types Exp Obs (Obs Exp) 2 /Exp Total (approx) Thus, X (θ 0 ) You should check that 005 P -value 0 I fact, the P -value is much closer to 0 So, at the asymptotic level 95%, we do ot have eough evidece agaist H 0 Some thigs to thik about: I the totals for Exp, you will see that the total is ad ot Ca you thik of why this has happeed? Does this pheomeo reder our coclusios false (or weaker tha they may appear)? Do we eed to fix this by icludig more types? How ca we address it i a sesible way? Would addressig this chage the fial coclusio? 9
Chi-squared tests Math 6070, Spring 2014
Chi-squared tests Math 6070, Sprig 204 Davar Khoshevisa Uiversity of Utah March, 204 Cotets MLE for goodess-of fit 2 2 The Multivariate ormal distributio 3 3 Cetral limit theorems 5 4 Applicatio to goodess-of-fit
More informationGoodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen)
Goodess-of-Fit Tests ad Categorical Data Aalysis (Devore Chapter Fourtee) MATH-252-01: Probability ad Statistics II Sprig 2019 Cotets 1 Chi-Squared Tests with Kow Probabilities 1 1.1 Chi-Squared Testig................
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More information5. Likelihood Ratio Tests
1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More information( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2
82 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Defiitio: et X be a radom sample with joit p.m/d.f. f X x θ. The geeralised likelihood ratio test g.l.r.t. of the NH : θ H 0 agaist the alterative AH : θ H 1,
More informationDiscrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22
CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationA quick activity - Central Limit Theorem and Proportions. Lecture 21: Testing Proportions. Results from the GSS. Statistics and the General Population
A quick activity - Cetral Limit Theorem ad Proportios Lecture 21: Testig Proportios Statistics 10 Coli Rudel Flip a coi 30 times this is goig to get loud! Record the umber of heads you obtaied ad calculate
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationApril 18, 2017 CONFIDENCE INTERVALS AND HYPOTHESIS TESTING, UNDERGRADUATE MATH 526 STYLE
April 18, 2017 CONFIDENCE INTERVALS AND HYPOTHESIS TESTING, UNDERGRADUATE MATH 526 STYLE TERRY SOO Abstract These otes are adapted from whe I taught Math 526 ad meat to give a quick itroductio to cofidece
More informationLast Lecture. Wald Test
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More information1 Inferential Methods for Correlation and Regression Analysis
1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationStatistics 511 Additional Materials
Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationConfidence intervals summary Conservative and approximate confidence intervals for a binomial p Examples. MATH1005 Statistics. Lecture 24. M.
MATH1005 Statistics Lecture 24 M. Stewart School of Mathematics ad Statistics Uiversity of Sydey Outlie Cofidece itervals summary Coservative ad approximate cofidece itervals for a biomial p The aïve iterval
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More informationChapter 22. Comparing Two Proportions. Copyright 2010, 2007, 2004 Pearson Education, Inc.
Chapter 22 Comparig Two Proportios Copyright 2010, 2007, 2004 Pearso Educatio, Ic. Comparig Two Proportios Read the first two paragraphs of pg 504. Comparisos betwee two percetages are much more commo
More informationStat 319 Theory of Statistics (2) Exercises
Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationAgreement of CI and HT. Lecture 13 - Tests of Proportions. Example - Waiting Times
Sigificace level vs. cofidece level Agreemet of CI ad HT Lecture 13 - Tests of Proportios Sta102 / BME102 Coli Rudel October 15, 2014 Cofidece itervals ad hypothesis tests (almost) always agree, as log
More informationTopic 18: Composite Hypotheses
Toc 18: November, 211 Simple hypotheses limit us to a decisio betwee oe of two possible states of ature. This limitatio does ot allow us, uder the procedures of hypothesis testig to address the basic questio:
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationDirection: This test is worth 150 points. You are required to complete this test within 55 minutes.
Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem
More informationSummary. Recap ... Last Lecture. Summary. Theorem
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca
More informationDiscrete Mathematics for CS Spring 2005 Clancy/Wagner Notes 21. Some Important Distributions
CS 70 Discrete Mathematics for CS Sprig 2005 Clacy/Wager Notes 21 Some Importat Distributios Questio: A biased coi with Heads probability p is tossed repeatedly util the first Head appears. What is the
More informationt distribution [34] : used to test a mean against an hypothesized value (H 0 : µ = µ 0 ) or the difference
EXST30 Backgroud material Page From the textbook The Statistical Sleuth Mea [0]: I your text the word mea deotes a populatio mea (µ) while the work average deotes a sample average ( ). Variace [0]: The
More informationEfficient GMM LECTURE 12 GMM II
DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationFrequentist Inference
Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for
More informationBinomial Distribution
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible
More informationDS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10
DS 00: Priciples ad Techiques of Data Sciece Date: April 3, 208 Name: Hypothesis Testig Discussio #0. Defie these terms below as they relate to hypothesis testig. a) Data Geeratio Model: Solutio: A set
More informationMath 152. Rumbos Fall Solutions to Review Problems for Exam #2. Number of Heads Frequency
Math 152. Rumbos Fall 2009 1 Solutios to Review Problems for Exam #2 1. I the book Experimetatio ad Measuremet, by W. J. Youde ad published by the by the Natioal Sciece Teachers Associatio i 1962, the
More informationSTAC51: Categorical data Analysis
STAC51: Categorical data Aalysis Mahida Samarakoo Jauary 28, 2016 Mahida Samarakoo STAC51: Categorical data Aalysis 1 / 35 Table of cotets Iferece for Proportios 1 Iferece for Proportios Mahida Samarakoo
More informationSection 9.2. Tests About a Population Proportion 12/17/2014. Carrying Out a Significance Test H A N T. Parameters & Hypothesis
Sectio 9.2 Tests About a Populatio Proportio P H A N T O M S Parameters Hypothesis Assess Coditios Name the Test Test Statistic (Calculate) Obtai P value Make a decisio State coclusio Sectio 9.2 Tests
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationChapter 22. Comparing Two Proportions. Copyright 2010 Pearson Education, Inc.
Chapter 22 Comparig Two Proportios Copyright 2010 Pearso Educatio, Ic. Comparig Two Proportios Comparisos betwee two percetages are much more commo tha questios about isolated percetages. Ad they are more
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationIntroductory statistics
CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key
More informationKurskod: TAMS11 Provkod: TENB 21 March 2015, 14:00-18:00. English Version (no Swedish Version)
Kurskod: TAMS Provkod: TENB 2 March 205, 4:00-8:00 Examier: Xiagfeg Yag (Tel: 070 2234765). Please aswer i ENGLISH if you ca. a. You are allowed to use: a calculator; formel -och tabellsamlig i matematisk
More informationMATH/STAT 352: Lecture 15
MATH/STAT 352: Lecture 15 Sectios 5.2 ad 5.3. Large sample CI for a proportio ad small sample CI for a mea. 1 5.2: Cofidece Iterval for a Proportio Estimatig proportio of successes i a biomial experimet
More informationLecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
More informationSimulation. Two Rule For Inverting A Distribution Function
Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump
More informationFACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures
FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationProperties and Hypothesis Testing
Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.
More informationTests of Hypotheses Based on a Single Sample (Devore Chapter Eight)
Tests of Hypotheses Based o a Sigle Sample Devore Chapter Eight MATH-252-01: Probability ad Statistics II Sprig 2018 Cotets 1 Hypothesis Tests illustrated with z-tests 1 1.1 Overview of Hypothesis Testig..........
More informationBIOS 4110: Introduction to Biostatistics. Breheny. Lab #9
BIOS 4110: Itroductio to Biostatistics Brehey Lab #9 The Cetral Limit Theorem is very importat i the realm of statistics, ad today's lab will explore the applicatio of it i both categorical ad cotiuous
More informationSTATISTICAL INFERENCE
STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample
More informationLecture 6 Simple alternatives and the Neyman-Pearson lemma
STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull
More informationA statistical method to determine sample size to estimate characteristic value of soil parameters
A statistical method to determie sample size to estimate characteristic value of soil parameters Y. Hojo, B. Setiawa 2 ad M. Suzuki 3 Abstract Sample size is a importat factor to be cosidered i determiig
More information11 Correlation and Regression
11 Correlatio Regressio 11.1 Multivariate Data Ofte we look at data where several variables are recorded for the same idividuals or samplig uits. For example, at a coastal weather statio, we might record
More informationLecture Notes 15 Hypothesis Testing (Chapter 10)
1 Itroductio Lecture Notes 15 Hypothesis Testig Chapter 10) Let X 1,..., X p θ x). Suppose we we wat to kow if θ = θ 0 or ot, where θ 0 is a specific value of θ. For example, if we are flippig a coi, we
More informationTAMS24: Notations and Formulas
TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =
More informationReview Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn
Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom
More informationTable 12.1: Contingency table. Feature b. 1 N 11 N 12 N 1b 2 N 21 N 22 N 2b. ... a N a1 N a2 N ab
Sectio 12 Tests of idepedece ad homogeeity I this lecture we will cosider a situatio whe our observatios are classified by two differet features ad we would like to test if these features are idepedet
More informationStat 200 -Testing Summary Page 1
Stat 00 -Testig Summary Page 1 Mathematicias are like Frechme; whatever you say to them, they traslate it ito their ow laguage ad forthwith it is somethig etirely differet Goethe 1 Large Sample Cofidece
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More informationThis is an introductory course in Analysis of Variance and Design of Experiments.
1 Notes for M 384E, Wedesday, Jauary 21, 2009 (Please ote: I will ot pass out hard-copy class otes i future classes. If there are writte class otes, they will be posted o the web by the ight before class
More informationModule 1 Fundamentals in statistics
Normal Distributio Repeated observatios that differ because of experimetal error ofte vary about some cetral value i a roughly symmetrical distributio i which small deviatios occur much more frequetly
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationEconomics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator
Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters
More informationChapter 8: Estimating with Confidence
Chapter 8: Estimatig with Cofidece Sectio 8.2 The Practice of Statistics, 4 th editio For AP* STARNES, YATES, MOORE Chapter 8 Estimatig with Cofidece 8.1 Cofidece Itervals: The Basics 8.2 8.3 Estimatig
More informationLECTURE 14 NOTES. A sequence of α-level tests {ϕ n (x)} is consistent if
LECTURE 14 NOTES 1. Asymptotic power of tests. Defiitio 1.1. A sequece of -level tests {ϕ x)} is cosistet if β θ) := E θ [ ϕ x) ] 1 as, for ay θ Θ 1. Just like cosistecy of a sequece of estimators, Defiitio
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationProbability and statistics: basic terms
Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample
More information2 1. The r.s., of size n2, from population 2 will be. 2 and 2. 2) The two populations are independent. This implies that all of the n1 n2
Chapter 8 Comparig Two Treatmets Iferece about Two Populatio Meas We wat to compare the meas of two populatios to see whether they differ. There are two situatios to cosider, as show i the followig examples:
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationSection 14. Simple linear regression.
Sectio 14 Simple liear regressio. Let us look at the cigarette dataset from [1] (available to dowload from joural s website) ad []. The cigarette dataset cotais measuremets of tar, icotie, weight ad carbo
More informationRecall the study where we estimated the difference between mean systolic blood pressure levels of users of oral contraceptives and non-users, x - y.
Testig Statistical Hypotheses Recall the study where we estimated the differece betwee mea systolic blood pressure levels of users of oral cotraceptives ad o-users, x - y. Such studies are sometimes viewed
More informationStat 421-SP2012 Interval Estimation Section
Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible
More informationChapter 11: Asking and Answering Questions About the Difference of Two Proportions
Chapter 11: Askig ad Aswerig Questios About the Differece of Two Proportios These otes reflect material from our text, Statistics, Learig from Data, First Editio, by Roxy Peck, published by CENGAGE Learig,
More informationChapter 6 Sampling Distributions
Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationMaximum Likelihood Estimation
Chapter 9 Maximum Likelihood Estimatio 9.1 The Likelihood Fuctio The maximum likelihood estimator is the most widely used estimatio method. This chapter discusses the most importat cocepts behid maximum
More informationBayesian Methods: Introduction to Multi-parameter Models
Bayesia Methods: Itroductio to Multi-parameter Models Parameter: θ = ( θ, θ) Give Likelihood p(y θ) ad prior p(θ ), the posterior p proportioal to p(y θ) x p(θ ) Margial posterior ( θ, θ y) is Iterested
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More information11 THE GMM ESTIMATION
Cotets THE GMM ESTIMATION 2. Cosistecy ad Asymptotic Normality..................... 3.2 Regularity Coditios ad Idetificatio..................... 4.3 The GMM Iterpretatio of the OLS Estimatio.................
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More information1 Approximating Integrals using Taylor Polynomials
Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationLecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett
Lecture Note 8 Poit Estimators ad Poit Estimatio Methods MIT 14.30 Sprig 2006 Herma Beett Give a parameter with ukow value, the goal of poit estimatio is to use a sample to compute a umber that represets
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More informationInverse Matrix. A meaning that matrix B is an inverse of matrix A.
Iverse Matrix Two square matrices A ad B of dimesios are called iverses to oe aother if the followig holds, AB BA I (11) The otio is dual but we ofte write 1 B A meaig that matrix B is a iverse of matrix
More informationGoodness-Of-Fit For The Generalized Exponential Distribution. Abstract
Goodess-Of-Fit For The Geeralized Expoetial Distributio By Amal S. Hassa stitute of Statistical Studies & Research Cairo Uiversity Abstract Recetly a ew distributio called geeralized expoetial or expoetiated
More informationST 305: Exam 3 ( ) = P(A)P(B A) ( ) = P(A) + P(B) ( ) = 1 P( A) ( ) = P(A) P(B) σ X 2 = σ a+bx. σ ˆp. σ X +Y. σ X Y. σ X. σ Y. σ n.
ST 305: Exam 3 By hadig i this completed exam, I state that I have either give or received assistace from aother perso durig the exam period. I have used o resources other tha the exam itself ad the basic
More informationRandom Matrices with Blocks of Intermediate Scale Strongly Correlated Band Matrices
Radom Matrices with Blocks of Itermediate Scale Strogly Correlated Bad Matrices Jiayi Tog Advisor: Dr. Todd Kemp May 30, 07 Departmet of Mathematics Uiversity of Califoria, Sa Diego Cotets Itroductio Notatio
More informationConfidence Intervals for the Population Proportion p
Cofidece Itervals for the Populatio Proportio p The cocept of cofidece itervals for the populatio proportio p is the same as the oe for, the samplig distributio of the mea, x. The structure is idetical:
More information