Theorem. Assume the following (Cramér) conditions 1. θ θ
|
|
- Dustin Simpson
- 5 years ago
- Views:
Transcription
1 ML test i a slightly differet form ML Testig (Likelihood Ratio Testig) for o-gaussia models Surya Tokdar Model X f (x θ), θ Θ. Hypothesist H 0 : θ Θ 0 Good set: B c (x) ={θ : l x (θ) max θ Θ l x (θ) c2 2 } ML test = reject H 0 if B c (x) Θ 0 = Same as: reject H 0 if max l x(θ) max l x (θ) c 2 /2 θ Θ θ Θ 0 Or equivaletly: reject H 0 if Λ(x) = max θ Θ L x (θ) max θ Θ0 L x (θ) k 1/30 Size calculatio Asymptotic ormality of MLE Need to calculate max θ Θ0 P [X θ] (Λ(X ) > k) IID Will work for models of the form: X i g(x i θ), θ Θ Ad hypothesis: H 0 : A T θ = a 0 for a give p r matrix A ad give scalar a 0 (typically a 0 = 0) Uder regularity coditios: 2 log Λ(X ) d χ 2 (r) whe IID X i g(x i θ 0 )withθ 0 Θ 0. Follows from asymptotic ormality of ˆθ MLE (X ) Theorem. Assume the followig (Cramér) coditios 1. θ θ log g(x θ) is twice cotiuously differetiable. 2. θ g( θ) isidetifiable θ 3 log g(x i θ) < h(x), θ bhd(θ 0 ), E [X θ0]h(x 1 ) < 4. ˆθ MLE (x) is the uique solutio of l x (θ) =0foreveryx. IID If X i g(x i θ 0 )the (ˆθ MLE θ 0 ) d N (0, {I F 1 (θ 0 )} 1) where I F 1 (θ) = E [X i θ] 2 θ 2 log g(x 1 θ) = Fisher iformatio at θ. 2/30 3/30 A sketch of a proof Three large approximatios to Λ(x) First order Taylor approximatio: 0= 1 lx (ˆθ MLE ) 1 lx (θ 0 )+ l X (θ 0 ) T Write T i = θ log g(x i θ). ETi = 0, VarT i = I F 1 (θ 0) 1 l X (θ 0 )= ( T 0) d N(0, I F 1 (θ 0)) [by CLT] Write W i = 2 log g(x θ 2 i θ) atθ = θ 0 EWi = I1 F (θ 0) lx (θ0) = W p I1 F (θ 0) Rest is Slutksy s theorem (ˆθ MLE θ 0 ) Assume θ 0 Θ 0,i.e.,A T θ 0 = a 0 Notatio: ˆθ H0 = argmax θ Θ0 l x (θ) (must have A T ˆθ H0 = a 0 ) Quadratic, Wald, Rao approximatios F (X )=(A T ˆθ MLE a 0) T {A T Ix 1 A} 1 (A T ˆθ MLE a 0) 2logΛ(X ) W (X )=(A T ˆθ MLE a 0) T {A T I1 F (ˆθ MLE) 1 A} 1 (A T ˆθ MLE a 0) S(X )= 1 l X (ˆθ H0 ) T {I1 F (ˆθ H0 )} 1 l X (ˆθ H0 ) 2 log Λ(X ) F (X ) p 0, etc. Each of 2 log Λ(X ), F (X ), W (X ), S(X ) d χ 2 (r) 4/30 5/30
2 Sketches of proof if you cared - part I Sketches of proof if you cared - part II 2 log Λ(X ) F (X ) follows from Quadratic approximatio: l x (θ) l x (ˆθ MLE ) 1 2 (θ ˆθ MLE ) T I X (θ ˆθ MLE ) ad the correspodig profile log-likelihood of η = A T θ: l x(η) l x (ˆθ MLE ) 1 2 (η AT ˆθ MLE ) T {A T I 1 X A} 1 (η A T ˆθ MLE ) ad by otig Λ(x) =l x(a T ˆθ H0 ) l x(a T ˆθ MLE ) 2 log Λ(X ) S(X ) follows because 0= l x (ˆθ MLE ) l x (ˆθ H0 )+ l x (ˆθ H0 ) T (ˆθ MLE ˆθ H0 ) l x (ˆθ H0 ) I F 1 (ˆθ H0 ) Ad so ˆθ MLE ˆθ H0 I F 1 (ˆθ H0 ) 1 l x (ˆθ H0 ) Plug this ito the origial quadratic approximatio ad use I X I F 1 (θ 0) I F 1 (ˆθ H0 )becauseˆθ MLE θ 0 2 log Λ(X ) W (X ) follows similarly, use I X I F 1 (ˆθ MLE ). 6/30 7/30 Sketches of proof if you cared - part III Back to size calculatio It is easiest to prove W (X ) d χ 2 (r) By asymptotic ormality of MLE ad Slutsky s theorem W (X ) d Z T {A T I1 F (θ 0 ) 1 A} 1 Z where Z N r (0, {A T I1 F (θ 0) 1 A}) But Z N r (0, Σ) measz T Σ 1 Z χ 2 (r). Recall: Each of 2 log Λ(X ), F (X ), W (X ), S(X ) d χ 2 (r) So each of the followig test procedures has size α ML/LRT: Reject H0 if 2 log Λ(x) > q χ 2(1 α, r) Quadratic: Reject H 0 if F (x) > q χ 2(1 α, r) Wald: Reject H0 if W (x) > q χ 2(1 α, r) Rao: Reject H 0 if S(x) > q χ 2(1 α, r) Here q χ 2(u, r) =F 1 χ 2 (u, r) wheref χ 2(x, r) = CDF of χ 2 (r) Also, oe ca calculate p-values as 1 F χ 2(2 log Λ(x), r) etc. 8/30 9/30 Which oe to use? Example: low birthweight Depeds o which is easier to compute To get W (x) you do t eed ˆθ H0, oly eed ˆθ MLE. To get S(x) you oly eed ˆθ H0, do t eed ˆθ MLE. I may cases F (X )=W (X )becausei X = I F 1 (ˆθ MLE ) This happes i most geeralized liear models: IND Y i g(y i β, z i )=h(z i, y i )e ξ(β)y i zi T β A(β,z i ) ad Wald s tests are popular choices Multiomial model of category couts, S(X ) = Pearso s-χ 2 Natality data = 500 records o US births i Jue Y i =1ifi-th birth record has birthweight < 2500g, Y i =0 otherwise. z i = (Cigarettes i, Black i ) records daily umber of cigarettes ad the race of the mother. Model P(Y i = 1) log 1 P(Y i = 1) = β 1 + β 2 Cigarettes i + β 3 Black i + β 4 Cigarettes i Black i. Ca compute ˆβ MLE ad I X (umerically). 10 / / 30
3 MLE ad curvature From glm() fuctio i R ˆβ MLE = I 1 X = How would you test the ull hypothesis that for a black mother, probability of low birthweight depeds o the umber of cigarettes? For a o-black mother? How would you test that this depedece is differet for black ad o-black mothers? Categorical data & Multiomial models Data: Couts of categories formed by oe or more attributes Tables could be oe-way, two-way, etc., depedig o how may attributes are used to decide categories. Hair color Eye color Blue Gree Brow Black Total Blode Red Brow Black Total / / 30 Multiomial model Multiomial model (cotd.) Data are k category couts of objects X =(X 1,, X k ), X j 0, X 1 + X X k =. Model X Multiomial(, p) wherep =(p 1,, p k ) k k is k-dim simplex, cotais k-dim prob vectors b =(b 1,, b k ), i.e., b j 0 j ad j b j = 1) Multiomial(, p) haspmf ( ) f (x p) = p x1 1 x 1 x px k k k This is a extesio of the biomial distributios. I X we record the couts from idepedet trials with k outcomes with probabilities give by p. Cosequetly X j Bi(, p j ) for ay sigle category 1 j k. Similarly for two categories j 1 j 2, X j1 + X j2 Bi(, p j1 + p j2 ), ad so o. for x =(x 1,, x k )withx i 0ad i x i =, f (x p) =0 otherwise. 14 / / 30 Multiomial model (cotd.) MLE For two-way tables, with k 1 rows ad k 2 colums, we ca write X ad p either as k 1 k 2 -dim vectors, or more commoly as k 1 k 2 matrices. Eve whe they re writte as matrices, we ll write X Multiomial(, p) to mea that the multiomial distributio is placed o the vector forms of X ad p. For multi-way table we ll use arrays of appropriate dimesios to represet X ad p. To obtai the MLE of p based o data X = x, weca maximize the log-likelihood fuctio i p with a additioal Lagrage compoet to accout for the costrait k p j = 1: l x (p, λ) = cost + k k x j log p j + λ( p j 1) The solutio i p equals: ( x1 ˆp MLE =,, x ) k 16 / / 30
4 Hypothesis testig: Medel s peas Formalizatio Medel, the fouder of moder geetics, studied how physical characteristics are iherited i plats. His studies led him to propose the laws of segregatio ad idepedet assortmet. We ll test this i a simple cotext. Uder Medel s laws, whe pure roud-yellow ad pure gree-wrikled pea plats are cross-bred, the ext geeratio of plat seeds should exhibit a 9:3:3:1 ratio of roud-yellow, roud-gree, wrikled-yellow ad wrikled-gree combiatios of shape ad color. I a sample of 556 plats from the ext geeratio the observed couts for these combiatios are (315, 108, 101, 32). Does the data support Medel s laws? Data X =(X 1, X 2, X 3, X 4 ) givig the category couts of the four types of plats Model X Multiomial( = 556, p), p 4 Test H 0 : p =( 9 16, 3 16, 3 16, 1 16 ) 18 / / 30 Hardy-Weiberg equilibrium Formalizatio The spottig o the wigs of Scarlet tiger moths are cotrolled by a gee that comes i two varieties (alleles) whose combiatios (moths have pairs of chromosomes) produce three varieties of spottig patter: white spotted, little spotted ad itermediate. If the moth populatio is i Hardy-Weiberg equilibrium (o curret selectio drift), the these varieties should be i the ratio a 2 :(1 a) 2 :2a(1 a), where a (0, 1) deotes the abudace of the domiat white spottig allele. I a sample of 1612 moths, the three varieties were couted to be 1469, 5 ad 138. Is the moth populatio i HW equilibrium? Data X =(X 1, X 2, X 3 ) the category couts of the three spottig patters, model X Multiomial( = 1612, p), p 3, Test whether H 0 : p HW 3, HW 3 is a subset of 3 cotaiig all vectors of the form (a 2, (1 a) 2, 2a(1 a)) for some a (0, 1). 20 / / 30 Hair ad eye color ML testig Are hair color ad eye color two idepedet attributes? I this case, writig p i the matrix form p =((p ij )), 1 i k 1 ad 1 j k 2,wewattotestifp ij = p (1) i p (2) j for some p (1) k1 ad p (2) k2. It s elemetary that if p ij factors as above, the p (1) i = k 2 p ij ad p (2) j = k 1 i=1 p ij. Writig the row ad colum totals as p i ad p j, the test of idepedece is ofte represeted by H 0 : p ij = p i p j, i, j dim( k )=k 1, as vector p k satisfy k i=1 p i = 1. Hypotheses H 0 : p Θ 0, Θ 0 is a sub-simplex of k of dimesio q < k. Medel s peas: Θ0 =( 9 16, 3 16, 3 ), q =0 16, 1 16 HW equilibrium: Θ0 = HW 3, q =1 Idepedece: Θ 0 = I k1,k2, q = k 1 + k 2 2. Could use ay of ML, quadratic, Wald or Rao tests We ll use I k 1,k 2 to deote this set. 22 / / 30
5 Pearso s χ 2 tests Example 1: poit ull Rao s test statistics boils dow to a very coveiet form: S(X )= k (X j ˆp H0,j) 2 ˆp H0,j = (Observed Expected) 2 Expected For categorical data, this was earlier discovered by Pearso who also ascertaied its approximate χ 2 distributio. Because this is Rao statistic, we have S(X ) d χ 2 (k 1 q) For testig H 0 : p = p 0 agaist p p 0,where p 0 =(p 0,1,, p 0,k ) k is a fixed probability vector of iterest (as i Medel s peas example), the S(X )= k (X j p 0,j ) 2 p 0,j which is asymptotically χ 2 (k 1 0) = χ 2 (k 1). Size-α Pearso s test rejects H 0 if S(X ) > q χ 2(1 α, k 1). 24 / / 30 Example 2: parametric form Example 2: parametric form (cotd.) HW test: H 0 : p 0 =(η 2, η(1 η), η(1 η), (1 η) 2 ), for some 0 < η < 1 To compute ˆp H0 it is equivalet to write the likelihood fuctio i η ad maximize: L X (η) = cost. {η 2 } x1 {η(1 η)} x2 {η(1 η)} x3 {(1 η) 2 } x4 = cost. η 2x1+x2+x3 (1 η) x2+x3+2x4 ad so ˆη MLE = 2x1+x2+x3 2. Oce we have ˆη MLE, we ca costruct ˆp H0 =(ˆη 2 MLE, ˆη MLE (1 ˆη MLE ), ˆη MLE (1 ˆη MLE ), (1 ˆη MLE ) 2 )ad evaluate S(X ). Because Θ 0 has dimesio q = 1 (oly a sigle umber η eeds to be kow), the asymptotic distributio of S(X ) is χ 2 (k 2). The same calculatios carry through for a more geeral parametric form: H 0 : p =(g 1 (η),, g k (η)) where η E is q-dim vector ad g 1 (η),, g k (η) are fuctios such that for every η E, (g 1 (η),, g k (η)) k. 26 / / 30 Example 3: idepedece Example 3: idepedece (cotd.) Cosider testig H 0 : p ij = p i p j, i, j To get ˆp H0, we write the likelihood fuctio i terms of p i, 1 i k 1 ad p j,1 j k 2 : k 1 k 2 L x (p 1,, p k1, p 1,, p k2 ) = cost. (p i p j ) x ij i=1 { k1 = cost. i=1 p x i i }{ k2 where x i = k 2 x ij ad x j = k 1 i=1 x ij are the margi couts of our two-way table. p x j j } Because (p 1,, p k1 ) k1 ad (p 1,, p k2 ) k2,the maximizer is give by: Ad so ˆp H0 ˆp i = x i, ˆp j = x j, 1 i k 1, 1 j k 2 has coordiates: ˆp H0,ij = x i x j 2 k 1 k 2 S(X )= i=1 givig (X ij X i X j ) 2 X i X j Because dim(θ 0 )=q = k 1 1+k 2 1, S(X ) d χ 2 (k 1 k 2 1 k 1 +1 k 2 + 1) = χ 2 ((k 1 1)(k 2 1)). 28 / / 30
6 Hair-eye color Hair color Eye color Blue Gree Brow Black Total Blode 20 (13.9) 15 (13.6) 18 (28.4) 14 (11.0) 67 Red 11(8.5) 4 (8.3) 24 (17.4) 2 (6.7) 41 Brow 9 (15.4) 11 (15.1) 36 (31.4) 18 (12.2) 74 Black 8 (10.2) 17 (10.0) 20 (20.8) 4 (8.1) 49 Total S(x) = So the p-value is 1 F (4 1)(4 1) (30.9) = 1 F 9 (30.9) / 30
ML Testing (Likelihood Ratio Testing) for non-gaussian models
ML Testing (Likelihood Ratio Testing) for non-gaussian models Surya Tokdar ML test in a slightly different form Model X f (x θ), θ Θ. Hypothesist H 0 : θ Θ 0 Good set: B c (x) = {θ : l x (θ) max θ Θ l
More informationGoodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen)
Goodess-of-Fit Tests ad Categorical Data Aalysis (Devore Chapter Fourtee) MATH-252-01: Probability ad Statistics II Sprig 2019 Cotets 1 Chi-Squared Tests with Kow Probabilities 1 1.1 Chi-Squared Testig................
More informationStatistics 3858 : Likelihood Ratio for Multinomial Models
Statistics 3858 : Likelihood Ratio for Multiomial Models Suppose X is multiomial o M categories, that is X Multiomial, p), where p p 1, p 2,..., p M ) A, ad the parameter space is A {p : p j 0, p j 1 }
More information( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2
82 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Defiitio: et X be a radom sample with joit p.m/d.f. f X x θ. The geeralised likelihood ratio test g.l.r.t. of the NH : θ H 0 agaist the alterative AH : θ H 1,
More informationLecture Notes 15 Hypothesis Testing (Chapter 10)
1 Itroductio Lecture Notes 15 Hypothesis Testig Chapter 10) Let X 1,..., X p θ x). Suppose we we wat to kow if θ = θ 0 or ot, where θ 0 is a specific value of θ. For example, if we are flippig a coi, we
More informationMultinomial likelihood. Multinomial MLE. NIST data and genetic fingerprints. θ = (p 1,..., p m ) with j p j = 1 and p j 0. Point probabilities
Multiomial distributio Let Y,..., Y be iid, uiformly sampled from a fiite populatio ad X i deotes a property of the idividual i. Label the properties,..., m. p j = PX i = j) = umber of idividuals with
More informationIntroductory statistics
CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key
More informationLast Lecture. Wald Test
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig
More informationChi-Squared Tests Math 6070, Spring 2006
Chi-Squared Tests Math 6070, Sprig 2006 Davar Khoshevisa Uiversity of Utah February XXX, 2006 Cotets MLE for Goodess-of Fit 2 2 The Multiomial Distributio 3 3 Applicatio to Goodess-of-Fit 6 3 Testig for
More information6. Sufficient, Complete, and Ancillary Statistics
Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary
More informationTAMS24: Notations and Formulas
TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =
More informationt distribution [34] : used to test a mean against an hypothesized value (H 0 : µ = µ 0 ) or the difference
EXST30 Backgroud material Page From the textbook The Statistical Sleuth Mea [0]: I your text the word mea deotes a populatio mea (µ) while the work average deotes a sample average ( ). Variace [0]: The
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationTable 12.1: Contingency table. Feature b. 1 N 11 N 12 N 1b 2 N 21 N 22 N 2b. ... a N a1 N a2 N ab
Sectio 12 Tests of idepedece ad homogeeity I this lecture we will cosider a situatio whe our observatios are classified by two differet features ad we would like to test if these features are idepedet
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More informationSummary. Recap ... Last Lecture. Summary. Theorem
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationLECTURE 14 NOTES. A sequence of α-level tests {ϕ n (x)} is consistent if
LECTURE 14 NOTES 1. Asymptotic power of tests. Defiitio 1.1. A sequece of -level tests {ϕ x)} is cosistet if β θ) := E θ [ ϕ x) ] 1 as, for ay θ Θ 1. Just like cosistecy of a sequece of estimators, Defiitio
More informationEfficient GMM LECTURE 12 GMM II
DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet
More informationStatistical inference: example 1. Inferential Statistics
Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationAsymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values
of the secod half Biostatistics 6 - Statistical Iferece Lecture 6 Fial Exam & Practice Problems for the Fial Hyu Mi Kag Apil 3rd, 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Rao-Blackwell
More informationA quick activity - Central Limit Theorem and Proportions. Lecture 21: Testing Proportions. Results from the GSS. Statistics and the General Population
A quick activity - Cetral Limit Theorem ad Proportios Lecture 21: Testig Proportios Statistics 10 Coli Rudel Flip a coi 30 times this is goig to get loud! Record the umber of heads you obtaied ad calculate
More information5. Likelihood Ratio Tests
1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationStatistical and Mathematical Methods DS-GA 1002 December 8, Sample Final Problems Solutions
Statistical ad Mathematical Methods DS-GA 00 December 8, 05. Short questios Sample Fial Problems Solutios a. Ax b has a solutio if b is i the rage of A. The dimesio of the rage of A is because A has liearly-idepedet
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationLecture 12: November 13, 2018
Mathematical Toolkit Autum 2018 Lecturer: Madhur Tulsiai Lecture 12: November 13, 2018 1 Radomized polyomial idetity testig We will use our kowledge of coditioal probability to prove the followig lemma,
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationProperties and Hypothesis Testing
Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.
More informationKurskod: TAMS11 Provkod: TENB 21 March 2015, 14:00-18:00. English Version (no Swedish Version)
Kurskod: TAMS Provkod: TENB 2 March 205, 4:00-8:00 Examier: Xiagfeg Yag (Tel: 070 2234765). Please aswer i ENGLISH if you ca. a. You are allowed to use: a calculator; formel -och tabellsamlig i matematisk
More information4. Hypothesis testing (Hotelling s T 2 -statistic)
4. Hypothesis testig (Hotellig s T -statistic) Cosider the test of hypothesis H 0 : = 0 H A = 6= 0 4. The Uio-Itersectio Priciple W accept the hypothesis H 0 as valid if ad oly if H 0 (a) : a T = a T 0
More informationLecture 23: Minimal sufficiency
Lecture 23: Miimal sufficiecy Maximal reductio without loss of iformatio There are may sufficiet statistics for a give problem. I fact, X (the whole data set) is sufficiet. If T is a sufficiet statistic
More informationChimica Inorganica 3
himica Iorgaica Irreducible Represetatios ad haracter Tables Rather tha usig geometrical operatios, it is ofte much more coveiet to employ a ew set of group elemets which are matrices ad to make the rule
More informationSTAC51: Categorical data Analysis
STAC51: Categorical data Aalysis Mahida Samarakoo Jauary 28, 2016 Mahida Samarakoo STAC51: Categorical data Aalysis 1 / 35 Table of cotets Iferece for Proportios 1 Iferece for Proportios Mahida Samarakoo
More information11 THE GMM ESTIMATION
Cotets THE GMM ESTIMATION 2. Cosistecy ad Asymptotic Normality..................... 3.2 Regularity Coditios ad Idetificatio..................... 4.3 The GMM Iterpretatio of the OLS Estimatio.................
More informationFACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures
FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals
More information1 Models for Matched Pairs
1 Models for Matched Pairs Matched pairs occur whe we aalyse samples such that for each measuremet i oe of the samples there is a measuremet i the other sample that directly relates to the measuremet i
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationMATH/STAT 352: Lecture 15
MATH/STAT 352: Lecture 15 Sectios 5.2 ad 5.3. Large sample CI for a proportio ad small sample CI for a mea. 1 5.2: Cofidece Iterval for a Proportio Estimatig proportio of successes i a biomial experimet
More informationStatistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
More informationStat410 Probability and Statistics II (F16)
Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationLecture 6 Simple alternatives and the Neyman-Pearson lemma
STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull
More informationIt is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.
MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied
More informationFinal Examination Statistics 200C. T. Ferguson June 10, 2010
Fial Examiatio Statistics 00C T. Ferguso Jue 0, 00. (a State the Borel-Catelli Lemma ad its coverse. (b Let X,X,... be i.i.d. from a distributio with desity, f(x =θx (θ+ o the iterval (,. For what value
More informationEconomics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator
Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters
More informationRandom Matrices with Blocks of Intermediate Scale Strongly Correlated Band Matrices
Radom Matrices with Blocks of Itermediate Scale Strogly Correlated Bad Matrices Jiayi Tog Advisor: Dr. Todd Kemp May 30, 07 Departmet of Mathematics Uiversity of Califoria, Sa Diego Cotets Itroductio Notatio
More informationSTA Learning Objectives. Population Proportions. Module 10 Comparing Two Proportions. Upon completing this module, you should be able to:
STA 2023 Module 10 Comparig Two Proportios Learig Objectives Upo completig this module, you should be able to: 1. Perform large-sample ifereces (hypothesis test ad cofidece itervals) to compare two populatio
More informationStatistical Inference Based on Extremum Estimators
T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0
More informationB Supplemental Notes 2 Hypergeometric, Binomial, Poisson and Multinomial Random Variables and Borel Sets
B671-672 Supplemetal otes 2 Hypergeometric, Biomial, Poisso ad Multiomial Radom Variables ad Borel Sets 1 Biomial Approximatio to the Hypergeometric Recall that the Hypergeometric istributio is fx = x
More informationDS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10
DS 00: Priciples ad Techiques of Data Sciece Date: April 3, 208 Name: Hypothesis Testig Discussio #0. Defie these terms below as they relate to hypothesis testig. a) Data Geeratio Model: Solutio: A set
More informationStatistical Hypothesis Testing. STAT 536: Genetic Statistics. Statistical Hypothesis Testing - Terminology. Hardy-Weinberg Disequilibrium
Statistical Hypothesis Testig STAT 536: Geetic Statistics Kari S. Dorma Departmet of Statistics Iowa State Uiversity September 7, 006 Idetify a hypothesis, a idea you wat to test for its applicability
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationStat 319 Theory of Statistics (2) Exercises
Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.
More information10-701/ Machine Learning Mid-term Exam Solution
0-70/5-78 Machie Learig Mid-term Exam Solutio Your Name: Your Adrew ID: True or False (Give oe setece explaatio) (20%). (F) For a cotiuous radom variable x ad its probability distributio fuctio p(x), it
More informationSTATISTICAL INFERENCE
STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample
More informationAgreement of CI and HT. Lecture 13 - Tests of Proportions. Example - Waiting Times
Sigificace level vs. cofidece level Agreemet of CI ad HT Lecture 13 - Tests of Proportios Sta102 / BME102 Coli Rudel October 15, 2014 Cofidece itervals ad hypothesis tests (almost) always agree, as log
More informationChapter 22. Comparing Two Proportions. Copyright 2010 Pearson Education, Inc.
Chapter 22 Comparig Two Proportios Copyright 2010 Pearso Educatio, Ic. Comparig Two Proportios Comparisos betwee two percetages are much more commo tha questios about isolated percetages. Ad they are more
More informationFrequentist Inference
Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for
More informationChapter 22. Comparing Two Proportions. Copyright 2010, 2007, 2004 Pearson Education, Inc.
Chapter 22 Comparig Two Proportios Copyright 2010, 2007, 2004 Pearso Educatio, Ic. Comparig Two Proportios Read the first two paragraphs of pg 504. Comparisos betwee two percetages are much more commo
More informationLecture 24: Variable selection in linear models
Lecture 24: Variable selectio i liear models Cosider liear model X = Z β + ε, β R p ad Varε = σ 2 I. Like the LSE, the ridge regressio estimator does ot give 0 estimate to a compoet of β eve if that compoet
More information6 Sample Size Calculations
6 Sample Size Calculatios Oe of the major resposibilities of a cliical trial statisticia is to aid the ivestigators i determiig the sample size required to coduct a study The most commo procedure for determiig
More informationSupport vector machine revisited
6.867 Machie learig, lecture 8 (Jaakkola) 1 Lecture topics: Support vector machie ad kerels Kerel optimizatio, selectio Support vector machie revisited Our task here is to first tur the support vector
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationRank tests and regression rank scores tests in measurement error models
Rak tests ad regressio rak scores tests i measuremet error models J. Jurečková ad A.K.Md.E. Saleh Charles Uiversity i Prague ad Carleto Uiversity i Ottawa Abstract The rak ad regressio rak score tests
More informationStatistics 511 Additional Materials
Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationChapter 13: Tests of Hypothesis Section 13.1 Introduction
Chapter 13: Tests of Hypothesis Sectio 13.1 Itroductio RECAP: Chapter 1 discussed the Likelihood Ratio Method as a geeral approach to fid good test procedures. Testig for the Normal Mea Example, discussed
More informationTMA4245 Statistics. Corrected 30 May and 4 June Norwegian University of Science and Technology Department of Mathematical Sciences.
Norwegia Uiversity of Sciece ad Techology Departmet of Mathematical Scieces Corrected 3 May ad 4 Jue Solutios TMA445 Statistics Saturday 6 May 9: 3: Problem Sow desity a The probability is.9.5 6x x dx
More information1 Approximating Integrals using Taylor Polynomials
Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationChapter Vectors
Chapter 4. Vectors fter readig this chapter you should be able to:. defie a vector. add ad subtract vectors. fid liear combiatios of vectors ad their relatioship to a set of equatios 4. explai what it
More informationPower and Type II Error
Statistical Methods I (EXST 7005) Page 57 Power ad Type II Error Sice we do't actually kow the value of the true mea (or we would't be hypothesizig somethig else), we caot kow i practice the type II error
More informationStatistical Theory MT 2008 Problems 1: Solution sketches
Statistical Theory MT 008 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. a) Let 0 < θ < ad put fx, θ) = θ)θ x ; x = 0,,,... b) c) where α
More informationDirection: This test is worth 150 points. You are required to complete this test within 55 minutes.
Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem
More informationA statistical method to determine sample size to estimate characteristic value of soil parameters
A statistical method to determie sample size to estimate characteristic value of soil parameters Y. Hojo, B. Setiawa 2 ad M. Suzuki 3 Abstract Sample size is a importat factor to be cosidered i determiig
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More information11 Correlation and Regression
11 Correlatio Regressio 11.1 Multivariate Data Ofte we look at data where several variables are recorded for the same idividuals or samplig uits. For example, at a coastal weather statio, we might record
More informationKLMED8004 Medical statistics. Part I, autumn Estimation. We have previously learned: Population and sample. New questions
We have previously leared: KLMED8004 Medical statistics Part I, autum 00 How kow probability distributios (e.g. biomial distributio, ormal distributio) with kow populatio parameters (mea, variace) ca give
More informationIf, for instance, we were required to test whether the population mean μ could be equal to a certain value μ
STATISTICAL INFERENCE INTRODUCTION Statistical iferece is that brach of Statistics i which oe typically makes a statemet about a populatio based upo the results of a sample. I oesample testig, we essetially
More information(3) If you replace row i of A by its sum with a multiple of another row, then the determinant is unchanged! Expand across the i th row:
Math 5-4 Tue Feb 4 Cotiue with sectio 36 Determiats The effective way to compute determiats for larger-sized matrices without lots of zeroes is to ot use the defiitio, but rather to use the followig facts,
More informationLecture 3. Properties of Summary Statistics: Sampling Distribution
Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary
More informationTests of Hypotheses Based on a Single Sample (Devore Chapter Eight)
Tests of Hypotheses Based o a Sigle Sample Devore Chapter Eight MATH-252-01: Probability ad Statistics II Sprig 2018 Cotets 1 Hypothesis Tests illustrated with z-tests 1 1.1 Overview of Hypothesis Testig..........
More informationBinomial Distribution
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible
More informationTesting Statistical Hypotheses for Compare. Means with Vague Data
Iteratioal Mathematical Forum 5 o. 3 65-6 Testig Statistical Hypotheses for Compare Meas with Vague Data E. Baloui Jamkhaeh ad A. adi Ghara Departmet of Statistics Islamic Azad iversity Ghaemshahr Brach
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More information[412] A TEST FOR HOMOGENEITY OF THE MARGINAL DISTRIBUTIONS IN A TWO-WAY CLASSIFICATION
[412] A TEST FOR HOMOGENEITY OF THE MARGINAL DISTRIBUTIONS IN A TWO-WAY CLASSIFICATION BY ALAN STUART Divisio of Research Techiques, Lodo School of Ecoomics 1. INTRODUCTION There are several circumstaces
More informationMath 152. Rumbos Fall Solutions to Review Problems for Exam #2. Number of Heads Frequency
Math 152. Rumbos Fall 2009 1 Solutios to Review Problems for Exam #2 1. I the book Experimetatio ad Measuremet, by W. J. Youde ad published by the by the Natioal Sciece Teachers Associatio i 1962, the
More informationLinear Support Vector Machines
Liear Support Vector Machies David S. Roseberg The Support Vector Machie For a liear support vector machie (SVM), we use the hypothesis space of affie fuctios F = { f(x) = w T x + b w R d, b R } ad evaluate
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More information, then cv V. Differential Equations Elements of Lineaer Algebra Name: Consider the differential equation. and y2 cos( kx)
Cosider the differetial equatio y '' k y 0 has particular solutios y1 si( kx) ad y cos( kx) I geeral, ay liear combiatio of y1 ad y, cy 1 1 cy where c1, c is also a solutio to the equatio above The reaso
More informationStatistical Inference
Statistical Iferece Statistical Iferece Statistical iferece provides us with the process of testig hypotheses uder ivestigatio from data. Procedures of statistical iferece are give for i parameter estimates
More information