Statistical Inference with Monotone Incomplete Multivariate Normal Data
|
|
- Rosamond Booth
- 5 years ago
- Views:
Transcription
1 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 1/4 Statistical Inference with Monotone Incomplete Multivariate Normal Data This talk is based on joint work with my wonderful co-authors: Wan-Ying Chang (Dept. of Fish & Wildlife, WA, USA) Tomoya Yamada (Sapporo Gakuin University, Japan) Megan Romer (Penn State University)
2 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 2/4
3 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 3/4 Background We have a population of patients We draw a random sample of N patients, and measure m variables on each patient: 1 Visual acuity 2 LDL (low-density lipoprotein) cholesterol 3 Systolic blood pressure 4 Glucose intolerance 5 Insulin response to oral glucose 6 Actual weight Expected weight. m. White blood cell count
4 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 4/4 We obtain data: Patient N v 1,1 v 1,2. v 2,1 v 2,2. v 3,1 v 3,2. v N,1 v N,2. v 1,m v 2,m v 3,m v N,m Vector notation: V 1,V 2,...,V N V 1 : The measurements on patient 1, stacked into a column etc.
5 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 5/4 Classical multivariate analysis Statistical analysis of N m-dimensional data vectors Common assumption: The population has a multivariate normal distribution V : The vector of measurements on a randomly chosen patient Multivariate normal populations are characterized by: µ: The population mean vector Σ: The population covariance matrix For a given data set, µ and Σ are unknown
6 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 6/4 We wish to perform inference about µ and Σ Construct confidence regions for, and test hypotheses about, µ and Σ Anderson (2003). An Introduction to Multivariate Statistical Analysis, third edition Eaton (1984). Multivariate Statistics: A Vector-Space Approach Johnson and Wichern (2002). Applied Multivariate Statistical Analysis Muirhead (1982). Aspects of Multivariate Statistical Theory
7 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 7/4 Standard notation: V N p (µ,σ) The probability density function of V : For v R m, f(v) = (2π) m/2 Σ 1/2 exp ( 1 2 (v µ) Σ 1 (v µ) ) V 1,V 2,...,V N : Measurements on N randomly chosen patients Estimate µ and Σ using Fisher s maximum likelihood principle Likelihood function: L(µ,Σ) = N j=1 f(v j) Maximum likelihood estimator: The value of (µ, Σ) that maximizes L
8 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 8/4 µ = 1 N N j=1 V j: The sample mean and MLE of µ Σ = 1 N n j=1 (V j V )(V j V ) : The MLE of Σ What are the probability distributions of µ and Σ? µ N p (µ, 1 N Σ) LLN: As N, 1 N Σ 0 and hence µ µ, a.s. N Σ has a Wishart distribution, a generalization of the χ 2 µ and Σ also are mutually independent
9 Monotone incomplete data Some patients were not measured completely The resulting data set, with denoting a missing observation v 1,1 v 1,2 v 1,3. v 1,m v 2,2 v 2,3. v 2,m v 3,2. v 3,m. v N,m Monotone data: Each is followed by s only We may need to renumber patients to display the data in monotone form Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 9/4
10 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 10/4 Physical Fitness Data A well-known data set from a SAS manual on missing data Patients: Men taking a physical fitness course at NCSU Three variables were measured: Oxygen intake rate (ml. per kg. body weight per minute) RunTime (time taken, in minutes, to run 1.5 miles) RunPulse (heart rate while running)
11 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 11/4 Oxygen RunTime RunPulse * * * * * * * * * * *
12 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 12/4 Monotone data have a staircase pattern; we will consider the two-step pattern Partition V into an incomplete part of dimension p and a complete part of dimension q ( X 1 Y 1 ), ( X 2 Y 2 ),..., ( X n Y n ), ( Y n+1 ), ( Y n+2 ),..., ( Y N ) Assume that the individual vectors are independent and are drawn from N m (µ,σ) Goal: Maximum likelihood inference for µ and Σ, with analytical results as extensive and as explicit as in the classical setting
13 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 13/4 Where do monotone incomplete data arise? Panel survey data (Census Bureau, Bureau of Labor Statistics) Astronomy Early detection of diseases Wildlife survey research Covert communications Mental health research Climate and atmospheric studies.
14 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 14/4 We have n observations on ( X Y ) and N n additional observations on Y Difficulty: The likelihood function is more complicated L = = = n N f X,Y (x i,y i ) f Y (y i ) i=1 i=n+1 n f Y (y i )f X Y (x i ) i=1 N f Y (y i ) i=1 i=1 N i=n+1 n f X Y (x i ) f Y (y i )
15 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 15/4 Partition µ and Σ similarly: ( ) ( ) µ1 Σ11 Σ 12 µ =, Σ = µ 2 Σ 21 Σ 22 Let µ 1 2 = µ 1 Σ 12 Σ 1 22 (Y µ 2), Σ 11 2 = Σ 11 Σ 12 Σ 1 22 Σ 21 Y N q (µ 2,Σ 22 ), X Y N p (µ 1 2,Σ 11 2 ) µ and Σ: Wilks, Anderson, Morrison, Olkin, Jinadasa, Tracy,... Anderson and Olkin (1985): An elegant derivation of Σ
16 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 16/4 Sample means: X = 1 n X j, n j=1 Ȳ 2 = 1 N n N j=n+1 Y j, Ȳ 1 = 1 n Ȳ = 1 N n j=1 N j=1 Y j Y j Sample covariance matrices: n A 11 = (X j X)(X j X) n, A 12 = (X j X)(Y j Ȳ1) j=1 j=1 n A 22,n = (Y j Ȳ 1 )(Y j Ȳ 1 ), A 22,N = j=1 j=1 N (Y j Ȳ )(Y j Ȳ )
17 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 17/4 The MLE s of µ and Σ Notation: τ = n/n, τ = 1 τ µ 1 = X τa 12 A 1 22,n (Ȳ 1 Ȳ 2 ), µ 2 = Ȳ µ 1 is called the regression estimator of µ 1 In sample surveys, extra observations on a subset of variables are used to improve estimation of a parameter Σ is more complicated: Σ 11 = 1 n (A 11 A 12 A 1 22,n A 21) + 1 N A 12A 1 22,n A 22,NA 1 22,n A 21 Σ 12 = 1 N A 12A 1 22,n A 22,N Σ 22 = 1 N A 22,N
18 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 18/4 Seventy year-old unsolved problems Explicit confidence levels for elliptical confidence regions for µ In testing hypotheses on µ or Σ, are the LRT statistics unbiased? Calculate the higher moments of the components of µ Determine the asymptotic behavior of µ as n or N The Stein phenomenon for µ? The crucial obstacle: The exact distribution of µ
19 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 19/4 The Exact Distribution of µ Theorem (Chang and D.R.): For n > p + q, µ = L µ + V 1 + ( ( ) 1 n 1 ) 1/2 ( Q2 ) 1/2 V2 N Q 1, 0 where V 1, V 2, Q 1, and Q 2 are independent; V 1 N p+q (0,Ω), V 2 N p (0,Σ 11 2 ), Q 1 χ 2 n q, Q 2 χ 2 q; Ω = 1 N Σ + ( 1 n 1 ) ( ) Σ N. 0 0 Corollary: µ is an unbiased estimator of µ. Also, µ 1 and µ 2 are independent iff Σ 12 = 0. A comment on the power of the method of characteristic functions; Terence s Stuff
20 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 20/4 Computation of the higher moments of µ now is straightforward Due to the term 1/Q 1, even moments exist only up to order n q The covariance matrix of µ: Cov( µ) = 1 N Σ + (n 2) τ n(n q 2) ( ) Σ
21 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 21/4 Asymptotics for µ Let n,n with N/n δ 1. Then ( ( )) L Σ N( µ µ) Np+q 0,Σ + (δ 1) 0 0 Many other asymptotic results can be obtained from the exact distribution of µ; e.g., If n and N with n/n 0 then µ 2 µ 2, almost surely
22 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 22/4 The analog of Hotelling s T 2 -statistic T 2 = ( µ µ) Ĉov( µ) 1 ( µ µ) where Ĉov( µ) = 1 N Σ + (n 2) τ n(n q 2) ) ( Σ An obvious ellipsoidal confidence region for µ is { } ν R p+q : ( µ ν) Ĉov( µ) 1 ( µ ν) c Calculate the corresponding confidence level
23 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 23/4 Chang and D.R.: Probability inequalities for T 2 µ 1 = X τa 12 A 1 22,n (Ȳ1 Ȳ2), µ 2 = Ȳ A crucial identity (Anderson, p. 63): µ Σ 1 µ = ( µ 1 Σ 12 Σ 1 22 µ 1 2) Σ 11 2 ( µ 1 Σ 12 Σ 1 22 µ 2) + µ 2 Σ 1 22 µ 2 Romer (2009, thesis): An exact stochastic representation for the T 2 -statistic Romer s result opens the way toward Stein-estimation of µ when Σ is unknown
24 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 24/4 A decomposition of Σ Notation: A 11 2,n := A 11 A 12 A 1 22,n A 21 ( ) ( A 11 A 12 n Σ = τ + τ A 21 A 22,n ( ) ( A 12 A 1 22,n 0 +τ 0 I q ) A 11 2,n B B B B ) ( A 1 22,n A I q ) where ( ) A 11 A 12 A 21 A 22,n W p+q (n 1,Σ) and B W q (N n,σ 22 ) are independent. Also, N Σ 22 W q (N 1,Σ 22 )
25 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 25/4 A 22,N = n (Y j Ȳ1 + Ȳ1 Ȳ )(Y j Ȳ1 + Ȳ1 Ȳ ) j=1 N + (Y j Ȳ2 + Ȳ2 Ȳ )(Y j Ȳ2 + Ȳ2 Ȳ ) j=n+1 A 22,N = A 22,n + B B = N j=n+1 (Y j Ȳ2)(Y j Ȳ2) + n(n n) N (Ȳ1 Ȳ2)(Ȳ1 Ȳ2) Verify that the terms in the decomposition of Σ are independent
26 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 26/4 The marginal distribution of Σ 11 is non-trivial If Σ 12 = 0 then A 22,n, B, A 11 2,n, A 12 A 1 22,n A 21, X, Ȳ 1, and Ȳ2 are independent Matrix F -distribution: F (q) a,b = W 1/2 2 W 1 W 1/2 2 where W 1 W q (a,σ 22 ) and W 2 W q (b,σ 22 )
27 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 27/4 Theorem: Suppose that Σ 12 = 0. Then Σ 1/2 11 Σ 11 Σ 1/2 11 L = 1 n W N W 1/2 where W 1, W 2, and F are independent, and 2 ( Ip + F ) W 1/2 2 W 1 W p (n q 1,I p ), F F (p) N n,n q+p 1 W 2 W p (q,i p ), and NΣ 1/2 11 Σ 11 Σ 1/2 11 L = N n Σ 1/2 11 A 11 2,n Σ 1/ Σ 1/2 11 A 12 A 1 22,n (A 22,n + B)A 1 22,n A 21Σ 1/2 11
28 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 28/4 Theorem: With no assumptions on Σ 12, Σ 12 Σ 1 22 L = Σ 12 Σ Σ1/ W 1/2 KΣ 1/2 22 where W and K are independent, and W W p (n q + p 1,I p ), K N pq (0,I p I q ) In particular, Σ 12 Σ 1 22 is an unbiased estimator of Σ 12Σ 1 22 The general distribution of Σ requires the hypergeometric functions of matrix argument Saddlepoint approximations
29 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 29/4 The distribution of Σ is much simpler: Σ = Σ 11 2 Σ 22 Σ 11 2 and Σ 22 are independent; each is a product of independent χ 2 variables Hao and Krishnamoorthy (2001): p Σ = L n p N q Σ χ 2 n q j j=1 q j=1 χ 2 N j It now is plausible that tests of hypothesis on Σ are unbiased
30 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 30/4 Testing Σ = Σ 0 Data: Two-step, monotone incomplete sample Σ 0 : A given, positive definite matrix Test H 0 : Σ = Σ 0 vs. H a : Σ Σ 0 (WLOG, Σ 0 = I p+q ) Hao and Krishnamoorthy (2001): The LRT statistic is λ 1 A 22,N N/2 exp ( 1 2 tra ) 22,N A 11 2,n n/2 exp ( 1 2 tra ) 11 2,n exp ( 1 2 tra 12A 1 22,n A 21) ). Is the LRT unbiased? If C is a critical region of size α, is P(λ 1 C H a ) P(λ 1 C H 0 )?
31 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 31/4 E. J. G. Pitman: With complete data, λ 1 is not unbiased λ 1 becomes unbiased if sample sizes are replaced by degrees of freedom With two-step monotone data, perhaps a similarly modified statistic, λ 2, is unbiased? Answer: Still unknown. Theorem: If Σ 11 < 1 then λ 2 is unbiased With monotone incomplete data, further modification is needed
32 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 32/4 Theorem: The modified LRT, λ 3 A 22,N (N 1)/2 exp ( 1 2 tra ) 22,N A 11 2,n (n q 1)/2 exp ( 1 2 tra ) 11 2,n A 12 A 1 22,n A 21 q/2 exp ( 1 2 tra 12A 1 22,n A 21), is unbiased. Also, λ 1 is not unbiased For diagonal Σ = diag(σ jj ), the power function of λ 3 increases monotonically as any σ jj 1 increases, j = 1,...,p + q.
33 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 33/4 With monotone two-step data, test H 0 : (µ,σ) = (µ 0,Σ 0 ) vs. H a : (µ,σ) (µ 0,Σ 0 ) where µ 0 and Σ 0 are given. The LRT statistic is λ 4 = λ 1 exp ( 1 2 (n X X + NȲ Ȳ ) ) Remarkably, λ 4 is unbiased The sphericity test, H 0 : Σ I p+q vs. H a : I p+q The unbiasedness of the LRT statistic is an open problem
34 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 34/4 The Stein phenomenon for µ µ: The mean of a complete sample from N m (µ,i m ) Quadratic loss function: L( µ,µ) = µ µ 2 Risk function: R( µ) = E L( µ,µ) C. Stein: µ is inadmissible for m 3 Berkeley Symposium, 1956
35 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 35/4 The James-Stein estimator for shrinking µ to ν R m : ( ) c µ c = 1 µ ν 2 ( µ ν) + ν Baranchik s positive-part shrinkage estimator: ( ) µ + c c = 1 µ ν 2 ( µ ν) + ν + With complete data from N m (µ,i m ), R( µ) > R( µ c ) > R( µ + c ) for 0 < c < 2(m 2)
36 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 36/4 We collect a monotone incomplete sample from N p+q (µ,σ) Does the Stein phenomenon hold for µ, the MLE of µ? The phenomenon seems almost universal: It holds for many loss functions, inference problems, and distributions Various results available on shrinkage estimation of Σ with incomplete data, but no such results available for µ The crucial impediment: The distribution of µ was unknown
37 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 37/4 Theorem (Yamada and D.R.): With two-step monotone incomplete data from N p+q (µ,i p+q ) with p 2 and n q + 3, both µ and µ c are inadmissible: R( µ) > R( µ c ) > R( µ + c ) for all ν R p+q and all c (0,2c ), where c = p 2 n + q N. Non-radial loss functions Replace µ ν 2 by non-radial functions of µ ν Shrinkage to a random vector ν, calculated from the data
38 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 38/4 The Fundamental Open Problems Extension to general, k-step, monotone incomplete data The crucial impediment: The distribution of µ Conjecture: The Stein phenomenon holds for all monotone missingness patterns under normality Does Stein s phenomenon hold for arbitrary missingness patterns? Will we have reduced risk with similar estimators?
39 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 39/4 Testing for multivariate normality Monotone incomplete data, i.i.d., unknown population: ( ) ( ) ( ) ( ) ( ) ( X 1,,...,,,,..., Y 1 X 2 Y 2 X n Y n Y n+1 Y n+2 A generalization of Mardia s statistic for testing for kurtosis: Y N ) ˆβ = [( n (Xj ) ) ( (Xj ) )] 2 µ Σ 1 µ j=1 + N j=n+1 Y j Y j [ ] 2 1 (Y j µ 2 ) Σ 22 (Y j µ 2 )
40 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 40/4 An alternative to ˆβ Impute each missing X j using linear regression: { X j, 1 j n X j = µ 1 + Σ 12 Σ 1 22 (Y j µ 2 ), n + 1 j N Construct β = [( N ) ( Xj Y j=1 j µ ) Σ 1 ( ) ( Xj Y j µ )] 2 Remarkably, β β Theorem (Yamada, Romer, D.R.): With certain regularity conditions, constants c 1, c 2, ( β c 1 )/c 2 L N(0,1)
41 Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 41/4 References Chang and R. (2009). Finite-sample inference with monotone incomplete multivariate normal data, I. J. Multivariate Analysis, 100, Chang and R. (2010). Finite-sample inference with monotone incomplete multivariate normal data, II. J. Multivariate Analysis, 101, Romer (2009). The Statistical Analysis of Monotone Incomplete Data. Ph.D. thesis, Penn State University. R. and Yamada (2010). The Stein phenomenon for monotone incomplete multivariate normal data. J. Multivariate Analysis, 101, Romer and R. (2010). Maximum likelihood estimation of the mean of a multivariate normal population with monotone incomplete data. Statist. Probab. Lett., 80, Yamada, Romer, and R. (2010). Measures of multivariate kurtosis with monotone incomplete data. Preprint, Penn State University. Romer and R. (2010). Finite-sample inference with monotone incomplete multivariate normal data, III. Preprint, Penn State University.
Statistical Inference with Monotone Incomplete Multivariate Normal Data
Statistical Inference with Monotone Incomplete Multivariate Normal Data p. 1/4 Statistical Inference with Monotone Incomplete Multivariate Normal Data This talk is based on joint work with my wonderful
More informationFinite-Sample Inference with Monotone Incomplete Multivariate Normal Data, I
Finite-Sample Inference with Monotone Incomplete Multivariate Normal Data, I Wan-Ying Chang and Donald St.P. Richards July 23, 2008 Abstract We consider problems in finite-sample inference with two-step,
More informationTHE STATISTICAL ANALYSIS OF MONOTONE INCOMPLETE MULTIVARIATE NORMAL DATA
The Pennsylvania State University The Graduate School Department of Statistics THE STATISTICAL ANALYSIS OF MONOTONE INCOMPLETE MULTIVARIATE NORMAL DATA A Dissertation in Statistics by Megan M. Romer c
More informationInferences on a Normal Covariance Matrix and Generalized Variance with Monotone Missing Data
Journal of Multivariate Analysis 78, 6282 (2001) doi:10.1006jmva.2000.1939, available online at http:www.idealibrary.com on Inferences on a Normal Covariance Matrix and Generalized Variance with Monotone
More informationFinite-Sample Inference with Monotone Incomplete Multivariate Normal Data, II
Finite-Sample Inference with Monotone Incomplete Multivariate Normal Data, II Wan-Ying Chang and Donald St. P. Richards July 4, 008 Abstract We continue our recent work on finite-sample, i.e., non-asymptotic,
More informationLecture 3. Inference about multivariate normal distribution
Lecture 3. Inference about multivariate normal distribution 3.1 Point and Interval Estimation Let X 1,..., X n be i.i.d. N p (µ, Σ). We are interested in evaluation of the maximum likelihood estimates
More informationTesting equality of two mean vectors with unequal sample sizes for populations with correlation
Testing equality of two mean vectors with unequal sample sizes for populations with correlation Aya Shinozaki Naoya Okamoto 2 and Takashi Seo Department of Mathematical Information Science Tokyo University
More informationAn Introduction to Multivariate Statistical Analysis
An Introduction to Multivariate Statistical Analysis Third Edition T. W. ANDERSON Stanford University Department of Statistics Stanford, CA WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION Contents
More informationMultivariate Statistics
Multivariate Statistics Chapter 2: Multivariate distributions and inference Pedro Galeano Departamento de Estadística Universidad Carlos III de Madrid pedro.galeano@uc3m.es Course 2016/2017 Master in Mathematical
More informationProblems. Suppose both models are fitted to the same data. Show that SS Res, A SS Res, B
Simple Linear Regression 35 Problems 1 Consider a set of data (x i, y i ), i =1, 2,,n, and the following two regression models: y i = β 0 + β 1 x i + ε, (i =1, 2,,n), Model A y i = γ 0 + γ 1 x i + γ 2
More informationStatistical Inference On the High-dimensional Gaussian Covarianc
Statistical Inference On the High-dimensional Gaussian Covariance Matrix Department of Mathematical Sciences, Clemson University June 6, 2011 Outline Introduction Problem Setup Statistical Inference High-Dimensional
More informationLikelihood Ratio Criterion for Testing Sphericity from a Multivariate Normal Sample with 2-step Monotone Missing Data Pattern
The Korean Communications in Statistics Vol. 12 No. 2, 2005 pp. 473-481 Likelihood Ratio Criterion for Testing Sphericity from a Multivariate Normal Sample with 2-step Monotone Missing Data Pattern Byungjin
More informationYou can compute the maximum likelihood estimate for the correlation
Stat 50 Solutions Comments on Assignment Spring 005. (a) _ 37.6 X = 6.5 5.8 97.84 Σ = 9.70 4.9 9.70 75.05 7.80 4.9 7.80 4.96 (b) 08.7 0 S = Σ = 03 9 6.58 03 305.6 30.89 6.58 30.89 5.5 (c) You can compute
More informationANALYSIS OF VARIANCE AND QUADRATIC FORMS
4 ANALYSIS OF VARIANCE AND QUADRATIC FORMS The previous chapter developed the regression results involving linear functions of the dependent variable, β, Ŷ, and e. All were shown to be normally distributed
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 9 for Applied Multivariate Analysis Outline Addressing ourliers 1 Addressing ourliers 2 Outliers in Multivariate samples (1) For
More informationReview of probability and statistics 1 / 31
Review of probability and statistics 1 / 31 2 / 31 Why? This chapter follows Stock and Watson (all graphs are from Stock and Watson). You may as well refer to the appendix in Wooldridge or any other introduction
More informationMultinomial Data. f(y θ) θ y i. where θ i is the probability that a given trial results in category i, i = 1,..., k. The parameter space is
Multinomial Data The multinomial distribution is a generalization of the binomial for the situation in which each trial results in one and only one of several categories, as opposed to just two, as in
More informationPeter Hoff Linear and multilinear models April 3, GLS for multivariate regression 5. 3 Covariance estimation for the GLM 8
Contents 1 Linear model 1 2 GLS for multivariate regression 5 3 Covariance estimation for the GLM 8 4 Testing the GLH 11 A reference for some of this material can be found somewhere. 1 Linear model Recall
More informationLinear Models and Estimation by Least Squares
Linear Models and Estimation by Least Squares Jin-Lung Lin 1 Introduction Causal relation investigation lies in the heart of economics. Effect (Dependent variable) cause (Independent variable) Example:
More informationInferences about Parameters of Trivariate Normal Distribution with Missing Data
Florida International University FIU Digital Commons FIU Electronic Theses and Dissertations University Graduate School 7-5-3 Inferences about Parameters of Trivariate Normal Distribution with Missing
More informationReview of Classical Least Squares. James L. Powell Department of Economics University of California, Berkeley
Review of Classical Least Squares James L. Powell Department of Economics University of California, Berkeley The Classical Linear Model The object of least squares regression methods is to model and estimate
More informationNotes on Random Vectors and Multivariate Normal
MATH 590 Spring 06 Notes on Random Vectors and Multivariate Normal Properties of Random Vectors If X,, X n are random variables, then X = X,, X n ) is a random vector, with the cumulative distribution
More informationSummer School in Statistics for Astronomers V June 1 - June 6, Regression. Mosuk Chow Statistics Department Penn State University.
Summer School in Statistics for Astronomers V June 1 - June 6, 2009 Regression Mosuk Chow Statistics Department Penn State University. Adapted from notes prepared by RL Karandikar Mean and variance Recall
More informationTesting a Normal Covariance Matrix for Small Samples with Monotone Missing Data
Applied Mathematical Sciences, Vol 3, 009, no 54, 695-70 Testing a Normal Covariance Matrix for Small Samples with Monotone Missing Data Evelina Veleva Rousse University A Kanchev Department of Numerical
More informationTesting Some Covariance Structures under a Growth Curve Model in High Dimension
Department of Mathematics Testing Some Covariance Structures under a Growth Curve Model in High Dimension Muni S. Srivastava and Martin Singull LiTH-MAT-R--2015/03--SE Department of Mathematics Linköping
More informationMath 423/533: The Main Theoretical Topics
Math 423/533: The Main Theoretical Topics Notation sample size n, data index i number of predictors, p (p = 2 for simple linear regression) y i : response for individual i x i = (x i1,..., x ip ) (1 p)
More informationInferences about a Mean Vector
Inferences about a Mean Vector Edps/Soc 584, Psych 594 Carolyn J. Anderson Department of Educational Psychology I L L I N O I S university of illinois at urbana-champaign c Board of Trustees, University
More informationA Multivariate Two-Sample Mean Test for Small Sample Size and Missing Data
A Multivariate Two-Sample Mean Test for Small Sample Size and Missing Data Yujun Wu, Marc G. Genton, 1 and Leonard A. Stefanski 2 Department of Biostatistics, School of Public Health, University of Medicine
More informationMultivariate Analysis and Likelihood Inference
Multivariate Analysis and Likelihood Inference Outline 1 Joint Distribution of Random Variables 2 Principal Component Analysis (PCA) 3 Multivariate Normal Distribution 4 Likelihood Inference Joint density
More informationLecture 5: Hypothesis tests for more than one sample
1/23 Lecture 5: Hypothesis tests for more than one sample Måns Thulin Department of Mathematics, Uppsala University thulin@math.uu.se Multivariate Methods 8/4 2011 2/23 Outline Paired comparisons Repeated
More informationPh.D. Qualifying Exam Friday Saturday, January 3 4, 2014
Ph.D. Qualifying Exam Friday Saturday, January 3 4, 2014 Put your solution to each problem on a separate sheet of paper. Problem 1. (5166) Assume that two random samples {x i } and {y i } are independently
More informationChapter 9. Hotelling s T 2 Test. 9.1 One Sample. The one sample Hotelling s T 2 test is used to test H 0 : µ = µ 0 versus
Chapter 9 Hotelling s T 2 Test 9.1 One Sample The one sample Hotelling s T 2 test is used to test H 0 : µ = µ 0 versus H A : µ µ 0. The test rejects H 0 if T 2 H = n(x µ 0 ) T S 1 (x µ 0 ) > n p F p,n
More informationThis model of the conditional expectation is linear in the parameters. A more practical and relaxed attitude towards linear regression is to say that
Linear Regression For (X, Y ) a pair of random variables with values in R p R we assume that E(Y X) = β 0 + with β R p+1. p X j β j = (1, X T )β j=1 This model of the conditional expectation is linear
More informationThe Delta Method and Applications
Chapter 5 The Delta Method and Applications 5.1 Local linear approximations Suppose that a particular random sequence converges in distribution to a particular constant. The idea of using a first-order
More information[y i α βx i ] 2 (2) Q = i=1
Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation
More informationStatistical Methods. Missing Data snijders/sm.htm. Tom A.B. Snijders. November, University of Oxford 1 / 23
1 / 23 Statistical Methods Missing Data http://www.stats.ox.ac.uk/ snijders/sm.htm Tom A.B. Snijders University of Oxford November, 2011 2 / 23 Literature: Joseph L. Schafer and John W. Graham, Missing
More informationNonconcave Penalized Likelihood with A Diverging Number of Parameters
Nonconcave Penalized Likelihood with A Diverging Number of Parameters Jianqing Fan and Heng Peng Presenter: Jiale Xu March 12, 2010 Jianqing Fan and Heng Peng Presenter: JialeNonconcave Xu () Penalized
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationSTAT 501 Assignment 2 NAME Spring Chapter 5, and Sections in Johnson & Wichern.
STAT 01 Assignment NAME Spring 00 Reading Assignment: Written Assignment: Chapter, and Sections 6.1-6.3 in Johnson & Wichern. Due Monday, February 1, in class. You should be able to do the first four problems
More informationThe purpose of this section is to derive the asymptotic distribution of the Pearson chi-square statistic. k (n j np j ) 2. np j.
Chapter 9 Pearson s chi-square test 9. Null hypothesis asymptotics Let X, X 2, be independent from a multinomial(, p) distribution, where p is a k-vector with nonnegative entries that sum to one. That
More informationCh 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
More informationTEST FOR INDEPENDENCE OF THE VARIABLES WITH MISSING ELEMENTS IN ONE AND THE SAME COLUMN OF THE EMPIRICAL CORRELATION MATRIX.
Serdica Math J 34 (008, 509 530 TEST FOR INDEPENDENCE OF THE VARIABLES WITH MISSING ELEMENTS IN ONE AND THE SAME COLUMN OF THE EMPIRICAL CORRELATION MATRIX Evelina Veleva Communicated by N Yanev Abstract
More informationPart 6: Multivariate Normal and Linear Models
Part 6: Multivariate Normal and Linear Models 1 Multiple measurements Up until now all of our statistical models have been univariate models models for a single measurement on each member of a sample of
More informationDepartment of Statistical Science FIRST YEAR EXAM - SPRING 2017
Department of Statistical Science Duke University FIRST YEAR EXAM - SPRING 017 Monday May 8th 017, 9:00 AM 1:00 PM NOTES: PLEASE READ CAREFULLY BEFORE BEGINNING EXAM! 1. Do not write solutions on the exam;
More informationSo far our focus has been on estimation of the parameter vector β in the. y = Xβ + u
Interval estimation and hypothesis tests So far our focus has been on estimation of the parameter vector β in the linear model y i = β 1 x 1i + β 2 x 2i +... + β K x Ki + u i = x iβ + u i for i = 1, 2,...,
More informationFall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
More informationEC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix)
1 EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix) Taisuke Otsu London School of Economics Summer 2018 A.1. Summation operator (Wooldridge, App. A.1) 2 3 Summation operator For
More informationMultivariate Distributions
IEOR E4602: Quantitative Risk Management Spring 2016 c 2016 by Martin Haugh Multivariate Distributions We will study multivariate distributions in these notes, focusing 1 in particular on multivariate
More information1 Data Arrays and Decompositions
1 Data Arrays and Decompositions 1.1 Variance Matrices and Eigenstructure Consider a p p positive definite and symmetric matrix V - a model parameter or a sample variance matrix. The eigenstructure is
More informationSTAT 730 Chapter 5: Hypothesis Testing
STAT 730 Chapter 5: Hypothesis Testing Timothy Hanson Department of Statistics, University of South Carolina Stat 730: Multivariate Analysis 1 / 28 Likelihood ratio test def n: Data X depend on θ. The
More informationGibbs Sampling in Linear Models #2
Gibbs Sampling in Linear Models #2 Econ 690 Purdue University Outline 1 Linear Regression Model with a Changepoint Example with Temperature Data 2 The Seemingly Unrelated Regressions Model 3 Gibbs sampling
More informationT 2 Type Test Statistic and Simultaneous Confidence Intervals for Sub-mean Vectors in k-sample Problem
T Type Test Statistic and Simultaneous Confidence Intervals for Sub-mean Vectors in k-sample Problem Toshiki aito a, Tamae Kawasaki b and Takashi Seo b a Department of Applied Mathematics, Graduate School
More informationStat 579: Generalized Linear Models and Extensions
Stat 579: Generalized Linear Models and Extensions Mixed models Yan Lu March, 2018, week 8 1 / 32 Restricted Maximum Likelihood (REML) REML: uses a likelihood function calculated from the transformed set
More informationLecture 7 Introduction to Statistical Decision Theory
Lecture 7 Introduction to Statistical Decision Theory I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 20, 2016 1 / 55 I-Hsiang Wang IT Lecture 7
More informationApplied Multivariate and Longitudinal Data Analysis
Applied Multivariate and Longitudinal Data Analysis Chapter 2: Inference about the mean vector(s) Ana-Maria Staicu SAS Hall 5220; 919-515-0644; astaicu@ncsu.edu 1 In this chapter we will discuss inference
More informationExpected probabilities of misclassification in linear discriminant analysis based on 2-Step monotone missing samples
Expected probabilities of misclassification in linear discriminant analysis based on 2-Step monotone missing samples Nobumichi Shutoh, Masashi Hyodo and Takashi Seo 2 Department of Mathematics, Graduate
More informationMultivariate Regression Generalized Likelihood Ratio Tests for FMRI Activation
Multivariate Regression Generalized Likelihood Ratio Tests for FMRI Activation Daniel B Rowe Division of Biostatistics Medical College of Wisconsin Technical Report 40 November 00 Division of Biostatistics
More information1 Appendix A: Matrix Algebra
Appendix A: Matrix Algebra. Definitions Matrix A =[ ]=[A] Symmetric matrix: = for all and Diagonal matrix: 6=0if = but =0if 6= Scalar matrix: the diagonal matrix of = Identity matrix: the scalar matrix
More informationQualifying Exam in Probability and Statistics. https://www.soa.org/files/edu/edu-exam-p-sample-quest.pdf
Part 1: Sample Problems for the Elementary Section of Qualifying Exam in Probability and Statistics https://www.soa.org/files/edu/edu-exam-p-sample-quest.pdf Part 2: Sample Problems for the Advanced Section
More informationSpatial inference. Spatial inference. Accounting for spatial correlation. Multivariate normal distributions
Spatial inference I will start with a simple model, using species diversity data Strong spatial dependence, Î = 0.79 what is the mean diversity? How precise is our estimate? Sampling discussion: The 64
More informationCh 3: Multiple Linear Regression
Ch 3: Multiple Linear Regression 1. Multiple Linear Regression Model Multiple regression model has more than one regressor. For example, we have one response variable and two regressor variables: 1. delivery
More informationg-priors for Linear Regression
Stat60: Bayesian Modeling and Inference Lecture Date: March 15, 010 g-priors for Linear Regression Lecturer: Michael I. Jordan Scribe: Andrew H. Chan 1 Linear regression and g-priors In the last lecture,
More informationRegression and Statistical Inference
Regression and Statistical Inference Walid Mnif wmnif@uwo.ca Department of Applied Mathematics The University of Western Ontario, London, Canada 1 Elements of Probability 2 Elements of Probability CDF&PDF
More informationLecture Note 1: Probability Theory and Statistics
Univ. of Michigan - NAME 568/EECS 568/ROB 530 Winter 2018 Lecture Note 1: Probability Theory and Statistics Lecturer: Maani Ghaffari Jadidi Date: April 6, 2018 For this and all future notes, if you would
More information1 Exercises for lecture 1
1 Exercises for lecture 1 Exercise 1 a) Show that if F is symmetric with respect to µ, and E( X )
More informationCentral Limit Theorem ( 5.3)
Central Limit Theorem ( 5.3) Let X 1, X 2,... be a sequence of independent random variables, each having n mean µ and variance σ 2. Then the distribution of the partial sum S n = X i i=1 becomes approximately
More informationGaussian processes. Chuong B. Do (updated by Honglak Lee) November 22, 2008
Gaussian processes Chuong B Do (updated by Honglak Lee) November 22, 2008 Many of the classical machine learning algorithms that we talked about during the first half of this course fit the following pattern:
More informationCovariance function estimation in Gaussian process regression
Covariance function estimation in Gaussian process regression François Bachoc Department of Statistics and Operations Research, University of Vienna WU Research Seminar - May 2015 François Bachoc Gaussian
More informationDependence. Practitioner Course: Portfolio Optimization. John Dodson. September 10, Dependence. John Dodson. Outline.
Practitioner Course: Portfolio Optimization September 10, 2008 Before we define dependence, it is useful to define Random variables X and Y are independent iff For all x, y. In particular, F (X,Y ) (x,
More informationNonparametric Inference In Functional Data
Nonparametric Inference In Functional Data Zuofeng Shang Purdue University Joint work with Guang Cheng from Purdue Univ. An Example Consider the functional linear model: Y = α + where 1 0 X(t)β(t)dt +
More informationCanonical Correlation Analysis of Longitudinal Data
Biometrics Section JSM 2008 Canonical Correlation Analysis of Longitudinal Data Jayesh Srivastava Dayanand N Naik Abstract Studying the relationship between two sets of variables is an important multivariate
More informationEstimation of a multivariate normal covariance matrix with staircase pattern data
AISM (2007) 59: 211 233 DOI 101007/s10463-006-0044-x Xiaoqian Sun Dongchu Sun Estimation of a multivariate normal covariance matrix with staircase pattern data Received: 20 January 2005 / Revised: 1 November
More informationNotes on the Multivariate Normal and Related Topics
Version: July 10, 2013 Notes on the Multivariate Normal and Related Topics Let me refresh your memory about the distinctions between population and sample; parameters and statistics; population distributions
More informationSTATISTICS SYLLABUS UNIT I
STATISTICS SYLLABUS UNIT I (Probability Theory) Definition Classical and axiomatic approaches.laws of total and compound probability, conditional probability, Bayes Theorem. Random variable and its distribution
More informationLecture 8: Information Theory and Statistics
Lecture 8: Information Theory and Statistics Part II: Hypothesis Testing and I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 23, 2015 1 / 50 I-Hsiang
More informationConjugate Analysis for the Linear Model
Conjugate Analysis for the Linear Model If we have good prior knowledge that can help us specify priors for β and σ 2, we can use conjugate priors. Following the procedure in Christensen, Johnson, Branscum,
More informationParameter estimation! and! forecasting! Cristiano Porciani! AIfA, Uni-Bonn!
Parameter estimation! and! forecasting! Cristiano Porciani! AIfA, Uni-Bonn! Questions?! C. Porciani! Estimation & forecasting! 2! Cosmological parameters! A branch of modern cosmological research focuses
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 9 for Applied Multivariate Analysis Outline Two sample T 2 test 1 Two sample T 2 test 2 Analogous to the univariate context, we
More information12 Discriminant Analysis
12 Discriminant Analysis Discriminant analysis is used in situations where the clusters are known a priori. The aim of discriminant analysis is to classify an observation, or several observations, into
More informationEstimation of the large covariance matrix with two-step monotone missing data
Estimation of the large covariance matrix with two-ste monotone missing data Masashi Hyodo, Nobumichi Shutoh 2, Takashi Seo, and Tatjana Pavlenko 3 Deartment of Mathematical Information Science, Tokyo
More informationMULTIVARIATE ANALYSIS OF VARIANCE UNDER MULTIPLICITY José A. Díaz-García. Comunicación Técnica No I-07-13/ (PE/CIMAT)
MULTIVARIATE ANALYSIS OF VARIANCE UNDER MULTIPLICITY José A. Díaz-García Comunicación Técnica No I-07-13/11-09-2007 (PE/CIMAT) Multivariate analysis of variance under multiplicity José A. Díaz-García Universidad
More informationApplied Multivariate and Longitudinal Data Analysis
Applied Multivariate and Longitudinal Data Analysis Chapter 2: Inference about the mean vector(s) II Ana-Maria Staicu SAS Hall 5220; 919-515-0644; astaicu@ncsu.edu 1 1 Compare Means from More Than Two
More informationEstimation theory. Parametric estimation. Properties of estimators. Minimum variance estimator. Cramer-Rao bound. Maximum likelihood estimators
Estimation theory Parametric estimation Properties of estimators Minimum variance estimator Cramer-Rao bound Maximum likelihood estimators Confidence intervals Bayesian estimation 1 Random Variables Let
More informationwhere x and ȳ are the sample means of x 1,, x n
y y Animal Studies of Side Effects Simple Linear Regression Basic Ideas In simple linear regression there is an approximately linear relation between two variables say y = pressure in the pancreas x =
More informationLifetime Dependence Modelling using a Generalized Multivariate Pareto Distribution
Lifetime Dependence Modelling using a Generalized Multivariate Pareto Distribution Daniel Alai Zinoviy Landsman Centre of Excellence in Population Ageing Research (CEPAR) School of Mathematics, Statistics
More informationPh.D. Qualifying Exam Friday Saturday, January 6 7, 2017
Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Let X 1, X 2,, X n be a sequence of i.i.d. observations from a
More informationz = β βσβ Statistical Analysis of MV Data Example : µ=0 (Σ known) consider Y = β X~ N 1 (β µ, β Σβ) test statistic for H 0β is
Example X~N p (µ,σ); H 0 : µ=0 (Σ known) consider Y = β X~ N 1 (β µ, β Σβ) H 0β : β µ = 0 test statistic for H 0β is y z = β βσβ /n And reject H 0β if z β > c [suitable critical value] 301 Reject H 0 if
More informationStatement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.
MATHEMATICAL STATISTICS Homework assignment Instructions Please turn in the homework with this cover page. You do not need to edit the solutions. Just make sure the handwriting is legible. You may discuss
More informationEstimation: Problems & Solutions
Estimation: Problems & Solutions Edps/Psych/Stat 587 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2017 Outline 1. Introduction: Estimation of
More informationSample mean, covariance and T 2 statistic of the skew elliptical model
Journal of Multivariate Analysis 97 (2006) 1675 1690 www.elsevier.com/locate/jmva Sample mean, covariance and T 2 statistic of the skew elliptical model B.Q. Fang,1 Institute of Applied Mathematics, Academy
More informationProblem Selected Scores
Statistics Ph.D. Qualifying Exam: Part II November 20, 2010 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. Problem 1 2 3 4 5 6 7 8 9 10 11 12 Selected
More informationAnalysis of variance, multivariate (MANOVA)
Analysis of variance, multivariate (MANOVA) Abstract: A designed experiment is set up in which the system studied is under the control of an investigator. The individuals, the treatments, the variables
More informationMean squared error matrix comparison of least aquares and Stein-rule estimators for regression coefficients under non-normal disturbances
METRON - International Journal of Statistics 2008, vol. LXVI, n. 3, pp. 285-298 SHALABH HELGE TOUTENBURG CHRISTIAN HEUMANN Mean squared error matrix comparison of least aquares and Stein-rule estimators
More informationMultivariate Linear Regression Models
Multivariate Linear Regression Models Regression analysis is used to predict the value of one or more responses from a set of predictors. It can also be used to estimate the linear association between
More informationDepartment of Statistics
Research Report Department of Statistics Research Report Department of Statistics No. 05: Testing in multivariate normal models with block circular covariance structures Yuli Liang Dietrich von Rosen Tatjana
More informationSTAT 512 sp 2018 Summary Sheet
STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}
More informationStat 206: Sampling theory, sample moments, mahalanobis
Stat 206: Sampling theory, sample moments, mahalanobis topology James Johndrow (adapted from Iain Johnstone s notes) 2016-11-02 Notation My notation is different from the book s. This is partly because
More informationLeast Squares Estimation
Least Squares Estimation Using the least squares estimator for β we can obtain predicted values and compute residuals: Ŷ = Z ˆβ = Z(Z Z) 1 Z Y ˆɛ = Y Ŷ = Y Z(Z Z) 1 Z Y = [I Z(Z Z) 1 Z ]Y. The usual decomposition
More informationLECTURES IN ECONOMETRIC THEORY. John S. Chipman. University of Minnesota
LCTURS IN CONOMTRIC THORY John S. Chipman University of Minnesota Chapter 5. Minimax estimation 5.. Stein s theorem and the regression model. It was pointed out in Chapter 2, section 2.2, that if no a
More informationStudentization and Prediction in a Multivariate Normal Setting
Studentization and Prediction in a Multivariate Normal Setting Morris L. Eaton University of Minnesota School of Statistics 33 Ford Hall 4 Church Street S.E. Minneapolis, MN 55455 USA eaton@stat.umn.edu
More information