Random Matrices and Multivariate Statistical Analysis
|
|
- Dorothy Greene
- 5 years ago
- Views:
Transcription
1 Random Matrices and Multivariate Statistical Analysis Iain Johnstone, Statistics, Stanford SEA p.1
2 Agenda Classical multivariate techniques Principal Component Analysis Canonical Correlations Multivariate Regression Hypothesis Testing: Single and Double Wishart Eigenvalue densities Linear Statistics Single Wishart Double Wishart Largest Eigenvalue Single Wishart Double Wishart Concluding Remarks SEA p.2
3 Classical Multivariate Statistics Canonical methods are based on spectral decompositions: One matrix (Wishart) Principal Component analysis Factor analysis Multidimensional scaling Two matrices(independent Wisharts) Multivariate Analysis of Variance (MANOVA) Multivarate regression analysis Discriminant analysis Canonical correlation analysis Tests of equality of covariance matrices SEA p.3
4 Gaussian data matrices cases. = = Independent rows: x i N p (0, Σ), i =1,...n or: X N(0,I n Σ p ) variables Zero mean no centering in sample covariance matrix: S =(S kk ), S = 1 n XT X, S kk = 1 n x ik x ik n ns W p (n, Σ) i=1 SEA 06@MIT p.4
5 Principal Components Analysis Hotelling, 1933 X 1,...,X n N p (µ, Σ), Low dim. subspace explaining most variance : l i =max{u Su : u u =1,u u j =0,j <i} Eigenvalues of Wishart: A = ns W p (n, Σ): Au i = l i u i l 1... l p 0. Key question: How many l i are significant? "scree" plot of singular values of phoneme data SEA 06@MIT p.5
6 Canonical Correlations ( ) ( ) X 1 X n jointly p + q variatenormal. Y 1 Y n Most predictable criterion : (Hotelling, 1935, 1936). max u i,v i Corr (u i X, v i Y ) Av i = r 2 i (A + B)v i, r r 2 p. Two independent Wishart distributions: A W p (q, Σ), B W p (n q, Σ). SEA 06@MIT p.6
7 Multivariate Multiple Regression Y = XB + U n p n q q p n p U N p (0,I Σ) n = #observations;p = # response variables; q = # predictor variables P = X(X T X) 1 X T projection on span{cols(x)} Y T Y = Y T PY + Y T (I P )Y H : hypothesis SSP + E : error SSP H W p (q, Σ) indep of E W p (n q, Σ) SEA 06@MIT p.7
8 Agenda Classical multivariate techniques Hypothesis Testing: Single and Double Wishart Eigenvalue densities Linear Statistics Largest Eigenvalue Concluding Remarks SEA p.8
9 Hypothesis Testing Null hypothesis H 0,nestedwithinAlternative hypothesis H A Test Statistics: functions of eigenvalues: T = T (l 1,...,l p ). Null hypothesis distribution: P (T >t H 0 true). RMT offers tools for evaluation, and approximation based on p Single Wishart A W p (n, I) e-vals det(a l i I)=0. Test H 0 :Σ=I (or λi) versush A :Σunrestricted. Double Wishart H W p (q, Σ),E W p (n q, Σ) independently. Eigenvalues det(h l i (E + H)) = 0. Typical hypothesis test (e.g. from Y = XB + U): H 0 : B =0versus H A : B unrestricted SEA 06@MIT p.9
10 Likelihood Ratio Test If X N p (0,I p Σ), the density f Σ (X) = det(2πσ) n/2 exp{ (n/2)trσ 1 S} Log likelihood Σ l(σ X) = log f Σ (X) =c np n 2 log det Σ n 2 trσ 1 S Maximum likelihood occurs at ˆΣ =S: max Σ l(σ X) =c np n 2 log det S Likelihood ratio test of H 0 :Σ=I vs. H A :Σunrestricted: log LR =max Σ H 0 l(σ X) max Σ H A l(σ X) = c np + n 2 ( i log l i i l i ) Linear statistics in eigenvalues of S: i log l i, i l i. SEA 06@MIT p.10
11 (Union-) Intersection Principle Combine univariate test statistics: H 0 :Σ=I a =1 H 0a : a T Σa =1. Var(a T X)=a T Σa, so reject H 0a if Var(a T X)=a T Sa > c a Reject H 0 reject some H 0a max a a T Sa > c max l max (S) >c max Summary: Likelihood ratio principle linear statistics in eigenvalues Intersection principle extreme eigenvalues SEA 06@MIT p.11
12 Agenda Classical multivariate techniques Hypothesis Testing: Single and Double Wishart Eigenvalue densities Linear Statistics Largest Eigenvalue Concluding Remarks SEA p.12
13 Eigenvalue densities - single Wishart Statistics (n, p): c N j e l i/2 j<k l j l k Laguerre OE (N,α): c p i=1 x α 2 j e x i/2 j<k x j x k ( ) ( ) N p α n p 1 i=1 l n p 1 2 Notation change has significance! p =#variables n = sample size Statistics: no necessary relation between p and n; traditional approximation uses p fixed, n. RMT: N with α fixed is most natural. (in Stat, fixing n p would be less natural). SEA 06@MIT p.13
14 Eigenvalue densities - double Wishart Statistics: If H W p (q, I) and E W p (n q, I) are indep, then joint density of eigenvalues {u i } of H(H + E) 1 is f(u) =c With p i=1 u (q p 1)/2 i (1 u i ) (n q p 1)/2 p n q p q p N +1 α β, recover the Jacobi orthogonal ensemble m (u i u j ). i<j u =(1+x)/2, f(x) =c N+1 N+1 (1 x i ) (α 1)/2 (1 + x i ) (β 1)/2 i=1 i<j x i x j. SEA 06@MIT p.14
15 Convergence of Empirical Spectra For e-values {l i } p i=1 G p (t) =p 1 #{l i t} G(t) =g(t)dt. Single Wishart (Marčenko-Pastur, 67) A W p (n, I) If p/n c>0, g MP (b+ t)(t b ) (t) =, b ± =(1± c) 2. 2πct Double Wishart (Wachter, 80) det(h l i (H + E)) = 0. If p q, p/n c =sin 2 (γ/2) > 0, q/n sin 2 (φ/2), g W (t) = (b+ t)(t b ) 2πct(1 t), b ± =sin 2 ( φ ± γ 2 ). SEA 06@MIT p.15
16 Agenda Classical multivariate techniques Hypothesis Testing: Single and Double Wishart Eigenvalue densities Linear Statistics Single Wishart Double Wishart Largest Eigenvalue Concluding Remarks SEA p.16
17 Linear Statistics: Single Wishart Approximate distributions: Statistics: Typically p fixed; standard χ 2 approximation, improvements by Bartlett correction RMT: Central Limit Theorems (p large) for linear statistics of eigenvalues. Large literature Jonsson (1982): S W p (n, I), p/n c>0 With d(c) =(1 c 1 )log(1 c) 1, log det S pd(c) D N( 1 2 log(1 c), 2log(1 c)) (1) trs p D N(0, 2c) Surprise: quality of approximation in (1) for p small (e.g. 2!) SEA 06@MIT p.17
18 Small p asymptotics Quantiles of Input Sample QQ Plot of Sample Data versus Standard Normal Standard Normal Quantiles n p qtile pfix pbais SEA 06@MIT p.18
19 CLT for Likelihood Ratio distribution Bai-Silverstein(2004) p f(l i ) p 1 Cov(X f,x g )= 1 2π 2 f(x)g MP (x)dx D X f N(EX f, Cov(X f )), Γ 1 f(z(m 1 ))g(z(m 2 )) Γ 2 (m 1 m 2 ) 2 dm 1 dm 2 CLT for null distribution of the LR test of H 0 :Σ=I, p (log l i l i +1) D N(pd(c)+ 1 2 log(1 c), 2[log(1 c) 1 c]). 1 SEA 06@MIT p.19
20 Linear Statistics: Double Wishart Hypothesis tests based on e-vals u i of H(H + E) 1,i.e. e-vals w i = u i /(1 u i ) of HE 1. Many standard tests are linear statistics S N (g) = p 1 g(u i): Wilks Λ: log Λ = p 1 log(1 u i) [Likelihood ratio test] Pillai s trace = p 1 u i Hotelling-Lawley trace = u i /(1 u i )= p 1 w i Roy s largest root = u (1). Basor-Chen (05) Unitary case, formal; N α, β fixed. S N (g) (2N + α + β)a g D N(0,bg g ), [a g = 1 2π R 1 1 g(x) q1 x 2 dx, b g g = 1 2π 2 R 1 1 g(x) q 1 x 2 P R 1 1 q 1 y 2 y x g(y)dydx] SEA 06@MIT p.20
21 Agenda Classical multivariate techniques Hypothesis Testing: Single and Double Wishart Eigenvalue densities Linear Statistics Largest Eigenvalue Single Wishart Double Wishart Concluding Remarks SEA p.21
22 Largest Eigenvalue - Single Wishart Usual approach to maxima is (classically) infeasible: {l (1) x} = p i=1 I{l i x} Key role: determinants, not independence: (l i l j )=det[li k 1 ] 1 i,k p i<j p I{l i x} = i=1 p ( ) p k ( 1) k I{l i >x}. k k=0 P { max 1 i p l i t} = i=1 det(i K p χ [t, ) ) K p (x, y) is (2 2 matrix) kernel uses {Laguerre, Jacobi} orthogonal polynomials via Christoffel-Darboux summation. SEA 06@MIT p.22
23 Tracy-Widom Limit For real (β =1,IMJ)orcomplex(β =2, Johansson) data, if n/p c (0, ): with F p (s) =P {l 1 µ np + σ np s} F β (s), µ np =( n + p) 2, σ np =( n + ( 1 p) n + 1 ) 1/3 p El Karoui (2004) In complex case, for refined µ np,σ np, F p (s) F 2 (s) Ce s p 2/3. Also, results for N,p separately, and under alternative hypotheses. SEA 06@MIT p.23
24 Painlevé II and Tracy-Widom Painlevé II: 0.8 q = xq +2q 3 q(x) Ai(x) as x Tracy-Widom distributions: F 2 (s) = exp{ F 1 (s) = (F 2 (s)) 1/2 exp{ 1 2 s (x s)q 2 (x)dx} s q(x)dx} SEA 06@MIT p.24
25 Largest Root - Double Wishart Assume p, q(p),n(p). Simply, γ p 2 =sin 1 r p.5 n 1, µ ± =cos 2 π 2 φ p ± γ p 2, σ 3 p+ = r φ p q.5 2 =sin 1 n 1. 1 sin 4 (φ p + γ p ) (2n 2) 2. sin φ p sin γ p u 1 µ + σ + D W1 F 1 More precisely, logit transform l(u) = log(u/(1 u)) : (IMJ,PJF) P {l(u 1 ) l(µ + )+sσ + l (µ + )} F 1 (s) Ce s/4 p 2/3 corrections (.5, 1, 2) improve approx n for p, q small, error is O(p 2/3 ) [insteadofo(p 1/3 )] SEA 06@MIT p.25
26 Approximation vs. Tables for p =5 Table vs. Approx at 95th %tile; mc = (q p 1)/2; nc = (n q p 1)/2 1 nc = nc = 5 Tables: William Chen, IRS, (2002) m c = q p 1 [0, 15], 2 n c = n q p 1 [1, 1000] 2 Squared Correlation nc = 10 nc = 20 nc = 40 Tracy Widom Chen Table p=5 nc = nc = mc SEA 06@MIT p.26
27 Remarks p 2/3 scale of variability for u 1 95th %tile. = µ p+ + σ p+, 99th %tile. = µ p+ +2σ p+ if µ p+ >.7, logit scale v i =logu i /(1 u i ) better. Smallest eigenvalue: with previous assumptions and γ 0 <φ 0, σp 3 1 sin = 4 (φ p γ p ) (2n 2) 2 sin φ p sin γ p then µ p u p σ p D W1 (W 2 ) Corresponding limit distributions for u 2 u k, u p k u p 1, k fixed SEA 06@MIT p.27
28 Agenda Classical multivariate techniques Hypothesis Testing: Single and Double Wishart Eigenvalue densities Linear Statistics Largest Eigenvalue Concluding Remarks SEA p.28
29 Concluding Remarks Numerous other topics deserve attention: distributions under alternative hypotheses: integral representations, matrix hypergeometric functions empirical distributions and graphical display (Wachter) computational advances: (Dumitriu, Edelman, Koev, Rao) operations on random matrices multivariate orthogonal polynomials matrix hypergeometric functions estimation and testing for eigenvectors (Paul) technical role for RMT in other statistical areas: e.g. via large deviations results SEA p.29
30 Back-Up Slides SEA p.30
31 Upper Bound in SAS Approximate n q q u 1 1 u 1 by F q,n q Table vs. Approx at 95th %tile; mc = (q p 1)/2; nc = (n q p 1)/2 1 nc = nc = 5 Squared Correlation nc = 10 nc = 20 Tracy Widom Chen Table p=5 nc = SAS 40 F approx nc = nc = mc SEA 06@MIT p.31
32 Testing Subsequent Correlations Suppose: Σ XY = ρ ρ 2 p 0 0 If largest r correlations are large, test H r :ρ r+1 = ρ r+2 =...= ρ p =0? Comparison Lemma (from SVD interlacing) L(u r+1 p, q, n; S XY H r ) st < L(u 1 p, q r, n; I) conservative P values for H r via TW(p, q r, n) approx n to RHS p q, n p [Aside: L(u 1 p r, q r, n; I) may be better, but no bounds] SEA 06@MIT p.32
Large sample covariance matrices and the T 2 statistic
Large sample covariance matrices and the T 2 statistic EURANDOM, the Netherlands Joint work with W. Zhou Outline 1 2 Basic setting Let {X ij }, i, j =, be i.i.d. r.v. Write n s j = (X 1j,, X pj ) T and
More informationUniversality of distribution functions in random matrix theory Arno Kuijlaars Katholieke Universiteit Leuven, Belgium
Universality of distribution functions in random matrix theory Arno Kuijlaars Katholieke Universiteit Leuven, Belgium SEA 06@MIT, Workshop on Stochastic Eigen-Analysis and its Applications, MIT, Cambridge,
More informationMultivariate Linear Models
Multivariate Linear Models Stanley Sawyer Washington University November 7, 2001 1. Introduction. Suppose that we have n observations, each of which has d components. For example, we may have d measurements
More informationPainlevé Representations for Distribution Functions for Next-Largest, Next-Next-Largest, etc., Eigenvalues of GOE, GUE and GSE
Painlevé Representations for Distribution Functions for Next-Largest, Next-Next-Largest, etc., Eigenvalues of GOE, GUE and GSE Craig A. Tracy UC Davis RHPIA 2005 SISSA, Trieste 1 Figure 1: Paul Painlevé,
More informationProgress in the method of Ghosts and Shadows for Beta Ensembles
Progress in the method of Ghosts and Shadows for Beta Ensembles Alan Edelman (MIT) Alex Dubbs (MIT) and Plamen Koev (SJS) Oct 8, 2012 1/47 Wishart Matrices (arbitrary covariance) G=mxn matrix of Gaussians
More informationSTT 843 Key to Homework 1 Spring 2018
STT 843 Key to Homework Spring 208 Due date: Feb 4, 208 42 (a Because σ = 2, σ 22 = and ρ 2 = 05, we have σ 2 = ρ 2 σ σ22 = 2/2 Then, the mean and covariance of the bivariate normal is µ = ( 0 2 and Σ
More informationChapter 7, continued: MANOVA
Chapter 7, continued: MANOVA The Multivariate Analysis of Variance (MANOVA) technique extends Hotelling T 2 test that compares two mean vectors to the setting in which there are m 2 groups. We wish to
More informationz = β βσβ Statistical Analysis of MV Data Example : µ=0 (Σ known) consider Y = β X~ N 1 (β µ, β Σβ) test statistic for H 0β is
Example X~N p (µ,σ); H 0 : µ=0 (Σ known) consider Y = β X~ N 1 (β µ, β Σβ) H 0β : β µ = 0 test statistic for H 0β is y z = β βσβ /n And reject H 0β if z β > c [suitable critical value] 301 Reject H 0 if
More informationAn Introduction to Multivariate Statistical Analysis
An Introduction to Multivariate Statistical Analysis Third Edition T. W. ANDERSON Stanford University Department of Statistics Stanford, CA WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION Contents
More informationExponential tail inequalities for eigenvalues of random matrices
Exponential tail inequalities for eigenvalues of random matrices M. Ledoux Institut de Mathématiques de Toulouse, France exponential tail inequalities classical theme in probability and statistics quantify
More informationConcentration Inequalities for Random Matrices
Concentration Inequalities for Random Matrices M. Ledoux Institut de Mathématiques de Toulouse, France exponential tail inequalities classical theme in probability and statistics quantify the asymptotic
More informationJournal of Multivariate Analysis. Sphericity test in a GMANOVA MANOVA model with normal error
Journal of Multivariate Analysis 00 (009) 305 3 Contents lists available at ScienceDirect Journal of Multivariate Analysis journal homepage: www.elsevier.com/locate/jmva Sphericity test in a GMANOVA MANOVA
More informationAnalysis of variance, multivariate (MANOVA)
Analysis of variance, multivariate (MANOVA) Abstract: A designed experiment is set up in which the system studied is under the control of an investigator. The individuals, the treatments, the variables
More information1 Intro to RMT (Gene)
M705 Spring 2013 Summary for Week 2 1 Intro to RMT (Gene) (Also see the Anderson - Guionnet - Zeitouni book, pp.6-11(?) ) We start with two independent families of R.V.s, {Z i,j } 1 i
More informationc 2005 Society for Industrial and Applied Mathematics
SIAM J. MATRIX ANAL. APPL. Vol. XX, No. X, pp. XX XX c 005 Society for Industrial and Applied Mathematics DISTRIBUTIONS OF THE EXTREME EIGENVALUES OF THE COMPLEX JACOBI RANDOM MATRIX ENSEMBLE PLAMEN KOEV
More informationFluctuations from the Semicircle Law Lecture 4
Fluctuations from the Semicircle Law Lecture 4 Ioana Dumitriu University of Washington Women and Math, IAS 2014 May 23, 2014 Ioana Dumitriu (UW) Fluctuations from the Semicircle Law Lecture 4 May 23, 2014
More informationCanonical Correlation Analysis of Longitudinal Data
Biometrics Section JSM 2008 Canonical Correlation Analysis of Longitudinal Data Jayesh Srivastava Dayanand N Naik Abstract Studying the relationship between two sets of variables is an important multivariate
More informationSTAT 730 Chapter 5: Hypothesis Testing
STAT 730 Chapter 5: Hypothesis Testing Timothy Hanson Department of Statistics, University of South Carolina Stat 730: Multivariate Analysis 1 / 28 Likelihood ratio test def n: Data X depend on θ. The
More informationExam 2. Jeremy Morris. March 23, 2006
Exam Jeremy Morris March 3, 006 4. Consider a bivariate normal population with µ 0, µ, σ, σ and ρ.5. a Write out the bivariate normal density. The multivariate normal density is defined by the following
More informationABOUT PRINCIPAL COMPONENTS UNDER SINGULARITY
ABOUT PRINCIPAL COMPONENTS UNDER SINGULARITY José A. Díaz-García and Raúl Alberto Pérez-Agamez Comunicación Técnica No I-05-11/08-09-005 (PE/CIMAT) About principal components under singularity José A.
More informationI L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
Canonical Edps/Soc 584 and Psych 594 Applied Multivariate Statistics Carolyn J. Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN Canonical Slide
More informationOn corrections of classical multivariate tests for high-dimensional data
On corrections of classical multivariate tests for high-dimensional data Jian-feng Yao with Zhidong Bai, Dandan Jiang, Shurong Zheng Overview Introduction High-dimensional data and new challenge in statistics
More informationHypothesis Testing. Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA
Hypothesis Testing Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA An Example Mardia et al. (979, p. ) reprint data from Frets (9) giving the length and breadth (in
More informationI L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
Comparisons of Several Multivariate Populations Edps/Soc 584 and Psych 594 Applied Multivariate Statistics Carolyn J Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS
More informationRegularized Estimation of High Dimensional Covariance Matrices. Peter Bickel. January, 2008
Regularized Estimation of High Dimensional Covariance Matrices Peter Bickel Cambridge January, 2008 With Thanks to E. Levina (Joint collaboration, slides) I. M. Johnstone (Slides) Choongsoon Bae (Slides)
More informationLecture 3. Inference about multivariate normal distribution
Lecture 3. Inference about multivariate normal distribution 3.1 Point and Interval Estimation Let X 1,..., X n be i.i.d. N p (µ, Σ). We are interested in evaluation of the maximum likelihood estimates
More informationInferences about a Mean Vector
Inferences about a Mean Vector Edps/Soc 584, Psych 594 Carolyn J. Anderson Department of Educational Psychology I L L I N O I S university of illinois at urbana-champaign c Board of Trustees, University
More informationTesting Some Covariance Structures under a Growth Curve Model in High Dimension
Department of Mathematics Testing Some Covariance Structures under a Growth Curve Model in High Dimension Muni S. Srivastava and Martin Singull LiTH-MAT-R--2015/03--SE Department of Mathematics Linköping
More informationEstimation of large dimensional sparse covariance matrices
Estimation of large dimensional sparse covariance matrices Department of Statistics UC, Berkeley May 5, 2009 Sample covariance matrix and its eigenvalues Data: n p matrix X n (independent identically distributed)
More informationMultivariate Analysis and Jacobi Ensembles: Largest eigenvalue, Tracy Widom Limits and Rates of Convergence
Multivariate Analysis and Jacobi Ensembles: Largest eigenvalue, Tracy Widom Limits and Rates of Convergence Iain M. Johnstone Stanford University ovember 18, 007 Abstract Let A and B be independent, central
More informationUpdate on the beta ensembles
Update on the beta ensembles Brian Rider Temple University with M. Krishnapur IISC, J. Ramírez Universidad Costa Rica, B. Virág University of Toronto The Tracy-Widom laws Consider a random Hermitian n
More informationMultivariate Regression
Multivariate Regression The so-called supervised learning problem is the following: we want to approximate the random variable Y with an appropriate function of the random variables X 1,..., X p with the
More informationMP 472 Quantum Information and Computation
MP 472 Quantum Information and Computation http://www.thphys.may.ie/staff/jvala/mp472.htm Outline Open quantum systems The density operator ensemble of quantum states general properties the reduced density
More informationA new test for the proportionality of two large-dimensional covariance matrices. Citation Journal of Multivariate Analysis, 2014, v. 131, p.
Title A new test for the proportionality of two large-dimensional covariance matrices Authors) Liu, B; Xu, L; Zheng, S; Tian, G Citation Journal of Multivariate Analysis, 04, v. 3, p. 93-308 Issued Date
More informationLinear Regression. In this problem sheet, we consider the problem of linear regression with p predictors and one intercept,
Linear Regression In this problem sheet, we consider the problem of linear regression with p predictors and one intercept, y = Xβ + ɛ, where y t = (y 1,..., y n ) is the column vector of target values,
More informationUniversal Fluctuation Formulae for one-cut β-ensembles
Universal Fluctuation Formulae for one-cut β-ensembles with a combinatorial touch Pierpaolo Vivo with F. D. Cunden Phys. Rev. Lett. 113, 070202 (2014) with F.D. Cunden and F. Mezzadri J. Phys. A 48, 315204
More informationMultivariate Gaussian Analysis
BS2 Statistical Inference, Lecture 7, Hilary Term 2009 February 13, 2009 Marginal and conditional distributions For a positive definite covariance matrix Σ, the multivariate Gaussian distribution has density
More informationTEST FOR INDEPENDENCE OF THE VARIABLES WITH MISSING ELEMENTS IN ONE AND THE SAME COLUMN OF THE EMPIRICAL CORRELATION MATRIX.
Serdica Math J 34 (008, 509 530 TEST FOR INDEPENDENCE OF THE VARIABLES WITH MISSING ELEMENTS IN ONE AND THE SAME COLUMN OF THE EMPIRICAL CORRELATION MATRIX Evelina Veleva Communicated by N Yanev Abstract
More informationMultivariate Analysis and Likelihood Inference
Multivariate Analysis and Likelihood Inference Outline 1 Joint Distribution of Random Variables 2 Principal Component Analysis (PCA) 3 Multivariate Normal Distribution 4 Likelihood Inference Joint density
More informationMULTIVARIATE ANALYSIS OF VARIANCE UNDER MULTIPLICITY José A. Díaz-García. Comunicación Técnica No I-07-13/ (PE/CIMAT)
MULTIVARIATE ANALYSIS OF VARIANCE UNDER MULTIPLICITY José A. Díaz-García Comunicación Técnica No I-07-13/11-09-2007 (PE/CIMAT) Multivariate analysis of variance under multiplicity José A. Díaz-García Universidad
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 4 for Applied Multivariate Analysis Outline 1 Eigen values and eigen vectors Characteristic equation Some properties of eigendecompositions
More informationHigh-dimensional asymptotic expansions for the distributions of canonical correlations
Journal of Multivariate Analysis 100 2009) 231 242 Contents lists available at ScienceDirect Journal of Multivariate Analysis journal homepage: www.elsevier.com/locate/jmva High-dimensional asymptotic
More information1 Data Arrays and Decompositions
1 Data Arrays and Decompositions 1.1 Variance Matrices and Eigenstructure Consider a p p positive definite and symmetric matrix V - a model parameter or a sample variance matrix. The eigenstructure is
More informationOn corrections of classical multivariate tests for high-dimensional data. Jian-feng. Yao Université de Rennes 1, IRMAR
Introduction a two sample problem Marčenko-Pastur distributions and one-sample problems Random Fisher matrices and two-sample problems Testing cova On corrections of classical multivariate tests for high-dimensional
More informationReview (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology
Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Some slides have been adopted from Prof. H.R. Rabiee s and also Prof. R. Gutierrez-Osuna
More informationExtreme eigenvalue fluctutations for GUE
Extreme eigenvalue fluctutations for GUE C. Donati-Martin 204 Program Women and Mathematics, IAS Introduction andom matrices were introduced in multivariate statistics, in the thirties by Wishart [Wis]
More informationRandom Vectors 1. STA442/2101 Fall See last slide for copyright information. 1 / 30
Random Vectors 1 STA442/2101 Fall 2017 1 See last slide for copyright information. 1 / 30 Background Reading: Renscher and Schaalje s Linear models in statistics Chapter 3 on Random Vectors and Matrices
More informationDeterminantal point processes and random matrix theory in a nutshell
Determinantal point processes and random matrix theory in a nutshell part II Manuela Girotti based on M. Girotti s PhD thesis, A. Kuijlaars notes from Les Houches Winter School 202 and B. Eynard s notes
More informationNumerical Solutions to the General Marcenko Pastur Equation
Numerical Solutions to the General Marcenko Pastur Equation Miriam Huntley May 2, 23 Motivation: Data Analysis A frequent goal in real world data analysis is estimation of the covariance matrix of some
More informationSTAT C206A / MATH C223A : Stein s method and applications 1. Lecture 31
STAT C26A / MATH C223A : Stein s method and applications Lecture 3 Lecture date: Nov. 7, 27 Scribe: Anand Sarwate Gaussian concentration recap If W, T ) is a pair of random variables such that for all
More informationMultivariate Regression (Chapter 10)
Multivariate Regression (Chapter 10) This week we ll cover multivariate regression and maybe a bit of canonical correlation. Today we ll mostly review univariate multivariate regression. With multivariate
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationComparisons of Several Multivariate Populations
Comparisons of Several Multivariate Populations Edps/Soc 584, Psych 594 Carolyn J Anderson Department of Educational Psychology I L L I N O I S university of illinois at urbana-champaign c Board of Trustees,
More informationNumerical Methods for Random Matrices
Numerical Methods for Random Matrices MIT 18.95 IAP Lecture Series Per-Olof Persson (persson@mit.edu) January 23, 26 1.9.8.7 Random Matrix Eigenvalue Distribution.7.6.5 β=1 β=2 β=4 Probability.6.5.4.4
More informationMarkov operators, classical orthogonal polynomial ensembles, and random matrices
Markov operators, classical orthogonal polynomial ensembles, and random matrices M. Ledoux, Institut de Mathématiques de Toulouse, France 5ecm Amsterdam, July 2008 recent study of random matrix and random
More informationLecture 6: Selection on Multiple Traits
Lecture 6: Selection on Multiple Traits Bruce Walsh lecture notes Introduction to Quantitative Genetics SISG, Seattle 16 18 July 2018 1 Genetic vs. Phenotypic correlations Within an individual, trait values
More informationTriangular matrices and biorthogonal ensembles
/26 Triangular matrices and biorthogonal ensembles Dimitris Cheliotis Department of Mathematics University of Athens UK Easter Probability Meeting April 8, 206 2/26 Special densities on R n Example. n
More informationMultivariate Linear Regression Models
Multivariate Linear Regression Models Regression analysis is used to predict the value of one or more responses from a set of predictors. It can also be used to estimate the linear association between
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationMATH5745 Multivariate Methods Lecture 07
MATH5745 Multivariate Methods Lecture 07 Tests of hypothesis on covariance matrix March 16, 2018 MATH5745 Multivariate Methods Lecture 07 March 16, 2018 1 / 39 Test on covariance matrices: Introduction
More informationCentral Limit Theorems for Classical Likelihood Ratio Tests for High-Dimensional Normal Distributions
Central Limit Theorems for Classical Likelihood Ratio Tests for High-Dimensional Normal Distributions Tiefeng Jiang 1 and Fan Yang 1, University of Minnesota Abstract For random samples of size n obtained
More informationCalculating determinants for larger matrices
Day 26 Calculating determinants for larger matrices We now proceed to define det A for n n matrices A As before, we are looking for a function of A that satisfies the product formula det(ab) = det A det
More informationTHE UNIVERSITY OF CHICAGO Booth School of Business Business 41912, Spring Quarter 2012, Mr. Ruey S. Tsay
THE UNIVERSITY OF CHICAGO Booth School of Business Business 41912, Spring Quarter 2012, Mr. Ruey S. Tsay Lecture 3: Comparisons between several multivariate means Key concepts: 1. Paired comparison & repeated
More informationExample 1 describes the results from analyzing these data for three groups and two variables contained in test file manova1.tf3.
Simfit Tutorials and worked examples for simulation, curve fitting, statistical analysis, and plotting. http://www.simfit.org.uk MANOVA examples From the main SimFIT menu choose [Statistcs], [Multivariate],
More informationCovariance estimation using random matrix theory
Covariance estimation using random matrix theory Randolf Altmeyer joint work with Mathias Trabs Mathematical Statistics, Humboldt-Universität zu Berlin Statistics Mathematics and Applications, Fréjus 03
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 17 for Applied Multivariate Analysis Outline Multivariate Analysis of Variance 1 Multivariate Analysis of Variance The hypotheses:
More informationTHE UNIVERSITY OF CHICAGO Graduate School of Business Business 41912, Spring Quarter 2008, Mr. Ruey S. Tsay. Solutions to Final Exam
THE UNIVERSITY OF CHICAGO Graduate School of Business Business 41912, Spring Quarter 2008, Mr. Ruey S. Tsay Solutions to Final Exam 1. (13 pts) Consider the monthly log returns, in percentages, of five
More informationMATH 320: PRACTICE PROBLEMS FOR THE FINAL AND SOLUTIONS
MATH 320: PRACTICE PROBLEMS FOR THE FINAL AND SOLUTIONS There will be eight problems on the final. The following are sample problems. Problem 1. Let F be the vector space of all real valued functions on
More informationEigenvalues and Singular Values of Random Matrices: A Tutorial Introduction
Random Matrix Theory and its applications to Statistics and Wireless Communications Eigenvalues and Singular Values of Random Matrices: A Tutorial Introduction Sergio Verdú Princeton University National
More informationAdvances in Random Matrix Theory: Let there be tools
Advances in Random Matrix Theory: Let there be tools Alan Edelman Brian Sutton, Plamen Koev, Ioana Dumitriu, Raj Rao and others MIT: Dept of Mathematics, Computer Science AI Laboratories World Congress,
More informationIr O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )
Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O
More informationPrincipal Component Analysis for a Spiked Covariance Model with Largest Eigenvalues of the Same Asymptotic Order of Magnitude
Principal Component Analysis for a Spiked Covariance Model with Largest Eigenvalues of the Same Asymptotic Order of Magnitude Addy M. Boĺıvar Cimé Centro de Investigación en Matemáticas A.C. May 1, 2010
More informationEigenvector stability: Random Matrix Theory and Financial Applications
Eigenvector stability: Random Matrix Theory and Financial Applications J.P Bouchaud with: R. Allez, M. Potters http://www.cfm.fr Portfolio theory: Basics Portfolio weights w i, Asset returns X t i If expected/predicted
More informationLecture 5: Hypothesis tests for more than one sample
1/23 Lecture 5: Hypothesis tests for more than one sample Måns Thulin Department of Mathematics, Uppsala University thulin@math.uu.se Multivariate Methods 8/4 2011 2/23 Outline Paired comparisons Repeated
More informationCS 143 Linear Algebra Review
CS 143 Linear Algebra Review Stefan Roth September 29, 2003 Introductory Remarks This review does not aim at mathematical rigor very much, but instead at ease of understanding and conciseness. Please see
More informationYou can compute the maximum likelihood estimate for the correlation
Stat 50 Solutions Comments on Assignment Spring 005. (a) _ 37.6 X = 6.5 5.8 97.84 Σ = 9.70 4.9 9.70 75.05 7.80 4.9 7.80 4.96 (b) 08.7 0 S = Σ = 03 9 6.58 03 305.6 30.89 6.58 30.89 5.5 (c) You can compute
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 9 for Applied Multivariate Analysis Outline Two sample T 2 test 1 Two sample T 2 test 2 Analogous to the univariate context, we
More informationDepartment of Statistics
Research Report Department of Statistics Research Report Department of Statistics No. 05: Testing in multivariate normal models with block circular covariance structures Yuli Liang Dietrich von Rosen Tatjana
More informationComparison Method in Random Matrix Theory
Comparison Method in Random Matrix Theory Jun Yin UW-Madison Valparaíso, Chile, July - 2015 Joint work with A. Knowles. 1 Some random matrices Wigner Matrix: H is N N square matrix, H : H ij = H ji, EH
More informationAn introduction to multivariate data
An introduction to multivariate data Angela Montanari 1 The data matrix The starting point of any analysis of multivariate data is a data matrix, i.e. a collection of n observations on a set of p characters
More informationVectors and Matrices Statistics with Vectors and Matrices
Vectors and Matrices Statistics with Vectors and Matrices Lecture 3 September 7, 005 Analysis Lecture #3-9/7/005 Slide 1 of 55 Today s Lecture Vectors and Matrices (Supplement A - augmented with SAS proc
More informationThe purpose of this section is to derive the asymptotic distribution of the Pearson chi-square statistic. k (n j np j ) 2. np j.
Chapter 9 Pearson s chi-square test 9. Null hypothesis asymptotics Let X, X 2, be independent from a multinomial(, p) distribution, where p is a k-vector with nonnegative entries that sum to one. That
More informationThere are six more problems on the next two pages
Math 435 bg & bu: Topics in linear algebra Summer 25 Final exam Wed., 8/3/5. Justify all your work to receive full credit. Name:. Let A 3 2 5 Find a permutation matrix P, a lower triangular matrix L with
More informationSingular Value Decomposition and Principal Component Analysis (PCA) I
Singular Value Decomposition and Principal Component Analysis (PCA) I Prof Ned Wingreen MOL 40/50 Microarray review Data per array: 0000 genes, I (green) i,i (red) i 000 000+ data points! The expression
More informationEcon 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationInsights into Large Complex Systems via Random Matrix Theory
Insights into Large Complex Systems via Random Matrix Theory Lu Wei SEAS, Harvard 06/23/2016 @ UTC Institute for Advanced Systems Engineering University of Connecticut Lu Wei Random Matrix Theory and Large
More information1. Introduction to Multivariate Analysis
1. Introduction to Multivariate Analysis Isabel M. Rodrigues 1 / 44 1.1 Overview of multivariate methods and main objectives. WHY MULTIVARIATE ANALYSIS? Multivariate statistical analysis is concerned with
More informationThe largest eigenvalues of the sample covariance matrix. in the heavy-tail case
The largest eigenvalues of the sample covariance matrix 1 in the heavy-tail case Thomas Mikosch University of Copenhagen Joint work with Richard A. Davis (Columbia NY), Johannes Heiny (Aarhus University)
More informationRandom Correlation Matrices, Top Eigenvalue with Heavy Tails and Financial Applications
Random Correlation Matrices, Top Eigenvalue with Heavy Tails and Financial Applications J.P Bouchaud with: M. Potters, G. Biroli, L. Laloux, M. A. Miceli http://www.cfm.fr Portfolio theory: Basics Portfolio
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 9 for Applied Multivariate Analysis Outline Addressing ourliers 1 Addressing ourliers 2 Outliers in Multivariate samples (1) For
More informationHYPOTHESIS TESTING ON LINEAR STRUCTURES OF HIGH DIMENSIONAL COVARIANCE MATRIX
Submitted to the Annals of Statistics HYPOTHESIS TESTING ON LINEAR STRUCTURES OF HIGH DIMENSIONAL COVARIANCE MATRIX By Shurong Zheng, Zhao Chen, Hengjian Cui and Runze Li Northeast Normal University, Fudan
More informationNon white sample covariance matrices.
Non white sample covariance matrices. S. Péché, Université Grenoble 1, joint work with O. Ledoit, Uni. Zurich 17-21/05/2010, Université Marne la Vallée Workshop Probability and Geometry in High Dimensions
More informationLocal semicircle law, Wegner estimate and level repulsion for Wigner random matrices
Local semicircle law, Wegner estimate and level repulsion for Wigner random matrices László Erdős University of Munich Oberwolfach, 2008 Dec Joint work with H.T. Yau (Harvard), B. Schlein (Cambrigde) Goal:
More informationCointegrated VAR s. Eduardo Rossi University of Pavia. November Rossi Cointegrated VAR s Financial Econometrics / 56
Cointegrated VAR s Eduardo Rossi University of Pavia November 2013 Rossi Cointegrated VAR s Financial Econometrics - 2013 1 / 56 VAR y t = (y 1t,..., y nt ) is (n 1) vector. y t VAR(p): Φ(L)y t = ɛ t The
More informationAdvanced Econometrics I
Lecture Notes Autumn 2010 Dr. Getinet Haile, University of Mannheim 1. Introduction Introduction & CLRM, Autumn Term 2010 1 What is econometrics? Econometrics = economic statistics economic theory mathematics
More informationRandom Matrix Theory Lecture 1 Introduction, Ensembles and Basic Laws. Symeon Chatzinotas February 11, 2013 Luxembourg
Random Matrix Theory Lecture 1 Introduction, Ensembles and Basic Laws Symeon Chatzinotas February 11, 2013 Luxembourg Outline 1. Random Matrix Theory 1. Definition 2. Applications 3. Asymptotics 2. Ensembles
More informationPeter Hoff Linear and multilinear models April 3, GLS for multivariate regression 5. 3 Covariance estimation for the GLM 8
Contents 1 Linear model 1 2 GLS for multivariate regression 5 3 Covariance estimation for the GLM 8 4 Testing the GLH 11 A reference for some of this material can be found somewhere. 1 Linear model Recall
More informationPrincipal component analysis and the asymptotic distribution of high-dimensional sample eigenvectors
Principal component analysis and the asymptotic distribution of high-dimensional sample eigenvectors Kristoffer Hellton Department of Mathematics, University of Oslo May 12, 2015 K. Hellton (UiO) Distribution
More information9.1 Orthogonal factor model.
36 Chapter 9 Factor Analysis Factor analysis may be viewed as a refinement of the principal component analysis The objective is, like the PC analysis, to describe the relevant variables in study in terms
More information