Random Matrix Theory and its Applications to Econometrics
|
|
- Lionel Owens
- 5 years ago
- Views:
Transcription
1 Random Matrix Theory and its Applications to Econometrics Hyungsik Roger Moon University of Southern California Conference to Celebrate Peter Phillips 40 Years at Yale, October 2018
2 Spectral Analysis of Large Dimensional Random Matrices The main goal of the Random Matrix Theory is to provide understanding of the diverse properties (most notably, statistics of matrix eigenvalues) of matrices with entries drawn randomly from various probability distributions traditionally referred to as the random matrix ensembles. from Scholarpedia. Dated back to Wishart and James in statistics. Wigner - studied the spectra of large dimensinoal random matrices in nuclear physics.
3 Empirical Spectral Distribution Empirical spectral distribution (ESD): Let A be an N N symmetric matrix and µ r (A) be the r th largest eigenvalue of A. The ESD of A is 1 N N I(µ r (A) x). r=1 Finite sample properties of ESD are hard to derive when the dimension of A is large. Find the limiting properties of the ESD. Suppose that e is the N T matrix of iid N(0, 1). Let µ r (e e) be the largest eigenvalue of e. Let κ := lim N T. F e e (x) := 1 N I(µ r (e e) Nx) F (x), T r=1 where F (x) =: f (x) = κ 2πx (b x)(x a), b = (1 + κ 1/2 ) 2, and a = (1 κ 1/2 ) 2. Wigner s semicircular law, Bai and Yin (1988), Marcenko and Pastur (1967), and Postur (1972, 1973),...
4 The Largest Eigenvalue Bounds on extreme eigenvalues. Suppose that e is the N T matrix of iid N(0, 1). Let λ 1(e e) be the largest eigenvalue of e. Let κ := lim N T. Starting from German (1980), many researchers derived the limit of the largest eigenvalue of the sample covariance matrix. For example, German (1980): 1 N λ1(e e) a.s. (1 + κ 1/2 ) 2 Johnstone (2001) and Soshnikov (2002): the properly normalized λ 1(e e) converge to the TracyWidom law, λ 1(e e) µ σ W 1, where µ = ( N 1 + T ) 2 and σ = ( N 1 + T )(1/ N 1 + 1/ T ) 1/3.
5 Expectation of Operator Norm of Random Matrix Suppose that u it are independent across i, t, and E(u it ) = 0 Let U = [u it ] R N T, the N T random matrix consisting of u it. Let U be the operator norm of U, that is, the largest singular value of U, or equivalently the square root of the largest eigenvalue of U U: U = ( 1/2 sup v U Uv) = sup w Uv. v =1 w =1, v =1 One of the well known results in the random matrix theory is E( U ) max(n, T ) if sup i,t E(u 4 it) M. (e.g., Latala (2005)). This implies that U = O p( max(n, T )).
6 Expectation of Operator Norm of Random Matrix: Extension I Suppose that ε it are independent across i, t with Eε it = 0, Eε 2 it = 1, and Eε 4 it uniformly bounded. (i) The u it MA( ) u it = ψ iτ ε i,t τ, for i = 1... N, t = 1... T, (1) τ=0 and the coefficients ψ iτ satisfy τ τ=0 max i=1...n ψ2 iτ < B, τ=0 max ψ iτ < B, (2) i=1...n for a finite constant B which is independent of N and T.
7 Expectation of Operator Norm of Random Matrix: Extension II (ii) The error matrix U is generated as U = σ 1/2 ε Σ 1/2. Here σ is the N N cross-sectional covariance matrix, and Σ is the T T time-serial covariance matrix, and they satisfy max j=1...n N i=1 σ ij < B, max τ=1...t T Σ tτ < B, (3) for some finite constant B which is independent of N and T. In this example we have Eu it u jτ = σ ij Σ tτ. See Moon and Weidner (2017, Appendix). t=1
8 Application 1: Factor Analysis Suppose that Y = λf + U, where λ R N R, f R T R λ and plim λ N N, plim T f f T > 0. Estimation R when N, T : Bai and Ng (2002), Onatski (2006), Ahn and Horenstein (2013) among others. We can estimate the number of the factors using the singular values of the observed samples in Y.
9 Factor Analysis: Cont d - I Let s i (A) be the i th largest singular of matrix A. Suppose that A and B are two matrices of dimension N T. For i = 1,..., n, we have s i (A + B) s i (A) s 1(B) (= B ). (Ky Fan s singular value inequality) Y = λf + U. We have rank(λf ) = R. So s i (λf ), if i R s i (λf ) = 0, if i R + 1. If U satisfies the previous conditions, U = O p( max(n, T )).
10 Factor Analysis: Cont d - II Therefore, ( ) Y s R ( ) Y s R+1 ( ) λf + U = s R ( ) λf s R U ( ) ( ) λf 1 s R O p min(n, T ) ( ) λf + U = s R ( ) 1 O p. min(n, T ) Choose ψ 0 with ψ min(n, T ). Then, R = min(n,t ) i=1 ( I ( ) ) Y s r ψ
11 Factor Analysis: Cont d - III This hints that we can distinguish weak factors or weak factor loadings (or local factor) whose variation is larger then max(n, T ).
12 Application 2: Interactive Fixed Effects Panel Regression with Missing Observations Moon and Weidner (2018). The latent model is Y = β 0X + Γ 0 + U rank(γ 0) = R 0 is fixed, that is, Γ 0 = λ 0f 0. S it : random selection dummy. Observe only (S it Y it, S it X it ). Goal: Estimate β 0. Computational Difficulty: min β,λ,f 1 2 i,t S it(y it βx it λ i f t) 2 is a non-convex optimization problem. Due to missing observations, we cannot use the principle component method of Bai (2009) and Moon and Weidner (2015,2017) No closed form profile objective function.
13 Nuclear Norm Reguralization Regularization: β ψ := argmin β min Γ 1 2 =: argmin Q ψ (β). β S it (Y it βx it Γ it ) 2 + Here Γ = min(n,t ) r=1 s r (Γ) = sup A 1 Tr(Γ A) is the nuclear norm of A. i,t Q ψ (β) is a convex function in β. ψ Γ Estimation of the reduced rank matrix Γ with missing observations: matrix completion problem (e.g., Netflix problem) in CS and ML literature. The parameter of interest is β 0. For simplicity, assume that X it has mean zero, and... Assume that E(S it ) = b > 0 and E(X 2 it ) > 0.
14 Consistency of β ψ - I Choose ψ such that ψ 0 and min(n, T )ψ 1/2. Let B ψ := {β B : β β 0 = Mψ 1/2 }. Since Q ψ (β) is convex, if argmin β Bψ Q ψ (β) Q ψ (β 0, Γ 0 ) > 0, then β ψ β 0 Mψ 1/2. A low bound of Q ψ (β) : By definition, for any A with A 1, S it (Y it βx it Γ it ) 2 + ψ Γ i,t S it (U it (β β 0 )X it (Γ it Γ 0,it )) 2 + i,t Choose A = 1 ψ (S it U it S it X it (β β 0 )). Then, A 1 1 [S it U it ] + β β 0 [S it X it ] ψ ψ ψ Tr(Γ A). ( ) ( ) 1 M O p + O p = o min(n, T )ψ min(n, T )ψ 1/2 p(1).
15 Consistency of β ψ - II Then, Q ψ (β) Q ψ (β 0, Γ 0 ) = min 1 S it (Y it βx it Γ it ) 2 + ψ Γ Γ 2 i,t min 1 S it (U it (β β 0 )X it (Γ it Γ 0,it )) 2 + ψ Tr(Γ A) Γ 2 i,t = 1 2 S it (U it (β β 0 )X it ) 2 + i,t 1 S it (U it (β β 0 )X it ) 2 2 i,t M S it U it X it i,t ψ1/2 + M2 i,t > 0 wp1 by choosing M large ψ Tr(Γ 0 A) ψ Γ 0 A 1 2 S it X 2 it ψ ψ Γ 0 S it Uit 2 i,t
16 Extension: Expectation of Operator Norm of Random Element Matrix Let u it (β) be a sequence of stochastic processes indexed by β B. Assume that the index set B is equipped with a pseudo metric d β (, ). Assume that u it (β) l (B) are independent across i, t. U(β) = [u it (β)] l (B) N T. Define U(β) B := sup U(β) = sup w U(β)v. β B β B, w =1, v =1 The main contribution: establish a bound of E( U(β) B ). We use some known results - covering numbers, chaining argument, concentration inequalities,...
17 Assumptions Assumptions (i) There exists a finite constant σ 2 such that sup log E (exp (λu it (β))) λ 2 σ 2 /2 β B for all λ > 0. (ii) For all (i, t) and (β 1, β 2 ) B B, ( x 2 ) P { u it (β 1 ) u it (β 2 ) > x} 2 exp 2d β (β 1, β 2 ) 2. (iii) The index set B is totally bounded with respect to d β (, ). Under the Assumption (i), sup β B E [max i,t u it (β) ] 2σ 2 log( ). u it (β) is a sub-gaussian process.
18 Main Result Suppose that N β (B, d β, ɛ) is the covering number of (B, d β ). Theorem E [ U(β) B ] log(max(n, T )) max(n, T ) + diam(b) 0 + max(n, T ) log N β (B, d β, ν)dν diam(b) 0 log N β (B, d β, ν)dν. If N β (B, d β, ɛ) ɛ K for some finite K, then E [ U(β) B ] log(max(n, T )) max(n, T ).
19 Sketch of Proof I Let g it iid N(0, 1), and independent of u it (β). Let G := [g it ]. Let Z(β) := U(β) G, the element by element product (the Hadamard product) of U(β) and G. Step 1: Using the symmetrization argument, we show that there exists a finite constant C such that E U(β) B C E Z(β) B. Let W := {x R N : x = 1} and V := {x R T : x = 1}. Let θ := (β, w, v ) Θ, where Θ := B W V. by definition we can express Z(β) B := sup sup β B w W, v V w Z(β)v. Let N w and N v be the ɛ-nets of (W, d w ) and (V, d v ), respectively.
20 Sketch of Proof II Step2: From w Z(β)v being bi-linear in w, v and by definition, we show ( ) ( ) sup Z(β) 2ɛ sup Z(β) + sup max w Z(β)v. β B β B β B (w,v) N w N v From this, we deduce [ ] E sup Z(β) β B [ 1 ( 1 2ɛ E sup β B max w Z(β)v (w,v) N w N v ) ]. Apply the chaining argument (to control sup β B ) and the maximal inequality (to control max (w,v) Nw N v ). Also, we use the fact that N w (W, d w, ɛ) Step 3 We deduce the desired result. ( ) N 3, N v (V, d v, ɛ) ɛ ( ) T 3. ɛ
21 Application: Least Operator Norm Estimation I Suppose that ε it (β) are moment functions such that E(ε it (β 0)) = 0. For simpliticy, assume that ε it (β) R and β B R. Let ε(β) = [ε it (β)], the N T matrix of moment functions. The conventional method of moment estimator solves 1 ( ) β = argmin ε it (β) β B = argmin 1 N ε(β) 1T, N T i,t where 1 N is the N-vector of ones. The least operator norm estimator minimizes the operator norm of the moment function matrix ε(β), β := argmin β B = argmin β B ε(β) max w =1, v =1 β B ( ) ε(β) w v.
22 Consistency I For consistency of ˆβ, we need the following conditions. (i) B is bounded. (ii) ε it (β) E(ε it (β)) satisfies the sub-gaussian process assumption. (iii) For any ɛ > 0, there exists δ > 0 such that inf β β0 ɛ E(ɛ(β)) > 2δ. The consistency of β follows if for any ɛ > 0, there exists δ > 0 such that ε(β) inf ε(β0) > δ wp1. (4) β β 0 ɛ Notice by the triangle inequailty, ε(β) E(ε(β)) ε(β) E(ε(β)) inf inf sup β β 0 ɛ β β 0 ɛ β B E(ε(β)) = inf + o p(1) β β 0 ɛ δ wp1. This shows that the least operator norm estimator β is consistent.
23 Extensions I If ε it (β) are iid, E(ε(β)) = E(ε it (β) 1 1 N T = E(εit (β). In this case, the identification condition (iii) becomes the usual identificaiton condition that for any ɛ > 0, there exists δ > 0 such that inf β β0 ɛ E(ε it (β)) > 2δ. Suppose that ε it (β) = (ε 1,it (β),..., ε L,it (β)) R L. For the least operator norm objective function, we may consider L l=1 ω l ε l (β). Suppose that R slowly satifying objective function: R 1 s r (ε(β)), r=1 R 0. Extend the min(n,t ) where s r (A) is the r th largest singular valeu of matrix A.
24 Extensions II Since ε(β) E(ε(β)) = s 1(ε(β) E(ε(β))), we have sup β B R 1 r=1 s r (ε(β) E(ε(β))) ( ) ε(β) E(ε(β)) R R = O p = o p(1). min(n, T )
25 Conclusion In today s talk, we derived a uniform bound of the expectation of the operator norm of random element matrix. As an application, we invesigate a new estimator that minimizes the operator norm of the moment function matrix, and show its consistency. Future study: convergence rate and asymptotic distribution.
NUCLEAR NORM PENALIZED ESTIMATION OF INTERACTIVE FIXED EFFECT MODELS. Incomplete and Work in Progress. 1. Introduction
NUCLEAR NORM PENALIZED ESTIMATION OF IERACTIVE FIXED EFFECT MODELS HYUNGSIK ROGER MOON AND MARTIN WEIDNER Incomplete and Work in Progress. Introduction Interactive fixed effects panel regression models
More informationLikelihood Expansion for Panel Regression Models with Factors
Likelihood Expansion for Panel Regression Models with Factors Hyungsik Roger Moon Martin Weidner May 2009 Abstract In this paper we provide a new methodology to analyze the Gaussian profile quasi likelihood
More informationLikelihood Expansion for Panel Regression Models with Factors
Likelihood Expansion for Panel Regression Models with Factors Hyungsik Roger Moon Martin Weidner April 29 Abstract In this paper we provide a new methodology to analyze the Gaussian profile quasi likelihood
More informationConcentration Inequalities for Random Matrices
Concentration Inequalities for Random Matrices M. Ledoux Institut de Mathématiques de Toulouse, France exponential tail inequalities classical theme in probability and statistics quantify the asymptotic
More informationAssessing the dependence of high-dimensional time series via sample autocovariances and correlations
Assessing the dependence of high-dimensional time series via sample autocovariances and correlations Johannes Heiny University of Aarhus Joint work with Thomas Mikosch (Copenhagen), Richard Davis (Columbia),
More informationLarge sample covariance matrices and the T 2 statistic
Large sample covariance matrices and the T 2 statistic EURANDOM, the Netherlands Joint work with W. Zhou Outline 1 2 Basic setting Let {X ij }, i, j =, be i.i.d. r.v. Write n s j = (X 1j,, X pj ) T and
More informationRandom Matrices: Beyond Wigner and Marchenko-Pastur Laws
Random Matrices: Beyond Wigner and Marchenko-Pastur Laws Nathan Noiry Modal X, Université Paris Nanterre May 3, 2018 Wigner s Matrices Wishart s Matrices Generalizations Wigner s Matrices ij, (n, i, j
More informationInvertibility of random matrices
University of Michigan February 2011, Princeton University Origins of Random Matrix Theory Statistics (Wishart matrices) PCA of a multivariate Gaussian distribution. [Gaël Varoquaux s blog gael-varoquaux.info]
More informationLinear regression for panel with unknown number of factors as interactive fixed effects
Linear regression for panel with unknown number of factors as interactive fixed effects Hyungsik Roger Moon Martin Weidner The Institute for Fiscal Studies Department of Economics, UCL cemmap working paper
More informationLecture I: Asymptotics for large GUE random matrices
Lecture I: Asymptotics for large GUE random matrices Steen Thorbjørnsen, University of Aarhus andom Matrices Definition. Let (Ω, F, P) be a probability space and let n be a positive integer. Then a random
More information1-Bit Matrix Completion
1-Bit Matrix Completion Mark A. Davenport School of Electrical and Computer Engineering Georgia Institute of Technology Yaniv Plan Mary Wootters Ewout van den Berg Matrix Completion d When is it possible
More informationNuclear Norm Regularized Estimation of Panel Regression Models
Nuclear Norm Regularized Estimation of Panel Regression Models Hyungsik Roger Moon Martin Weidner February 3, 08 Abstract In this paper we investigate panel regression models with interactive fixed effects.
More informationLinear Regression for Panel with Unknown Number of Factors as Interactive Fixed Effects
Linear Regression for Panel with Unknown Number of Factors as Interactive Fixed Effects Hyungsik Roger Moon Martin Weidner December 26, 2014 Abstract In this paper we study the least squares (LS) estimator
More informationRandom matrices: Distribution of the least singular value (via Property Testing)
Random matrices: Distribution of the least singular value (via Property Testing) Van H. Vu Department of Mathematics Rutgers vanvu@math.rutgers.edu (joint work with T. Tao, UCLA) 1 Let ξ be a real or complex-valued
More informationHomework 1. Yuan Yao. September 18, 2011
Homework 1 Yuan Yao September 18, 2011 1. Singular Value Decomposition: The goal of this exercise is to refresh your memory about the singular value decomposition and matrix norms. A good reference to
More informationPCA with random noise. Van Ha Vu. Department of Mathematics Yale University
PCA with random noise Van Ha Vu Department of Mathematics Yale University An important problem that appears in various areas of applied mathematics (in particular statistics, computer science and numerical
More informationThe circular law. Lewis Memorial Lecture / DIMACS minicourse March 19, Terence Tao (UCLA)
The circular law Lewis Memorial Lecture / DIMACS minicourse March 19, 2008 Terence Tao (UCLA) 1 Eigenvalue distributions Let M = (a ij ) 1 i n;1 j n be a square matrix. Then one has n (generalised) eigenvalues
More information1 Intro to RMT (Gene)
M705 Spring 2013 Summary for Week 2 1 Intro to RMT (Gene) (Also see the Anderson - Guionnet - Zeitouni book, pp.6-11(?) ) We start with two independent families of R.V.s, {Z i,j } 1 i
More informationEstimation of the Global Minimum Variance Portfolio in High Dimensions
Estimation of the Global Minimum Variance Portfolio in High Dimensions Taras Bodnar, Nestor Parolya and Wolfgang Schmid 07.FEBRUARY 2014 1 / 25 Outline Introduction Random Matrix Theory: Preliminary Results
More informationRandom regular digraphs: singularity and spectrum
Random regular digraphs: singularity and spectrum Nick Cook, UCLA Probability Seminar, Stanford University November 2, 2015 Universality Circular law Singularity probability Talk outline 1 Universality
More informationExponential tail inequalities for eigenvalues of random matrices
Exponential tail inequalities for eigenvalues of random matrices M. Ledoux Institut de Mathématiques de Toulouse, France exponential tail inequalities classical theme in probability and statistics quantify
More informationFluctuations from the Semicircle Law Lecture 4
Fluctuations from the Semicircle Law Lecture 4 Ioana Dumitriu University of Washington Women and Math, IAS 2014 May 23, 2014 Ioana Dumitriu (UW) Fluctuations from the Semicircle Law Lecture 4 May 23, 2014
More informationStatistics & Data Sciences: First Year Prelim Exam May 2018
Statistics & Data Sciences: First Year Prelim Exam May 2018 Instructions: 1. Do not turn this page until instructed to do so. 2. Start each new question on a new sheet of paper. 3. This is a closed book
More informationRandom Matrix Theory Lecture 1 Introduction, Ensembles and Basic Laws. Symeon Chatzinotas February 11, 2013 Luxembourg
Random Matrix Theory Lecture 1 Introduction, Ensembles and Basic Laws Symeon Chatzinotas February 11, 2013 Luxembourg Outline 1. Random Matrix Theory 1. Definition 2. Applications 3. Asymptotics 2. Ensembles
More informationDynamic Linear Panel Regression Models with Interactive Fixed Effects
Dynamic Linear Panel Regression Models with Interactive Fixed Effects Hyungsik Roger Moon Martin Weidner The Institute for Fiscal Studies Department of Economics, UCL cemmap working paper CWP47/4 Dynamic
More informationOn the distinguishability of random quantum states
1 1 Department of Computer Science University of Bristol Bristol, UK quant-ph/0607011 Distinguishing quantum states Distinguishing quantum states This talk Question Consider a known ensemble E of n quantum
More informationOn corrections of classical multivariate tests for high-dimensional data
On corrections of classical multivariate tests for high-dimensional data Jian-feng Yao with Zhidong Bai, Dandan Jiang, Shurong Zheng Overview Introduction High-dimensional data and new challenge in statistics
More informationsparse and low-rank tensor recovery Cubic-Sketching
Sparse and Low-Ran Tensor Recovery via Cubic-Setching Guang Cheng Department of Statistics Purdue University www.science.purdue.edu/bigdata CCAM@Purdue Math Oct. 27, 2017 Joint wor with Botao Hao and Anru
More informationA note on a Marčenko-Pastur type theorem for time series. Jianfeng. Yao
A note on a Marčenko-Pastur type theorem for time series Jianfeng Yao Workshop on High-dimensional statistics The University of Hong Kong, October 2011 Overview 1 High-dimensional data and the sample covariance
More informationComparison Method in Random Matrix Theory
Comparison Method in Random Matrix Theory Jun Yin UW-Madison Valparaíso, Chile, July - 2015 Joint work with A. Knowles. 1 Some random matrices Wigner Matrix: H is N N square matrix, H : H ij = H ji, EH
More informationOn corrections of classical multivariate tests for high-dimensional data. Jian-feng. Yao Université de Rennes 1, IRMAR
Introduction a two sample problem Marčenko-Pastur distributions and one-sample problems Random Fisher matrices and two-sample problems Testing cova On corrections of classical multivariate tests for high-dimensional
More informationA new method to bound rate of convergence
A new method to bound rate of convergence Arup Bose Indian Statistical Institute, Kolkata, abose@isical.ac.in Sourav Chatterjee University of California, Berkeley Empirical Spectral Distribution Distribution
More informationInference For High Dimensional M-estimates. Fixed Design Results
: Fixed Design Results Lihua Lei Advisors: Peter J. Bickel, Michael I. Jordan joint work with Peter J. Bickel and Noureddine El Karoui Dec. 8, 2016 1/57 Table of Contents 1 Background 2 Main Results and
More informationModel Selection and Geometry
Model Selection and Geometry Pascal Massart Université Paris-Sud, Orsay Leipzig, February Purpose of the talk! Concentration of measure plays a fundamental role in the theory of model selection! Model
More informationUpper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices 1
Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices 1 Feng Wei 2 University of Michigan July 29, 2016 1 This presentation is based a project under the supervision of M. Rudelson.
More informationThe properties of L p -GMM estimators
The properties of L p -GMM estimators Robert de Jong and Chirok Han Michigan State University February 2000 Abstract This paper considers Generalized Method of Moment-type estimators for which a criterion
More informationPreface to the Second Edition...vii Preface to the First Edition... ix
Contents Preface to the Second Edition...vii Preface to the First Edition........... ix 1 Introduction.............................................. 1 1.1 Large Dimensional Data Analysis.........................
More informationRandom Correlation Matrices, Top Eigenvalue with Heavy Tails and Financial Applications
Random Correlation Matrices, Top Eigenvalue with Heavy Tails and Financial Applications J.P Bouchaud with: M. Potters, G. Biroli, L. Laloux, M. A. Miceli http://www.cfm.fr Portfolio theory: Basics Portfolio
More informationGaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
More information1-Bit Matrix Completion
1-Bit Matrix Completion Mark A. Davenport School of Electrical and Computer Engineering Georgia Institute of Technology Yaniv Plan Mary Wootters Ewout van den Berg Matrix Completion d When is it possible
More informationDissertation Defense
Clustering Algorithms for Random and Pseudo-random Structures Dissertation Defense Pradipta Mitra 1 1 Department of Computer Science Yale University April 23, 2008 Mitra (Yale University) Dissertation
More informationRandom Methods for Linear Algebra
Gittens gittens@acm.caltech.edu Applied and Computational Mathematics California Institue of Technology October 2, 2009 Outline The Johnson-Lindenstrauss Transform 1 The Johnson-Lindenstrauss Transform
More informationChapter 4: Asymptotic Properties of the MLE
Chapter 4: Asymptotic Properties of the MLE Daniel O. Scharfstein 09/19/13 1 / 1 Maximum Likelihood Maximum likelihood is the most powerful tool for estimation. In this part of the course, we will consider
More informationMulti-stage convex relaxation approach for low-rank structured PSD matrix recovery
Multi-stage convex relaxation approach for low-rank structured PSD matrix recovery Department of Mathematics & Risk Management Institute National University of Singapore (Based on a joint work with Shujun
More informationarxiv: v5 [math.na] 16 Nov 2017
RANDOM PERTURBATION OF LOW RANK MATRICES: IMPROVING CLASSICAL BOUNDS arxiv:3.657v5 [math.na] 6 Nov 07 SEAN O ROURKE, VAN VU, AND KE WANG Abstract. Matrix perturbation inequalities, such as Weyl s theorem
More informationPainlevé Representations for Distribution Functions for Next-Largest, Next-Next-Largest, etc., Eigenvalues of GOE, GUE and GSE
Painlevé Representations for Distribution Functions for Next-Largest, Next-Next-Largest, etc., Eigenvalues of GOE, GUE and GSE Craig A. Tracy UC Davis RHPIA 2005 SISSA, Trieste 1 Figure 1: Paul Painlevé,
More informationThe norm of polynomials in large random matrices
The norm of polynomials in large random matrices Camille Mâle, École Normale Supérieure de Lyon, Ph.D. Student under the direction of Alice Guionnet. with a significant contribution by Dimitri Shlyakhtenko.
More informationLecture 2. We now introduce some fundamental tools in martingale theory, which are useful in controlling the fluctuation of martingales.
Lecture 2 1 Martingales We now introduce some fundamental tools in martingale theory, which are useful in controlling the fluctuation of martingales. 1.1 Doob s inequality We have the following maximal
More informationThe largest eigenvalues of the sample covariance matrix. in the heavy-tail case
The largest eigenvalues of the sample covariance matrix 1 in the heavy-tail case Thomas Mikosch University of Copenhagen Joint work with Richard A. Davis (Columbia NY), Johannes Heiny (Aarhus University)
More informationDistributed Optimization over Networks Gossip-Based Algorithms
Distributed Optimization over Networks Gossip-Based Algorithms Angelia Nedić angelia@illinois.edu ISE Department and Coordinated Science Laboratory University of Illinois at Urbana-Champaign Outline Random
More information(Part 1) High-dimensional statistics May / 41
Theory for the Lasso Recall the linear model Y i = p j=1 β j X (j) i + ɛ i, i = 1,..., n, or, in matrix notation, Y = Xβ + ɛ, To simplify, we assume that the design X is fixed, and that ɛ is N (0, σ 2
More informationCS281 Section 4: Factor Analysis and PCA
CS81 Section 4: Factor Analysis and PCA Scott Linderman At this point we have seen a variety of machine learning models, with a particular emphasis on models for supervised learning. In particular, we
More informationarxiv: v1 [math.pr] 22 Dec 2018
arxiv:1812.09618v1 [math.pr] 22 Dec 2018 Operator norm upper bound for sub-gaussian tailed random matrices Eric Benhamou Jamal Atif Rida Laraki December 27, 2018 Abstract This paper investigates an upper
More informationAsymptotic Distributions of Instrumental Variables Statistics with Many Instruments
CHAPTER 6 Asymptotic Distributions of Instrumental Variables Statistics with Many Instruments James H. Stock and Motohiro Yogo ABSTRACT This paper extends Staiger and Stock s (1997) weak instrument asymptotic
More informationRECENT DEVELOPMENTS IN LARGE DIMENSIONAL FACTOR ANALYSIS 58
RECENT DEVELOPMENTS IN LARGE DIMENSIONAL FACTOR ANALYSIS Serena Ng June 2007 SCE Meeting, Montreal June 2007 SCE Meeting, Montreal 1 / Factor Model: for i = 1,... N, t = 1,... T, x it = λ if t + e it F
More informationNuclear Norm Regularized Estimation of Panel Regression Models arxiv: v1 [econ.em] 25 Oct 2018
Nuclear Norm Regularized Estimation of Panel Regression Models arxiv:80.0987v [econ.em] 25 Oct 208 Hyungsik Roger Moon Martin Weidner October 26, 208 Abstract In this paper we investigate panel regression
More informationMatrix estimation by Universal Singular Value Thresholding
Matrix estimation by Universal Singular Value Thresholding Courant Institute, NYU Let us begin with an example: Suppose that we have an undirected random graph G on n vertices. Model: There is a real symmetric
More informationDe-biasing the Lasso: Optimal Sample Size for Gaussian Designs
De-biasing the Lasso: Optimal Sample Size for Gaussian Designs Adel Javanmard USC Marshall School of Business Data Science and Operations department Based on joint work with Andrea Montanari Oct 2015 Adel
More informationInference For High Dimensional M-estimates: Fixed Design Results
Inference For High Dimensional M-estimates: Fixed Design Results Lihua Lei, Peter Bickel and Noureddine El Karoui Department of Statistics, UC Berkeley Berkeley-Stanford Econometrics Jamboree, 2017 1/49
More informationEconometrics Master in Business and Quantitative Methods
Econometrics Master in Business and Quantitative Methods Helena Veiga Universidad Carlos III de Madrid Models with discrete dependent variables and applications of panel data methods in all fields of economics
More informationOn singular values distribution of a matrix large auto-covariance in the ultra-dimensional regime. Title
itle On singular values distribution of a matrix large auto-covariance in the ultra-dimensional regime Authors Wang, Q; Yao, JJ Citation Random Matrices: heory and Applications, 205, v. 4, p. article no.
More informationAssumptions for the theoretical properties of the
Statistica Sinica: Supplement IMPUTED FACTOR REGRESSION FOR HIGH- DIMENSIONAL BLOCK-WISE MISSING DATA Yanqing Zhang, Niansheng Tang and Annie Qu Yunnan University and University of Illinois at Urbana-Champaign
More informationRandom Fermionic Systems
Random Fermionic Systems Fabio Cunden Anna Maltsev Francesco Mezzadri University of Bristol December 9, 2016 Maltsev (University of Bristol) Random Fermionic Systems December 9, 2016 1 / 27 Background
More informationECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis
ECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis Lecture 7: Matrix completion Yuejie Chi The Ohio State University Page 1 Reference Guaranteed Minimum-Rank Solutions of Linear
More informationECO Class 6 Nonparametric Econometrics
ECO 523 - Class 6 Nonparametric Econometrics Carolina Caetano Contents 1 Nonparametric instrumental variable regression 1 2 Nonparametric Estimation of Average Treatment Effects 3 2.1 Asymptotic results................................
More informationRegret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Part I. Sébastien Bubeck Theory Group
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Part I Sébastien Bubeck Theory Group i.i.d. multi-armed bandit, Robbins [1952] i.i.d. multi-armed bandit, Robbins [1952] Known
More informationGARCH Models Estimation and Inference
GARCH Models Estimation and Inference Eduardo Rossi University of Pavia December 013 Rossi GARCH Financial Econometrics - 013 1 / 1 Likelihood function The procedure most often used in estimating θ 0 in
More informationStatistics 300B Winter 2018 Final Exam Due 24 Hours after receiving it
Statistics 300B Winter 08 Final Exam Due 4 Hours after receiving it Directions: This test is open book and open internet, but must be done without consulting other students. Any consultation of other students
More informationLeast squares under convex constraint
Stanford University Questions Let Z be an n-dimensional standard Gaussian random vector. Let µ be a point in R n and let Y = Z + µ. We are interested in estimating µ from the data vector Y, under the assumption
More informationEcon 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the
More informationLinear Regression for Panel with Unknown Number of Factors as Interactive Fixed Effects
Linear Regression for Panel with Unnown Number of Factors as Interactive Fixed Effects Hyungsi Roger Moon Martin Weidne February 27, 22 Abstract In this paper we study the Gaussian quasi maximum lielihood
More informationStrong Convergence of the Empirical Distribution of Eigenvalues of Large Dimensional Random Matrices
Strong Convergence of the Empirical Distribution of Eigenvalues of Large Dimensional Random Matrices by Jack W. Silverstein* Department of Mathematics Box 8205 North Carolina State University Raleigh,
More informationEigenvalues and Singular Values of Random Matrices: A Tutorial Introduction
Random Matrix Theory and its applications to Statistics and Wireless Communications Eigenvalues and Singular Values of Random Matrices: A Tutorial Introduction Sergio Verdú Princeton University National
More informationOn Bayesian Computation
On Bayesian Computation Michael I. Jordan with Elaine Angelino, Maxim Rabinovich, Martin Wainwright and Yun Yang Previous Work: Information Constraints on Inference Minimize the minimax risk under constraints
More informationSTAT 200C: High-dimensional Statistics
STAT 200C: High-dimensional Statistics Arash A. Amini May 30, 2018 1 / 57 Table of Contents 1 Sparse linear models Basis Pursuit and restricted null space property Sufficient conditions for RNS 2 / 57
More informationQuantile Processes for Semi and Nonparametric Regression
Quantile Processes for Semi and Nonparametric Regression Shih-Kang Chao Department of Statistics Purdue University IMS-APRM 2016 A joint work with Stanislav Volgushev and Guang Cheng Quantile Response
More informationEigenvalue variance bounds for Wigner and covariance random matrices
Eigenvalue variance bounds for Wigner and covariance random matrices S. Dallaporta University of Toulouse, France Abstract. This work is concerned with finite range bounds on the variance of individual
More informationSupplementary Material. Dynamic Linear Panel Regression Models with Interactive Fixed Effects
Supplementary Material Dynamic Linear Panel Regression Models with Interactive Fixed Effects Hyungsik Roger Moon Martin Weidner August 3, 05 S. Proof of Identification Theorem. Proof of Theorem.. Let Qβ,
More informationRandom Matrices: Invertibility, Structure, and Applications
Random Matrices: Invertibility, Structure, and Applications Roman Vershynin University of Michigan Colloquium, October 11, 2011 Roman Vershynin (University of Michigan) Random Matrices Colloquium 1 / 37
More informationKernel Method: Data Analysis with Positive Definite Kernels
Kernel Method: Data Analysis with Positive Definite Kernels 2. Positive Definite Kernel and Reproducing Kernel Hilbert Space Kenji Fukumizu The Institute of Statistical Mathematics. Graduate University
More informationFreeness and the Transpose
Freeness and the Transpose Jamie Mingo (Queen s University) (joint work with Mihai Popa and Roland Speicher) ICM Satellite Conference on Operator Algebras and Applications Cheongpung, August 8, 04 / 6
More informationOptimality and Sub-optimality of Principal Component Analysis for Spiked Random Matrices
Optimality and Sub-optimality of Principal Component Analysis for Spiked Random Matrices Alex Wein MIT Joint work with: Amelia Perry (MIT), Afonso Bandeira (Courant NYU), Ankur Moitra (MIT) 1 / 22 Random
More informationHigh Dimensional Inverse Covariate Matrix Estimation via Linear Programming
High Dimensional Inverse Covariate Matrix Estimation via Linear Programming Ming Yuan October 24, 2011 Gaussian Graphical Model X = (X 1,..., X p ) indep. N(µ, Σ) Inverse covariance matrix Σ 1 = Ω = (ω
More informationConsistency of the maximum likelihood estimator for general hidden Markov models
Consistency of the maximum likelihood estimator for general hidden Markov models Jimmy Olsson Centre for Mathematical Sciences Lund University Nordstat 2012 Umeå, Sweden Collaborators Hidden Markov models
More informationAsymptotic Statistics-III. Changliang Zou
Asymptotic Statistics-III Changliang Zou The multivariate central limit theorem Theorem (Multivariate CLT for iid case) Let X i be iid random p-vectors with mean µ and and covariance matrix Σ. Then n (
More informationInference for High Dimensional Robust Regression
Department of Statistics UC Berkeley Stanford-Berkeley Joint Colloquium, 2015 Table of Contents 1 Background 2 Main Results 3 OLS: A Motivating Example Table of Contents 1 Background 2 Main Results 3 OLS:
More informationKernels to detect abrupt changes in time series
1 UMR 8524 CNRS - Université Lille 1 2 Modal INRIA team-project 3 SSB group Paris joint work with S. Arlot, Z. Harchaoui, G. Rigaill, and G. Marot Computational and statistical trade-offs in learning IHES
More informationBinary matrix completion
Binary matrix completion Yaniv Plan University of Michigan SAMSI, LDHD workshop, 2013 Joint work with (a) Mark Davenport (b) Ewout van den Berg (c) Mary Wootters Yaniv Plan (U. Mich.) Binary matrix completion
More informationEstimation of Dynamic Regression Models
University of Pavia 2007 Estimation of Dynamic Regression Models Eduardo Rossi University of Pavia Factorization of the density DGP: D t (x t χ t 1, d t ; Ψ) x t represent all the variables in the economy.
More informationConsistent and equivariant estimation in errors-in-variables models with dependent errors
Consistent and equivariant estimation in errors-in-variables models with dependent errors Michal Pešta Charles University in Prague Department of Probability and Mathematical Statistics ROBUST 2010 February
More informationConfidence Intervals for Low-dimensional Parameters with High-dimensional Data
Confidence Intervals for Low-dimensional Parameters with High-dimensional Data Cun-Hui Zhang and Stephanie S. Zhang Rutgers University and Columbia University September 14, 2012 Outline Introduction Methodology
More informationConcentration inequalities: basics and some new challenges
Concentration inequalities: basics and some new challenges M. Ledoux University of Toulouse, France & Institut Universitaire de France Measure concentration geometric functional analysis, probability theory,
More informationLecture 13 October 6, Covering Numbers and Maurey s Empirical Method
CS 395T: Sublinear Algorithms Fall 2016 Prof. Eric Price Lecture 13 October 6, 2016 Scribe: Kiyeon Jeon and Loc Hoang 1 Overview In the last lecture we covered the lower bound for p th moment (p > 2) and
More informationSparsity Models. Tong Zhang. Rutgers University. T. Zhang (Rutgers) Sparsity Models 1 / 28
Sparsity Models Tong Zhang Rutgers University T. Zhang (Rutgers) Sparsity Models 1 / 28 Topics Standard sparse regression model algorithms: convex relaxation and greedy algorithm sparse recovery analysis:
More informationRank minimization via the γ 2 norm
Rank minimization via the γ 2 norm Troy Lee Columbia University Adi Shraibman Weizmann Institute Rank Minimization Problem Consider the following problem min X rank(x) A i, X b i for i = 1,..., k Arises
More informationCENTER FOR LAW, ECONOMICS AND ORGANIZATION RESEARCH PAPER SERIES
Maximum Score Estimation of a Nonstationary Binary Choice Model Hyungsik Roger Moon USC Center for Law, Economics & Organization Research Paper No. C3-15 CENTER FOR LAW, ECONOMICS AND ORGANIZATION RESEARCH
More informationSmall Ball Probability, Arithmetic Structure and Random Matrices
Small Ball Probability, Arithmetic Structure and Random Matrices Roman Vershynin University of California, Davis April 23, 2008 Distance Problems How far is a random vector X from a given subspace H in
More informationNon white sample covariance matrices.
Non white sample covariance matrices. S. Péché, Université Grenoble 1, joint work with O. Ledoit, Uni. Zurich 17-21/05/2010, Université Marne la Vallée Workshop Probability and Geometry in High Dimensions
More informationLinear Regression. In this problem sheet, we consider the problem of linear regression with p predictors and one intercept,
Linear Regression In this problem sheet, we consider the problem of linear regression with p predictors and one intercept, y = Xβ + ɛ, where y t = (y 1,..., y n ) is the column vector of target values,
More information1 Tridiagonal matrices
Lecture Notes: β-ensembles Bálint Virág Notes with Diane Holcomb 1 Tridiagonal matrices Definition 1. Suppose you have a symmetric matrix A, we can define its spectral measure (at the first coordinate
More information