Random Matrix Theory Lecture 1 Introduction, Ensembles and Basic Laws Symeon Chatzinotas February 11, 2013 Luxembourg
Outline 1. Random Matrix Theory 1. Definition 2. Applications 3. Asymptotics 2. Ensembles & Basic Laws 1. Gaussian=> Full-circle, Quarter-circle 2. Wigner=> Semi-circle 3. Wishart => Marcenko-Pastur 4. Haar => Haar 3. Examples 1. MIMO communications 2. Spectrum Sensing
Random Matrix Theory What is a Random Matrix? A matrix whose elements are random numbers from a specified distribution For example Multivariate statistics Sometimes all we need is independent identically distributed elements 3
Decompositions Eigenvalue Decomposition A square NxN matrix can be decomposed as follows: Eigenvectos in U Ordered eigenvalues in Λ Singular Value Decomposition A rectangular matrix NxM can be decomposed as: Square roots of non-zero eigenvalues of BB H The Matrix Cookbook http://orion.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf 4
Metrics of Interest Eigenvalue distribution Average eigenvalue Maximal eigevalue Minimal eigenvalue Eigenvalue Spacing Random values for random matrices 5
Pdf and Moments The kth moment of random matrix X is defined as: The pdf of random matrix X f X (A) is defined as the probability of an elementary volume around point A 6
Applications Nuclear Physics model the spectra of heavy atoms [Eugene Wigner] spacings between the lines in the spectrum of a heavy atom should resemble the spacings between the eigenvalues of a random matrix Matrix ensemble named after him Eigenvalues of random matrices related to real-world problems Wigner, E. (1955). "Characteristic vectors of bordered matrices with infinite dimensions". Ann. Of Math. 62(3): 548 564. doi:10.2307/1970079 7
Applications Statistical Finance Return statistics and portfolio theory Principal component analysis Decompose fluctuations into decorrelated contributions of descreasing variance Portfolio Risk J.P. Bouchaud, M. Potters, Financial Applications of Random Matrix Theory: a short review, http://arxiv.org/abs/0910.1205 8
Applications Signal Processing Multi-dimensional channel matrices Antennas Carriers Samples Users Basic Linear Model 9
Signal Processing Performance Evaluation Channel capacity MMSE error Spectrum Sensing 10
Why asymptotics? Finite vs asymptotic RMT Law of large numbers One or both dimensions of random matrix go to infinity Discover deterministic rules for metrics of interest Convergence almost surely or almost everywhere or with probability 1 or strongly 11
Why asymptotics? 12
Gaussian Matrices A standard real/complex Gaussian m n matrix H has i.i.d. real/complex zero-mean Gaussian entries with identical variance σ 2 = 1/m. The p.d.f. of a complex Gaussian matrix ith i.i.d. zero-mean Gaussian entries with variance σ 2 is The probability of an elementary volume around point H 13
Full-circle Law Let H be an N x N complex random matrix whose entries are independent random variables with identical mean and variance and finite kth moments for k 4. Assume that the joint distribution of the entries real and imaginary parts of the entries have uniformly bounded support densities. The asymptotic spectrum of H converges almost surely to the circular law, namely the uniform distribution over the unit disk on the complex plane, whose density is given by: 14
Full-circle Matlab code 15
Full-circle Density 16
Quarter-circle Law Consider an N x N matrix H whose entries are independent zero mean complex (or real) random variable with variance 1/N, the asymptotic distribution of the singular values converges to: Square roots of eigenvalues of HH H 17
Quarter-circle Matlab code 18
Quarter-circle Density 19
Wigner Matrices An n n Hermitian matrix W is a Wigner matrix if its upper-triangular entries are independent zero-mean random variables with identical variance. If the variance is 1/n, then W is a standard Wigner matrix. Pdf for i.i.d. zero-mean Gaussian entries with unit variance 20
Semicircle Law Consider an N N standard Wigner matrix W with bounded fourth moment The empirical distribution of W converges almost surely to the semicircle law whose density is 21
Semicircle Matlab code 22
Semicircle Density 23
Wishart Matrices The m m random matrix A = HH H is a (central) real/complex Wishart matrix with n degrees of freedom and covariance matrix Σ, (A W m (n,σ)), if the columns of the m n matrix H are zero-mean independent real/complex Gaussian vectors with covariance matrix Σ. The p.d.f. of a complex Wishart matrix A W m (n,σ) for n m is 24
Convergence N=2 N=3 N=4 N=5 25
Wishart Properties For a central Wishart matrix W W m (n, I) 26
MP Law Consider an N x K matrix H whose entries are independent zero-mean complex (or real) random variables with variance 1/N and fourth moments of order O( 1/N 2 ). As K,N with K/N a, the empirical distribution of H H H converges almost surely to a nonrandom limiting distribution with density What about HH H??? V. A. Marchenko and L. A. Pastur, Distributions of eigenvalues for some sets of random matrices, Math USSR-Sbornik, vol.1 pp.457-483, 1967. 27
MP Matlab code 28
MP Density 29
Haar Matrices A square matrix U is unitary if An n n random matrix U is a Haar matrix if it is uniformly distributed on the set, U(n), of n n unitary matrices. Its density function on U(n) is given by 30
Haar distribution Let the random matrix W be square N x N with independent identically complex Gaussian distributed c.c.s. i.i.d. entries with zero mean and finite variance. Then the empirical distribution of of the eigenvalues of the unitary random matrix converges asymptotically to: 31
Haar Matlab Code 32
Haar density 33
Examples MIMO Communications Mutual information MMSE Multidimensional Spectrum Sensing Scaled largest eigenvalue Standard Condition number Spherical test John s detector 34
MIMO communications Basic linear I/O model x: input symbols (Gaussian) z: additive noise (Gaussian) y: output symbols SNR: variance ratio of x over z 35
MIMO Mutual Information Can be expressed as an integral over aepdf 36
MIMO Mutual Information Assuming Gaussian channel matrix and using MP law 37
MIMO MMSE Can be expressed as an integral over aepdf 38
MIMO MMSE Assuming Gaussian channel matrix and using MP law 39
Spectrum Sensing Still using the linear model, but matrix I/O Hypothesis testing: Sample covariance matrix: Decision based on whether 40
Spectrum Sensing Scaled largest eigenvalue For Wishart 41
Spectrum Sensing Standard Condition number For Wishart 42
Spectrum Sensing Spherical test For Wishart 43
Spectrum Sensing John s detector For Wishart 44
Summary The statistics of certain random matrices crystallize for large dimensions Metrics of interest can often be defined in closedform Applications in analyzing the statistics of functions that have random matrix arguments Using asymptotics, the randomness disappears and results provide good estimates for finite cases as well 45
Questions? Random Matrix Theory Lecture 1 Introduction, Ensembles and Basic Laws 46