Applications and fundamental results on random Vandermon

Transcription

1 Applications and fundamental results on random Vandermonde matrices May 2008

2 Some important concepts from classical probability Random variables are functions (i.e. they commute w.r.t. multiplication) with a given p.d.f. (denoted f ) Expectation (denoted E) is integration Independence Additive convolution ( ) and the logarithm of the Fourier transform Multiplicative convolution Central limit law, with special role of the Gaussian law Poisson distribution P c : The limit of (( 1 n c ) δ(0) + c n δ(1) ) n as n. Divisibility: For a given a, nd i.i.d. b 1,..., b n such that f a = f b1 + +b n.

3 Free probability Naples 2008 A more general theory, where the random variables are matrices (or more generally, elements in a unital -algebra (denoted A), typically B(H)), with their eigenvalue distribution (spectrum) taking the role as the p.d.f. The above mentioned concepts have their analogues in this theory. For instance, the expectation (denoted φ) is a normalized linear functional on A. The pair (A, φ) is called a noncommutative probability space. For (random) matrices, φ will be the (expected) trace: φ(a) = tr n (A) = 1 n a n ii (φ(a) = E(tr n (A)). i=1 What should it mean that two random matrices are "free" (=analogue of independent, to be dened)? Think of as two independent random matrices, where eigenvectors of one point in all directions with equal probability (unitary invariance).

4 The semicircle law Naples 2008 Free probability has a "Gaussian distribution counterpart": A = (1/sqrt(2000)) * (randn(1000,1000) + j*randn(1000,1000)); A = (sqrt(2)/2)*(a+a'); hist(eig(a),40)

5 Motivation for free probability Assume that X n, Y n are independent, Gaussian n n-matrices. One can show that the limits φ ( X i 1 Y j1 X i l Y j ) l := lim tr ( n n X i 1n Y j 1 n X i l ny j ) l n exist. If we linearly extend the linear functional φ to all polynomials in A and B, the following can be shown: Theorem If P i, Q i are polynomials in X and Y respectively, with 1 i l, and φ(p i (X )) = 0, φ(q i (Y )) = 0 for all i, then φ (P 1 (X )Q 1 (Y ) P l (X )Q l (Y )) = 0. This motivates the denition of freeness (=analogue of independence):

6 Denition of freeness Naples 2008 Denition A family of unital -subalgebras (A i ) i I is called a free family if a j A ij i 1 i 2, i 2 i 3,, i n 1 i n φ(a 1 a n ) = 0. (1) φ(a 1 ) = φ(a 2 ) = = φ(a n ) = 0 A family of random variables a i is called a free family if the algebras they generate form a free family. (1) is also called the freeness relation, and can be viewed as a rule for computing the mixed moments φ(a 1 a n ) of the a i from their individual moments φ(a m i ). In random matrix settings, it relates the moments (E(tr n (X m )) of random matrices.

7 Additive and multiplicative free convolution Additive/multiplicative free convolution ( / ) corresponds to summing/multiplying free random variables. Can also be viewed as operations on measures, by associating the moments with a probability measure. In random matrix settings, additive free convolution cooresponds to estimating the eigenvalue distribution of the sum of two large, independent random matrices, where one is unitarily invariant. Alternative functional equations exist for computing additive/multiplicative free convolution. Uses the Stieltjes transform. One of the main questions in my papers: Let X and Y be random matrices. How can we make a good prediction of the eigenvalue distribution of X when one has the eigenvalue distribution of XY and Y (i.e. problem turned around to a deconvolution problem)? Simplest case is Y Gaussian.

8 Recent developments in free probability Second order freeness. Tools to estimate higher order cumulants of the moments of random matrices, not only their rst order cumulants (moments). Free rectangular convolution. Can be used to estimate the singular law of a sum of rectangular matrices, from their individual singular laws. Numerical tools for computing free convolution How to use to estimate channel capacity [1].

9 Does other types of random matrices (i.e. non-unitarily invariant) t into a framework similar to free probability? We have investigated this for Vandermonde matrices [2, 3], which are widely used. They have the form 1 1 x 1 x L V =... x1 N 1 xl N 1 It is straightforward to show that square Vandermonde matrices have determinant det(v) = (x l x k ). 1 k<l N In particular, V is nonsingular if the x k are dierent.

10 Various results exist on the distribution of the determinant of Vandermonde matrices (Gaussian entries (Metha), entries with B-distribution (Selberg)), but there are many open problems (below, V H V is used since V is rectangular in general): How can we nd the moments of the Vandermonde matrices (i.e. tr L ( (V H V ) k ) ) (not the determinant itself)? Deconvolution problem: How to estimate the moments of D from mixed moments DV H V? Mixed moments of independent Vandermonde matrices? Asymptotic results? If X is an N N standard, complex, Gaussian matrix, then lim 1 N N log 2 det ( I + ρ ( 1 N XX H)) = 2 log 2 (1 + ρ 1 ( ) ) 2 4 4ρ log 2 e ( ) 2 4ρ 4ρ (which is the expression for the capacity). We are not aware of similar asymptotic expressions for the determinant/capacity of Vandermonde matrices.

11 Random Vandermonde matrices We will consider Vandermonde matrices V of dimension N L of the form 1 1 V = 1 e jω 1 e jω L (2) N..... e j(n 1)ω 1 e j(n 1)ω L (i.e. we assume that the x i lie on the unit circle). The ω i are called phase distributions. We will limit the study of Vandermonde matrices to cases where The phase distributions are i.i.d. The asymptotic case N, L with lim L N N = c. The normalizing factor 1 N is included to ensure limiting asymptotic behaviour.

12 Where can such Vandermonde matrices appear? Consider a multi-path channel of the form: h(τ) = L α i g(τ τ i ) i=1 α i are i.d. Gaussian random variables with power P i, τ i are uniformly distributed delays over [0, T ], g is the low pass transmit lter. L is the number of paths In the frequency domain, the channel is given by: L c(f ) = α i G(f )e j2πf τ i i=1 We suppose the transmit lter to be ideal (G(f ) = 1).

13 Sampling the continuous frequency signal at f i = i W N (N is the number of frequency samples) where W is the bandwidth, our model becomes r = VP 1 2 α 1 n 1 +. α L. n N, (3) where V is a random Vandermonde matrix of the type (2), and P is the L L diagonal power matrix, n i is independent, additive, white, zero mean Gaussian noise of variance σ N.

14 Main result Naples 2008 Denition Dene K ρ,ω,n = 1 N n+1 ρ (0,2π) ρ nk=1 1 e jn(ω b(k 1) ω b(k) ) 1 e j(ω b(k 1) ω b(k) ), dω 1 dω ρ, (4) where ω ρ 1,..., ω ρ ρ are i.i.d. (indexed by the blocks of ρ), all with the same distribution as ω, and where b(k) is the block of ρ which contains k (where notation is cyclic, i.e. b( 1) = b(n)). If the limit K ρ,ω = lim N K ρ,ω,n exists, then K ρ,ω is called a Vandermonde mixed moment expansion coecient.

15 Main result 2 Naples 2008 Assume that {D r (N)} 1 r n are diagonal L L matrices which have a joint limit distribution as N, L N c. We would like to express the limits M n = lim N E[tr L(D 1 (N)V H VD 2 (N)V H V D n (N)V H V)]. (5) It turns out that this is feasible when all Vandermonde mixed moment expansion coecients K ρ,ω exist.

16 For convenience, dene m n = (cm) n = c lim N E [ (( tr L D(N)V H V ) n)], d n = (cd) n = c lim N tr L (D n (N)), Theorem Assume D 1 (N) = D 2 (N) = = D n (N). When ω = u, m 1 = d 1 m 2 = d 2 + d1 2 m 3 = d 3 + 3d 2 d 1 + d1 3 m 4 = d 4 + 4d 3 d 1 + 8/3d d 2 d1 2 + d1 4 m 5 = d 5 + 5d 4 d /3d 3 d d 3 d /3d2 2 d d 2 d1 3 + d1 5 m 6 = d 6 + 6d 5 d d 4 d d 4 d /20d d 3 d 2 d 1 +20d 3 d d d2 2 d d 2 d1 4 + d1 6 m 7 = d 7 + 7d 6 d /3d 5 d d 5 d /20d 4 d d 4 d 2 d 1 +35d 4 d /20d3 2 d /10d 3 d d 3 d 2 d d 3 d d2 3 d /3d2 2 d d 2 d1 5 + d1 7. (6)

17 Comparison Naples 2008 The Gaussian equivalent of this is m 1 = d 1 m 2 = d 2 + d1 2 m 3 = d 3 + 3d 2 d 1 + d1 3 m 4 = d 4 + 4d 3 d 1 + 3d d 2d1 2 + d4 1 m 5 = d 5 + 5d 4 d 1 + 5d 3 d d 3 d d2 2 d d 2 d1 3 + d5 1 m 6 = d 6 + 6d 5 d 1 + 6d 4 d d 4 d d d 3d 2 d 1 +20d 3 d d d2 2 d d 2d1 4 + d6 1 m 7 = d 7 + 7d 6 d 1 + 7d 5 d d 5 d d 4d d 4 d 2 d 1 +35d 4 d d2 3 d d 3 d d 3d 2 d d 3 d d3 2 d d2 2 d d 2d1 5 + d7 1, (7) when we replace V H V with N 1 XXH, with X an L N complex, standard, Gaussian matrix.

18 Sketch of proof Naples 2008 We can write [ ( E tr L D1 (N)V H VD 2 (N)V H V D n (N)V H V )] (8) as L 1 i1 j,...,in E( D 1 (N)(j 1, j 1 )V H (j 1, i 2 )V(i 2, j 2 ) 1,...,j n D 2 (N)(j 2, j 2 )V H (j 2, i 3 )V(i 3, j 3 ) (9). D n (N)(j n, j n )V H (j n, i 1 )V(i 1, j 1 )) The (j 1,..., j n ) give rise to a partition ρ of {1,..., n}, where each block ρ j consists of equal values, i.e. ρ j = {k j k = j}. This ρ will actually represent the ρ used in the denition of K ρ,ω,n. The rest of the proof goes by carefully computing this limit quantity using much combinatorics.

19 Comparisons Denote by (( ν(λ, α) = lim 1 λ ) δ 0 + λ ) n n n n δ α the (classical) Poisson distribution of rate λ and jump size α. Denote also by (( µ(λ, α) = lim 1 λ ) δ 0 + λ ) n n n n δ α the free Poisson distribution of rate λ and jump size α (also called the Marchenko Pastur law). Denote also µ c = µ( 1 c, c), ν c = ν(c, 1).

20 Comparisons 2 Naples 2008 Corollary Assume that V has uniformly distributed phases. Then the limit moment [ (( M n = lim E tr L V H V ) n)] N satsies the inequality φ(a n 1) M n 1 c E(an 2), where a 1 µ c, a 2 ν c. In particular, equality occurs for m = 1, 2, 3 and c = 1.

21 Comparisons (a) V H V, with V a Vandermonde matrix with uniformly displex, standard, Gaussian matrix. (b) N 1 XXH, with X an comtributed phases. Figure: Histogram of mean eigenvalue distributions.

22 Other results Naples 2008 Mixed moments of (more than one) independent Vandermonde matrices. Generalized Vandermonde matrices: These have the form V = ( e jα ) kβl 1 k N,1 l L. It is known that V is nonsingular i all α k are dierent, and all β l are dierent. The papers also contain results on the asymptotics of generalized Vandermonde matrices. Exact moments of lower order Vandermonde matrices. Reveals slower convergence. Computation of the asymptotic moments when the phase distribution is not uniform. Phase distributions with continous density, and phase distributions with singularities.

23 Application 1: Estimation of the number of paths Return to the multi-path channel model (3). For simplicity, set W = T = 1, so that the phase distribution of the Vandermonde matrix is uniform. We take K observations of (3) and form the observation matrix Y = [r 1 r K ] = VP 1 2 α 1 (1) α 1 (K) α L (1) α L (K) n 1 (1) n 1 (K)..... n N (1) n N (K), (10) It is now possible to combine the deconvolution result for Vandermonde matrices with known deconvolution results for Gaussian matrices to estimate L from a number of observations (assuming P is known). All values of L are tried, and the one which "best matches" the observed values is chosen:

24 Estimation of the number of paths 2 Proposition Assume that V has uniformly distributed phases, and let mp i be the moments of P, and m iˆr = tr N(^R i ) the moments of the sample covariance matrix ^R = 1 K YYH. Dene also c 1 = K N, c 2 = N L, and c 3 = K L. Then E [ ] mˆr = c 2 mp 1 + σ2 [ E m 2ˆR ] ( = c ) mp N 2 + c 2(c 2 + c 3 )(mp 1 )2 E [ m 3ˆR ] = +2σ 2 (c 2 + c 3 )m 1 P + σ4 (1 + c 1 )

25 L L Naples 2008 Estimation of the number of paths 3 70 Estimate of L Actual value of L 70 Estimate of L Actual value of L Number of observations (a) K = Number of observations (b) K = 10 Figure: Estimate for the number of paths. Actual value of L is 36. Also, σ = 0.1, N = 100.

26 Application 2: Wireless capacity Analysis For a general matrix W, the mean capacity is dened as C N = N 1 E ( log 2 det ( I N + 1 σ 2 WW H)) = N 1 Nk=1 E ( ( log ( σ 2 λ k WW H ))) = ( log σ 2 t ) µ(dt) (11) where µ is the mean empirical eigenvalue distribution of WW H. Substituting the Taylor series log 2 (1 + t) = ln 1 k=1 2 ( 1) k+1 tk k, we obtain C N = ln 1 k=1 ( 1) k+1 m k (µ)ρ k 2 k, (12) where ρ is SNR, and where m k (µ) = t k dµ(t) for k Z + However, if W is a Vandermonde matrix, many more moments are required for precise estimation of capacity than we can provide with the formulas for the rst 7 moments.

27 Wireless capacity Analysis 2 3 Asymptotic capacity sample capacity Capacity ρ Figure: Several realizations of the capacity 1 N log 2 det ( I + ρ 1 N XXH) when X is standard, complex, Gaussian. Matrices of size were used. The known expression for the asymptotic capacity is also shown.

28 Wireless capacity Analysis Capacity 1.5 Capacity ρ ρ (a) Realizations of 1 N log 2 det ( I + ρvv H) (b) Realizations of when ω 1 N log 2 det ( I + ρvv H) when ω has uniform phase distribution. has a certain non-uniform phase distribution. Figure: Several realizations of the capacity for Vandermonde matrices for two dierent phase distributions. Matrices of size were used.

29 This talk is available at oyvindry/talks.shtml. My publications are listed at oyvindry/publications.shtml THANK YOU!

30 Ø. Ryan and M. Debbah, Channel capacity estimation using free probability theory, To appear in IEEE Trans. Signal Process., 2007, Random Vandermonde matrices-part I: Fundamental results, Submitted to IEEE Trans. on Information Theory, 2008., Random Vandermonde matrices-part II: Applications, Submitted to IEEE Trans. on Information Theory, 2008.