A new method to bound rate of convergence

Transcription

1 A new method to bound rate of convergence Arup Bose Indian Statistical Institute, Kolkata, Sourav Chatterjee University of California, Berkeley

2 Empirical Spectral Distribution Distribution (LSD) Different Types of Convergence Rates of Convergence A new technique of bounding the rates of convergence Application to the matrices: Wigner, Sample Covariance, Reverse Circulant,, of our approach Random matrices - p. 2/33

3 A n is an n n Hermitian matrix with random entries. Eigenvalues (characteristic roots): λ 0, λ 1,...,λ n 1. The empirical spectral distribution (ESD) function of A n : n 1 F n (x) = n 1 i=0 I{λ i x}. The empirical spectral measure: the corresponding probability measure P n The expected spectral distribution function: E(F n ( )). Expected spectral measure: The corresponding probability measure of our approach Random matrices - p. 3/33

4 The Distribution (or measure) (LSD): weak limit of {P n }, in some probabilistic sense, such as almost surely or in probability. The weak convergence of E(F n ) often serves as an intermediate step in showing the weak convergence of F n. LSD known for Wigner, Sample covariance,,, Reverse Circulant... Two main tools to establish the LSD: the moment method and the Stieltjes transform method. of our approach Random matrices - p. 4/33

5 When the LSD exists, how does one obtain the rate of convergence? So far the main method to establish such rates is the use of Stieltjes transform. : 1. Establish some general results useful in establishing the probabilistic weak convergence of F n from the convergence of E(F n ) and the corresponding rates of convergence. 2. Apply these to establish some new rates of convergence. of our approach Random matrices - p. 5/33

6 ϕ= of A: the characteristic function of the empirical spectral distribution of A. ϕ(t) = 1 n tr(eita ) where n= the order of A and e M = k=0 1 k! Mk. In general, for any distribution function G (which may be random) on R, its (random) characteristic function is defined as ϕ G (t) = e itx dg(x). of our approach Random matrices - p. 6/33

7 of our approach STEP 1. Obtain bounds for and its partial derivatives. (For a complex random variable X, its variance is defined to be E X E(X) 2.) STEP 2. Deduce the concentration of the spectral measure near its mean, and also get the magnitude of concentration using the bounds. of our approach Random matrices - p. 7/33

8 Kolmogorov : F G = sup F(x) G(x). x R Theorem 1. Suppose F is a random distribution function on R with (random) characteristic function ϕ. Suppose Ct 2 for each t. If G is a nonrandom distribution function on R, such that sup x R G (x) λ, then E F G 2 E(F) G + 8(3)1/2 λ 1/2 C 1/4. π This will be eventually used to establish rates of convergence for the ESD. C determines the rate. of our approach Random matrices - p. 8/33

9 The convolution F G of F and G is defined in the usual way. That is, F G(x) = F(x y)dg(y) = G(x y)df(y). Define the probability density h L (x) = 1 coslx πlx 2. Let H L be the corresponding distribution function. The characteristic function of H L is given by Note that ψ L (t) = (1 t L )I( t L). ψ L (t) dt = L. of our approach Random matrices - p. 9/33

10 Let F 0 = E(F), and let η be the characteristic function of F 0. Then by assumption, E ϕ(t) η(t) C t. By Esseen s lemma, F G 2 F H L G H L + 24λ πl Now F H L G H L F 0 H L G H L + F H L F 0 H L This in turn is bounded by F 0 G + F H L F 0 H L. of our approach Random matrices - p. 10/33

11 So by applying the inversion formula (see Feller 1966, page ) and the hypothesis about, E F H L F 0 H L 1 π C1/2 L π. ψ L(t) E ϕ(t) η(t) t dt Combining all these observations, we have E F G 2 F 0 G + 2 CL π + 24λ πl Choosing L 2 = 12λC 1/2 gives the desired conclusion. of our approach Random matrices - p. 11/33

12 Note the condition on the variance in Theorem 1. Note that the matrix A is a function of independent real random variables x 1, x 2,...,x m, (note that m is large). Write ϕ(t) as a function of x 1, x 2,...,x m. Hoeffding (unpublished work), Efron and Stein (1981), Steele (1986) and Devroye (1991): See Györfi et al. (2002) for proof. Theorem 2. (Efron-Stein type ). Suppose Z 1,...,Z n, Z1,...,Zn vectors where Z i has the same distribution as Zi f : R m.n R is a square integrable function. Then are independent m-dimensional random for all i. Suppose that Var(f(Z 1,...,Z n )) 1 2 n k=1 E f(z 1,...,Z n ) f(z 1,...,Z k 1, Z k, Z k+1,...,z n ) 2. of our approach Random matrices - p. 12/33

13 Theorem 3. Suppose f(x 1, x 2,...,x n ) is a coordinatewise differentiable map from R n into C and M j, 1 j n are constants such that sup f M j for j = 1, 2,...,n. x x j Then for independent complex random variables X 1, X 2,...,X n, n Var(f(X 1, X 2,...,X n )) Mj 2 Var(X j ). j=1 of our approach Random matrices - p. 13/33

14 POIN There exists a constant K > 0 such that if X F, and g : R R is any differentiable map, then Var(g(X)) KE g (X) 2. Log-concave densities (i.e. densities of the form e U(x) where U is a concave function) satisfy POIN: The standard normal: Chernoff (1981) with K = 1. Double exponential, and uniform distributions See also Chen (1982). of our approach Random matrices - p. 14/33

15 The complete characterization of all absolutely continuous distributions which satisfy POIN is given in the following theorem, proved in a more general form in Muckenhoupt (1972). Theorem 4. A distribution function F with F(0) = 1/2 and density f w.r.t. Lebesgue measure satisfies POIN if and only if B = max{sup x 0 (1 F(x)) x 0 1 f(t) dt, sup x<0 F(x) 0 x Moreover, the smallest constant K admissible in POIN satisfies 1 2 B K 4B. 1 f(t) dt} <. of our approach Random matrices - p. 15/33

16 The next theorem follows easily from the Efron-Stein : Theorem 5. If X 1, X 2,...,X n i.i.d F where F has property POIN with constant K, then for any coordinatewise differentiable map f : R n C, Var(f(X 1,...,X n )) Kσ 2 E f(x 1,...,X n ) 2 where denotes the Euclidean norm, and σ 2 = Var(X 1 ). of our approach Random matrices - p. 16/33

17 Use the identity ϕ(t) = 1 x j n tr ( it A ) e ita x j Use the fact that e ita is a unitary matrix. Either the partial derivatives are bounded by small numbers in sup norm Then use an Efron-Stein type. Or the expected value of the norm-squared of ϕ(t) is small. Then use a Poincaré type of our approach Random matrices - p. 17/33

18 Suppose A(u) is a matrix with complex entries. Derivative of product formula: d du (A(u)B(u)) = A (u)b(u) + A(u)B (u). Lemma 1. If A(u) is an elementwise differentiable map from R or C into C n n then ( ) d da du tr(ea ) = tr du ea Lemma 2. If A is Hermitian and t is real, then e ita is a unitary matrix. In particular, for any vector x, e ita x = x, (where denotes the Euclidean norm) and also all entries of e ita have modulus 1. of our approach Random matrices - p. 18/33

19 W n is a Wigner matrix with independent entries (X (n) jk ) having common variance 1. We shall drop the superscript n. In many situations, the LSD of n 1/2 W n exists and is given by F(x) = 1 2π x Note that sup 2 x 2 F (x) = π 1. Let F n be the ESD of n 1/2 W n. (4 y 2 ) 1/2 I [ 2, 2] dy. of our approach Random matrices - p. 19/33

20 (W1) E(X jk ) = 0, E(Xjk 2 ) = 1. (W2) sup i,j,n EXij 4 ( <. ) (W3) E Xij 4 I { X ij ɛn 1/2 } = o(n 2 ) for any ɛ > 0. Bai (1993a). Bai, Miao and Tsay (1997). Bai, Miao and Tsay (2002). (W3*) F n F = O p (n 2/5 ). E(X 8 ij I { Xij ɛn 1/2 }) = o(n 2 ) for any ɛ > 0. Then E(F n ) F = O(n 1/2 ). (W3**) sup n sup ij E X ij k < for every k 1. Then F n F = O(n 2/5+η ) almost surely for every η > 0. of our approach Random matrices - p. 20/33

21 W n : R n(n+1)/2 R n n Takes a tuple (a jk ) to the Wigner matrix with (j, k)-th entry n 1/2 a jk (j k), and n 1/2 a kj otherwise. Then W jk n = W n a jk is a constant matrix whose only nonzero entries are (j, k)-th and (k, j)-th entries (=n 1/2 ). ϕ t n= the empirical characteristic function of W n. Then ϕ t ( n = n 1 tr it W ) n e itw n = n 1 tr ( itwn jk e itw ) n. a jk a jk Let B = e itw n. Note that B is unitary. Then ϕ t n = it(b jk + b kj ) a jk n = 2itb jk n n n ϕ t n 2 = j k ϕt n a jk 2 j,k 4t 2 b jk 2 n 3 = 4t2 n 2. of our approach Random matrices - p. 21/33

22 Stronger conditions new and stronger results. Note that sup 2 x 2 F (x) = π 1. Theorem 6. If W n is a random real Wigner matrix whose entries on and above the diagonal are i.i.d. with variance 1, drawn from a distribution satisfying POIN with constant K, then Var(ϕ n (t)) 4Kt 2 /n 2. Consequently, by Theorem 1, if F denotes the semicircular law, then E F n F 2 EF n F + 8(6)1/2 K 1/4 π 3/2 n 1/2 where F n denotes the empirical c.d.f. of n 1/2 W n. of our approach Random matrices - p. 22/33

23 Minimal conditions weaker rate results. Lemma 2: the elements of e itw n are bounded in modulus by 1. Hence ϕ t n a jk 2 t n 3/2. So we can invoke Theorems 3 and 1: Theorem 7. If {W n } is a sequence of random Wigner matrices, whose entries on and above the diagonal are independent with variance uniformly bounded by 1, then Var(ϕ n (t)) 4t2 n 3 j k Var(a jk ) 4t2 n Hence if F n denotes the empirical c.d.f. of n 1/2 W n and F denotes the semicircular law, then E F n F 2 EF n F + 8(6)1/2 K 1/4 π 3/2 n 1/4. of our approach Random matrices - p. 23/33

24 Suppose X is a real p n matrix with entries x jk, which are i.i.d. real random variables with mean zero and unit variance. Let S = 1 n XXT. If y n = p/n y (0, 1] then the ESD of S n converges almost surely to the law F y ( ) with the Marčenko-Pastur density f y (x) = 1 2πxy (b x)(x a) if a x b, 0 otherwise. where a = a(y) = (1 y) 2 and b = b(y) = (1 + y) 2. It can be easily shown that the density is bounded by λ = [ π y(1 y) ] 1. In cases where y > 1, the LSD exists but has a point mass at the origin. If y = 0, then a scaling and a centering are required for the LSD of S n to exist. See Bai (1999) or Bose et al. (2003) for the precise results. We do not consider these cases. (1) of our approach Random matrices - p. 24/33

25 Stieltjes transform method: Bai (1993b) proved that E(F n ) converges to F y at a rate O(n 1/4 ) or O(n 5/48 ) depending on how close y n is to 1. The same rates were obtained for convergence in probability of F n to F y in Bai, Miao and Tsay (1997). Bai, Miao and Yao (2003) prove several results under the conditions given in Wigner case: If y n remains bounded away from 1 and suitable combinations of the above conditions hold then E(F n ) F y = O(n 1/2 ), F n F y = O P (n 2/5 ) and F n F y = O a.s. (n 2/5+η ). of our approach Random matrices - p. 25/33

26 Let Y = X/ x jk = e j e T k x k = kth column of X ϕ t n = empirical characteristic function evaluated at t. S jk := S x jk = 1 n (Y XT + XY T ) ϕ t n x jk = p 1 tr(its jk e its ) = it(np) 1 tr(y X T e its + XY T e its ) = it(np) 1 tr(e j x T ke its + x k e T j e its ) = it(np) 1 (x T ke its e j + e T j e its x k ) = 2itz kj np where we have written z kj for the jth component of the vector z k := e its x k. Note that since e its is unitary, z k = x k. of our approach Random matrices - p. 26/33

27 S matrix rate with POIN j,k ϕt n x jk 2 4t2 n 2 p 2 n k=1 z k 2 = 4t2 n 2 p 2 n k=1 x k 2 = 4t2 n 2 p 2 j,k x jk 2 a.s.. So, if j, k, E x jk 2 M 2, then E ϕ t n 2 4M2 t 2 np. Applying Theorems 5 and 1, Theorem 8. If {x jk } are i.i.d. from a distribution satisfying POIN with constant K, mean 0 and variance 1, then Var(ϕ n (t)) 4Kt2 np. Consequently, if y n = p/n (0, 1) and F y denotes the Marčenko-Pastur distribution with parameter y, then E F n,p F yn 2 EF n,p F yn + 8(6)1/2 K 1/4 π 3/2 [y n (1 y n )] 1/2 n 1/2 where F n,p denotes the ESD of S. of our approach Random matrices - p. 27/33

28 S matrix rate without POIN If we don t assume POIN but instead impose j, k, x jk M a.s., then ϕt n 2 t (np) 1 z k = 2 t (np) 1 x k 2M t x jk n p Thus, if the variance of x jk is bounded by 1, then Var(ϕ n (t)) 4M2 t 2 n 2 p np = 4M2 t 2. n Hence we get Theorem 9. Suppose y = p/n (0, 1) and x jk are independent with mean zero and variance bounded by 1. Suppose M is such that P( x jk M) = 1. Then E F n F y 2 EF n F y + where F n is the ESD of S n. a.s. 8(6) 1/2 M 1/2 π 3/2 y 1/4 (1 y) 1/2n 1/4. of our approach Random matrices - p. 28/33

29 : LSD Reverse Circulant matrix A n = ((x (i+j 2) mod n )). Visually, x 0 x 1 x 2 x n 1 x 1 x 2 x n 1 x 0 A n = x 2 x n 1 x 0 x 1. x n 1 x 0 x 1 x n 2 Bose and Mitra (2002), Bose, Chatterjee and Gangopadhyay (2003): If {x i } are i.i.d. with mean zero and variance 1 then ESD of X n = n 1/2 A n converges (in probability) to the LSD with density f given by f(x) = x exp( x 2 ), < x <. of our approach Random matrices - p. 29/33

30 rate with POIN Let B = e ita n. Then b ij 2 = n. Also for each k, there are at most 2n pairs of (i, j) such that i + j 2 = k mod n. Let ϕ t n= empirical characteristic function of A n evaluated at t. ϕ t n x k = k it n n i+j 2=kmod n ϕt n 2 t2 x k n 3 [2n k b ji i+j 2=kmod n b ji 2 ] = 2t2 n If x k are i.i.d. from a density satisfying POIN, then E F n F 2 EF n F + O(n 1/4 ). The eigenvalues can be explicitly obtained and using them, EF n F is of a much smaller order than n 1/4. of our approach Random matrices - p. 30/33

31 H n = ((x i+j 2 )) is called a matrix. Bryc Jiang and Dembo: LSD of n 1/2 H n. Not much known about the LSD. IF the limit has a bounded density, then we can use our technique. The computations for our method are very similar to the previous example. In fact, here ϕ t n x k = it n n i+j 2=k b ji So, similar computations show that under POIN E F n F 2 EF n F + O(n 1/4 ). of our approach Random matrices - p. 31/33

32 T n = ((x i j )) is called a matrix of order n. Bryc Dembo and Jiang: LSD exists and is unimodal. IF the limiting distribution has a bounded density: Exactly the same kind of computations as in the preceding examples show that in this case, too, Under POIN, E F n F 2 EF n F + O(n 1/4 ). of our approach Random matrices - p. 32/33

33 Do the and the limits have bounded densities? How optimal are the rates? Sharpen the method to obtain better results? Develop other characteristic function based methods? Higher order estimates? of our approach Random matrices - p. 33/33