Stein s Method and Characteristic Functions

Transcription

1 Stein s Method and Characteristic Functions Alexander Tikhomirov Komi Science Center of Ural Division of RAS, Syktyvkar, Russia; Singapore, NUS, May 2015 Workshop New Directions in Stein s method

2 Topics Central Limit Theorems for Sum of Independent and Dependent Random variables CLT for Maximal Sum CLT for Multivariate Random Variables CLT for Hilbert Space Valued Random Variables Semi circular Law for Random Matrices CLT for Linear Statistics of Eigenvalues of Random Matrices

3 Topics Central Limit Theorems for Sum of Independent and Dependent Random variables CLT for Maximal Sum CLT for Multivariate Random Variables CLT for Hilbert Space Valued Random Variables Semi circular Law for Random Matrices CLT for Linear Statistics of Eigenvalues of Random Matrices

4 Topics Central Limit Theorems for Sum of Independent and Dependent Random variables CLT for Maximal Sum CLT for Multivariate Random Variables CLT for Hilbert Space Valued Random Variables Semi circular Law for Random Matrices CLT for Linear Statistics of Eigenvalues of Random Matrices

5 Topics Central Limit Theorems for Sum of Independent and Dependent Random variables CLT for Maximal Sum CLT for Multivariate Random Variables CLT for Hilbert Space Valued Random Variables Semi circular Law for Random Matrices CLT for Linear Statistics of Eigenvalues of Random Matrices

6 Topics Central Limit Theorems for Sum of Independent and Dependent Random variables CLT for Maximal Sum CLT for Multivariate Random Variables CLT for Hilbert Space Valued Random Variables Semi circular Law for Random Matrices CLT for Linear Statistics of Eigenvalues of Random Matrices

7 Topics Central Limit Theorems for Sum of Independent and Dependent Random variables CLT for Maximal Sum CLT for Multivariate Random Variables CLT for Hilbert Space Valued Random Variables Semi circular Law for Random Matrices CLT for Linear Statistics of Eigenvalues of Random Matrices

8 Stein Comments The abstract normal approximation theorem of Section 2 is elementary in the sense that it uses only the basic properties of conditional expectation and elements of analysis, including the solution of a first order linear differential equation. It also direct, in the sense the expectation of a fairly arbitrary function of the random variable W in question is approximated directly without going through characteristic function.

9 Stein Comments. Continue Because of the clumsiness of the technique used on this point, it is likely that slightly better results could be obtained by using the basic identity, Lemma 2.1, to approximate the characteristic function of W and by standard procedures, to go from this to an approximation for the distribution of W. However, I believe that, in the long run, better results will be obtained by direct methods. Ch. Stein, 1972

10 The Notation Let {Ω, M, Pr} be probability space and let, for any n 1, F n M be sub σ-algebra of M s.t. F n F n+1. Let (G n ) n=1 be a sequence of r.v. s with E G n <. Let W n := E{G n F n }. For any sequence of r.v. s Wn with E Wn < define the following quantities: D n := W n Wn, (1)

11 The Notation. Continue ( e D n 1 ) δ n (t) := E G n 1, it ε n1 (t) := E G n e itw n, { (Gn ε n2 (t) := E E (e Dn 1) E G n (e Dn 1) ) }, Fn ε n (t) := ε n1 (t) + ε n2 (t), δn (T ) := sup δ n (t). t: t T

12 The main Lemma Lemma For any 0γ < 1 and for any T > 0 s. t. δ n (T ) γ, there exists an absolute constant C > 0 s.t. the following inequality holds sup Pr{W n x} Φ(x) C x T + δ n (T ) 1 γ where Φ(x) = 1 2π x e u2 2 du. + 1 T 1 γ 0 e t2 2 t t 0 ε n (t) e u2 2 dudt,

13 The proof of Main Lemma Let f n (t) := E e i twn. Hence E W n < and E G n < we may write f n(t) = i E W n e itwn = i E G n e itwn. (2) Continuing, we get f n(t) = i E G n e i tw n + i f n (t) E G n (1 e i t Dn ) [ { + E E G n (1 e i tdn ) E G n (1 e i tdn ) F n }]e i twn.

14 The Continuing Let θ( ) (with an index or without) denote any function s.t. θ( ) 1, By symbol C we shall denote an absolute constant, which can be changed from row to row. We may rewrite now the previous equation for ch. f. in the form f n(t) = t(1 δ n ( t))f n (t) + θ n (t)ε n (t).

15 Solving this equation with initial condition f n (0) = 1, we get t f n (t) = exp { t2 2 + ( 1 + Furthermore, note that t u } uδ n (u)du 0 t 0 { u 2 u } ) θ n (u)ε n (u) exp 2 vδ n (v)dv du. 0 vδ n (v)dv γ2 2 (t2 u 2 ), for u t T. (3)

16 The last relation immediately implies that, for t T, f n (t) = exp { t2 2 (1 + θ n1(t) δ n (T )) } + θ n2 (t) t From here it follows that f n (t) e t2 2 δn (T)t 2 e (1 γ)t ε n (u) exp { (t2 u 2 )(1 γ) } du. 2 t 0 ε n (u) e (1 γ)(t2 u 2 ) 2 du. (4)

17 Applying the famous Esseen s inequality, we get sup Pr{W n x} Φ(x) C x T + δ n (T ) 1 γ + 1 T e (1 γ)t2 /2 ( t 1 γ 0 t 0 ) ε(u) e (1 γ)u2/2 du dt. (5)

18 Corollary Corollary Assume that δ n (t) 0 and ε n (t) 0 as n uniformly in all bounded interval [0, T ]. Then lim sup Pr{W n x} Φ(x) = 0. (6) n x

19 Applications of Main Lemma. The Sums of Weakly Dependent R.V. s Let X 1, X 2,... be a stationary sequence of random variables with E X 1 = 0 and E X1 2 = 1. Let Mb a denote σ-algebra generated by X j, when j [a, b]. We denote the strong mixing coefficient of the sequence (X j ) j=1 by the equality α(m) := sup Pr(AB) Pr(A) Pr(B) (7) A M k 1, B M k+m

20 Theorem Let the following conditions hold for some δ > 0, Then m=1 [α(m)] δ 2+δ <, E X 1 2+δ <, n lim n n 1 E( X j ) 2 = σ 2 > 0. lim sup Pr{B 1 n n x j=1 n X j x} Φ(x) = 0 (8) j=1

21 To prove this result we use our Lemma. Let I be uniformly distributed on the set {1,..., n} random variable. Put G = nb 1 n X I. Then W = E{G F}. Put W = B 1 Then we have the estimation n j: j I >m X j. ε n1 (t) = E Ge itw CnB 1 n E δ 2+δ X1 2+δ [α(m)] 1+δ 2+δ. (9) Furthermore, δ n (t) 1 B n n j=1 where D nj = B 1 n e itd nj 1 it E X j C t 1+δ m 1+δ E X 1 2+δ, it n k: j k m X j. (10)

22 For ε n2 (t) we have the following estimation ε n2 = E 1 n X j (e itd nj 1) E X j (e itd nj 1) C t δ mδ. (11) B n j=1 n δ 2

23 CLT for Quadratic forms Let X 1, X 2,... be a sequence of r.v. s s.t. E X k = 0, E X 2 k = 1 and let the sequence be uniformly integrated, i.e. sup E Xk 2 I{ X k > M} 0, as M 0. (12) k 1 We consider the quadratic form Q = n j,k=1 a (n) jk X jx k, where the matrix of coefficients A n = (a (n) jk )n j,k=1 is supposed symmetric and diagonal entries of A are equal zero, a (n) kk = 0.

24 Let A denote the operator norm of matrix A and let A 2 denote the Hilbert Schmidt norm of matrix A. We shall assume that lim n This condition is equivalent to A n A n 2 = 0. (13) λ (n) 1 lim = 0, (14) n σ n where Λ (n) 1 is the maximum modulus eigenvalue of A n and σ 2 n = E Q 2 n = 2 A n 2 2.

25 Theorem Assume conditions (13) and (12). Then sup Pr{σn 1 Q n x} Φ(x) 0 as n. (15) x

26 To apply main Lemma, we denote by T j := {1,..., n} \ {j} and introduce r.v. s ξ j = X j k T j a jk X k. Note that Q = n j=1 ξ j. We put G = nξ I. We introduce W := Q (I), where Q (I) := k,l T I a kl X k X l. Note that W W = 2ξ I First we assume that X j M for all j = 1,..., n. We may estimate now δ n (t) and ε n (t). For instance n n ε n1 (it) 1 E ξ l (e 2itξ l 12 itξ l ) C t 2 E ξ l 3 1=1 Ct 2 M 2 l=1 n ( a kl 2 ) 3 2 C t 2 λ n. (16) k T l l=1

27 Asymptotic distribution of quadratic forms Consider the quadratic forms Q = 1 j k n a jk X j X k + n j=1 a jj (X 2 j 1) Let V 2 = n j=1 a2 jj, L2 j = n k=1 a2 jk, A 2 2 = n j,k=1 a2 jk.

28 Let µ k = E X k 1. Let A(0) = (a jk ) n j,k=1 with A(0) jj = 0, and let d = (a 11,..., a nn ). Let Y and Y be two independent copies of standard Gaussian vector in R n. Let G := µ 2 < A (0) Y, Y > +µ 3 µ < d, Y > + µ 4 µ 2 2 < d, Y >.

29 Characterized equation Denote by R t the resolvent matrix of A 0., R t = (I 2itµ 2 A (0) ) 1 Introduce the function G(t) = µ 2 Tr (R t A (0) ) t 2 µ 2 3 < d, R2 t A (0) d > +it(µ 4 µ 2 2 ) < d, d >. (17)

30 The characteristic function f n (t) = E exp{itq n } satisfy the equation f n(t) = ig(t)f (t) + ε n (t), (18) where the function ε n (t) convergence to 0 uniformly in all bounded intervals.

31 CLT for Generalized U-statistics Consider statistics of type U 2 = 1 l l n g lj(x l, X k ). Denote by σ 2 kj := E g 2 k,j (X k, X j ) <, g k,l (x, y) we shall assume (a) g lj (x, y) = g jl (y, x); (b) E g kj (X k, X j ) X j ) = 0; (c) (d) lim lim max σ 2 M 1 k j k n σn 2 n (e) lim σn 2 n σn 2 = 2 k,j σ2 kj. About the functions kj E gkj 2 (X k, X j )I{ g kj (X k, X j ) M} = 0; E g lk (X l, X k )g j,k (X j, X k ) 2 = 0; l j k k σ2 jk = 0.

32 Theorem Assuming conditions (a) (e), we have sup Pr{σn 1 U 2 x} Φ(x) 0 as n. x Moreover, there exist an absolute positive constant C > 0 s.t. sup x ( Pr{σn 1 U 2 x} Φ(x) C +σ 4 n k n j,k=1 E g jk (X j, X k ) 4 + σ 4 n +σ 4 n k n j ( k n E k=1 σ 3 n k n j,k=1 k n j,k=1 E g jk (X j, X k ) 3 k n E g kl (X k, X l )g jl (X j, X l ) l=1 E{gjk 2 (X j, X k ) ) 2 ) 1 2 X j }. 2

33 Maximal sum It is well-known that the limit distribution of the maximal sum of independent random variables is 2 x G(x) = e u2 /2 dui{x 0}, (19) π 0 i.e. the distribution of module of standard Gaussian r.v. Note that Φ(x) = (1 G( x) + G(x))/2 and f (t) = Re g(t), where f (t), g(t) are characteristic functions of Φ(x) and G(x) respectively. That mean we may proof CLT for the symmetrization of maximal sum using our Lemma.

34 CLT for Hilbert-valued r.v. s Let X1, X2,... be a sequence of random elements of the Hilbert space H with zero mean, EX j = O, and covariation operators A j = cov(x j ). Suppose the sequence {Xj : j = 1...} satisfies the uniformly strong mixing condition with coefficient ϕ(m). Let S n = n j=1 X j, B n = cov(s n ), Z n = S n Bn. (20)

35 The well-known equations for the characteristic function f (t) = E exp{it ξ + a 2 }): ( f (t) = i Tr B(I 2itB) 1) f (t)+ < a, (I 2B) 2 a > f (t).

36 The main result is the following theorem: Theorem Let E X j 3 L < and constants K, β > 0 exist such that ϕ(m) < K exp{ βm}, for all m 1. Let also the following conditions hold: A A n lim n n = A and ϕ(m) < 1. (21) Then there exists a constant c depending on the first eigenvalues of the operator A such that sup Pr{ Z n +a < r} Pr{ Y n +a < r} cln 1/2 log{n(1+ a )}, r where Y n, is a Gaussian r. v. with covariance operator equal to the covariance A. Tikhomirov, operator Syktyvkar, ofrussia Zn andstein with Metod mean zero.

37 Semi Circular Law for Random Matrices Let p(x) denote the density of semi-circular law, p(x) = 1 2π 4 x 2 I [ 2,2] (x), and let f (t) be the characteristic function of semicircular law, f (t) = eitx p(x)dx. Lemma The characteristic function of semi-circular law is the unit solution of the equation f (t) = t 0 f (u)f (t u)du, f (0) = 1. (22)

38 Recall that the moments {α k } of semi-circular law satisfies the relation We have representation 0 k 1 α k+1 = α ν α k ν. (23) f (t) = i ν=0 k=1 ν=0 µ=0 (it) k k! α k+1 From the other hand side t t (iu) µ (i(t u)) µ f (u)f (t u)du = α ν α µ du ν!µ! = ν=0 µ=0 0 α ν α µ i µ+ν t µ+ν+1 (µ + ν + 1)!

39 Changing the order of summing, we get t 0 f (u)f (t u)du = s=0 i s t s+1 (s + 1)! s α ν α s ν. ν=0 Continuing this equality and applying relation (23), we obtain t 0 f (u)f (t u)du = s=0 i s t s+1 (s + 1)! α s+2 = i s=1 (it) s s! α s+1 = f (t).

40 Lemma Let for the characteristic functions f n (t) the following representation holds t f n(t) = f n (s)f n (t s)ds + ε n (t), (24) 0 where ε n (t) 0 as n uniformly for all bounded intervals. Then lim f n(t) = f (t), (25) n where f (t) is characteristic functions of semi-circular law.

41 We represent f n (t) in the form t t f n (t) = 1 + s 0 f n(u)f n (s u)duds + ε n (s)ds. (26) 0 0 If we put now g n (t) = g(t) then we get g n (t) = t s 0 0 t g n (u)g n (s u) + ε n (s)ds. (27) 0 Let g n (t) = sup s [0,t] g n (s) and ε n (t) = sup s [0,t] ε n (s) It is straightforward to check that, for any k 1 g n (t) 2 t k k! g n (t) + e t ε n (t). (28) The last inequality implies the claim.

42 How we may used the last Lemma? Let W = 1 n (X jk ) n j,k=1 be n n Hermitian matrix with independent entries (up to symmetry). Let U(t) = e itw. Then f n (t) = 1 ntr U(t). We have E f n(t) = i n E Tr WU(t) = i n j,k=1 E X jku kj (t). It is well-known n n U pq (t) X jk = i(1 + δ jk ) 1 (U jp U kq (t) + U pk U jq (t)), (29) where f g(t) := t 0 f (s)g(t s)ds.

43 Using the last relations, we get t E f n(t) = E 0 ( 1 n 2 n j,k=1 ) t (U jj (s)u kk (t s)ds +ε n (t) = 0 E f n (s)f n (t (30)

44 For the Marchenko Pastur Law we may used representation for characteristic function of symmetrizing law f (t) = y t 0 t f (s)f (t s)ds (1 y) f (s)ds. (31) 0

45 Thank you for your attention!