Asymptotic normality of randomly truncated stochastic algorithms

Transcription

1 Asymptotic normality of randomly truncated stochastic algorithms Jérôme Lelong To cite this version: Jérôme Lelong. Asymptotic normality of randomly truncated stochastic algorithms. ESAIM: Probability and Statistics, EDP Sciences, 3, 7, pp.5-9. <.5/ps/>. <hal-46438v> HAL Id: hal Submitted on May HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

2 Asymptotic normality of randomly truncated stochastic algorithms Jérôme Lelong May, Abstract We study the convergence rate of randomly truncated stochastic algorithms, which consist in the truncation of the standard Robbins-Monro procedure on an increasing sequence of compact sets. Such a truncation is often required in practice to ensure convergence when standard algorithms fail because the expected-value function grows too fast. In this work, we give a self contained proof of a central limit theorem for this algorithm under local assumptions on the expected-value function, which are fairly easy to check in practice. Key words: stochastic approximation, central limit theorem, randomly truncated stochastic algorithms, martingale arrays. Introduction The use of stochastic algorithms is widespread for solving stochastic optimization problems. These algorithms are extremely valuable for a practical use and particularly well suited to localize the zero of a function u. Such algorithms go back to the pioneering work of Robbins and Monro [95], who considered the sequence X n+ = X n γ n+ u(x n γ n+ δm n+ ( to estimate the zero of the function u. The sequence (γ n n classically denotes the gain or step sequence of the algorithm and (δm n n depicts a random measurement error. Nevertheless, the assumptions required to ensure the convergence basically, a sub-linear growth of u on average are barely satisfied in practice, which dramatically reduces the range of applications. Chen and Zhu [986] proposed a modified algorithm to deal with fast growing functions. Their new algorithm can be summed up as X n+ = X n γ n+ u(x n γ n+ δm n+ +γ n+ p n+ ( where (p n n is a truncation term ensuring that the sequence (X n n cannot jump too far ahead in one step. In this paper, we are concerned with the rate of convergence of Equation (. Numerous results are known for the sequence defined by Equation (, which is known to converge at Laboratoire Jean Kuntzmann, Université de Grenoble et CNRS, BP 53, 384 Grenoble Cédex 9, FRANCE, jerome.lelong@imag.fr

3 Randomly truncated stochastic algorithms the rate γ n when γ n is of the form γ n with / < α (see Delyon [996], Duflo [997] α or Buche and Kushner [] for instance. Let x denote the unique root of u, u(x =. When γ n = γ n and u(x is of the form λi, Duflo [997] showed that the convergence rate depends on the relative position of λ and γ. A functional central limit theorem for this algorithm was proved by Bouton [985] and Benveniste et al. [99]. The convergence rate of constrained algorithms was studied by Kushner and Yin [3]. The problem of multiple targets was tackled by Pelletier [998] who proved a Central Limit Theorem. However, very few results are known about the convergence rate of the algorithm devised by Chen and Zhu [986]. Chen [] briefly studied the convergence rate under global hypotheses on the noise sequence (δm n n. Here, we aim at giving a clarified, self-contained and elementary proof of this result under local assumptions (see Section.3 for a detailed comparison of the two results. Besides giving a clarified and self-contained proof of the central limit theorem for randomly truncated algorithm, the improvement brought by our work is the use of the local condition sup n E[ δm n +ρ { Xn x η}] < with some ρ > and η > replacing the global condition sup n E[ δm n +ρ ] <. First, we define the general framework and explain the algorithm developed by Chen and Zhu [986]. Our main results are stated in Theorems. and. (see page 4 depending on the decreasing speed of the sequence (γ n n. In Sections.3 and.4, we discuss the improvements brought by our new results and we give a concrete example to show the benefits of using local assumptions. Section 3 is devoted to the proof of the main results. Notations If z is a complex number, we denote by Re(z (resp. Im(z its real (resp. imaginary part. The prime notation denotes the transpose operator, eg. A stands for the transpose of A. For any vector x, x denotes its Eucildean norm and the associated scalar product. For any square matrix A, A denotes the matrix norm associated to, ie. A = sup x Ax. For any matrix A, A is equal tothe squareroot of thelargest eigenvalue of A A. If A is a square matrix, we denote by Sp(A the eigenvalues of A. We say that a square matrix A is repulsive if all the eigenvalues of A have positive real parts. A CLT for randomly truncated stochastic algorithms It is quite common to look for the root of a continuous function u: x R d u(x R d, which is not easily tractable. We assume that we can only access u up to a measurement error embodied in the following by the sequence (δm n n and that the norm u(x grows faster than x such that the standard Robbins-Monro algorithm (see Equation ( quickly fails. Instead, we consider the alternative procedure introduced by Chen and Zhu[986]. This

4 J. Lelong 3 technique consists in forcing the algorithm to remain in an increasing sequence of compact sets (K j j such that K j = R d and j, K j int(k j+. j= It prevents the algorithm from blowing up during the first iterates. Let (γ n n be a decreasing sequence of positive real numbers satisfying n γ n = and n γ n <. For X R d and σ =, we define the sequences of random variables (X n n and (σ n n by X n+ = X n γ n+ u(x n γ n+ δm n+, if X n+ K σn X n+ = X n+ and σ n+ = σ n, if X n+ / K σn X n+ = X and σ n+ = σ n +. Let F n denote the σ algebra generated by (δm k,k n, F n = σ(δm k,k n. We assume that (δm n n is a sequence of martingale increments, i.e. E(δM n+ F n =. Remark. X n+ is actually drawn from the dynamics of the Robbins-Monro algorithm (see Equation (. If the standard algorithm wants to jump too far ahead it is reset to a fixed value. When X n+ / K σn, one can set X n+ to any measurable function of (X,...,X n with values in a given compact set. The existence of such a compact set is crucial to prove the a.s. convergence of (X n n. It is more convenient to rewrite Equation (3 as follows X n+ = X n γ n+ u(x n γ n+ δm n+ +γ n+ p n+ (4 where ( p n+ = u(x n +δm n+ + (X X n { }. γ n+ X n+ / K σn In this paper, we only consider gain sequences of the type γ n = γ (n+, with / < α. If α α =, we obtain a slightly different limit. For values of α outside this range, the almost sure convergence is not even guarantied.. Hypotheses In the following, the prime notation stands for the transpose operator. We introduce the following hypotheses. (A i. x R d s.t. u(x = and x R d, x x, (x x u(x >. ii. There exist a function y : R d R d d satisfying lim x y(x = and a repulsive matrix A such that u(x = A(x x +y(x x (x x. (A For any q >, the series n γ n+δm n+ { Xn x q} converges almost surely. (3 (A3 i. There exist two real numbers ρ > and η > such that ( κ = supe δm n +ρ { Xn x η} <. n

5 4 Randomly truncated stochastic algorithms ii. There exists a symmetric positive definite matrix Σ such that E ( δm n δm n P Fn { Xn x η} Σ. n (A4 There exists µ > such that n, d(x, K n µ. Remark. Comments on the assumptions.. Hypothesis (A-i is satisfied as soon as u can beinterpreted as thegradient of a strictly convex function. The Hypothesis (A-ii is equivalent to saying that u is differentiable at x.. Hypothesis (A ensures that X n x a.s. and σ n is almost surely finite, see Lelong [9] for a proof of this result. 3. Hypothesis (A3-i corresponds to some local uniform integrability condition and reminds of Lindeberg s condition. (A3-ii guaranties the convergence of the angle bracket of the martingale of interest. 4. Hypothesis (A4 is only required for technical reasons but one does not need to be concerned with it in practical applications. It reminds of the case of constrained stochastic algorithms for which the CLT can only be proved for non saturated constraints. The idea of using local assumptions, ie. assumptions to be checked in the neighbourhood of x was already used in the results of Duflo [996] but unfortunately her proof cannot be easily adapted as being in a neighbourhood of x does not ensure that there is no projection anymore.. Main results For n, we define the renormalized and centered error n = X n x γn. A CLT for / < α < Theorem.. If we assume Hypotheses (A to (A4, the sequence ( n n converges in distribution to a normal random variable with mean and covariance A CLT for α = V = exp( AtΣexp( A tdt. Theorem.. We assume Hypotheses (A to (A4 and (A5 γa I is repulsive

6 J. Lelong 5 Then, the sequence ( n n converges in distribution to a normal random variable with mean and covariance (( (( I I V = γ exp γa t Σexp γa t dt. Remark 3. Hypothesis (A5 involves the gradient of function u at the point x, which is seldom tractable from a practical point of view but one can definitely not avoid it. The condition Re(Sp(γA I > is the border of two different convergence regimes as already noted by Duflo [997] for the Robbins-Monro algorithm..3 Discussion around the assumptions of Theorem. Theorem. is actually an extension of Chen [, Theorem 3.3.]. The main improvements brought by our new result concern the conditions imposed on the noise term. Our Assumption (A3 is weaker than the one imposed by Chen [, A3.3.3] since we only assume local conditions on the noise terms; namely, unlike Chen, we only need to monitor the behavior of (δm n n in a small neighborhood of the optimum x (see Assumptions (A3-i and (A3-ii. Moreover, we only assume the local convergence in probability of the angle bracket of the martingale of interest built with (δm n n whereas Chen [, Equation (3.3.] requires the almost sure convergence which may be a little harder to prove in practical applications. Concerning Assumption (A-ii, it essentially means that u must be differentiable at x. This is to be compared to the Hölder continuity property of the remainder of the first order expansion of u at x required by Chen [, A3.3.4], which is not so obvious to check in practice. Our goal in this work was not only to state a theorem with weaker assumptions but also to present a self contained and elementary proof of a central limit theorem for truncated stochastic algorithms. In particular, Lemma 3. provides a smart way of handling the truncation terms..4 Example of applications In this part, we present an application of Algorithm(3 to adaptive variance reduction and show the improvement brought by the localisation of the assumptions. The adaptive Monte Carlo framework recalled hereafter was investigated by Arouna [4] and later revisited by Lemaire and Pagès [], Lapeyre and Lelong []. Let G be a d dimensional standard normal random vector. For any measurable function h : R d R such that E( h(g <, one has for all θ R d E(h(G = E (e θ G θ h(g+θ. (5 Assume we want to compute E(f(G using Monte Carlo simulations for a measurable function f : R d R such that P(f(G > and there exists ε >, s.t. E( f(g 4+ε <. By applying equality (5 to h = f and h(x = f (xe θ x+ θ, one obtains that the θ G θ expectation and the variance of the random variable f(g + θe are respectively equal to E(f(G and v(θ E (f(g where v(θ = E (f (Ge θ G+ θ.

7 6 Randomly truncated stochastic algorithms The function v can be proved to be of infinitely continuously differentiable and strongly convex. Moreover, v(θ = E (f (Ge θ G+ θ (θ G. Hence, v has a unique minimizer characterised by v(θ =. As the computation of the expectation E(f(G will be carried out using simulations, it is advised to make the most of the free parameter θ in Equation (5 by choosing θ = θ. Obviously, this heavily relies on the ability to compute θ numerically. Note that v can be written as an expectation of a function U : R d R d R d defined by U(θ,x = f (xe θ x+ θ (θ x. If we consider Algorithm (3 for our problem, we get θ n+ = θ n γ n+ v(θ n γ n+ δm n+, if θ n+ K σn θ n+ = θ n+ and σ n+ = σ n, if θ n+ / K σn θ n+ = θ and σ n+ = σ n + with δm n+ = U(θ n,g n+ v(θ n where (G n n is a sequence of.i.i.d. standard normal random vectors. Remark 4. Note that using Hölder s inequality and the assumptions on f, one can show that for any ρ < ε/ and A >, E(sup θ <A U(θ,G +ρ < Let us have a look at the assumptions of Theorems. and.. Assumption (A is satisfied as v is strongly convex and the matrix A appearing in the assumption is symmetric positive definite. Let q >. The sequence Y n = n i= γ i+δm i+ { θi θ q} is a martingale with angle bracket Y n = n i= γ i+ E(δM i+δm i+ F i { θi θ q}. As γi <, by Remark 4, E( Y < which implies, usingthestronglaw of large numbersforsquareintegrable martingales, that Y n converges almost surely. Let η > and ρ < ε/. C denotes a positive constant. ( E( δm n +ρ { θn θ η} F n C E( sup U(θ, G +ρ + sup v(θ +ρ θ θ η θ θ η The second term on the r.h.s is bounded as v is continuous and the first term on the r.h.s. is bounded by applying Remark 4. The indicator of the set { θ n θ η} is definitely essential to prove the boundedness. Assumptions (A and (A imply that θ n converges almost surely to θ. E(δM i+ δm i+ F i = E(U(θ,GU(θ,G θ=θi v(θ i v(θ i. Thecontinuity of v ensurestheconvergence of v(θ i v(θ i to v(θ v(θ. Moreover thanks to Remark 4, the expectation E(U(θ,GU(θ,G is continuous w.r.t. θ. Hence, the matrix Σ of Assumption (Aii is given by Var(U(θ,G which is therefore positive definite. As we have just seen it, having local assumptions makes them pretty easy to check in practical situations.

8 J. Lelong 7 3 Proofs of Theorems. and. In this section, we prove the Theorems presented in Section. through a series of three lemmas. The proofs of these lemmas are postponed to Section Technical lemmas For any fixed n >, we introduce s n,k = k i= γ n+i for k > and we set s n, =. (s n,k k can be interpreted as a discretisation grid of [, because lim k s n,k =. Theorems. and. are based on the following three lemmas. Lemma 3.. Let ε > and η > as in Hypothesis (A3. There exists N >, such that if we define for n N { } A n = sup X m x η n m N, then ( P(A n ε n N and sup E n {An} <. n N Lemma 3.. For any integers t > and n > t t n+t = e sn,tq n e Q(s n,k s n,t γn+k+ δm n+k+ e Q(s n,k s n,t γ n+k R n+k,(6 where if α =, Q = A γ I R m = y(x m x m + γm+ p m+ +γ m (a m I +b m (A+y(X m x +O(γ m m, (7 if / < α <, Q = A R m = y(x m x m γm+ p m+ (8 mγ m (a m I +b m γ n (A+y(X m x m +O(γ m m with (a n n and (b n n two real valued and bounded sequences. Moreover, the last term in (6 tends to zero in probability. Lemma 3.3. In Equation (6, the sequence ( t eq(s n,k s n,t γn+k+ δm n+k+ t converges in distribution to N(,V for any fixed n when t goes to infinity, where V = e Qu Σe Q u du.

9 8 Randomly truncated stochastic algorithms Proof of Theorems. and.. Let us consider Equation (6 for a fixed n > N, where N is defined in Lemma 3.. Because the matrix Q is definite positive and n is almost surely finite, e sn,tq n tends to zero almost surely when t goes to infinity. Thanks to Lemma 3., the last term in Equation (6 tends to zero in probability when t goes to infinity. Combining these two convergences in probability to zero with Lemma 3.3 yields the convergence in distribution of ( n+t t to a normal random variable with mean and variance V when t goes to infinity, where V is defined in Lemma 3.3. Plugging the value of the matrix Q (see Equations (7 and (8 in the expression of V yields the result. Note that the proof for the classical Robbins Monro algorithm is much simpler since we do not need to introduce the A n sets, which are only used here to handle the truncation terms. 3. Proofs of the lemmas 3.. Proof of Lemma 3. We only do the proof in the case α =, as in the other case, it is sufficient to slightly modify a few Taylor expansions and the same results still hold. From Equation (4, we have the following recursive relation n+ = X n+ x γn = n γ n+ (u(x n +δm n+ p n+. γn+ γ n+ Using Hypothesis (A-ii, the previous equation becomes n+ = ( γn I γ n+ γ n (A+y(X n x n γ n+ δm n+ + γ n+ p n+. (9 γ n+ The following Taylor expansions hold γn = + γ n γ n+ γ +O( γn and γn γ n+ = γ n +O ( γn. ( There exist two real valued and bounded sequences (a n n and (b n n such that γn γ n+ = + γ n γ +γ na n and γ n γ n+ = γ n +γ nb n. This enables us to simplify Equation (9 n+ = n γ n Q n γ n y(x n x n γ n+ δm n+ + γ n+ p n+ +γ n (a ni +b n (A+y(X n x n, (

10 J. Lelong 9 where Q = A I γ. Let n+ = X n+ x, where X n+, defined by Equation (3, is the γn+ value of the new iterate obtained before truncation. n+ = n γ n Q n γ n y(x n x n γ n+ δm n+ +γ n(a n I +b n (A+y(X n x I n n γ n Q n γ n y(x n x n +γ n+ δm n+ +γ 4 n (a n I +b n (A+y(X n x I n +γ n ( n γ n Q n γ n y(x n x n δm n+ +γ 4 n((a n I +b n (A+y(X n x I n ( n γ n Q n γ n y(x n x n +γ 5/ n δm n+ (a ni +b n (A+y(X n x I M n. If we take the conditional expectation with respect to F n denoted E n in the previous equality, we find n+ E n n γ n Q n γ n y(x n x n +γ n+ E n δm n+ +γ 4 n a ni +b n (A+y(X n x I n +γ 4 n (a n I +b n (A+y(X n x I (+γ n Q+y(X n x n. ( n+ E n n (I γ nq n +γ n y(x n x n +γ n+ E n δm n+ +O ( γ n (+ y(xn x n. ( Note that in the previous equation the quantity O ( γ n is non random. Let ε >. Since (X n n converges almost surely to x, there exists a rank N such that P( sup m>n X m x > η < ε. Hence, P(A n ε for all n > N. Let λ = min{re(l;l Sp(Q} > and < ζ < λ. We can assume that for n > N, γ n /ζ, then it ensues from Proposition A. that I γ n Q γ n ζ. Moreover, since lim x y(x =, there exists η > such that for all x < η, y(x < ζ/. We assume that this value of η satisfies Hypothesis (A3. On the set A n, n (I γ nq n +γ n y(x n x n (( γ n ζ+γ n ζ/ n ( γ n ζ n. Hence, we can deduce from Equation ( that E( n+ ( {An} E n {An} ( γ n ζe n {An} +γ n κ +O(γ n(+ ζe ( n {An} We can assume that for n > N, O(γn (+ζ/ γ nζ/. Hence we get, for n N, E( n+ ( {An} E n ζ ( {An} γ n E n {An} +γ n κ.

11 Randomly truncated stochastic algorithms Since A n+ A n, E( n+ {An+ } Now, we would like to replace n+ ( E n ζ ( {An} γ n E n {An} +κγ n, (3 by n+ in Equation (3. n+ = X x γ {pn+ } + n+ {pn+ =}, n+ n+ n+ + X x { }. γ n+ X n+ / K σn Taking the conditional expectation w.r.t. F n on the set A n gives E n n+ n+ E n + X x ( P X γ n+ n+ E n n+ n+ {An} E n {An} + X x ( E n+ {An+ } γ n+ n+ E( {An} + X x γ n+ / K σn F n, {An}P ( X n+ ( P A n {X n+ The probability on the right hand side can be rewritten ( P A n {X n+ / K σn } = E ( {γn+ u(x n+δm n+ d(x n, K σn } {An} / K σn F n, / K σn }.(4 Moreover using the triangle inequality, we have d(x n, K σn d(x, K σn X n x. Due to Hypothesis (A4, d(x, K σn µ and on A n, X n x η. Hence, d(x n, K σn µ η. One can choose η < µ/ for instance, so that d(x n, K σn > µ. ( P A n {X n+ ( / K σn } E (E n {γn+ u(x n+δm n+ µ } {An}, 8γ ( n+ µ E u(x n {An} + δm n+ {An}. (5 Thanks to Hypothesis (A3 and the continuity of u, the expectation on the r.h.s of (5 is bounded by a constant c > independent of n. So, we get ( P X n+ / K σn,a n cγn+. Hence, from Equation (4 we can deduce ( E n+ {An+ } E( n+ {An} By combining Equations (6 and (3, we come up with ( E n+ {An+ } ( ζ ( γ n E n {An} +cγ n, +cγ n. (6

12 J. Lelong where c = c+κ. ( } Let I = {i N : ζ E i {Ai } +c >, then ( supe i {Ai } < c i I ζ <. ( Note that we can always assume that c/ζ E N {AN }, such that the set I is non empty. Assume i / I, let i = sup{k < i : k I}. ( ( E i {Ai } E i {Ai } i k=i γ k (c ζ ( E k {Ak } Since all the terms for k = i +,...,i are negative and i I, we find ( E i {Ai } γ i c+ c ζ. Finally, we come with the following upper bound. ( sup E n {An} n N <. Remark 5 (case / < α <. This proof is still valid for α < if we replace the Taylor expansions of Equation ( by γn = + a n γ n+ n and γ n γ n+ = γ n + γ nb n n. Then, Equation ( becomes n+ = n γ n Q n γ n y(x n x n γ n+ δm n+ with Q = A this time, which is still positive definite. 3.. Proof of Lemma 3. + γ n+ p n+ + n (a ni +b n γ n (A+y(X n x n, Let us go back to Equation (. For any n > N and k >, we can write n+k = n+k γ n+k Q n+k γ n+k δm n+k +γ n+k R n+k where R m = y(x m x m + γm+ p m+ +γ m (a m I +b m (A+y(X m x m, We can actually notice that the previous equation pretty much looks like a discrete time ODE. Based on this remark, it is natural to multiply the previous equation by e s n,kq to find e s n,kq n+k (e s n,kq e s n,kq γ n+k Q n+k = e s n,kq γn+k δm n+k +γ n+k e s n,kq R n+k

13 Randomly truncated stochastic algorithms Note that e sn,kq e sn,kq γ n+k Q = e sn,k Q ( + O(γn+k. Hence, we come up with the following equation where e s n,kq n+k e Q n+k = e s n,kq γn+k δm n+k +γ n+k e s n,kq R n+k R m = y(x m x m + γm+ p m+ +γ m (a m I +b m (A+y(X m x +O( m, (7 When summing the previous equalities for k =,...,t for any integer t >, we get t t n+t = e sn,tq n e (s n,k s n,tq γn+k+ δm n+k+ e (s n,k s n,tq γ n+k R n+k, Let us a have a closer look at the different terms of Equation (7 lim m y(x m x m { Xm x >η} = a.s. thanks to the a.s. convergence of (X m m and using Lemma 3., the sequence (y(x m x m { Xm x η} m is uniformly integrable and tends to zero in probability because lim m y(x m x = a.s. p m is almost surely equal to for m large enough thanks to Remark, so γm p m = a.s. for m large enough. γ m (a m I + b m (A + y(x m x + O( m { Xm x >η} almost surely because for m large enough the indicator equals. The sequence γ m (a m I + b m (A + y(x m x + O( m { Xm x η} is uniformly integrable by Lemma 3. and tends to zero in probability because γ m. Hence, R m canbesplitintwoterms: onetendingtozeroalmostsurelyandanotheronewhich is uniformly integrable and tends to zero in probability. Then, we can apply Propositions A. and A.3 to prove the convergence in probability of ( t e(s n,k s n,tq γ n+k R n+k t. This last point ends the proof of Lemma Proof of Lemma 3.3 To prove Lemma 3.3, we need a result on the rate of convergence of martingale arrays. First, note that for γn δm n { Xn x >η} tends to a.s. when n goes to infinity because { Xn x >η} = for n large enough. Then, it ensues from Proposition A. that t k= eq(s n,k s n,t γn+k δm n+k { Xn+k x >η} converges to zero in probability when t goes to infinity. Henceforth, it is sufficient to prove a localized version of Lemma 3.3 by considering t eq(s n,k s n,t γn+k δm n+k { Xn+k x η}. We will use the following Central Limit Theorem for martingale arrays adapted from Duflo [997, Theorem..9]. Theorem 3.4. Suppose that {(F t l l t;t > } is a family of filtrations and {(N t l l t;t > } a square integrable martingale array with respect to the previous filtration. Assume that : (A6 there exists a symmetric positive definite matrix Γ such that N t t P t Γ.

14 J. Lelong 3 (A7 There exists ρ > such that l= N t E( l Nl t +ρ F t P l. t Then, N t t L t N(,Γ. Using this theorem, we can now prove Lemma 3.3. Proof of Lemma 3.3. Let us define N t l for all l t and t > N t l = l e (s n,k s n,tq γn+k δm n+k { Xn+k x η}. k= We should have written instead of s n,k, but as e ( s n,tq = e (s n,k s n,tq e γ n+kq and e γ n+kq converges to the identity matrix, we can make this little change without altering the rigor of the proof and this way, N t l naturally fits in the framework of Propositions A. and A.4. (N t l l p is obviously a martingale with respect to (F n+l l. Let us compute its angle bracket N t t = e (s n,k s n,tq γ n+k E ( δm n+k δm n+k { X n+k x η} F n+k e (s n,k s n,tq.(8 k= Thanks to Hypotheses (A3, the conditional expectation in (8 is uniformly integrable and converges in probability to Σ when k goes to infinity. Applying Proposition A.4 proves the convergence in probability of N t t to e Q u Σe Qu du. Let ρ be the real number defined in Theorem.. l= N t E( l Nl t +ρ = k= e (s n,k s n,tq +ρ γ +ρ n+k ( δm E n+k +ρ { Xn+k x η}. γ ρ n+k converges to when k goes to infinity and the sequence of expectations is bounded ( using Hypothesis (A3, so γ ρ n+k E δm n+k +ρ { Xn+k x η} tends to zero when k goes to infinity. Proposition A. proves that the l.h.s. of Equation (9 tends to when t goes to infinity. Hence, ( t N l= E t l N p +ρ l F t l tends to zero in L, and consequently in probability. Then, the Hypotheses of Theorem 3.4 are satisfied. Finally, we have proved that ( e Q(s n,k s n,t law γn+k δm n+k N, t e Qu Σe Q u du. (9

15 4 Randomly truncated stochastic algorithms 4 Conclusion In this work, we have proved a Central Limit Theorem with rate γ n for randomly truncated stochastic algorithms under local assumptions. We have also tried to clarify the proof of the convergence rate of randomly truncated stochastic algorithms under assumptions which can be easily verified in practice. The improvement brought by this new set of assumptions is that all they should only be checked in a neighbourhood of the target value x, which means that in the case where u(x = E(U(x,Z the assumptions can be reformulated in terms of some local regularity properties of U. A Some elementary results Proposition A.. Let Q be a square matrix such that all its eigenvalues have positive real parts. Let λ > = min{re(µ;µ Sp(Q}. Then, for all < λ < λ, there exists < γ < /λ, such that for all γ γ, I γq ( γλ. Proof. We consider the Jordan decomposition J of Q such that Q = PJP. I γq I γj. Let M = sup{im(µ;µ Sp(Q} and γ = max{re(µ;µ Sp(Q}. If γ < max(γ,/λ, the diagonal terms of I γj are bounded by ( γλ +(γm = ( γλ(+o(γ. The conclusion follows easily. Proposition A.. Let (Y n n be a sequence of random vectors of R d converging almost surely to a non random vector x R d. For any fixed integer n > and repulsive matrix Q R d d, we define, for all integers t, Z t = t eq(s n,k s n,t γ n+k Y n+k. Then, lim t Z t = e Qu x du almost surely. Proof. It is clear that lim t e Qu du x = e Qu du x. Hence, it is sufficient to consider Z t e Qu du x e Q(s γ n,k s n,t n+k Y n+k x + γ n+k e Q(s n,k s n,t e Qu du x. ( Let q = min{re(λ;λ Sp(Q} > and q = max{re(λ;λ Sp(Q} >. Step : We will prove that the first term in Equation ( tends to almost surely. e Q(s γ n,k s n,t n+k Y n+k x e q(s n,k s n,t Y n+k x du e q(u sn,t e qγ n+tn(u Y n+tn(u x du ( where for any real number u >, t n (u is the largest integer k such that u < s n,k. Note that lim u + t n (u = +. lim u + e qγ n+tn(u Y n+tn(u x = a.s., hence it is obvious that the term on the r.h.s of Equation ( tend to almost surely.

16 J. Lelong 5 Step : We will now prove that the second term in Equation ( tends to. We use the convention s n, = and recall that s n,k = + γ n+k. Note that sn,t e Qu du = s n,t e Q(u sn,t du, hence the following inequality holds γ n+k e Q(s n,k s n,t e Qu du e Q(s n,k s n,t e Q(u sn,t du e Q(u sn,t e Q(s n,k u I du. e q(u sn,t (e qγ n+k du. ( Let ε >, there exits T > such that for all t T, (e qγ n+t ε, hence for all t > T, T e q(u sn,t (e qγ n+k du sn,t e q(u sn,t (e q du+ε e q(u sn,t (e q du+ε (e q(s n,t s n,t e qsn,t eq q k=t + s n,t e q(u sn,t du, +ε q. e q(u sn,t du, There exists T > T such that for all t > T, (e q(s n,t s n,t e qsn,t (e q ε, hence for all t > T, e q(u sn,t (e qγ n+k du ε q. ThisendstoprovethatthesecondterminEquation(tendstowhentgoestoinfinity. Proposition A.3. Let (Y n n be a sequence of random vectors of R d uniformly integrable and converging in probability to a non random vector x R d. For any fixed integer n > and repulsive matrix Q R d d, we define, for all integers t, Z t = t eq(s n,k s n,t γ n+k Y n+k. Then, lim t Z t = e Qu x du in probability. Proof. We recall the decomposition given by Equation ( Z t e Qu du x e Q(s γ n,k s n,t n+k Y n+k x + γ n+k e Q(s n,k s n,t e Qu du x. Thelast termintheaboveequationhasalreadybeenprovedtotendtointheproofofproposition A. Step. So, we only need to prove that lim t u + γ n+k e Q(s n,k s n,t Yn+k x = in probability.

17 6 Randomly truncated stochastic algorithms Let q = min{re(λ;λ Sp(Q} > and q = max{re(λ;λ Sp(Q} >. e Q(s γ n,k s n,t n+k Y n+k x e q(s n,k s n,t Y n+k x du e q(u sn,t e qγ n+tn(u Y n+tn(u x du where for any real number u >, t n (u is the largest integer k such that u < s n,k. Let Y k = γ k Y k x. The sequence (Y k k tends to zero in probability, is uniformly integrable and positive. ( E e q(u sn,t Y n+tn(u du = e q(u sn,t E(Y n+tn(udu. (3 Since (Y k k is uniformly integrable and converges to in probability, lim u + E(Y n+tn(u =, hence the term on the r.h.s of Equation (3 tends to when t goes to infinity. This proves that lim t u + γ n+k e Q(s n,k s n,t Yn+k x = in L and in probability. Proposition A.4. Let (Y n n R d d be a sequence of random square matrices R d uniformely integrable and converging in probability to a non random matrix X R d d. For any fixed integer n > and repulsive matrix Q R d d, we define, for all integers t, Z t = t eq(s n,k s n,t γ n+k Y n+k e Q (s n,k s n,t. Then, lim t Z t = e Qu Xe Q u du in probability. Proof. The proof is very similar to the previous but for sake of completeness we make it. Let q = min{re(λ;λ Sp(Q} > and q = max{re(λ;λ Sp(Q} >. We use a decomposition similar to Equation ( Z t e Qu X e Q u du e Q(s γ n,k s n,t n+k (Y n+k Xe Q (s n,k s n,t + γ n+k e Q(s n,k s n,t X e Q (s n,k s n,t e Qu Xe Q u du γ n+k e q(s n,k s n,t Y n+k X + γ n+k e Q(s n,k s n,t X e Q (s n,k s n,t e Qu Xe Q u du The convergence in probability to zero of the first term on the r.h.s of the above equation is

18 J. Lelong 7 ensured by Proposition A.3. We rewrite the second term as follows γ n+k e Q(s n,k s n,t Xe Q (s n,k s n,t e Qu X e Q u du = e Q(s n,k s n,t X e Q (s n,k s n,t e Q(u sn,t X e Q (u s n,t du = e Q(u sn,t (e Q(sn,k u X e Q (s n,k u Xe Q (u s n,t du = e Q(u sn,t (e Q(sn,k u IX e Q (s n,k u +X(e Q (s n,k u Ie Q (u s n,t du { } (e q(sn,k u e q(sn,k u +(e q(sn,k u du X e q(u sn,t e q(u sn,t (e q(s n,k u (e q(s n,k u +du X The proof is ended by closely following the same reasoning as to prove that the r.h.s of Equation ( tends to. References B. Arouna. Adaptative Monte Carlo method, a variance reduction technique. Monte Carlo Methods Appl., (: 4, 4. A. Benveniste, M. Métivier, and P. Priouret. Adaptive algorithms and stochastic approximations, volume of Applications of Mathematics (New York. Springer-Verlag, Berlin, 99. Translated from the French by Stephen S. Wilson. C. Bouton. Approximation Gaussienne d algorithmes stochastiques à dynamique Markovienne. PhD thesis, Université Pierre et Marie Curie - Paris 6, 985. R. Buche and H. J. Kushner. Rate of convergence for constrained stochastic approximation algorithms. SIAM J. Control Optim., 4(4: 4 (electronic,. H. Chen and Y. Zhu. Stochastic Approximation Procedure with randomly varying truncations. Scientia Sinica Series, 986. H.-F. Chen. Stochastic approximation and its applications, volume 64 of Nonconvex Optimization and its Applications. Kluwer Academic Publishers, Dordrecht,. B. Delyon. General results on the convergence of stochastic algorithms. IEEE Transactions on Automatic Control, 4(9:45 55, 996. M. Duflo. Algorithmes stochastiques (Mathématiques et Applications. Springer, 996. M. Duflo. Random Iterative Models. Springer-Verlag Berlin and New York, 997.

19 8 Randomly truncated stochastic algorithms H. J. Kushner and G. G. Yin. Stochastic approximation and recursive algorithms and applications, volume 35 of Applications of Mathematics (New York. Springer-Verlag, New York, second edition, 3. Stochastic Modelling and Applied Probability. B. Lapeyre and J. Lelong. A framework for adaptive Monte Carlo procedures. Monte Carlo Methods Appl.,. J. Lelong. Almost sure convergence of randomly truncated stochastic agorithms under verifiable conditions. Statistics & Probability Letters, 78, 9. V. Lemaire and G. Pagès. Unconstrained Recursive Importance Sampling. Ann. Appl. Probab., (3:9 67,. M. Pelletier. Weak convergence rates for stochastic approximation with application to multiple targets and simulated annealing. Ann. Appl. Probab., 8(: 44, 998. H. Robbins and S. Monro. A stochastic approximation method. Ann. Math. Statistics, : 4 47, 95.