Gain Function Approximation in the Feedback Particle Filter

Transcription

1 Gain Function Approimation in the Feedback Particle Filter Amirhossein Taghvaei, Prashant G. Mehta Abstract This paper is concerned with numerical algorithms for gain function approimation in the feedback particle filter. The eact gain function is the solution of a Poisson equation involving a probability-weighted Laplacian. The problem is to approimate this solution using only particles sampled from the probability distribution. Two algorithms are presented: a Galerkin algorithm and a graph-based algorithm. Both the algorithms are adapted to the samples and do not require approimation of the probability distribution as an intermediate step. The paper contains error analysis for the algorithms as well as some comparative numerical results for a non-gaussian distribution. These algorithms are also applied and illustrated for a simple nonlinear filtering eample. I. ITRODUCTIO This paper is concerned with algorithms for numerically approimating the solution of a certain linear partial differential equation (pde) that arises in the problem of nonlinear filtering. In continuous time, the filtering problem pertains to the following stochastic differential equations (sdes): dx t = a(x t )dt + db t, dz t = h(x t )dt + dw t, (a) (b) where X t R d is the (hidden) state at time t, Z t R is the observation, and {B t }, {W t } are two mutually independent standard Wiener processes taking values in R d and R, respectively. The mappings a( ) R d R d and h( ) R d R are C functions. Unless noted otherwise, all probability distributions are assumed to be absolutely continuous with respect to the Lebesgue measure, and therefore will be identified with their densities. The choice of observation being scalar-valued (Z t R) is made for notational ease. The objective of the filtering problem is to estimate the posterior distribution of X t given the time history of observations (filtration) Z t = σ(z s s t). The density of the posterior distribution is denoted by p, so that for any measurable set A R d, A p (,t) d = P[X t A Z t ]. The filter is infinite-dimensional since it defines the evolution, in the space of probability measures, of {p (,t) t }. If a( ), h( ) are linear functions, the solution is given by the finite-dimensional Kalman-Bucy filter. The article [3] surveys numerical methods to approimate the nonlinear Financial support from the SF CMMI grants and is gratefully acknowledged. A. Taghvaei and P. G. Mehta are with the Coordinated Science Laboratory and the Department of Mechanical Science and Engineering at the University of Illinois at Urbana-Champaign (UIUC). taghvae@illinois.edu; mehtapg@illinois.edu; filter. One approach described in this survey is particle filtering. The particle filter is a simulation-based algorithm to approimate the filtering task [3]. The key step is the construction of stochastic processes {Xt i i }: The value Xt i R d is the state for the i-th particle at time t. For each time t, the empirical distribution formed by the particle population is used to approimate the posterior distribution. Recall that this is defined for any measurable set A R d by, p () (A,t) = i= [X i t A]. A common approach in particle filtering is called sequential importance sampling, where particles are generated according to their importance weight at every time step [], [3]. In our earlier papers [7], [6], [4], an alternative feedback control-based approach to the construction of a particle filter was introduced. The resulting particle filter, referred to as the feedback particle filter (FPF), is a controlled system. The dynamics of the i-th particle have the following gain feedback form, dxt i = a(xt i )dt + dbt i +K t (Xt i ) (dz t h(x t i )+ĥ t dt), () where {Bt} i are mutually independent standard Wiener processes and ĥ t = E[h(Xt i ) Z t ]. The initial condition X i is drawn from the initial density p (,) of X independent of {Bt}. i Both {Bt} i and {X i } are also assumed to be independent of X t,z t. The indicates that the sde is epressed in its Stratonovich form. The gain function K t is obtained by solving a weighted Poisson equation: For each fied time t, the function φ is the solution to a Poisson equation, BVP (p(,t) φ(,t)) = (h() ĥ)p(,t), φ(,t)p(,t) d = (zero-mean), for all R d where and denote the gradient and the divergence operators, respectively, and p denotes the conditional density of Xt i given Z t. In terms of the solution φ, the gain function is given by, (3) K t () = φ(,t). (4) ote that the gain function K t is vector-valued (with dimension d ) and it needs to be obtained for each fied time t. For the linear Gaussian case, the gain function is the Kalman gain. For the general nonlinear non-gaussian problem, the FPF () is eact, given an eact gain function and an eact initialization p(,) = p (,). Consequently, if the initial

2 conditions {X i} i= are drawn from the initial density p (,) of X, then, as, the empirical distribution of the particle system approimates the posterior density p (,t) for each t. A numerical implementation of the FPF () requires a numerical approimation of the gain function K t and the mean ĥ t at each time-step. The mean is approimated empirically, ĥ t i= h(xt i ) = ĥ () t. The gain function approimation - the focus of this paper - is a challenging problem because of two reasons: i) Apart from the linear Gaussian case, there are no known closed-form solutions of (3); ii) The density p(,t) is not eplicitly known. At each time-step, one only has samples X i. These are assumed to be i.i.d sampled from p. Apart from the FPF algorithm, solution of the Poisson equation is also central to a number of other algorithms for nonlinear filtering [5], [5]. In our prior work, we have obtained results on eistence, uniqueness and regularity of the solution to the Poisson equation, based on certain weak formulation of the Poisson equation. The weak formulation led to a Galerkin numerical algorithm. The main limitation of the Galerkin is that the algorithm requires a pre-defined set of basis functions - which scales poorly with the dimension d of the state. The Galerkin algorithm can also ehibit certain numerical issues related to the Gibb s phenomena. This can in turn lead to numerical instabilities in simulating the FPF. The contributions of this paper are as follows: We present a new basis-free kernel-based algorithm for approimating the solution of the gain function. The key step is to construct a Markov matri on a certain graph defined on the space of particles {X i } i=. The value of the function φ for the particles, φ(x i ), is then approimated by simply solving a fied-point problem involving the Markov matri. The fiedpoint problem is shown to be a contraction and the method of successive approimation applies to numerically obtain the solution. We present results on error analysis for both the Galerkin and the kernel-based method. These results are illustrated with the aid of an eample involving a multi-modal distribution. Finally, the two methods are compared for a filtering problem with a non-gaussian distribution. In the remainder of this paper, we epress the linear operator in (3) as a weighted Laplacian ρ φ = ρ (ρ φ) where additional assumptions on the density ρ appear in the main body of the paper. In recent years, this operator and the associated Markov semigroup have received considerable attention with several applications including spectral clustering, dimensionality reduction, supervised learning etc [4], [9]. For a mathematical treatment, see the monographs [7], []. Related specifically to control theory, there are important connections with stochastic stability of Markov operators [6], []. The outline of this paper is as follows: The mathematical preliminaries appear in Sec. II. The Galerkin and the Graphbased algorithms for the gain function approimation appear in Sec. III and Sec. IV, respectively. The nonlinear filtering eample appears in Sec. V. BVP II. MATHEMATICAL PRELIMIARIES The Poisson equation (3) is epressed as, ρ φ = h, φρ d = (zero-mean), where ρ is a probability density on R d, ρ φ = ρ (ρ φ) and, without loss of generality, it is assumed ĥ = hρ d =. Problem statement: Approimate the solution φ(x i ) and φ(x i ) given independent samples X i drawn from ρ. The density ρ is not eplicitly known. For the problem to be well-posed requires definition of the function spaces and additional assumptions on ρ and h enumerated net: Throughout this paper, µ is an absolutely continuous probability measure on R n with associated density ρ. L (R d,µ) is the Hilbert space of square integrable functions on R d equipped with the inner-product, < φ,ψ > = φ()ψ()dµ(). The associated norm is denoted as φ =< φ,φ >. The space H (R d,µ) is the space of square integrable functions whose derivative (defined in the weak sense) is in L (R d,µ). We use L and H to denote L (R d,µ) and H (R d,µ), respectively. For the zero-mean solution of interest, we additionally define the co-dimension subspace L = {φ L ; φ dµ = } and H = {φ H ; φ dµ = }. L is used to denote the space of functions that are bounded a.e. (Lebesgue) and the sup-norm of a function φ L is denoted as φ. The following assumptions are made throughout the paper: (i) Assumption A: The probability density function ρ is of the form ρ() = e ( µ)t Σ ( µ) V () where µ R d, Σ is a positive-definite matri and V C with V,DV,D V L. (ii) Assumption A: The function h L and hdµ =. Under assumption A, the density ρ admits a spectral gap (or Poincaré inequality)[], i.e λ > such that, (5) φ ρ d λ φ ρ d, φ H. (6) Furthermore, the spectrum is known to be discrete with an ordered sequence of eigenvalues = λ < λ λ and associated eigenfunctions {e n } that form a complete orthonormal basis of L [Corollary 4..9 in []]. The trivial eigenvalue λ = with associated eigenfunction e =. On the subspace of zero-mean functions, the spectral decomposition yields: For φ L, ρ φ = λ m < e m,φ > e m. m= The spectral gap condition (6) implies that λ >. Consequently, the semigroup e t ρ φ = e tλ m < e m,φ > e m (7) m=

3 8 φ () Fig.. The eact solution to the Poisson equation using the formula (8). The density ρ is the sum of two Gaussians (,σ ) and (+,σ ), and h() =. The density is depicted as the shaded curve in the background. is a strict contraction on the subspace L. It is also easy to see that µ is an invariant measure and e t ρ φ()dµ() = φ()dµ() = for all φ L. Eample : If the density ρ is Gaussian with mean µ R d and a positive-definite covariance matri Σ, the spectral gap constant (/λ ) equals the largest eigenvalue of Σ. The eigenfunctions are the Hermite functions. Given a linear function h() = H, the unique solution of the BVP (5) is given by, φ() = (ΣH) ( µ), where denotes the vector dot product in R d. In this case, K = φ = ΣH is the Kalman gain. Eample : In the scalar case (where d = ), the Poisson equation is: ρ() d d dφ (ρ() ()) = h. d Integrating twice yields the solution eplicitly, dφ d () = ρ() ρ(z)h(z)dz dy y (8) φ() = ρ(y) ρ(z)h(z)dz For the particular choice of ρ as the sum of two Gaussians (,σ ) and (+,σ ) with σ =.4 and h() =, the solution obtained using (8) is depicted in Fig.. III. GALERKI FOR GAI FUCTIO APPROXIMATIO Weak formulation: A function φ H is said to be a weak solution of Poisson s equation (5) if < φ, ψ >=< h,ψ >, ψ H. (9) It is shown in [] that, under Assumptions (A)-(A), there eists a unique weak solution of the Poisson equation. The Galerkin approimation involves solving (9) in a finite-dimensional subspace S H (Rd ;ρ). The solution φ is approimated as, M φ (M) () = c m ψ m (), () m= where {ψ m ()} M m= are a given set of basis functions. The finite-dimensional approimation of (9) is to choose constants {c m } M m= such that < φ (M), ψ >=< h,ψ >, ψ S, () where S = span{ψ,,ψ M } H. Denoting [A] ml =< ψ l ψ m >, b m =< h,ψ m >, and c = (c,c,,c M ) T, the finite-dimensional approimation () is epressed as a linear matri equation: Ac = b. () In a numerical implementation, the matri A and vector b are approimated as, [A] ml =< ψ l ψ m > b m =< h,ψ m > i= i= ψ l (X i ) ψ m (X i ) = [A] () ml, (3) h(x i )ψ m (X i ) = b () m. (4) The resulting solution of the matri equation (), with A = A () and b = b (), is denoted as c (), A () c () = b (). (5) Using (), we obtain the particle-based approimation of the solution: M φ (M,) () = m= c () m ψ m (). (6) In terms of this solution, the gain function is obtained as, M φ M, () = m= c () m ψ m (). Convergence analysis: The following is a summary of the approimations in the Galerkin algorithm: < φ, ψ > =< h,ψ >, ψ H Galerkin appro: < φ (M), ψ > =< h,ψ >, ψ S H Empirical approimation: i= φ (M,) (X i ) ψ(x i ) = i= h(x i )ψ(x i ), ψ S We are interested in the error analysis of these approimations as a function of both M and. ote that the approimation error φ (M) φ (M,) is random because X i are sampled randomly from the probability distribution µ. The following Proposition provides error bounds for the case where the basis functions are the eigenfunctions of the Laplacian. The proof appears in the Appendi.

4 8 6 φ () 4 modes 3 modes 5 modes 3 3 Fig.. Comparison of the eact solution and its empirical Galerkin approimation with M =,3,5 modes and = particles. The density is depicted as the shaded curve in the background. Proposition : Consider the empirical Galerkin approimation of the Poisson equation (9) on the space S = span{e,e,,e M } of the first M eigenfunctions. Fi M <. Then the solution for the matri equation (5) eists with probability approaching as. And there is a sequence of random variables { } such that where as a.s. φ (M,) φ λm h +, (7) Remark : In practice, the eigenfunctions of the Laplacian are not known. The basis functions are typically picked from the polynomial family, e.g., the Hermite functions. In this case, the bounds will provide qualitative assessment of the error provided the eigenfunctions associated with the first M eigenvalues are approimately in S. Quantitatively, the additional error may be bounded in terms of the projection error between the eigenfunction and the projection onto S. Eample 3: We net revisit the bimodal distribution first introduced in the Eample. Fig. depicts the empirical Galerkin approimation, with = samples, of the gain function. The basis functions are from the polynomial family - {,,, M } - and the figure depicts the gain functions with M =,3,5 modes. The (M = ) case is referred to as the constant-gain approimation where, dφ d () = h() dµ() = (const.) For the linear Gaussian case, the (const.) is the Kalman gain. The plots included in the figure demonstrate the Gibb s phenomena as more and more modes are included in the Galerkin. Particularly concerning is the change in the sign of the gain which can lead to numerical issues in simulating the filter. This is further discussed with the aid of a filtering eample in Sec. V. IV. SEMIGROUP APPROXIMATIO OF THE GAI Semigroup formulation: The semigroup formula (7) is used to construct the solution of the Poisson equation by solving the following fied-point equation for any fied positive value of t: φ = e t t ρ φ + es ρ hds. (8) A unique solution eists because e t ρ is a contraction on L. The approimation proposed in this section involves approimating the semigroup as a perturbed integral operator, for small positive values of t =. The following approimation of the semigroup appears in [4], [9]: where k () (,y) = T () φ() = k () (,y)φ(y)dµ(y) k (), (9) (,y)dµ(y) g () (,y) g () (,y)dµ(y) g () (,y)dµ() and g () (,y) = ep( y (4π) d 4 ) is the Gaussian kernel in R d. In terms of the perturbed integral operator, the fied-point equation (8) becomes, φ () = T () φ () + T (s) hds. () The superscript is used to distinguish the approimate (dependent) solution from the eact solution φ. As shown in the Appendi B, T () has an ergodic invariant measure µ () which approimates µ as. For any fied positive, we are interested in solutions that are zero-mean with respect to this measure. The eistence-uniqueness result for this solution is described net; the proof appears in the Appendi. Proposition : Consider the fied-point problem () with the perturbed operator T () defined according to (9). Fi >. Then there eists a unique solution φ () such that φ () dµ () =. In a numerical implementation, the solution φ is approimated directly for the particles: Φ (,) = (φ () (X ),φ () (X ),,φ () (X )). The integral operator T () is approimated as a Markov matri whose (i, j) entry is obtained empirically as, where k (,) (,y) = T (,) k (,) (X i,x j ) i, j = l= k(,) (X i,x l ), () g (,y) l= g (,X l ) l= g (y,x l ). The resulting finite-dimensional fied-point equation is given by, Φ (,) = T (,) Φ (,) + T (s,) H () ds, () where Φ (,) = (Φ (,),Φ (,),,Φ (,) ) R is the vectorvalued solution, H () = (h(x ),h(x ),,h(x )) R, and T (,) and T (s,) are matrices defined according to (). The eistence-uniqueness result for the zero-mean

5 solution of the finite-dimensional equation () is described net; its proof is given in the Appendi. The zero-mean property is the finite-dimensional counterpart of the zeromean condition φ dµ = for the original problem and φ dµ = for the perturbed problem. Proposition 3: Consider the fied-point problem () with the matri T (,) defined according to (). Then, with probability, there eists a unique zero-mean solution Φ (,) R d. Once Φ (,) is available, it is straightforward to etend it to the entire domain. For R d, φ (,) () = i= k (,) (,X i )Φ (,) i + i= k (,) (,X i ) T (s,) h()ds. (3) By construction φ (,) (X i ) = Φ (,) i for i =,,. The etension is not necessary for filtering because one only needs to solve for φ at X i. The formula for this is given by, φ (X i ) = l j= [T (,) i j Φ (,) j (X j l k= T (,) ik Xl k )] where X i l R is the l-component of X i R d. The algorithm is summarized in Algorithm. For numerical purposes we use H s T (s,) H () ds. Algorithm Kernel-based gain function approimation Input: {X i } i=, H = {h(x i )} i=, Output: Φ = {φ(x i )} i=, { φ(x i )} i= Calculate g i j = ep( X i X j /4) for i, j = to. Calculate k i j = Calculate T i j = g i j l g il l g jl for i, j = to. k i j l k il for i, j = to. Solve Φ = T Φ+H for Φ (Iteration). Calculate φ (X i ) = l j= [T i j Φ j (X j l k= T ikxl k )] for l = to d. Convergence: The following is a summary of the approimations with the kernel-based method: φ = e ρ φ + es ρ hds Kernel appro: φ () = T () φ () + T (s) hds Empirical appro: φ (,) = T (,) φ (,) + T (s,) hds We break the convergence analysis into two steps. The first step involves convergence of φ (,) to φ () as. The second step involves convergence of φ () to φ as. The following theorem states the convergence result for the first step; a sketch of the proof appears in Appendi D. Theorem : Consider the empirical kernel approimation of the fied-point equation (8). Fi >. Then, 8 φ () 6 4 ɛ =. ɛ =. ɛ =.4 ɛ = Fig. 3. Comparison of the eact solution and its kernel-based approimation with =.,.,.4,.8 and = particles. The density is depicted as the shaded curve in the background. (i) There eists a unique (zero-mean) solution φ () for the perturbed fied-point equation (). (ii) For any finite, a unique (zero-mean) solution Φ (,) for () eists with probability. For a compact set Ω R d, lim sup φ (,) () φ () () =, a.s, (4) Ω where φ (,) is the etension of the vector-valued solution Φ (,) (see (3)). The convergence analysis for step, as is subject of future work. In this regard, it is shown in [8] that for all functions f C 3 (Ω) where Ω R d is compact, Ω, T () f () = ρ f ()+O() Eample 4: Consider once more the bimodal distribution introduced in. Figure 3 depicts the kernel-based approimation of the gain function with = particles and range of. The kernel-based avoids the Gibbs phenomena observed with the Galerkin (see Fig. ). ote that the gain function is positive for any choice of. V. UMERICS In this section, we consider a filtering problem associated with the bimodal distribution introduced in Eample. The filtering model is given by, dx t =, (5) dz t = X t dt +σ w dw t, (6) where X t R, Z t R, {W t } is standard Wiener process, and the initial condition X is sampled from the bimodal distribution comprising of two Gaussians, (,σ ) and (+,σ ). As in Eample, the observation function h() = is linear. The

6 static case is considered because the posterior is given by the following eplicit formula: p (,t) (const.)ep(h()(z t Z ) h() t) p (). (7) The following filtering algorithms are implemented for comparison: ) Kalman filter; Galerkin with basis function {}; ) Feedback particle filter with the Galerkin approimation where S = span{,, 3, 4, 5 }; 3) Feedback particle filter with the kernel approimation. The performance metrics are as follows: ) Filter mean ˆX t ; ) Conditional probability P[ X t X < Z t]; The simulation parameters areas follows: The true initial state X =. The measurement noise parameter σ W =.3. The simulation is carried out over a finite time-horizon t [,T ] with T =.8 and a fied discretized time-step t =.. All the particle filters use = particles and have the same initialization, where particles are drawn with equal probability from one of the two Gaussians, (,σ ) or (+,σ ), where σ =.. For the Markov approimation, we use =.5. The simulation parameters are tabulated in Table I. Figure 4 depicts the particle trajectories and the associated distributions obtained using the Markov approimation and the Galerkin approimation. Figure 4 depicts a comparison of the simulation results for the two metrics. These metrics are computed empirically, e.g., the filter mean is the empirical mean of the population i= Xt i. Parameters µ σ σ w T Value TABLE I SIMULATIO PARAMETERS [] Vivian Hutson, J Pym, and M Cloud. Applications of functional analysis and operator theory, volume. Elsevier, 5. [] R. S. Laugesen, P. G. Mehta, S. P. Meyn, and M. Raginsky. Poisson s equation in nonlinear filtering. SIAM Journal on Control and Optimization, 53():5 55, 5. [] Sean P Meyn and Richard L Tweedie. Markov chains and stochastic stability. Springer Science & Business Media,. [3] Adrian Smith, Arnaud Doucet, ando de Freitas, and eil Gordon. Sequential Monte Carlo methods in practice. Springer Science & Business Media, 3. [4] T. Yang, P. G. Mehta, and S. P. Meyn. Feedback particle filter. IEEE Trans. Automatic Control, 58():465 48, October 3. [5] Tao Yang, Henk AP Blom, and Prashant G Mehta. The continuousdiscrete time feedback particle filter. In American Control Conference (ACC), 4, pages IEEE, 4. [6] Tao Yang, Prashant G Mehta, and Sean P Meyn. Feedback particle filter with mean-field coupling. In Decision and Control and European Control Conference (CDC-ECC), 5th IEEE Conference on, pages IEEE,. [7] Tao Yang, Prashant G Mehta, and Sean P Meyn. A mean-field controloriented approach to particle filtering. In American Control Conference (ACC),, pages IEEE,. A. Proof of Proposition APPEDIX Using the spectral representation, because h L, φ = ρ h = m= < e m,h > e m. λ m With basis functions as eigenfunctions, Therefore, φ (M) = φ φ (M) = M m= m=m+ < e m,h > e m. λ m λ m < e m,h > λm h. REFERECES [] Alan Bain and Dan Crisan. Fundamentals of stochastic filtering, volume 3. Springer, 9. [] Dominique Bakry, Ivan Gentil, and Michel Ledou. Analysis and geometry of Markov diffusion operators, volume 348. Springer Science & Business Media, 3. [3] Amarjit Budhiraja, Lingji Chen, and Chihoon Lee. A survey of numerical methods for nonlinear filtering problems. Physica D: onlinear Phenomena, 3():7 36, 7. [4] Ronald R Coifman and Stéphane Lafon. Diffusion maps. Applied and computational harmonic analysis, ():5 3, 6. [5] F. Daum, J. Huang, and A. oushin. particle flow for nonlinear filters. In SPIE Defense, Security, and Sensing, pages ,. [6] Peter W Glynn and Sean P Meyn. A liapounov bound for solutions of the poisson equation. The Annals of Probability, pages 96 93, 996. [7] Aleander GrigorâĂŹyan. Heat kernels on weighted manifolds and applications. Cont. Math, 398:93 9, 6. [8] Matthias Hein, Jean-Yves Audibert, and Ulrike Von Luburg. From graphs to manifolds weak and strong pointwise consistency of graph laplacians. In Learning theory, pages Springer, 5. [9] Matthias Hein, Jean-Yves Audibert, and Ulrike Von Luburg. Graph laplacians and their convergence on random neighborhood graphs. arxiv preprint math/685, 6. et, by the triangle inequality, φ φ (M,) φ φ (M) + φ (M) φ (M,). We show φ (M) φ (M,) a.s. as. Using the formulae () and (6) with basis-functions ψ m as eigenfunctions e m, φ (M) φ (M,) = c c, where denotes the Euclidean norm in R M. The vectors c and c () solve the matri equations (see () and (5)), Ac = b, A () c () = b (), where A () a.s. A, b () a.s. b by the strong law of large numbers. Consequently, because A = diag(λ,...,λ ) is invertible, c () c a.s. as.

7 Particles FPF (Kernel) True state Particles FPF (Galerkin) True state Xt i Xt i t t t t ρ(t,) ρ () (t,) ρ(t,) ρ () (t,) (a) (b). Filter mean P(X t >.5 Z t ) ˆX t.4.. FPF (Kernel) FPF (Galerkin) Kalman filter t (c).6 p.4. FPF (Kernel) FPF (Galerkin) Kalman filter t (d) Fig. 4. Summary of simulation results: (a) Posterior distribution and trajectory of particles for scenario I, Kernel-based approach, (b) Posterior distribution and trajectory of particles for scenario I, Galerkin approach, (c) Conditional mean of the state X t, (d) Conditional probability of X t larger than. B. Proof of Proposition Denote n () () = k () (,y)dµ(y), and re-write the operator T () as, T () f () = k () (,y) n () ()n () (y) f (y) dµ() (y), where dµ () () = n () ()dµ(). Denote L (µ () ) as the space of square integrable functions with respect to µ () and as before L (µ() ) = {φ L (µ () ) φ dµ () = } is the co-dimension subspace of mean-zero functions in L (µ () ). The technical part of proving the Proposition is to show that the operator T () is a strict contraction on the subspace. Lemma : Suppose ρ, the density of the probability measure µ, satisfies Assumption A. Then, (i) µ () is a finite measure. (ii) For sufficiently small values of, the operator T () L (µ () ) L (µ () ) is a compact Markov operator with an invariant measure µ (). (iii) T () L (µ() ) L (µ() ) is a strict contraction. Proof: (i) WLOG assume µ = in the Assumption A. For notational ease, denote ρ () () = g () (,y)ρ(y)dy, where recall that ρ () () is used to define the denominator

8 of the kernel. Then c ep( T Q ρ () () c ep( T Q ), where Q = (Σ+I) and c = (π) d Q e V and c = (π) d Q e V are positive constants that depend V. Therefore µ () (R d ) = k () (,y)dµ()dµ(y) = g () (,y) ρ () () ρ () (y) ρ()ρ(y)ddy c g () (,y)e T Q e yt Q y ddy, which is bounded because Q = Σ Q. This proves that µ () is a finite measure. (ii) The integral operator T is a Markov operator because the kernel k () (,y) > and T () = k () (,y) n () ()n () (y) dµ() (y) = T is compact because [Theorem 7..7 in []], k() (,y) n () dµ(y) =. () k () (,y) n () ()n () (y) dµ () ()dµ () (y) <. The inequality holds because the integrand can be bounded by a Gaussian: AMIR Finally, the measure µ is an invariant measure because for all functions f, T () f () dµ () () = f (y) dµ () (y). (iii) Since µ () is an invariant measure, T () L (µ() ) L (µ() ). et, for all f, g L (µ() ), f () dµ () ()+ g(y) dµ () (y) =, k () (,y) n () ()n () (y) f ()g(y)dµ() ()dµ () (y) k () (,y) n () ()n () (y) ( f () g(y)) dµ () ()dµ () (y) where, because the kernel is everywhere positive, the equality holds iff f () = g(y) = (const.) µ a.e. Therefore, substituting g = T () f in the inequality above, Therefore, T () is strictly contractive on L (µ() ). The proof of the Prop. now follows because T () is a contraction on L (µ() ). C. Proof of Proposition 3 By construction, T (,) is a stochastic matri whose entries are all positive with probability. The result follows. D. Proof sketch for Theorem Parts (i) and (ii) have already been proved as part of the Prop. and Prop. 3, respectively. The convergence result (4) leans on approimation theory for integral operators. In [Theorem in []], it is shown that if (i) T () is compact and T (,) is collectively compact; (ii) T () < ; (iii) T (,) converges to T () pointwise, i.e lim T (,) f T () f = a.s, f Then for large enough, (I T (,) ) is bounded and lim (I T(,) ) h (I T () ) h =, a.s h. For showing the convergence result, we consider the Banach space C (Ω) = { f C(Ω); f dµ() = } equipped with the norm. We consider T () and T (,) as linear operators on C (Ω). ote that T (,) here corresponds to the etension of the solution to R d using (3). The proof involves verification of each of the requirements stated above: (i) Collective compactness follows because k () is continuous [Theorem 7..6 []]. (ii) The norm condition holds because T () f () k () (,y) f (y) dµ(y) k () (,y)dµ(y) k () (,y)dµ(y) k () (,y)dµ(y) f f where the equality holds only when f is constant. This constant must be zero on C (Ω). (iii) Pointwise convergence follows from using the LL. For a fied continuous function f, LL implies convergence for every fied point R d. On a compact set Ω, pointwise convergence also implies uniform convergence. T () f f + T () f T () f f, where the L norms here are with respect to the invariant measure µ. ow, for f L (µ() ), f dµ () =, and thus the equality holds iff f () = (const.) =.