UFD u. It is also known that u = u(t; x) is the solution of a certain stochastic partial

Transcription

1 Nonlinear Filtering: Separation of Parameters and Oservations Using Galerkin Approximation and Wiener Chaos Decomposition C. P. Fung S. Lototsky y Astract The nonlinear ltering prolem is considered for the time homogeneous diffusion model. An algorithm is proposed for computing a recursive approximation of the unnormalized ltering density, and the error of the approximation is estimated. The algorithm can have high resolution in time and, unlike most existing algorithms for nonlinear ltering, can e implemented in real time for large dimensions of the sate process. The on-line part of the algorithm is simplied y performing the most time consuming operations o line. 1 Introduction In many prolems of stochastic analysis it is necessary to nd the est mean square estimate of moments or other similar functionals of a partially oserved diusion process. Assume that = (t); t ; is the unoservale component and the oservale component Y = Y (t); t ;is given y Y (t) = h((s))ds + W (t); where W = W (t);t ; is a Wiener process independent of the process. If f = f(x) is a measurale function satisfying Ejf((t))j < 1; t ; then it is known [11, 13, ] that under certain regularity assumptions the est mean square estimate ^f t of f((t)) given the trajectory Y (s); s t; can e written as ^f t = R f(x)u(t; x)dx R u(t; x)dx ; where u = u(t; x) is a random eld called the unnormalized ltering density (UFD). The prolem of estimating f((t)) is thus reduced to the prolem of computing the Center for Applied Mathematical Sciences, University of Southern California, Los Angeles, CA (chingpan@cams-.usc.edu). y Institute for Mathematics and its Applications, University of Minnesota, 514 Vincent Hall, 6 Church Street S.E., Minneapolis, MN (lototsky@ima.umn.edu). This work was partially supported y the ONR Grant N

2 UFD u. It is also known that u = u(t; x) is the solution of a certain stochastic partial dierential equation, called the Zakai equation, driven y the oservation process. The exact solution of the Zakai equation can e found only in a few special cases, and as a result the central part of the general nonlinear ltering prolem is the numerical solution of the equation. In some applications, e.g. target tracking, the solution must e computed in real time, which puts additional restriction on the corresponding numerical scheme. Most of the existing numerical schemes for the Zakai equation use various generalizations of the corresponding algorithms for the deterministic partial dierential equations and therefore cannot e implemented in real time when the dimension of the state process is more than three ecause of the large amount of computations. Examples of the corresponding algorithms can e found in Bennaton [1], Florchinger and LeGland [5], Ito [1], etc. When the parameters of the model are known in advance, an alternative approach is to separate the deterministic and stochastic components of the Zakai equation using the Wiener chaos decomposition and to do the computations related to the deterministic component in advance. Starting with the works of Kunita [1], Ocone [19], and Lo and Ng [14], this approach was further developed y Mikulevicius and Rozovskii [17, 18] and Budhiraja and Kallianpur []. The rst algorithm to solve the Zakai equation using this approach so that no partial dierential equations are solved on line was suggested in Lototsky et al. [15]. The ojective of the current work is to develop another algorithm for solving the Zakai equation in such a way that no dierential equations are solved on line and the on-line computations are performed recursively in time. The goal is achieved in two steps. First, the partial dierential equation satised y the UFD u(t; x) is approximated y a nite system of ordinary dierential equations (Galerkin approximation). After that, the solution of the system is approximated using the Cameron-Martin version of the Wiener chaos decomposition. The error of each approximation and the overall error are estimated in terms of the asymptotic parameters of the algorithm. The approach was rst suggested in [6] for a slightly dierent model, ut no explicit error ound was given. Filtering Prolem Let ~w = f ~w(t)g t and w = fw(t)g t e a d 1 - and an r-dimensional Wiener processes on a complete proaility space (; F; P). Consider the d-dimensional state (or signal) process = (t) of the form d i (t) = i ((t))dt + () = ; d 1 j=1 ij ((t))d ~w j (t); <tt; (.1) where =( i (x)) 1id is a d-dimensional vector function on IR d and =( ij (x)) 1id; 1jd1 is a d d 1 dimensional matrix function on IR d. The information aout the state process is availale only from the r-dimensional oservation

3 process Y = Y (t) given y Y (t) = h((s))ds + w(t); t T; (.) where h =(h l (x)) 1lr is an r-dimensional vector function on IR d. The ltering prolem studied in this paper is to compute the est mean square estimate ^f t of f((t)) given the oservation trajectory Y (s); s t, where f = f(x) is a function on IR d such that Ejf((t))j < 1 for all t T. The following is assumed aout the model (.1), (.): (A1) the functions,, and h are innitely dierentiale and ounded with all the derivatives; (A) the processes w and ~w are independent; (A3) the random vector is independent of oth w and ~w and has a density p = p(x); x IR d, so that the function p is innitely dierentiale and decays at innity with all the derivatives faster than every power of jxj. Let F y t e the -algera generated y Y (s), s t. Dene (t) = exp n r h l ((s))dw l (s) 1 r o jh l ((s))j ds : It is well known [11, 13] that the measure e P given y d e P = (T )dp is a proaility measure on (; F) with the properties: (i) On the reference proaility space (; F; e P), Y () isabrownian motion independent of(); (ii) The optimal lter ^f t = E[f((t))jF y t ]isgiven y E[f((t))(t) ^f t = ~ 1 jf y t ] ; (.3) ee[(t) 1 jf y t ] where e E is the expectation with respect to measure e P. If assumptions (A1 { A3) hold, then y Theorems 6..1 and 6.. in [] the unnormalized ltering measure t (dx) = e E[1 f(t)dxg (t) 1 jf y t ] admits the density u(t; x) = t (dx)=dx, called unnormalized ltering density (UFD), which is a solution of the Zakai equation du(t; x) =L u(t; x)dt + r h l (x)u(t; x)dy l (t); (.4) where L u := 1 j (( ) ij u) i ( i u): 3

4 By Theorem 4.3. in [], there is a unique solution u(t; x) of (.4) and for every positive integer n and a multi-index =( 1 ;:::; d sup tt ee Z IR d(1 + jxj ) n jd u(t; x)j dx < 1; (.5) where D jj 1 1 :::@x and jj = d 1 + :::+ d : d Throughout the paper, C will denote a constant depending on the parameters of the model, namely the length T of the time interval, dimensions d and r of the state and oservation processes, and various ounds on the coecients ; ; h and the initial density p and their derivatives. The value of C can e dierent at dierent places. The norm and inner product in L (IR d ) are denoted y kk and (; ), respectively. 3 Galerkin approximation In this section, a Galerkin approximation of equation (.4) will e studied using the Hermite asis in L (IR d ). The Hermite asis in L (IR d ) is constructed as follows. Let =( 1 ;:::; d )e amulti-index with i nonnegative integers. The set of all such multi-indices will e denoted y. For every dene e (x 1 ;:::;x d )= dy i=1 e i (x i ); (3.1) where e i (t) := 1 p n 1= n e t = H i (t) and H n (t) =( 1) n dn t e dt e t ; n ; (3.) n is the n-th Hermite polynomial. Then the set fe g is an orthonormal asis in L (IR d ) [7, 8] and will e referred to as the Hermite asis. For denote jj := P d i=1 i. Given a positive integer, dene the set := f :jjg:direct computations show that the numer of elements j j in the set is equal to ( + d)=(d). The Galerkin approximation u of the solution of (.4) is given y u (t; x) = P u (t)e (x), where the coecients fu g satisfy the system of stochastic ordinary dierential equations du (t) = r (L e ;e ) u (t)dt+ (h l e ;e ) u (t)dy l(t); <tt; u () = (p; e ) ; : 4 (3.3)

5 This is a linear system with constant coecients, so the solution fu (t)g exists and is unique [13, Theorem 4.6]. Even though the Galerkin approximation has een used in various numerical schemes for the Zakai equation (e.g. Bennaton [1], Ito [1]), no estimates of the approximation error have een given. In what follows, it is proved that under assumptions (A1 {A3) the approximation u converges to u in L (; P), e i.e. lim sup 1 tt and the rate of convergence is estalished. eeku(t; ) u (t; )k =; Theorem 3.1 If assumptions (A1 { A3) hold, then for every positive integer there is a constant C() depending only on and the parameters of the model so that sup tt eeku(t; ) u (t; )k C() : (3.4) Proof. If u (t) :=(u(t; );e ), then eeku(t; ) u (t; )k = Eju e (t) u (t)j + = e Eju (t)j : (3.5) By Lemma A.1 in Appendix P sup tt = e Eju (t)j C() : (3.6) For dene (t) :=u (t) Ej e j ; and also u (t);so that P e Eju (t) u (t)j = 1; (t):= = (L e ;e ) u (t); l ;(t):= = (h l e ;e ) u (t): Both 1; and l ; are well dened, ecause, according to Lemmas A.1 and A.3, Z T ee( 1; (s)) ds + r Z T ee( l ;(s)) ds C() : (3.7) As a result, d (t) = (L e ;e ) (t)dt + 1; ds + r () = ; ; r l ; dy l(t); <tt; (h l e ;e ) (t)dy l (t)+ (3.8) 5

6 and y the Ito formula, r e Ej (t)j = r ; ee It follows from Lemma A. that r ; (L e ;e ) e E (s) (s)ds+ (h l e ;e ) (s) ds + r (h l e ;e ) E e l ;(s) (s)ds + ; (L e ;e ) e E (s) (s)ds+ ee (h l e ;e ) (s) ds C ee 1; (s) (s)ds+ ee( l ;(s)) ds: ee( (s)) ds: Then (3.7), (3.9), (3.1), and the ovious inequality jaj a + imply Ej e (t)j C() so that y the Gronwall inequality sup tt + C eej (s)j ds; e Ej (t)j C() : Together with (3.5) and (3.6), the last inequality implies (3.4). (3.9) (3.1) 4 Approximate solution of the ODE By Theorem 3.1, to nd an approximate solution of the partial dierential equation (.4) it is sucient to solve the nite system of ordinary dierential equations (3.3). In this section, the system (3.3) is solved using the Cameron-Martin version of the Wiener chaos decomposition. An approximate solution is then constructed and the error of the approximation is estimated. Using the matrix-vector notation, system (3.3) can e written as du (t) =A u (t)dt + u () = p ; r B l u (t)dy l (t); <tt; (4.1) where u (t) is the column vector fu (t)g,p is the column vector f(p; e ) g, and for every vector IR j j, (A ) := (L e ;e ) ; (B l ) := r (h l e ;e ) : 6 (4.)

7 It follows from the denition that the matrices A and Bl do not, in general, commute with each other and therefore there is no computale explicit solution of (4.1). Below, an approximate solution is constructed using the Cameron-Martin version of the Wiener chaos decomposition, i.e. an expansion of u (t) with respect to a certain orthonormal asis in L (; F y t ; P). e The asis is constructed as follows. Let e an r-dimensional multi-index, i.e. a collection =( l) k 1lr; k1of nonnegative integers such that only nitely many of l kare dierent from zero. The set of all such multi-indices will e denoted y J. For J dene: jj := P l;k k l (length of ); d() :=maxfk1: l k := Q k;l( l k ). > for some 1 l rg (order of ); Fix < t T,choose a complete orthonormal system fm k g = fm k (s)g k1 in L ([;t]), and dene k;l = m k (s)dy l (s): Due to property (i) of the measure e P, k;l are independent Gaussian random variales with zero mean and unit variance. Theorem 4.1 (Cameron and Martin [3]) If H(t) := n= H n (t= p ); (4.3) where H n is the n-th Hermite polynomial (3.), then the collection 8 < : := Y H l k ( k;l ) q l k is a complete orthonormal system in L (; F y t ; e P). 1 9 = A ; J ; Theorem 4. For every s t the solution of (4.1) can e written as u (s) = J 1 p ' (s; p ) (P- a.s.) (4.4) The coecients of the expansion satisfy the recursive system of deterministic ordinary dierential equations d' (s; p ) ds = A ' (s; p )+ k;l ' (; p ) =p 1 fjj=g ; l k m k(s)b l ' (k;l)(s; p ); <st; (4.5) where =( l k) 1lr; k1 Jand (i; j) stands for the multi-index ~ =(~ l k) 1lr; k1 with ( ~ l = l k if k 6= i or l 6= j or oth k max(; j (4.6) i 1) if k = i and l = j. 7

8 The series in (4.4) converges in L (; e P) and L 1 (; P) and the following Parseval's equality holds: eeju (s)j = J 1 j' (s; p )j : (4.7) Proof. Since the initial condition u () and the matrices A and Bl are deterministic, the vector u (s) elongs to L (; F y t ; P) e for every s t y Theorem 4.6 in [13], and therefore y Theorem 4.1 u (s) = J ee[u (s) ] ; e Eju (s)j = J j e E[u (s) ]j <1: Denote ' (s) := p e E[u (s) ] (to simplify the notations, p in the argument of' will e omitted). Direct computations show that where ( P t (z) = exp = P t(z) z= ; r m l z (s)dy l(s) 1 r jm l z (s)j ds m l z = P k1 m k (s)z l k; fz l kg; l =1;:::;r; k =1;;:::; is a sequence of real numers such that P k;l jz l kj < 1, and := Y l k : (@zk l)l k Consequently, (s) = e E[u t (z)] z= e E[u s (z)] z= ; where the second equality follows from the martingale property ofp s (z)on(; e P):It also follows from the denition of P s (z) that dp s (z) = r Then (4.1) and the Ito formula imply u (t)p t (z) = p + m l z(s)p s (z)dy l (s); s t; P (z)=1: r A u (s)+ r B l u (s) m l z(s)p s (z)ds+ B l u (s)+u (s)m l z(s) P s (z)dy l (s): 8

9 Taking expectation e E on oth sides of the last equality and setting ' (s; z) := eeu (s)p s (z) results (s; z) = A (s)+ P r ml z(s)bl ' (s; z); <st ' (;z) =p : Applying the operator p and setting z = yields Convergence of (4.4) in L 1 (; P) follows from its convergence in L (; P) e and the inequality Ejj = e Ej(T)jq eejj q ee (T)Cq eejj ; valid for every L (; F y t ; P) e due to the oundedness of h l. System (4.5) is recursiveinjj: once the functions ' are known for all of length jj = k, it is possile to compute all ' for jj = k + 1. The computations can e performed o line ecause (4.5) does not involve the oservation process Y. It seems natural to use these features of expansion (4.4) in a numerical algorithm for solving (4.1). Since in all such algorithms the innite summation in (4.4) must e truncated and the error due to the truncation can e expected to grow with t, it is desirale to have a recursive version of Theorem 4.. Let = t <t 1 < ::: < t M = T e a uniform partition of the interval [;T] with step (so that t i = i; i=;:::;m) and let fm k g = fm k (t)g k1 e an orthonormal asis in L ([; ]). Dene random variales 8 < : i := Y H l k ( i k;l) q l k 1 9 = A ; J ; ; where and H n is dened in (4.3). i k;l = i t i 1 m k (s t i 1 )dy l (s) (4.8) Theorem 4.3 If u (t ):=p and t i 1 s t i, then u (s) = J 1 p ' (s t i ;u (t i 1 )) i ; i =1;:::;M (P- a.s.) (4.9) and the coecients ' satisfy (4.5) with the corresponding initial condition. This series converges in L (; P) e and L 1 (; P) and the following Parseval's equality holds: eeju 1 (s)j = Ej' e (s t i ;u (t i 1 ))j ; i=1;:::;m: J 9

10 Proof. Since the coecients of (4.1) do not depend on time, for every t i the vector u (s) satises 1 s t i u (s) = u (t i 1 )+ r Z s ti 1 Z s ti 1 A u (t+t i 1 )dt+ B l u (t + t i 1 )d(y l (t + t i 1 ) Y l (t i 1 )): (4.1) Then oth statements of the theorem follow from the corresponding statements of Theorem 4., ecause, under the measure e P, Y l (t + t i 1 ) Y l (t i 1 ) is independent of F y t i 1. To study the truncation of (4.4) or (4.9), it is necessary to note that the summation P J is doule innite: PJ = P 1 P k= jj=k and there are innitely many multiindices with jj = k >. A possile way to truncate the summation is to restrict to the nite set JN n := f J : jj N; d() ng. The error due to this truncation is given in the following theorem. Theorem 4.4 Given the positive integers n and N and the uniform partition ft i g;i=;:::;m of [;T]with step, dene the sequence ofvectors fu (i; n; N)g IR j j ; i=;:::;m; y 1 u (;n;n):=p ; u (i; n; N) := p ' (;u (i 1;n;N)) i : (4.11) If fm k g is the Fourier cosine asis then m 1 (s) = 1 p ; max im m k (s)= eeju (t i ) J n N s cos (k 1)s ;k>1; s ; (4.1) u (i; n; N)j (C)N (N + 1) + C + : (4.13) n The proof of this theorem is rather long and is given in Section 6. For xed the error (4:13) grows with due to the stiness of matrix A. For xed, the error tends to zero with. In fact, since N 1 and n 1, (4.13) shows that the approximation u (i; n; N) has order of strong convergence in time equal to :5. If n = 1, then the approximation uses only the increments Y l (t i+1 ) Y l (t i )of the oservation process, and it is know [4] that :5 is the highest possile rate of convergence under the circumstances. On the other hand, if r = 1 or the matrices Bl commute (which, in general, can happen only if the functions h l are multiples of each other), then, as the proof of the theorem shows, the term C=n on the right hand side of (4.13) disappears and the order of convergence may e equal to 1:. If n>1and the oservations Y l (t) are availale continuously in time, then the random variales i can e computed with prescried accuracy y reducing the Wiener integrals (4.8) to Riemann integrals using integration y parts; the details can e found in [16, Section 4.5.1]. 1

11 5 The Algorithm The approximation u (i; n; N) from Theorem 4.4 still does not meet the criteria stated in the Introduction ecause, starting with i = 1, the initial condition of (4:5) is random and therefore the system of dierential equations must e solved on line. The following construction makes it possile to circumvent the diculty. If f g; ;is the standard unit asis in IR j j, meaning that =1if= and =if6=, then u (i; n; N)= P u (i; n; N). Recursive denition of u (i; n; N) and the linearity of system (4.5) imply that u (;n;n)=p ; u (i; n; N) = Q ; u (i 1;n;N)i ; (5.1) where JN n Q ; = ' ;(; ): (5.) Using the matrix-vector notations, (5.1) can e written as u (i; n; N) = Q u (i 1;n;N) i : (5.3) J n N Since the matrices Q can e computed o line, the on-line part of the algorithm is reduced to computing random variales i (which, in principle, can e implemented on the hardware level) and evaluating the sum in (5.3). If u(t; x) is the exact solution of the Zakai equation (.4) and u n;n(i; x) := P u (i; n; N)e (x), then, y Theorems 3.1 and 4.4, the overall error of approximation is given y sup im eeku(i; ) u n;n(i; )k C() + (C)N (N + 1) + C + : (5.4) n The following is the complete description of the algorithm. 1. Of f line (efore the oservations are availale) compute the matrices A and B l according to (4.); the matrices Q ; J N n ; according to (5.). the coecients u (;n;n)= Z IR dp (x)e (x)dx, ; set u n;n(;x)= P u (;n;n)e (x).. On line; i th step (as the oservations ecome availale): compute u (i; n; N) = injn n Q u (i 1;n;N)i ; (5.5) then, if necessary, compute u n;n(i; x) = u (i; n; N)e (x): (5.6) 11

12 The main advantage of this algorithm as compared to the traditional schemes for solving the Zakai equation is that the time consuming computations of solving partial dierential equations are performed o line, while the on-line part is relatively simple even when the dimension d of the state process is large. Here are some other features of the algorithm: (1) The overall amount of the o-line computations does not depend on the numer of the on-line time steps; () Only the coecients u (i; n; N) must e computed at every time step while the approximate lter ~ f ti ; and/or UFD u n;n(i; x) can e computed as needed, e.g. at the nal time moment. (3) The on-line part of the algorithm can e easily parallelized. In the proposed algorithm, the Wiener chaos decomposition was used to solve the nite dimensional system otained y projecting the original Zakai equation (.4) on the suspace generated y fe g. The error estimate (5.4) shows that this approach should e used with suciently small time step and moderate values of, e.g. when good resolution in time is required and the spatial resolution is not important. In [15], an alternative approach was suggested, in which the Wiener chaos decomposition was applied directly to the Zakai equation with susequent projection of the result on the span of fe g. The error of the corresponding approximation is given y C() + C n + (C)N (N + 1) ; which means that this alternative approach should e ecient for moderate values of and suciently large, e.g. when good resolution in space is required and the time resolution is not important. The choice etween the two approaches will therefore depend on the specic features of the prolem. 6 Proof of Theorem 4.4 Introduce the following notations: t := e ta ; s k, the ordered set (s 1 ;:::;s k ); ds k := ds 1 :::ds k ; l k, the ordered set (l 1 ;:::;l k ); F (t; s k ; l k ):= t s k Bl k s k s k 1 :::Bl 1 s 1 p ; k1; Z (k;t)()ds Z sk Z s k := ::: ()ds 1 :::ds k ; 1

13 l k := r l 1 ;:::;l k =1. To simplify the notations, the superscript will e omitted throughout the rest of the proof. Every multi-index with jj = k can e identied with the set K = f(i 1 ;q 1);:::;(i k ;q k)g so that i 1 i ::: i k and if i j = i j+1, then qj q. The rst pair j+1 (i 1 ;q)ink 1 is the position numers (k; l) of the rst nonzero element l k of. The second pair is the same as the rst if the rst nonzero elementofis greater than one; otherwise, the second pair is the position numers of the second nonzero entry of and so on. As a result, if q j >, then exactly q j pairs in K are (j; q). The set K will e referred to as the characteristic set of multi-index. For example, if r = and = 1 3 ::: 1 1 ::: then the nonzero elements are = 1 1 = 6 =1; = 1 =; =3;and the characteristic set is K = f(1; ); (; 1); (; ); (; ); (4; 1); (4; 1); (5; 1); (5; 1); (5; 1); (6; )g. In the future, when there is no danger of confusion, the superscript in i and q will e omitted (i.e. (i j ;q j ) will e used instead of (i j ;q j)). Let P k e the permutation group of the set f1;:::;kg. For a given J with jj = k and the characteristic set f(i 1 ;q 1 );:::;(i k ;q k )g dene E (s k ; l k ):= m i1 (s (1) )1 fl(1) =q 1 g m ik (s (k) )1 fl(k) =q k g: (6.1) P k To prove (4.13) it suces to show that ; eeju(t i ) u(i; n; N)j +1 (C)N (N + 1) + C + 3 : (6.) n After that, (4.13) will follow directly from Gronwall's inequality. Moreover, (6.) will follow from +1 eeju(t 1 ) u(1;n;n)j (C)N (N + 1) + C + 3 jpj ; (6.3) n ecause similar arguments can then e used to show that eeju(t i ) u(i; n; N)j +1 (C)N (N + 1) + C + 3 eeju(ti )j ; n and y Lemma A.4, max im Eju(ti e )j C. As a result, it suces to prove (6.3). For the sake of simplicity, the argument pin ' will e omitted. Dene ' () 1 u(;n):= p : jjn 13

14 If it is known that eeju() u(;n)j (C)N +1 (N + 1) jpj (6.4) and eeju(;n) u(1;n;n)j + 3 C jpj ; (6.5) n then (6.3) will follow immediately from the inequality (a+) (a + ). Proof of (6.4). By Lemma A.5, jj=k j' ()j = l k Z (k;) jf (; s k ; l k )j ds k ; (6.6) where ' is the solution of (4.5) with an aritrary CONS fm k g in L ([; ]). Since 1 are uncorrelated under P, e eeju() j' u(;n)j ()j Z (k;) = = jf (; s k ; l k )j ds k : jj=k k>n l k It follows from the denition of F and Lemma A.5 that Finally, jf (; s k ; l k )j e C( s k ) j sk s k 1 B lk :::B l1 s1 pj :::C k e C( s k +s k :::+s s 1 +s 1) jpj C k jpj : Z (k;) ds k = k =k implies eeju() u(;n)j k>n (Cr) k k +1 (C)N (N + 1) ec ; and (6.4) follows. Note that it holds for every CONS fm k g. Proof of (6.5). If isamulti-index with jj = k and the characteristic set f(i 1 ;q 1 );:::;(i k ;q)g then k i = d(), the order of, and so the set k J N n can e descried as f J : jj N; i jj ng.thus eeju(1;n;n) u(;n)j = 1 N =n+1 k=1 The prolem is thus to estimate P 1 =n+1 P N k=1 j' ()j jj=k;i k = P jj=k;i k = j' ()j =: By Lemma A.5 the corresponding solution ' of (4.5) can e written as ' () = l k Z (k;) F (; s k ; l k )E (s k ;l k )ds k ; (6.7) where E is given y (6.1). Since y denition (4.6) the characteristic set of (i k ;q k ) is f(i 1 ;q 1 );:::;(i k 1 ;q k 1 )g, it is possile to write E (s k )= k j=1 m ik (s j )1 flj =q k ge (ik ;q k )(s k j ; l k j ); 14 :

15 where s k (resp. j lk) denotes the same set j (s 1;:::;s k ) (resp. (l 1 ;:::;l k )) with omitted s j (resp. l j ); for example, s k =(s 1 ;:::;s k ): Then (6.7) can e rewritten as ' ()= k j=1 l k j (kz1;) Z sj+1 s j F (; s k ; l k )m ik (s j )1 flj =q k gds j E (ik ;q k )(s k j ; l k j )ds k j ; (6.8) 1 where s := ; s k+1 :=. (Change the order of integration in the multiple integral.) Denote M k (s) := p (k 1) sin (k 1) s ; k>1; s; and also F j s k ; l k )=@s j. Then, as long as i k = >1, integration y parts in the inner integral on the right of (6.8) yields: Z sj+1 s j 1 F (; s k ; l k )m (s j )ds j = F (; s k ; l k )M (s j ) s j =s j+1 s j =s j 1 Z sj+1 s j 1 F j (; s k ; l k )M (s j )ds j : For each j, the remaining variales s k j in (6.8) are renamed as follows: t i := s i ;i j 1; t i := s i+1 ;i> j 1, or, symolically, t k 1 := s k j. Set t :=, t k := and denote y t k 1;j ;j =1;:::;k 1; the set t k 1 in which t j is repeated twice (e.g. t k 1;1 = (t 1 ;t 1 ;:::;t k 1 );etc.); also t k 1; := (t ;t 1 ;t ;:::;t k 1 ), t k 1;k := (t 1 ;:::;t k 1 ;t k ). The similar changes will also e made with the set l k : for xed j, there are k 1 free indices l 1 ;:::;l j 1 ;l j+1 ;:::;l k and they are renamed just like s k to form the set l k 1 (in this case, the same letters are used); symol l k 1;j denotes the set (l 1 ;:::;l j 1 ;q j ;l j ;:::;l k 1 ): After these transformations, E (ik ;q k )(s k j ; l k j ) ecomes E (ik ;q k )(t k 1 ; l k 1 ) - independent ofj, and F (; s k ; l k )1 flj =q k gm (s j ) s j=s j+1 = s j =s j 1 F (; t k 1;j ; l k 1;j )M (t j ) F (; t k 1;j 1 ; l k 1;j )M (t j 1 ); j=1;:::;k: Therefore, if d() =>1 and jj = k>, then ' () = l k 1 (kz1;) f (1) (; t k 1 ; l k 1 )+ f () (; t k 1 ; l k 1 ) E (ik ;q k )(t k 1 ; l k 1 )dt k 1 ; where f (1) (; t k 1 ; l k 1 )= P k j=1 F (; t k 1;j ; l k 1;j )M (t j ) F (; t k 1;j 1 ; l k 1;j )M (t j 1 ) ; if k>1; 15

16 f (1) =ifk= 1 { ecause M (t )=M (t k ) = (this is the only place where the choice of fm k g really makes the dierence), and f () (; t k 1 ; l k 1 )= 1 k 1 j j= k F 1 (; s; t k 1 ; q k ;l k 1 )M (s)ds t j 1 F j (; :::;t j 1 ;s;t j ;:::;l k 1;j )M (s)ds t k 1 F k (; t k 1 ;s;l k 1 ;q k )M (s)ds: Then, since j(i jj ;q jj )j=jj 1 and (i jj ;q jj ), j' ()j = jj=k;i k = r q k =1 jj=k;i k = r q k =1 jj=k 1 1 p l k 1 (kz1;) (f (1) + f () )E (;qk )dt k 1 1 Z (k 1;) (1) p (f + f () l k 1 )E dt k 1 and arguments similar to those used in the proof of Lemma A.5 show that the last expression is equal to r q k =1 l k 1 Z (k 1;) f (1) (; t k 1 ; l k 1 )+f () (; t k 1 ; l k 1 ) dt k 1 : (6.9) ; Denition of f (1) and Lemmas A. and A.4 imply jf (1) j =; k=1; jf (1) j kc ( 1) jpj ; k: (6.1) Next, direct computations yield F j (; s k ; l k )= sk B lk ::: sj+1 s j B lj A sj s j 1 ::: s1 p and then y Lemma A.4, sk B lk :::A sj+1 s j B lj sj s j 1 ::: s1 p; jf j (; s k ; l k )j C k jpj : After that the denition of f () and ovious inequalities (a 1 + :::+a k ) k(a 1 +:::+a k ) and Z x f(y)dy x Z x (f(y)) dy 16

17 imply: jf () j kc k e C jpj kck 3 ( 1) jpj ; Z (M (s)) ds so, since R (k 1;) dt k 1 = k 1 =(k 1), (6.9), (6.1), and the last inequality yield eeju(;n) u(1;n;n)j = Cjpj 4 k +(C) k k+1 k k C + 3 jpj : n N n+1 k=1 + 3 k j' ()j jj=k;i k = (k + 1)(C) k This completes the proof of (6.5) and the theorem as a whole. k 3 5 n 1 Acknowledgment. The authors would like to thank Professor B. L. Rozovskii who supervised the work. The second author is also thankful to the NSF for the nancial support awarded through the Institute for Mathematics and its Applications. References [1] J. F. Bennaton. Discrete Time Galerkin Approximation to the Nonlinear Filtering Solution. Journal of Mathematical Analysis and Applications, 11():364{383, [] A. Budhiraja and G. Kallianpur. Approximations to the Solution of the Zakai Equations Using Multiple Wiener and Stratonovich Integral Expansions. Stochastics and Stochastics Reports, 56:71{315, [3] R. H. Cameron and W. T. Martin. The Orthogonal Development of Nonlinear Functionals in a Series of Fourier-Hermite Functions. Annals of Mathematics, 48():385{39, [4] J. M. C. Clark and R. J. Cameron. The Maximum Rate of Convergence of Discrete Approximation for Stochastic Dierential Equations. Springer Lecture Notes in Control and Inform. Sc., 5:16{171, 198. [5] P. Florchinger and F. LeGland. Time Discretization of the Zakai Equation for Diffusion Processes Oserved in Correlated Noise. Stochastics and Stochastics Reports, 35(4):33{56, [6] C. P. Fung. New Numerical Algorithms for Nonlinear Filtering. PhD thesis, University of Southern California, Los Angeles, CA, 989, Nov [7] D. Gottlie and S. A. Orszag. Numerical Analysis of Spectral Methods: Theory and Applications. CBMS-NSF Regional Conference, Series in Applied Mathematics, Vol.6,

18 [8] E. Hille and R. S. Phillips. Functional Analysis and Semigroups. Amer. Math. Soc. Colloq. Pul., Vol. I, [9] K. Ito. Multiple Wiener integral. Journal of the Mathematical Society of Japan, 3:157{169, [1] K. Ito. Approximation of the Zakai Equation for Nonlinear Filtering. SIAM Journal on Control and Optimization, 34():6{634, [11] G. Kallianpur. Stochastic Filtering Theory. Springer, New York, 198. [1] H. Kunita. Cauchy Prolem for Stochastic Partial Dierential Equations Arising in Nonlinear Filtering Theory. System & Control Letters, 1(1):37{41, [13] R. S. Liptser and A. N. Shiryayev. Statistics of Random Processes. Springer, New York, 199. [14] J. T.-H. Lo and S.-K. Ng. Optimal Orthogonal Expansion For Estimation I: Signal in White Gaussian Noise. In R. Bucy and J. Moura, editors, Nonlinear Stochastic Prolems, pages 91{39, D. Reidel Pul. Company, [15] S. Lototsky, R. Mikulevicius, and B. L. Rozovskii. Nonlinear Filtering Revisited: A Spectral Approach. SIAM Journal on Control and Optimization, 35(), To appear. [16] S. V. Lototsky. Prolems in Statistics of Stochastic Dierential Equations. PhD thesis, University of Southern California, Los Angeles, CA 989, Aug [17] R. Mikulevicius and B. L. Rozovskii. Separation of Oservations and Parameters in Nonlinear Filtering. In Proceedings of the 3nd IEEE Conference ondecision and Control, Part, San Antonio, 1993, pages 1564{1569. IEEE Control Systems Society, [18] R. Mikulevicius and B. L. Rozovskii. Fourier-Hermite Expansion for Nonlinear Filtering. In A. A. Novikov, editor, Festschrift in honor of A. N. Shiryayev, [19] D. Ocone. Multiple Integral Expansions for Nonlinear Filtering. Stochastics, 1:1{3, [] B. L. Rozovskii. Stochastic Evolution Systems. Kluwer Academic Pulishers, 199. Appendix In what follows, fe g is the Hermite asis in L (IR d ) and = f : jjg. Lemma A.1 If function f is innitely dierentiale and decays at innity with all the derivatives faster than every power of jxj then = j(f; e ) j C() : (A.1) 18

19 In particular, sup tt = e Eju (t)j C() : (A.) Proof. Introduce the operator := r +1+jxj ;where r is the Laplace operator. Direct computations show that e =(jj+d+1)e := e : By assumption, f L (IR d ) for every positive integer and therefore (f; e ) = (f; e ) = (f; e ) = :::= ( f; e ) : This implies that j(f; e ) j k fk and (A.1) follows ecause j j d. After that, (A.) follows from (A.) and (.5). Lemma A. If assumption (A1) holds, then for every IR j j (L e ;e ) Cjj ; ; (h l e ;e ) Cjj : (A.3) (A.4) Proof. If is the L (IR d )-orthogonal projection on the span of fe (x)g and ~ := P e (x), then (L e ;e ) =(L ; ~ ) ~ ; ; (h l e ;e ) = k (h l )k ~ : Direct computations show that under assumption (A1) (L f; f) Ckfk ; kh lfk Ckfk for every suciently regular function f. Then estimates (A.4) follow, ecause k ~ k = = jj : Lemma A.3 Assumption (A1) implies the following estimates: kl e k Cjj; kh l e k C: 19

20 Proof. The second inequality isovious ecause ke k =1.To prove the rst, consider the operator = r +1+jxj. Then direct computations show that kl f k Ck(1 )f k C(kf k + kf k ) for every suciently regular function f. ke k Cjj, which implies the result. It follows from the proof of Lemma A.1 that Lemma A.4 If assumption (A1) holds, then for every IR j j je ta jjje Ct ; (constant C does not depend on ). r jb l jcjj; ja jcjj: Proof. Given IR j j, the vector v(t) :=e ta is the solution of Then y the denition of matrix A, djv(t)j = dv(t) =A v(t); v() = : ; (L e ;e ) v (t)v (t)dt and y Lemma A., djv(t)j Cjv(t)j dt. After that, Gronwall's inequality implies jv(t)j jj e Ct and the rst inequality follows. The second inequality follows directly from Lemma A. and the third { from Lemma A.3. Lemma A.5 If J is a multi-index with jj = k and the characteristic set f(i 1 ;q 1 );:::;(i k ;q k )g, then the corresponding solution ' (t; p ) of (4.5) is given y ' (t; p )= (k;t) Z P k l k ' (t; p )= F (t; s k ; l k )m i(k) (s k )1 flk =q (k) g m i(1) (s 1 )1 fl1 =q (1) gds k ; k>1; ' (t; p )= t p ; k=; t s 1 B q 1 s1 p m i1 (s 1 )ds 1 ; k =1; (A.5) (see eginning of Section 6 for notations). In addition, j' (t; p )j jj=k = l k Z (k;t) jf (t; s k ; l k j ds k : (A.6) Proof. For the sake of simplicity, the argument pin ' will e omitted. Representation (A.5) is oviously true for jj =. Then the general case jj 1 follows y induction from the variation of parameters formula.

21 To prove (A.6), rst of all note that m i(k) (s k )1 flk =q (k) g m i(1) (s 1 )1 fl1 =q (1) g = P k m ik (s (k) )1 fl(k) =q k g m i 1 (s (1) 1 fl(1) =q 1 g: P k Indeed, every term on the left corresponding to a given P k is equal to the term on the right corresponding to 1 P k. Then (A.5) can e written as ' (t) = l k Z (k;t) F (t; s k ; l k )E (s k ; l k )ds k ; where E is given y (6.1). Using the notation G (s k ; l k ):= P k t s (k) B l (k) ::: s () s (1) B l (1) s (1) p k 1 s(1) <:::<s (k) ; it can e rewritten as ' (t) = 1 k Z l k [;t] k G (s k ; l k )E (s k ; l k )ds k : (A.7) Since every component of the vector G is a symmetric function from L (([;t] f1;:::;rg) k ) and the collection fe = p k; jj = kg is a CONS for the symmetric part of the space, G c = E p k jj=k with some c IRj j. Then from (A.7) j' j = =jc j =k and so jj=k 1 k j' (t)j Z l k [;t] k = 1 k jj=k jc j = 1 k Z [;t] k jg (s k ; l k )j ds k = t s (k) Bl (k) ::: s () s (1) Bl (1) s (1) p k 1 s(1) <:::<s (k) ds k = P k l k Z (k;t) jf (t; s k ; l k )j ds k ; which proves (A.6). Remark. If jj = k, then y Theorem 3.1 in [9] = p 1 Z sk ::: Z s l k E (s k ;l k )dy(s 1 ) :::dy(s k ): This gives an alternative (ut equivalent) form of (4.4) in terms of multiple Wiener integrals. 1