Decentralized Particle Filter with Arbitrary State Decomposition

Transcription

1 1 Decenralized Paricle Filer wih Arbirary Sae Decomposiion Tianshi Chen, Member, IEEE, Thomas B. Schön, Member, IEEE, Henrik Ohlsson, Member, IEEE, and Lennar Ljung, Fellow, IEEE Absrac In his paper, a new paricle filer (PF) which we refer o as he decenralized PF (DPF) is proposed. By firs decomposing he sae ino wo pars, he DPF splis he filering problem ino wo nesed sub-problems and hen handles he wo nesed sub-problems using PFs. The DPF has he advanage over he regular PF ha he DPF can increase he level of parallelism of he PF. In paricular, par of he resampling in he DPF bears a parallel srucure and can hus be implemened in parallel. The parallel srucure of he DPF is creaed by decomposing he sae space, differing from he parallel srucure of he disribued PFs which is creaed by dividing he sample space. This difference resuls in a couple of unique feaures of he DPF in conras wih he eising disribued PFs. Simulaion resuls of wo eamples indicae ha he DPF has a poenial o achieve in a shorer eecuion ime he same level of performance as he regular PF. Inde Terms Paricle filering, parallel algorihms, nonlinear sysem, sae esimaion. I. INTRODUCTION In his paper, we sudy he filering problem for he following nonlinear discree-ime sysem ξ +1 = f (ξ, v ) y = h (ξ, e ) where is he discree-ime inde, ξ R n ξ is he sae a ime, y R ny is he measuremen oupu, v R nv and e R ne are independen noises whose known disribuions are independen of, ξ and y, and f ( ) and h ( ) are known funcions. The filering problem consiss of recursively esimaing he poserior densiy p(ξ y 0: ) where, y 0: {y 0,..., y }. Analyic soluions o he filering problem are only available for a relaively small and resriced class of sysems, he mos imporan being he Kalman filer [19] which assumes ha sysem (1) has a linear-gaussian srucure. A class of powerful numerical algorihms o he filering problem are paricle filers (PFs), which are sequenial Mone Carlo mehods based on paricle represenaions of probabiliy densiies [2]. Since he seminal work [15], PFs have become an imporan ool in handling he nonlinear non-gaussian filering problem, and have found many applicaions in saisical signal processing, economics and engineering; see e.g., [2, 12, 14, 16] for recen surveys of PFs. Copyrigh (c) 2010 IEEE. Personal use of his maerial is permied. However, permission o use his maerial for any oher purposes mus be obained from he IEEE by sending a reques o pubs-permissions@ieee.org. The auhors are wih he Division of Auomaic Conrol, Deparmen of Elecrical Engineering, Linköping Universiy, Linköping, 58183, Sweden. E- mail: {schen, schon, ohlsson, ljung}@isy.liu.se. Phone: +46 (0) Fa: +46 (0) (1) In his paper, a new PF, which we refer o as he decenralized PF (DPF), will be proposed. By firs decomposing he sae ino wo pars, he DPF splis he filering problem of sysem (1) ino wo nesed sub-problems and hen handles he wo nesed sub-problems using PFs. The DPF has he advanage over he regular PF ha he DPF can increase he level of parallelism of he PF in he sense ha besides he paricle generaion and he imporance weighs calculaion, par of he resampling in he DPF can also be implemened in parallel. As will be seen from he DPF algorihm, here are acually wo resampling seps in he DPF. The firs resampling in he DPF, like he resampling in he regular PF, canno be implemened in parallel, bu he second resampling bears a parallel srucure and can hus be implemened in parallel. Hence, he parallel implemenaion of he DPF can be used o shoren he eecuion ime of he PF. As poined ou in [5], he applicaion of PFs in real-ime sysems is limied due o is compuaional compleiy which is mainly caused by he resampling involved in he PF. The resampling is essenial in he implemenaion of he PF as wihou resampling he variance of he imporance weighs will increase over ime [13]. The resampling however inroduces a pracical problem. The resampling limis he opporuniy o parallelize since all he paricles mus be combined, alhough he paricle generaion and he imporance weighs calculaion of he PF can sill be realized in parallel [13]. Therefore, he resampling becomes a boleneck o shoren he eecuion ime of he PF. Recenly, some disribued resampling algorihms for parallel implemenaion of PFs have been proposed in [5, 22]. The idea of he disribued resampling is o divide he sample space ino several sraa or groups such ha he resampling can be performed independenly for each sraum or group and can hus be implemened in parallel. The effec of differen disribued resampling algorihms on he variance of he imporance weighs has been analyzed in [22]. Based on he inroduced disribued resampling algorihms, a couple of disribued PFs have been furher proposed in [5, 22], such as he disribued resampling wih proporional allocaion PF (DRPA-PF) and he disribued resampling wih nonproporional allocaion PF (DRNA-PF). The underlying idea of he DPF is differen from ha of he eising disribued PFs, while hey all have parallel srucure. The parallel srucure of he DPF is creaed by decomposing he sae space, differing from he parallel srucure of he disribued PFs which is creaed by dividing he sample space. This difference resuls in a couple of unique feaures of he DPF in conras wih he eising disribued PFs. Firs,

2 z z z a b c Fig. 1. Paerns for poins where he poserior densiies are compued/esimaed. a. A fied regular grid used in he poin-mass filer. b. Randomly allocaed poins following he regular PF equaions. c. Poins randomly allocaed o verical parallel lines (which hemselves are randomly locaed) used in he DPF. compared o he DRPA-PF, he DPF allows a simpler scheme for paricle rouing and acually reas each processing elemen as a paricle in he paricle rouing. Second, he DPF does no have he efficiency decrease problem of he DRPA-PF. Given a PF wih parallel srucure, i works mos efficienly if each processing elemen handles he same number of paricles. However, he efficiency of he DRPA-PF usually decreases, since he numbers of paricles produced by each processing elemen are no evenly bu randomly disribued among he processing elemens. Third, i will be verified by wo numerical eamples ha, he DPF has he poenial o achieve in a shorer eecuion ime he same level of performance as he boosrap PF. In conras, he DRNA-PF acually rades he PF performance for he speed improvemen [5]. Besides, he level of parallelism of he DPF can be furher increased in wo ways so ha he eecuion ime of he parallel implemenaion of he DPF can be furher shorened; he firs one is o uilize any of he disribued resampling algorihms proposed in [5, 22] o perform he firs resampling of he DPF, and he oher is based on an eension of he DPF. As a resul, he DPF is a new opion for he applicaion of PFs in real-ime sysems and he parallel implemenaion of PFs. The res of he paper is organized as follows. The problem formulaion is given in Secion II. In Secion III, he DPF algorihm is firs derived and hen summarized, and finally some issues regarding he implemenaion of he DPF are discussed. In Secion IV, some discussions abou he DPF and is comparison wih he eising disribued PFs are made. In Secion V, wo numerical eamples are elaboraed o show he efficacy of he DPF. Finally, we conclude he paper in Secion VI. A. Inuiive preview II. PROBLEM FORMULATION The formulas for paricle filering end o look comple, and i may be easy o ge los in indices and updae epressions. Le us herefore provide a simple and inuiive preview o ge he main ideas across. Filering is abou deermining he poserior densiies of he saes. If he sae is wo-dimensional wih componens and z, say, he densiy is a surface over he z plane, see Fig. 1. One way o esimae he densiy is o fi M poins in he plane in a regular grid (Fig. 1.a) and updae he values of he densiy according o Bayesian formulas. This is known as he poin-mass filer [3, 6]. Anoher way is o hrow M poins a he plane a random (Fig 1.b), and le hem move o imporan places in he plane, and updae he values of he densiies a he chosen poins, using Bayesian formulas. This is a simplified view of wha happens in he regular PF. A hird way is illusraed in Fig 1.c: Le he poins move o well chosen locaions, bu resric hem o be aligned parallel o one of he aes (he z-ais in he plo). The parallel lines can move freely, as can he poins on he lines, bu here is a resricion of he paern as depiced. The algorihm we develop in his paper (DPF) gives boh he movemens of he lines and he posiions of he poins on he lines, and he densiy values a he chosen poins, by applicaion of Bayesian formulas. I is well known ha he regular PF ouperforms he poinmass filer wih he same number of poins, since i can concenrae hem o imporan areas. One would hus epec ha he DPF would give worse accuracy han he regular PF wih he same number of poins, since i is less fleible in he allocaion of poins. On he oher hand, he srucure migh allow more efficien ways of calculaing new poin locaions and weighs. Tha is wha we will develop and sudy in he following secions. B. Problem saemen Consider sysem (1). Suppose ha he sae ξ can be decomposed as [ ] ξ = (2) z and accordingly ha sysem (1) can be decomposed as +1 = f (, z, v ) z +1 = f z (, z, v z ) y = h (, z, e ) where R n, z R nz, and v = [(v )T (v z )T ] T wih v R n v and v z R n v z. In he following, i is assumed for convenience ha he probabiliy densiies p( 0 ), p(z 0 0 ) and for 0, p( +1, z ), p(z +1 :+1, z ) and p(y, z ) are known. In his paper, we will sudy he filering problem of recursively esimaing he poserior densiy p(z, 0: y 0: ). According o he following facorizaion (3) p(z, 0: y 0: ) = p(z 0:, y 0: )p( 0: y 0: ) (4)

3 3 where 0: { 0,..., }, he filering problem (4) can be spli ino wo nesed sub-problems: 1) recursively esimaing he densiy p( 0: y 0: ); 2) recursively esimaing he densiy p(z 0:, y 0: ). Tha he wo sub-problems are nesed can be seen from Fig. 2 where we have skeched he five seps used o derive he recurrence relaion of he concepual soluion o he filering problem (4). Since here is in general no analyic soluion o he filering problem (4), a numerical algorihm, i.e., he DPF is inroduced o provide recursively he empirical approimaions o p( 0: y 0: ) and p(z 0:, y 0: ). The DPF acually handles he wo nesed sub-problems using PFs. Roughly speaking, he DPF solves he firs sub-problem using a PF wih N paricles ( (i) 0:, i = 1,..., N ) o esimae p( 0: y 0: ). Then he DPF handles he second sub-problem using N PFs wih N z paricles each o esimae p(z (i) 0:, y 0:), i = 1,..., N. As a resul of he nesedness of he wo sub-problems, i will be seen laer ha he seps of he PF used o esimae p( 0: y 0: ) is nesed wih ha of he N PFs used o esimae p(z (i) 0:, y 0:). Remark 2.1: The idea of decomposing he sae ino wo pars and accordingly spliing he filering problem ino wo nesed sub-problems is no new. Acually, i has been used in he Rao-Blackwellized PF (RBPF); see, e.g, [1, 8, 9, 11, 13, 25]. However, he RBPF imposes cerain racable subsrucure assumpion on he sysem considered and hence solves one of he sub-problem wih a number of opimal filers, such as he Kalman filer [19] or he HMM filer [24]. In paricular, he filering problem (4) has been previously sudied in [25] where sysem (3) is assumed o be condiionally (on ) linear in z and subjec o Gaussian noise. Due o hese assumpions, he sae z of sysem (3) is marginalized ou by using he Kalman filer. However, since here is no racable subsrucure assumpion made on sysem (3) in his paper, no par of he sae ξ is analyically racable as was he case in [1, 8, 9, 11, 13, 25]. In he following, le p() denoe ha is a sample drawn from he densiy p() of he random variable, le N(m, Σ) denoe he (mulivariae) Gaussian probabiliy densiy wih mean vecor m and covariance mari Σ and le P(A) denoe he probabiliy of he even A. For convenience, for each i = 1,..., N, α i = β i / N β j is denoed by α i β i ; N α i = 1 (5) where N is a naural number, α i, β i, i = 1,..., N, are posiive real numbers, and α i β i denoes ha α i is proporional o β i. p( 0: y 0: 1 ) p( 0: y 0: ) p( 0:+1 y 0: ) p(y 0:,y 0: 1 ) p(y,z ) p( +1 0:,y 0: ) p( +1,z ) p(z +1 :+1,z ) p(z 0:,y 0: 1 ) p(z 0:,y 0: ) p(z 0:+1,y 0: ) p(z +1 0:+1,y 0: ) Fig. 2. Skech of he seps used o derive he concepual soluion o he filering problem (4). Assume ha he probabiliy densiies p( 0 ), p(z 0 0 ) and for 0, p( +1, z ), p(z +1 :+1, z ) and p(y, z ) are known. Wih a sligh abuse of noaion le p( 0 y 0: 1 ) = p( 0 ) and p(z 0 0, y 0: 1 ) = p(z 0 0 ). Then five seps are needed o derive he recurrence relaion of he concepual soluion o he filering problem (4). The specific relaions beween he differen densiies can be obained by sraighforward applicaion of Bayesian formulas and hus are omied. Insead, he noaion p 1 ( ) p 2 ( ) is used o indicae ha he calculaion of he densiy p 2 ( ) makes use of he knowledge of he densiy p 1 ( ). hroughou he paper. Suppose we have P N (dξ ) = 1 N N δ (i) ξ (dξ ) (6) where δ (i) ξ (A) is a Dirac measure for a given ξ (i) and a measurable se A. Then he epecaion of any es funcion g(ξ ) wih respec o p(ξ ) can be approimaed by g(ξ )p(ξ )dξ g(ξ )P N (dξ ) = 1 N N g(ξ (i) ) (7) Alhough he disribuion P N (dξ ) does no have a well defined densiy wih respec o he Lebesgue measure, i is a common convenion in he paricle filering communiy [2, 15] o use p N (ξ ) = 1 N N δ(ξ ξ (i) ) (8) where δ( ) is a Dirac dela funcion, as if i is an empirical approimaion of he probabiliy densiy p(ξ ) wih respec o he Lebesgue measure. The noaions like (8) and he corresponding erms menioned above are, in he mos rigorous mahemaical sense, no correc. However, hey enable one o avoid he use of measure heory which indeed simplifies he represenaion a lo especially when a heoreical convergence proof is no he concern. III. DECENTRALIZED PARTICLE FILTER In his secion, he DPF algorihm is firs derived and hen summarized. Finally, some issues regarding he implemenaion of he DPF are discussed. We here make a commen regarding some noaions used A. Derivaion of he DPF algorihm Firs, we iniialize he paricles (i) 0 p( 0 ), i = 1,..., N, and for each (i) 0, he paricles z(i,j) 0 p(z 0 (i) 0 ), j = 1,..., N z. Then he derivaion of he DPF algorihm will be compleed in wo seps by he inducion principle. In he firs

4 4 sep, we show ha Inducive Assumpions 3.1 o 3.3 o be inroduced below hold a = 1. In he second sep, assume ha Inducive Assumpions 3.1 o 3.3 hold a for 1, hen we show ha Inducive Assumpions 3.1 o 3.3 hold recursively a + 1. In he following, we firs inroduce Inducive Assumpions 3.1 o 3.3 a. Assumpion 3.1: The DPF produce N paricles (i) 0: 1, i = 1,..., N and an unweighed empirical approimaion of p( 0: 1 y 0: 1 ) as p N ( 0: 1 y 0: 1 ) = 1 N N δ( 0: 1 (i) 0: 1 ) (9) and for each pah (i) 0: 1, i = 1,..., N, he DPF produce N z paricles z (i,j) 1, j = 1,..., N z, and a weighed empirical approimaion of p(z 1 (i) 0: 1, y 0: 1) as p Nz (z 1 (i) 0: 1, y 0: 1) = 1 δ(z 1 z (i,j) 1 ) (10) where 1 is he imporance weigh and is definiion will be given in he derivaion. Assumpion 3.2: The paricles (i), i = 1,..., N, are generaed according o he proposal funcion π( (i) 0: 1, y 0: 1) and for each (i), i = 1,..., N, he paricles z (i,j), j = 1,..., N z, are generaed according o he proposal funcion π(z (i) 0:, y 0: 1) where (i) 0: ((i) 0: 1, (i) ). Assumpion 3.3: For each i = 1,..., N, an approimaion p Nz (y (i) 0:, y 0: 1) of p(y (i) 0:, y 0: 1) can be obained as wih r (i,j) where p( z (i,j) p Nz (y (i) 0:, y 0: 1) = r (i,j) p(y (i), z (i,j) )/ = p Nz ( z (i,j) (i) l=1 r (i,l) (11) 0:, y 0: 1)/π( z (i,j) (i) 0:, y 0: 1), p Nz ( z (i,j) (i) 0:, y 0: 1) is an approimaion of 0:, y 0: 1) and is definiion will be given in he (i) derivaion. Remark 3.1: In Inducive Assumpions 3.1 and 3.3, p N ( 0: 1 y 0: 1 ) and p Nz (z 1 (i) 0: 1, y 0: 1) are empirical approimaions of p( 0: 1 y 0: 1 ) and p(z 1 (i) 0: 1, y 0: 1), respecively. These approimaions should of course be as close o he rue underlying densiy as possible. This closeness is ypically assessed via some es funcion g 1 ( ) used in he following sense, I(g 1 ) = g 1 (z 1, 0: 1 ) p(z 1, 0: 1 y 0: 1 )dz 1 d 0: 1 (12) where g 1 : R nz R n R is a es funcion. Wih he empirical approimaions defined in (9) and (10), an esimae I N,N z (g 1 ) of I(g 1 ) is obained as follows I N,N z (g 1 ) = 1 N N 1 g 1( z (i,j) 1, (i) 0: 1 ) (13) A convergence resul formalizing he closeness of (13) o (12), as N and N z end o infiniy, will be in line wih our earlier convergence resuls [18] of he PF for raher arbirary unbounded es funcions. Then in he firs sep we should show ha Inducive Assumpions 3.1 o 3.3 hold a = 1. However, since he firs sep is eacly he same as he second sep, we only consider he second sep here due o he space limiaion. In he second sep, assume ha Inducive Assumpions 3.1 o 3.3 hold a, hen we will show in seven seps ha Inducive Assumpions 3.1 o 3.3 recursively hold a ) Measuremen updae of 0: based on y By using imporance sampling, his sep aims o derive a weighed empirical approimaion of p( 0: y 0: ). Noe ha an unweighed empirical approimaion p N ( 0: 1 y 0: 1 ) of p( 0: 1 y 0: 1 ) has been given as (9) in Inducive Assumpion 3.1, hen simple calculaion shows ha he following equaion holds approimaely p( 0: y 0: ) p N ( 0: 1 y 0: 1 )p( 0: 1, y 0: 1 ) p(y 0:, y 0: 1 ) (14) From Inducive Assumpions 3.1 and 3.2, (i) 0: 1, i = 1,..., N, can be regarded as samples drawn from p N ( 0: 1 y 0: 1 ), and for i = 1,..., N, is he sample drawn from π( (i) 0: 1, y 0: 1). Therefore p N ( 0: 1 y 0: 1 )π( (i) 0: 1, y 0: 1) can be reaed as he imporance funcion. On he oher hand, from (10) and (i) p( 0: 1, y 0: 1 ) (15) = p(z 1 0: 1, y 0: 1 )p( 1, z 1 )dz an approimaion p Nz ( (i) (i) 0: 1, y 0: 1) p( (i) (i) 0: 1, y 0: 1) can be obained as p Nz ( (i) (i) 0: 1, y 0: 1) = 1 p( (i) (i) 1, z(i,j) 1 ) (16) Moreover, an approimaion p Nz (y (i) 0:, y 0: 1) of p(y (i) 0:, y 0: 1) has been given as (11) in Inducive Assumpion 3.3. Now using he imporance sampling yields a weighed empirical approimaion of p( 0: y 0: ) as where w (i) w (i) N p N ( 0: y 0: ) = w (i) δ( 0: (i) 0: ) (17) is evaluaed according o p N z (y (i) 0:, y 0: 1)p Nz ( (i) (i) 0: 1, y 0: 1) π( (i) (i) 0: 1, y 0: 1) N w (i) = 1 ; of (18)

5 5 2) Resampling of 0:, z(i,1), r (i,1),..., z (i,nz), r (i,nz) }, i = 1,..., N By using resampling, his sep aims o derive an unweighed empirical approimaion of p( 0: y 0: ). Resample N imes from he discree disribuion over { 0:, z(i,1), r (i,1),..., z (i,nz) wih probabiliy mass w (i), r (i,1) 0:, z(i,1), r (i,nz) }, i = 1,..., N } associaed wih he elemen, r (i,nz) } o generae sam-, r (i,nz) }, i = 1,..., N, ples { (i) 0:, z(i,1) so ha for any m, P{{ (m) 0:, z(m,1) 0:, z(i,1),..., z (i,nz), r (i,1), r (i,1), r (m,1),..., z (i,nz),..., z (m,nz), r (m,nz) } =, r (i,nz),..., z (i,nz) }} = w (i) (19) where r (i,j) can be defined as r (i,j) = p Nz ( z (i,j) (i) 0:, y 0: 1)/π( z (i,j) (i) 0:, y 0: 1) according o he definiion of r (i,j) in Inducive Assumpion 3.3 (See Secion III-C1 for more deailed eplanaion). As a resul, i follows from (17) ha an unweighed empirical approimaion of p( 0: y 0: ) is obained as p N ( 0: y 0: ) = 1 N δ( 0: (i) 0: N ) (20) 3) Measuremen updae of z based on y By using imporance sampling, his sep aims o derive a weighed empirical approimaion of p(z (i) 0:, y 0:). Simple calculaion shows ha p(z (i) 0:, y 0:) p(z (i) 0:, y 0: 1)p(y (i), z ) (21) From Inducive Assumpion 3.2, for each (i), i = 1,..., N, he paricles z (i,j), j = 1,..., N z, are generaed from he proposal funcion π(z (i) 0:, y 0: 1). Therefore π(z (i) 0:, y 0: 1) is chosen as he imporance funcion. On he oher hand, noe ha an approimaion p Nz ( z (i,j) (i) 0:, y 0: 1) of p( z (i,j) (i) 0:, y 0: 1) has been given in Inducive Assumpion 3.3. Then using imporance sampling and also noing he definiion of r (i,j) yields, for each i = 1,..., N, a weighed empirical approimaion of p(z (i) 0:, y 0:) as p Nz (z (i) 0:, y 0:) = where is evaluaed according o p(y (i), z (i,j) )r (i,j) ; δ(z z (i,j) ) (22) = 1 (23) 4) Generaion of paricles (i) +1, i = 1,..., N Assume ha he paricles (i) +1, i = 1,..., N, are generaed according o he proposal funcion π( +1 (i) 0:, y 0:). 5) Measuremen updae of z based on +1 By using imporance sampling, his sep aims o derive a weighed empirical approimaion of p(z (i) 0:+1, y 0:). Simple calculaion shows ha p(z (i) 0:+1, y 0:) p(z (i) 0:, y 0: 1)p(y (i), z ) p( (i) +1 (i), z ) (24) Analogously o sep 3), choose π(z (i) 0:, y 0: 1) as he imporance funcion and noe ha p Nz ( z (i,j) (i) 0:, y 0: 1) is an approimaion of p( z (i,j) (i) 0:, y 0: 1). Using imporance sampling and also noing he definiion of r (i,j) yields, for each i = 1,..., N, a weighed empirical approimaion of p(z (i) 0:+1, y 0:) as p Nz (z (i) 0:+1, y 0:) = where (i) 0:+1 according o q (i,j) ( (i) 0:, (i) +1 ) and q(i,j) p(y (i), z (i,j) )p( (i) q (i,j) = 1 q (i,j) δ(z z (i,j) ) (25) +1 (i), z (i,j) is evaluaed )r (i,j) ; (26) 6) Resampling of he paricles z (i,j), i = 1,..., N, j = 1,..., N z By using resampling, his sep aims o derive an unweighed empirical approimaion of p(z (i) 0:+1, y 0:). For each i = 1,..., N, resample N z imes from he discree disribuion over { z (i,j), j = 1,..., N z } wih probabiliy mass q (i,j) associaed wih he elemen z (i,j) o generae samples {z (i,j) any m, P{z (i,m), j = 1,..., N z }, so ha for = z (i,j) } = q (i,j) (27) As a resul, i follows from (25) ha for each i = 1,..., N, an unweighed empirical approimaion of p(z (i) 0:+1, y 0:) is as follows p Nz (z (i) 0:+1, y 0:) = 1 N z δ(z z (i,j) ) (28) 7) Generaion of paricles z (i,j) +1, i = 1,..., N, j = 1,..., N z Assume ha for each (i) +1, i = 1,..., N, he paricles z (i,j) +1, j = 1,..., N z, are generaed according o he proposal funcion π(z +1 (i) 0:+1, y 0:). By using imporance sampling, we ry o derive a weighed empirical approimaion of p(z +1 (i) 0:+1, y 0:). Firs, choose π(z +1 (i) 0:+1, y 0:) as he imporance funcion. Then from (28) and p(z +1 0:+1, y 0: ) (29) = p(z 0:+1, y 0: )p(z +1 :+1, z )dz an approimaion of p(z +1 (i) 0:+1, y 0:), i = 1,..., N, can be obained as p Nz (z +1 (i) 0:+1, y 0:) = 1 p(z +1 (i) :+1 N, (30) z(i,l) ) z l=1

6 6 Now using imporance sampling yields ha for i = 1,..., N, a weighed empirical approimaion of p(z +1 (i) 0:+1, y 0:) as follows p Nz (z +1 (i) 0:+1, y 0:) N z Nz = r (i,j) +1 δ(z +1 z (i,j) +1 )/ where r (i,j) +1 is evaluaed according o l=1 r (i,l) +1 (31) r (i,j) +1 = p N z ( z (i,j) +1 (i) 0:+1, y 0:)/π( z (i,j) +1 (i) 0:+1, y 0:) (32) Finally, from (31) and p(y +1 0:+1, y 0: ) (33) = p(z +1 0:+1, y 0: )p(y +1 +1, z +1 )dz +1 an approimaion p Nz (y +1 (i) 0:+1, y 0:) of p(y +1 (i) 0:+1, y 0:) can be obained as (11) wih replaced by + 1. In urn, i can be seen from (20), (22), sep 4), sep 7), and (32) ha Inducive Assumpions 3.1 o 3.3 hold wih replaced by + 1. Hence, we have compleed he derivaion of he DPF by he inducion principle. B. Summary of he filering algorihm Iniializaion Iniialize he paricles (i) 0 p( 0 ), i = 1,..., N, and for each (i) 0, he paricles z(i,j) 0 p(z 0 (i) 0 ), j = 1,..., N z. Wih a sligh abuse of noaion le p( 0 ) = p Nz ( 0 0: 1, y 0: 1 ) = π( 0 0: 1, y 0: 1 ) and p(z 0 0 ) = p Nz (z 0 0, y 0: 1 ) = π(z 0 0, y 0: 1 ). A each ime insan ( 0) 1) Measuremen updae of 0: based on y The imporance weighs w (i), i = 1,..., N, are evaluaed according o (18). 2) Resampling of 0:, z(i,1), r (i,1)..., z (i,nz), r (i,nz) }, i = 1,..., N According o (19), resample 0:, z(i,1), r (i,1)..., z (i,nz), r (i,nz) }, i = 1,..., N, o generae samples { (i) 0:, z(i,1), r (i,1)..., z (i,nz), r (i,nz) }, i = 1,..., N, where for 0, r (i,j) is defined according o (32). 3) Measuremen updae of z based on y For i = 1,..., N, he imporance weighs, j = 1,..., N z, are evaluaed according o (23). 4) Generaion of paricles (i) +1, i = 1,..., N For each i = 1,..., N, (i) +1 he proposal funcion π( +1 (i) 0:, y 0:). 5) Measuremen updae of z based on +1 is generaed according o For i = 1,..., N, he imporance weighs q (i,j), j = 1,..., N z, are evaluaed according o (26). 6) Resampling of he paricles z (i,j), i = 1,..., N, j = 1,..., N z According o (27), for each i = 1,..., N, resample he paricles z (i,j) z (i,j), j = 1,..., N z., j = 1,..., N z, o generae samples 7) Generaion of paricles z (i,j) +1, i = 1,..., N, j = 1,..., N z For each i = 1,..., N, he paricles z (i,j) +1, j = 1,..., N z, are generaed according o he proposal funcion π(z +1 (i) 0:+1, y 0:) where (i) 0:+1 ((i) 0:, (i) +1 ). C. Implemenaion issues 1) Two resampling seps: Unlike mos of PFs in he lieraure, he DPF has wo resampling seps, i.e., sep 2) and sep 6). Furhermore, he second resampling bears a parallel srucure. This is because he paricles z (i,j), i = 1,..., N, j = 1,..., N z, can be divided ino N independen groups in erms of he inde i. Therefore, he second resampling can be implemened in parallel. In he implemenaion of he firs resampling, i would be helpful o noe he following poins. For i = 1,..., N, { r (i,1),..., r (i,nz) } is associaed wih 0:, z(i,1),..., z (i,nz) }, according o he definiion r (i,j) = p Nz ( z (i,j) (i) 0:, y 0: 1)/π( z (i,j) (i) 0:, y 0: 1). Therefore, afer resampling of 0:, z(i,1),..., z (i,nz) }, i = 1,..., N, { r (i,1),..., r (i,nz) }, i = 1,..., N, should accordingly be resampled o obain {r (i,1),..., r (i,nz) }, i = 1,..., N. According o he definiion of r (i,j), r (i,j) can be defined as r (i,j) = p Nz ( z (i,j) (i) 0:, y 0: 1)/π( z (i,j) (i) 0:, y 0: 1). As a resul, 0:, z(i,1), r (i,1),..., z (i,nz), r (i,nz) }, i = 1,..., N, is resampled in sep 2). Moreover, since he paricles (i) 0: 1, i = 1,..., N, will no be used in he fuure, i is acually only necessary o resample, z (i,1), r (i,1),..., z (i,nz), r (i,nz) }, i = 1,..., N, o generae { (i), z (i,1), r (i,1),..., z (i,nz), r (i,nz) }, i = 1,..., N. 2) Consrucion of he proposal funcions: Like [15] where he prior is chosen as he proposal funcion, we ry o choose p( +1 (i) 0:, y 0:) and p(z +1 (i) 0:+1, y 0:) as he proposal funcions π( +1 (i) 0:, y 0:) and π(z +1 (i) 0:+1, y 0:), respecively. Unlike [15], however, p( +1 (i) 0:, y 0:) and p(z +1 (i) 0:+1, y 0:) are usually unknown. Therefore, we need o consruc approimaions o p( +1 (i) 0:, y 0:) and p(z +1 (i) 0:+1, y 0:) such ha he paricles (i) +1 and z(i,j) +1, j = 1,..., N z, can be sampled from he approimaions, respecively. A convenien way o consruc hose approimaions is given as follows. From (22), an approimaion of p( +1 (i) 0:, y 0:) can be obained as p Nz ( +1 (i) 0:, y 0:) = p( +1 (i), z (i,j) ) (34) In urn, a furher approimaion of p( +1 (i) 0:, y 0:) can be obained as N( (i) +1, Σ(i) +1 ) wih (i) +1 and Σ(i) +1, respecively, he mean and he covariance of he discree disribuion over { (i,j) +1, j = 1,..., N z} wih probabiliy mass associaed wih he elemen (i,j) +1 p( +1 (i), z (i,j) ). Therefore, for i = 1,..., N, he paricle (i) +1 can be generaed from π( +1 (i) 0:, y 0:) = N( (i) +1, Σ(i) +1 ) (35) On he oher hand, from (28) and (29), an approimaion p Nz (z +1 (i) 0:+1, y 0:) of p(z +1 (i) 0:+1, y 0:) has already been

7 7 given in (30). Then, i follows from (30) and he assumpion ha p(z +1 :+1, z ) is known ha, for each i = 1,..., N, he paricle z (i,j) +1, j = 1,..., N z can be generaed from π(z +1 (i) 0:+1, y 0:) = p Nz (z +1 (i) 0:+1, y 0:) (36) Remark 3.2: From (25) and (29), anoher approimaion of p(z +1 (i) 0:+1, y 0:) can be obained as p Nz (z +1 (i) 0:+1, y 0:) = q (i,j) p(z +1 (i) 0:+1, z(i,j), y 0: ) (37) This observaion shows ha he PF used o esimae p(z (i) 0:, y 0:) is closely relaed o he so-called marginal PF [21]. A marginal PF would sample he paricles z (i,j) +1, j = 1,..., N z, from π(z +1 (i) 0:+1, y 0:) = q (i,j) π(z +1 (i) 0:+1, z(i,j), y 0: ) (38) According o [21], sampling from (38) is precisely equivalen o firs resample he paricles z (i,j), i = 1,..., N, j = 1,..., N z according o (27) and hen sample he paricle z (i,j) +1 from π(z +1 (i) 0:+1, z(i,j), y 0: ). If (37) is chosen as he proposal funcion, i.e., π(z +1 (i) 0:+1, y 0:) = p Nz (z +1 (i) 0:+1, y 0:) in he marginal PF, hen sampling from (38) would be eacly he same as wha we did. In addiion, like he marginal PF, for each i = 1,..., N, he compuaional cos of r (i,j), j = 1,..., N z, is O(Nz 2 ). However, his should no be a problem as a small N z of paricles are usually used o approimae p(z (i) 0:, y 0:). Moreover, a couple of mehods have been given in [21] o reduce his compuaional cos. On he oher hand, if (36) is chosen as he proposal funcion, hen r (i,j) = 1, i = 1,..., N, j = 1,..., N z. As a resul, he resampling of { r (i,1),..., r (i,nz) }, i = 1,..., N, is no needed and he compuaion of (11), (23) and (26) is simplified. Remark 3.3: If sysem (3) has a special srucure, hen he consrucion of π( +1 (i) 0:, y 0:) can become simpler. We menion wo cases here. Firs, assume ha he -dynamics of sysem (3) is independen of z, hen p( +1 (i) 0:, y 0:) = p( +1 (i) ) and hus we can choose π( +1 (i) 0:, y 0:) = p( +1 (i) ). Sysems wih his special srucure have been sudied in he lieraure before, see e.g., [11]. Second, assume ha sysem (3) akes he following form +1 = f (, z ) + g ( )v z +1 = f z (, z ) + g z (, z )v z y = h (, z, e ) (39) where f ( ), f z ( ), g ( ), g z ( ) and h ( ) are known funcions, v is assumed whie and Gaussian disribued according o [ ] ( [ ]) v v = Q N 0, (Q z ) T (40) v z Q z Then from (34), a furher simplified approimaion of p( +1 (i) 0:, y 0:) can be obained as N( (i) +1, Σ(i) +1 + Q z g ((i) )Q (g ((i) )) T ) wih (i) +1 and Σ(i) +1, respecively, he mean and he covariance of he discree disribuion { (i,j) +1, j = 1,..., N z} wih probabiliy mass, z (i,j) asso- ). For ciaed wih he elemen (i,j) +1 p( +1 (i) his special case, π( +1 (i) 0:, y 0:) = N( (i) +1, Σ(i) g ((i) )Q (g ((i) )) T ) While we have assumed he proposal funcions in he form of π( (i) 0: 1, y 0: 1) and π(z (i) 0:, y 0: 1), i is possible o choose proposal funcions in more general forms. For eample, in he imporance sampling of (24), he proposal funcion π(z (i) 0:, y 0: 1) can be replaced by anoher proposal funcion in he form of π(z (i) 0:+1, y 0: 1). Moreover, his new proposal funcion, ogeher wih he wo proposal funcions π( (i) 0: 1, y 0: 1) and π(z (i) 0:, y 0: 1) can be furher made dependen on y. Tha is, in he imporance sampling of (14), (21) and (24), he proposal funcions in he form of π( (i) 0: 1, y 0:), π(z (i) 0:, y 0:) and π(z (i) 0:+1, y 0:) can be used, respecively. Due o he space limiaion, we do no discuss his issue here and we refer he ineresed reader o, for eample, [2, 7, 13, 23] for relevan discussions. Finally, we remaind ha if hese proposal funcions in more general forms are used, hen sligh modificaion is needed o make o he DPF algorihm. 3) Compuing he sae esimae: A common applicaion of a PF is o compue he sae esimae, i.e., he epeced mean of he sae. For sysem (3), he sae esimae of and z are defined as = E p( y 0:)( ), z = E p(z y 0:)(z ) (41) Then he approimaion of and z can be compued in he following way for he DPF. Noe from (17) ha p( y 0: ) has an empirical approimaion p N ( y 0: ) = N w(i) δ( (i) ). Then, an approimaion ˆ of can be calculaed in he following way N ˆ = E pn ( y 0:)( ) = w (i) (i) (42) Analogously, noe from (20) and (22) ha p(z y 0: ) has an empirical approimaion p N,N z (z y 0: ) = 1 N Nz N q(i,j) δ(z z (i,j) ). Then, an approimaion ẑ of z can be calculaed as ẑ = E pn,nz (z y 0:)(z ) = 1 N N IV. DISCUSSION z (i,j) (43) In he DPF algorihm, he summaion calculaion in he normalizing facor (he denominaor) of w (i) in (18) and he firs resampling, i.e., he resampling of, z (i,1), r (i,1),..., z (i,nz), r (i,nz) }, i = 1,..., N, are he only operaions ha canno be implemened in parallel. Besides, he remaining operaions including he second resampling, i.e., he resampling of he paricles z (i,j), i = 1,..., N, j = 1,..., N z, can be divided ino N independen pars in erms of he inde i and can hus be implemened in parallel. In his sense, we say ha he DPF has he advanage over he

8 8 CU PE PE MU W (k) MU y w i Fig. 3. The archiecure of he PF wih parallel srucure. I consiss of a cenral uni (CU) and a number of processing elemens (PE) where he CU handles he operaions ha canno be implemened in parallel and he PEs are run in parallel o deal wih he operaions ha can be implemened in parallel. PE Inra R N (k) Iner R TU z R z R regular PF in ha he DPF can increase he level of parallelism of he PF. The underlying idea of he DPF is differen from ha of he disribued PFs, such as he DRPA-PF and he DRNA- PF (see Secion I), while hey all have parallel srucure. This difference resuls in a couple of unique feaures of he DPF in conras wih he disribued PFs, which idenify he poenial of he DPF in he applicaion of PFs in real-ime sysems and he parallel implemenaion of PFs. In he remaining par of his secion, we firs briefly review he DRPA-PF and he DRNA-PF, and hen we poin ou he unique feaures of he DPF. Finally, we show ha here eis wo ways o furher increase he level of parallelism of he DPF. Before he discussion, i should be noed ha all he DPF, he DRPA-PF and he DRNA-PF have he archiecure as shown in Fig. 3. A. Review of he DRPA-PF and he DRNA-PF [5] Assume ha he sample space conains M paricles, where M is assumed o be he number of paricles ha is needed for he sampling imporance resampling PF (SIR-PF) [13] or he boosrap PF [15] o achieve a saisfacory performance for he filering problem of sysem (1). Then he sample space is divided ino K disjoin sraa where K is an ineger and saisfies 1 K M. Furher assume ha each sraum corresponds o a PE. Before resampling each PE hus has N paricles where N = M/K is an ineger. The sequence of operaions performed by he kh PE and he CU for he DRPA-PF is shown in Fig. 4. The iner-resampling is performed on he CU and is funcion is o calculae he number of paricles N (k) ha will be produced afer resampling for he kh PE. E(N (k) ) should be proporional o he weigh W (k) of he kh PE which is defined as he sum of he weighs of he paricles inside he kh PE. In paricular, N (k) is calculaed using he residual sysemaic resampling (RSR) algorihm proposed in [4]. Once N (k), k = 1,..., K, are known, resampling is performed inside he K PEs independenly which is referred o as he inra-resampling. Noe ha afer resampling, for k = 1,..., K, he kh PE has N (k) paricles and N (k) is a random number because i depends on he overall disribuion of he weighs W (k), k = 1,..., K. On he oher hand, noe ha each PE is supposed o be responsible for processing N paricles and o perform he same operaions in ime. Therefore afer resampling, he echange of paricles PR TU PE k DRPA-PF CU PR N (k) N ξ -paricles MU z TU MU y z PR PE i DPF CU PR See Remark 4.1 Fig. 4. Sequence of operaions performed by he specified PE and he CU for he DRPA-PF and he DPF. The daa ransmied beween he specified PE and he CU and beween differen PEs are marked. For he DRPA- PF, he abbreviaions are MU (measuremen updae of ξ 0: based on y ), Iner R (iner resampling), Inra R (inra resampling), PR (paricle rouing), and TU (generaion of paricles ξ (l) +1, l = 1,...,M). For he DPF, he abbreviaions are MU y (measuremen updae of 0: based on y ), R (resampling of, z (i,1), r (i,1),..., z (i,nz), r (i,nz) }, i = 1,...,N ), PR (paricle rouing), MU y z (measuremen updae of z based on y ), TU (generaion of paricles (i) +1, i = 1,...,N), MU z (measuremen updae of z based on +1 ), R z (resampling of z (i,j), i = 1,...,N, j = 1,...,N z) and TU z (generaion of paricles z (i,j) +1, i = 1,...,N, j = 1,...,Nz). among he PEs has o be performed such ha each PE has N paricles. This procedure is referred as paricle rouing and is conduced by he CU. According o [5], he DRPA- PF requires a complicaed scheme for paricle rouing due o he proporional allocaion rule. In order o shoren he delay caused by he complicaed paricle rouing scheme in he DRPA-PF, he DRNA is in urn proposed in [5]. B. Unique feaures of he DPF The parallel srucure of he DPF is creaed by decomposing he sae space, differing from he parallel srucure of he disribued PFs which is creaed by dividing he sample space. In he following, we will show ha his difference resuls in a couple of unique feaures of he DPF. Before he discussion, he sequence of operaions performed by he ih PE and he CU for he DPF is shown in Fig. 4. The summaion calculaion in he normalizing facor of w (i) in (18) and he resampling of, z (i,1), r (i,1),..., z (i,nz), r (i,nz) },

9 9 i = 1,..., N, are performed on he CU. Assume ha here are N PEs, and for each i = 1,..., N, he ih PE handles he ih independen par of he remaining operaions of he DPF. In paricular, for each i = 1,..., N, he ih PE handles he resampling of he paricles z (i,j), j = 1,..., N z. Therefore, he resampling of he paricles z (i,j), i = 1,..., N, j = 1,..., N z, is run in parallel on he PEs. Remark 4.1: In he paricle rouing of he DPF, he daa ransmied beween differen PEs depends on he resampling resul of, z (i,1), r (i,1),..., z (i,nz), r (i,nz) }, i = 1,..., N. More specifically, here will be no daa ransmied hrough he ih PE if, z (i,1), r (i,1),..., z (i,nz), r (i,nz) } is seleced only once in he resampling. Oherwise, he daa ransmied hrough he ih PE will be { (m), z (m,1), r (m,1),..., z (m,nz), r (m,nz) } for some m = 1,..., N. In paricular, if, z (i,1), r (i,1),..., z (i,nz), r (i,nz) } is seleced more han once, hen m = i; if, z (i,1), r (i,1),..., z (i,nz), r (i,nz) } is no seleced, hen m = 1,..., N and m i. Therefore, he daa ransmied beween any wo PEs, say, he i 1 h PE and he i 2 h PE, will be eiher zero or { (m), z (m,1), r (m,1),..., z (m,nz), r (m,nz) } for eiher m = i 1 or m = i 2. In conras o he DRPA-PF, he DPF allows a simpler paricle rouing scheme. For he DRPA-PF, since afer resampling he kh PE has N (k) paricles ha is a random number, a complicae scheme has o be used for he DRPA-PF o make all K PEs has equally N paricles. For he DPF, however, since afer he resampling of, z (i,1), r (i,1),..., z (i,nz), r (i,nz) }, i = 1,..., N, all N PEs sill have he same number of paricles, hen he DPF allows a simpler paricle rouing scheme and acually each PE can be reaed as a single paricle in he paricle rouing. Given a PF wih parallel srucure, i works mos efficienly if each PE handles he same number of paricles. The efficiency of he DRPA-PF usually decreases, since he numbers of paricles produced by each PE are no evenly bu randomly disribued among he PEs. To be specific, noe ha he ime used by he kh PE o produce N (k) paricles, k = 1,..., K, afer resampling are usually no he same. This observaion implies ha he ime used by he DRPA o produce he paricles afer resampling is deermined by he k h PE ha produces he larges N (k ). Clearly, he more unevenly he numbers of paricles produced by each PE are disribued, he more ime he DRPA akes o produce he paricles afer resampling. Especially, in he ereme case ha N (k ) N (k) wih k = 1,..., K, and k k, he efficiency of he DRPA- PF will be decreased significanly. However, for he DPF, he ih PE ha handles he resampling of paricles z (i,j), j = 1,..., N z, produces, afer resampling, he same number of paricles z (i,j), j = 1,..., N z. Therefore, he DPF does no have he efficiency decrease problem of he DRPA-PF. Besides, i will be verified by wo numerical eamples in he subsequen secion ha, he DPF has he poenial o achieve in a shorer eecuion ime he same level of performance as he boosrap PF. However, he DRNA-PF acually rades PF performance for speed improvemen [5, 22]. Remark 4.2: The ideal minimum eecuion ime T e of he DRPA-PF and he DRNA-PF have been given in [5, 22]. Analogously, we can also give he ideal minimum eecuion ime of he DPF. Like [5, 22], he following assumpions are made. Consider an implemenaion wih a pipelined processor. Assume ha he eecuion ime of he paricle generaion and he imporance weighs calculaion of every paricle is LT clk where L is he laency due o he pipelining and T clk is he clock period. Also assume ha he resampling akes he same amoun of ime as he paricle generaion and he imporance weighs calculaion. As a resul, we have he ideal minimum eecuion ime of he DPF as Te DPF = (2N z + L + N + M r + 1)T clk. Here, 2N z represens he delay due o he resampling of he paricles z (i,j), j = 1,..., N z, and he corresponding paricle generaion and imporance weighs calculaion, N represens he delay due o he resampling of, z (i,1), r (i,1),..., z (i,nz), r (i,nz) }, i = 1,..., N, M r is he delay due o he paricle rouing and he era one T clk is due o he paricle generaion and imporance weigh calculaion of he paricle (i). C. Two ways o furher increase he level of parallelism of he DPF The firs resampling of he DPF, i.e., resampling of, z (i,1), r (i,1),..., z (i,nz), r (i,nz) }, i = 1,..., N, is he major operaion ha canno be implemened in parallel. If N is large, hen his resampling will cause a large delay. In order o furher increase he level of parallelism of he DPF and shoren he eecuion ime, i is valuable o find ways o handle his problem. Two possible ways will be given here. The firs one is sraighforward and is o employ any of he disribued resampling algorihms proposed in [5, 22] o perform he firs resampling of he DPF and besides, he remaining pars of he DPF say unchanged. Noneheless, we prefer he DRPA o he oher disribued resampling algorihms, since i can produce he same resul as he sysemaic resampling [20] according o [4]. Compared o he firs way, he second way only applies o high dimensional sysem (1) and i is based on an eension of he DPF. We have assumed above ha he sae ξ is decomposed ino wo pars according o (2). Acually, he DPF can be eended o handle he case where he sae ξ is decomposed ino more han wo (a mos n ξ ) pars. For illusraion, we briefly consider he case where he sae ξ is decomposed ino hree pars. The more general case can be sudied using he inducion principle. For convenience, assume ha he sae in (2) can be furher decomposed ino wo pars, i.e., = [ 1, 2, ] (44) and accordingly, sysem (3) can be furher decomposed ino he following form 1,+1 = f 1 ( 1,, 2,, z, v 1 ) 2,+1 = f 2 ( 1,, 2,, z, v 2 ) z +1 = f z ( 1,, 2,, z, v z ) y = h ( 1,, 2,, z, e ) (45)

10 10 where 1, R n 1, 2, R n 2, v = [(v 1 ) T (v 2 ) T ] T wih v 1 R n v 1, v 2 R n v 2. Assume ha he probabiliy densiies p( 1,0 y 1 ) = p( 1,0 ), p( 2,0 1,0, y 1 ) = p( 2,0 1,0 ), p(z 0 1,0, 2,0, y 1 ) = p(z 0 1,0, 2,0 ) and for 0, p( 1,+1 1,, 2,, z ), p( 2,+1 1,:+1, 2,, z ), p(z 1,:+1, 2,:+1, z ) and p(y 1,, 2,, z ) are known. The filering problem of sysem (45) can be spli ino hree nesed sub-problems according o he following facorizaion p(z, 1,0:, 2,0: y 0: ) = p(z 1,0:, 2,0:, y 0: ) p( 2,0: 1,0:, y 0: )p( 1,0: y 0: ) (46) where for i = 1, 2, i,0: { i,0,..., i, }. I can be shown ha he DPF can be eended o handle he filering problem of sysem (45) by using PFs o solve he hree nesed subproblems. Roughly speaking, a PF wih N 1 paricles ( (i) 1,0:, i = 1,..., N 1 ) will be used o esimae p( 1,0: y 0: ), and for each i = 1,..., N 1, a PF wih N 2 paricles ( (i,j) 2,0:, j = 1,..., N 2 ) will be used o esimae p( 2,0: (i) 1,0:, y 0:), and for each i = 1,..., N 1 and j = 1,..., N 2, a PF wih N z paricles will be used o esimae p(z (i) 1,0:, (i,j) 2,0:, y 0:). Similar o he DPF based on (4), he major operaion ha canno be implemened in parallel in he DPF based on (46) is is firs resampling, i.e., he resampling of N 1 composie paricles. If a saisfacory performance of he DPF based on (46) can be achieved wih N 1 +N 2 N, hen he number of composie paricles involved in he firs resampling of he DPF will be reduced from N o N 1. Therefore, in his way he level of parallelism of he DPF is furher increased. If he DPF is implemened in parallel, hen he eecuion ime of he DPF will be furher decreased as well. However, i should be noed ha N 1 N 2 PEs are required o fully eploi he parallelism of he DPF based on (46). Due o he space limiaion, we canno include he eension of he DPF in his paper and insead we refer he reader o [10] for he deails. V. NUMERICAL EXAMPLES In his secion we will es how he DPF performs on wo eamples. The simulaions are performed using Malab under he Linu operaing sysem. The plaform is a server consising of eigh Inel(R) Quad Xeon(R) CPUs (2.53GHz). A. Algorihms esed For he wo eamples, he boosrap PF is implemened in he sandard fashion, using differen number of paricles (M). The DPF is implemened according o Secion III-B for differen combinaions of and z paricles (N and N z ). The DRPA-PF according o [5] is esed as well, using differen number of PEs (K). The formulas of [5] has been closely followed, bu he implemenaion is our own, and i is of course possible ha i can be furher rimmed. In addiion, as suggesed in [17, 20] sysemaic resampling is chosen as he resampling algorihm for all algorihms esed. B. Performance evaluaion: Accuracy In he ess, he performance of all algorihms are evaluaed by Mone Carlo simulaions. Basically, he accuracy of he sae esimae is measured by he Roo Mean Square Error (RMSE) beween he rue sae and he sae esimae. For eample, he RMSE of ˆ is defined as RMSE of ˆ = = i ˆi 2 (47) where wih a sligh abuse of noaion, i denoes he rue sae a ime for he ih simulaion and ˆ i is he corresponding sae esimae. I is also esed how well he RMSE reflecs he accuracy of he esimaed poserior densiies (See Remark 5.1 for more deails). C. Performance evaluaion: Timing One objecive wih he simulaions is o assess he poenial efficiency of he parallel implemenaion of he DPF. For ha purpose, we record he following imes T si : This is he average eecuion ime of he sequenial implemenaion of a PF. T cp : This is he average ime used by he operaions ha canno be implemened in parallel in a PF. T pi : This is he poenial eecuion ime of parallel implemenaion of a PF. For he boosrap PF wih cenralized resampling and he DPF, i is calculaed according o T pi = T cp + (T si T cp )/N PE where N PE is he number of processing elemens. For he DPF, le N PE = N. For he boosrap PF wih cenralized resampling, le N PE be he maimal N in he simulaion of he corresponding eample. Here, he boosrap PF wih cenralized resampling means ha besides he resampling, he remaining paricle generaion and imporance weighs calculaion of he boosrap PF are implemened in parallel. For he DRPA-PF, T pi is calculaed according o T pi = T cp +T mir + (T si T cp T mir )/N PE where N PE = K and T mir is he average maimal inra-resampling ime for he DRPA-PF. D. Performance evaluaion: Divergence failures The rae r d is used o reveal how ofen a PF diverges in he Mone Carlo simulaions. The boosrap PF and he DRPA-PF are said o diverge if heir imporance weighs are all equal o zero in he simulaion. The DPF is said o diverge if w (i), i = 1,..., N, are all equal o zero in he simulaion. Once he divergence of a PF is deeced, he PF will be rerun. E. Skech of he simulaion For he wo eamples, he boosrap PF using M paricles is firs implemened and is accuracy measured by he RMSE will be reaed as he reference level. Then i is shown ha he DPF using suiable N and N z and z paricles can achieve he same level of accuracy. In urn, he DRPA-PF using M paricles, bu wih differen number of processing elemens is also implemened. Finally, he boosrap PF using 2M paricles is implemened.

11 11 TABLE I SIMULATION RESULT FOR SYSTEM (48) WITH (49) SEE SECTIONS V-B - V-D FOR EXPLANATIONS OF THE NUMBERS Case RMSE of [ˆ, ẑ ] T si (Sec) T cp (Sec) T pi (Sec) r d Boosrap PF, M = 1000 [2.0173, ] DPF, N = 100, N z = 19 [2.0104, ] DPF, N = 120, N z = 19 [1.9914, ] DPF, N = 110, N z = 24 [1.9907, ] DPF, N = 120, N z = 24 [1.9906, ] DRPA-PF, M = 1000, K = 40 [2.0222, ] DRPA-PF, M = 1000, K = 25 [2.0332, ] Boosrap PF, M = 2000 [1.9714, ] F. Two dimensional eample Consider he following wo dimensional nonlinear sysem 0.7 PF, M=1000 DPF, N =100, N z =19 z +1 = z 2 + v z +1 = + 0.5z + 25z 1 + z 2 y = aan( ) + z e + 8 cos(1.2) + v z (48) where [ 0 z 0 ] T is assumed Gaussian disribued wih [ 0 z 0 ] T N(0, I 2 2 ), v = [v v z]t and e are assumed whie and Gaussian disribued wih v N ( 0, [ ]), and e N(0, 1) (49) For he DPF, he proposal funcions are chosen according o Remark 3.3 and (36). The simulaion resul over [1 250] is shown in Table I, from which i can be seen ha he DPF has he poenial o achieve in a shorer eecuion ime he same level of accuracy as he boosrap PF. Remark 5.1: In Table I, he accuracy of he algorihms is measured enirely hrough he RMSE of he sae esimae (47). Since he PF acually compues esimaes of he poserior densiy p(z, 0: y 0: ) one may discuss if his would be a more appropriae quaniy o evaluae for comparison. Acually, he sae esimaes ˆ could be quie accurae even hough he esimae of he poserior densiy is poor. To evaluae ha, we compued an accurae value of he rue poserior densiy using he boosrap PF wih many (100000) paricles, and compared ha wih he esimaes of he poserior densiies using he boosrap PF and he DPF wih fewer (M = 1000 and N = 100, N z = 19) paricles. To avoid smoohing issues for he empirical approimaions of he poserior densiies, we made he comparisons for he poserior cumulaive disribuions (empirical approimaions of he poserior cumulaive disribuions). The resul is shown in Fig. 5. Moreover, le i denoe he rue mean of i, hen (47) wih i replaced by i is also calculaed: i is for he boosrap PF wih M = 1000, and for he DPF wih N =100 and N z =19. From he above simulaion resuls, we see ha he DPF is a leas as good as he boosrap PF in approimaing he poserior cumulaive disribuion. We conclude ha he RMSE (47) gives a fair evaluaion of he accuracy of he sae esimae produced by he DPF for sysem (48) wih (49) Time (Sec) Fig. 5. The disance F( y 0: ) F( y 0: ) beween he rue poserior cumulaive disribuion F( y 0: ) and he empirical approimaion of he poserior cumulaive disribuion F( y 0: ) obained by using he boosrap PF and he DPF (wih M = 1000 and N = 100, N z = 19 paricles, respecively) as a funcion of ime (Thin curve: he boosrap PF, Thick curve: he DPF). The resul is an average over simulaions. G. Four dimensional eample Consider he following four dimensional nonlinear sysem 1,+1 = 0.5 1, + 8 sin() + v 1 2,+1 = 0.4 1, , + v 2 z 1,+1 = z 1, + z 2, 1 + z2, 2 + v z1 z 2,+1 = z 1, + 0.5z 2, + 25z 2, 1 + z2, cos(1.2) + v z2 y = 1, + 2, , + aan(z 1, ) + z2 2, 20 + e (50) where = [ 1, 2, ] T and z = [z 1, z 2, ] T. [ T 0 zt 0 ]T is assumed Gaussian disribued wih [ T 0 zt 0 ]T N(0, I 4 4 ), v = [(v ) T (v z ) T ] T wih v = [v 1 v 2 ] T and v z = [v z1 v z2 ]T, and e are assumed whie and Gaussian disribued

12 12 TABLE II SIMULATION RESULT FOR SYSTEM (50) WITH (51) SEE SECTIONS V-B - V-D FOR EXPLANATIONS OF THE NUMBERS Case RMSE of [ˆ 1,, ˆ 2,, ẑ 1,, ẑ 2, ] T si (Sec) T cp (Sec) T pi (Sec) r d Boosrap PF, M = 1500 [1.1566, , , ] DPF, N = 50, N z = 29 [1.1707, , , ] DPF, N = 60, N z = 49 [1.1633, , , ] DPF, N = 75, N z = 39 [1.1610, , , ] DRPA-PF, M = 1500, K = 30 [1.1564, , , ] DRPA-PF, M = 1500, K = 50 [1.1566, , , ] Boosrap PF, M = 3000 [1.1518, , , ] wih v N 0, , and e N(0, 1) (51) Since he -dynamics does no depend on z, we le π( +1 (i) 0:, y 0:) = p( +1 (i) ) and choose he oher proposal funcion according o (36). The simulaion resul over [1 150] is shown in Table II, from which i can be seen ha he DPF has he poenial o achieve in a shorer eecuion ime he same level of accuracy as he boosrap PF. H. Summary Regarding he accuracy, comparison of he firs par of he RMSE column in Tables I and II shows ha wih suiably chosen N and N z, he DPF achieves he same level of accuracy as he boosrap PF. On he oher hand, wih comparable number of paricles (i is fair o compare M wih N (N z +1)) he accuracy is no much worse for he DPF han for he boosrap PF. In fac, in Table II he DPF even performs slighly beer han he PF for some of he saes (no saisical significance), illusraing ha allocaing poins as in Fig. 1.c could acually be beneficial for some sysems. Regarding iming, comparison of he T si and T pi column in Tables I and II shows ha he eecuion ime of he DPF can be shorened significanly if he DPF can be implemened in parallel. Moreover, he DPF has a poenial o offer beer accuracy in shorer eecuion ime. In paricular, he T pi of he DPF is less han ha of he boosrap PF wih cenralized resampling. I is valuable o noe ha he T pi of he DPF is even smaller han he T cp of he boosrap PF wih cenralized resampling, which is acually he lower bound of he eecuion ime of he boosrap PF wih cenralized resampling. In addiion, as discussed in Secion IV-C, he T pi of he DPF can be furher shorened by using any one of he disribued resampling algorihms o perform he resampling of, z (i,1), r (i,1),..., z (i,nz), r (i,nz) }, i = 1,..., N. As a resul, i is fair o say ha he parallel implemenaion of he DPF has he poenial o shoren he eecuion ime of he PF. VI. CONCLUSIONS In his paper we have proposed a new way of allocaing he paricles in paricle filering. By firs decomposing he sae ino wo pars, he DPF splis he filering problem ino wo nesed sub-problems and hen handles he wo nesed sub-problems using PFs. Reurning o he quesions in he inuiive preview in Secion II-A, we have seen ha he DPF can produce resuls of no much worse accuracy compared o he regular PF, wih comparable number of paricles. We have also seen ha he srucure gives a poenial for more efficien calculaions. The advanage of he DPF over he regular PF lies in ha he DPF can increase he level of parallelism of he PF in he sense ha par of he resampling has a parallel srucure. The parallel srucure of he DPF is creaed by decomposing he sae space, differing from he parallel srucure of he disribued PFs which is creaed by dividing he sample space. This difference resuls in a couple of unique feaures of he DPF in conras wih he eising disribued PFs. As a resul, we believe ha he DPF is a new opion for parallel implemenaion of PFs and he applicaion of PFs in real-ime sysems. An ineresing opic for fuure work is o sudy how o decompose he sae given a high dimensional sysem such ha he eecuion ime of he parallel implemenaion of he DPF can be maimally reduced. Anoher ineresing opic is o es he DPF in differen ypes of parallel hardware, for eample, graphical processing unis (GPU) and field-programmable gae arrays (FPGA). A furher opic of fuure invesigaions is o generalize he line paern in Fig. 1.c o oher lines and curves ha may pick up useful shapes in he poserior densiies o be esimaed. This essenially involves a change of sae variables before he DPF is applied. REFERENCES [1] C. Andrieu and A. Douce, Paricle filering for parially observed Gaussian sae space models, Journal of he Royal Saisical Sociey: Series B (Saisical Mehodology), vol. 64, pp , [2] M. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, A uorial on paricle filers for online nonlinear/non-gaussian Bayesian racking, IEEE Transacions on Signal Processing, vol. 50, pp , [3] N. Bergman, Recursive Bayesian esimaion: Navigaion and racking applicaions, PhD hesis No 579, Linköping Sudies in Science and Technology, SE Linköping, Sweden, May [4] M. Bolić, P. Djurić, and S. Hong, Resampling algorihms for paricle filers: A compuaional compleiy perspecive, EURASIP Journal on Applied Signal Processing, vol. 15, pp , [5], Resampling algorihms and archiecures for disribued paricle filers, IEEE Transacions on Signal Processing, vol. 53, no. 7, pp , [6] R. S. Bucy and K. D. Senne, Digial synhesis on nonlinear filers, Auomaica, vol. 7, pp , [7] J. Carpener, P. Clifford, and P. Fearnhead, Improved paricle filer for nonlinear problems, IEE Proceedings - Radar, Sonar and Navigaion, vol. 146, no. 1, pp. 2 7, [8] G. Casella and C. P. Rober, Rao-Blackwellisaion of sampling schemes, Biomerika, vol. 83, pp , 1996.

13 13 [9] R. Chen and J. Liu, Miure Kalman filers, Journal of he Royal Saisical Sociey: Series B (Saisical Mehodology), vol. 62, pp , [10] T. Chen, T. B. Schön, H. Ohlsson, and L. Ljung, An eension of he dencenralized paricle filer wih arbirary sae pariioning, Auomaic Conrol Division, Linköping Universiy, Tech. Rep. No. 2930, [11] A. Douce, N. de Freias, K. Murphy, and S. Russell, Rao-Blackwellised paricle filering for dynamic Bayesian neworks, in Proceedings of he 16h Conference on Uncerainy in Arificial Inelligence, 2000, pp [12] A. Douce, N. D. Freias, and N. Gordonn, Sequenial Mone Carlo mehods in pracice. New york: Springer, [13] A. Douce, S. Godsill, and C. Andrieu, On sequenial Mone Carlo sampling mehods for Bayesian filering, Saisics and Compuing, vol. 10, pp , [14] A. Douce and A. M. Johansen, A uorial on paricle filering and smoohing: Fifeen years laer, in Handbook of Nonlinear Filering, D. Crisan and B. Rozovsky, Eds. Oford, UK: Oford Universiy Press, [15] N. Gordon, D. Salmond, and A. Smih, Novel approach o nonlinear/non-gaussian Bayesian sae esimaion, Radar and Signal Processing, IEE Proceedings F, vol. 140, no. 2, pp , [16] F. Gusafsson, F. Gunnarsson, N. Bergman, U. Forssell, J. Jansson, R. Karlsson, and P.-J. Nordlund, Paricle filers for posiioning, navigaion, and racking, IEEE Transacions on Signal Processing, vol. 50, no. 2, pp , [17] J. D. Hol, T. B. Schön, and F. Gusafsson, On resampling algorihms for paricle filers, in IEEE Nonlinear Saisical Signal Processing Workshop, 2006, pp [18] X. Hu, T. B. Schön, and L. Ljung, A basic convergence resul for paricle filering, IEEE Transacions on Signal Processing, vol. 56, no. 4, pp , [19] R. Kalman, A new approach o linear filering and predicion problems, Journal of Basic Engineering, vol. 82, no. 1, pp , [20] G. Kiagawa, Mone Carlo filer and smooher for non-gaussian nonlinear sae space models, J. Compu. Graph. Sais., vol. 5, no. 1, pp. 1 25, [21] M. Klass, N. de Freias, and A. Douce, Towards pracical N 2 Mone Carlo: The marginal paricle filer, in Uncerainy in Arificial Inelligence, [22] J. Míguez, Analysis of parallelizable resampling algorihms for paricle filering, Signal Processing, vol. 87, pp , [23] M. K. Pi and N. Shephard, Filering via simulaion: Auiliary paricle filers, Journal of he American Saisical Associaion, vol. 94, no. 446, pp , [24] L. R. Rabiner and B. H. Juang, An inroducion o hidden Markov models, IEEE Acous., Speech, Signal Processing Mag., pp. 4 16, [25] T. B. Schön, F. Gusafsson, and P.-J. Nordlund, Marginalized paricle filers for mied linear/nonlinear sae-space models, IEEE Transacions on Signal Processing, vol. 53, pp , Tianshi Chen received he M.S. degree in 2005 from Harbin Insiue of Technology, and he Ph.D degree in December 2008 from Chinese Universiy of Hong Kong. Since April 2009, he has been a posdocoral researcher a he Division of Auomaic Conrol, Deparmen of Elecrical Engineering, Linköping Universiy, Linköping, Sweden. His research ineress include nonlinear conrol heory and applicaions, sysem idenificaion and saisical signal processing. Thomas B. Schön was born in Sweden in He received he M.Sc. degree in Applied Physics and Elecrical Engineering in Sep. 2001, he B.Sc. degree in Business Adminisraion and Economics in Feb and he Ph.D. degree in Auomaic Conrol in Feb. 2006, all from Linköping Universiy, Linköping, Sweden. He has held visiing posiions a he Universiy of Cambridge (UK) and he Universiy of Newcasle (Ausralia). His research ineress are mainly wihin he areas of signal processing, sysem idenificaion and machine learning, wih applicaions o he auomoive and he aerospace indusry. He is currenly an Associae Professor a Linköping Universiy. Henrik Ohlsson was born in Sweden in He received he M.Sc. degree in Applied Physics and Elecrical Engineering in Oc and his Liceniae degree in Auomaic Conrol in Dec. 2008, all from Linköping Universiy, Sweden. He has held visiing posiions a he Universiy of Cambridge (UK) and he Universiy of Massachuses (USA). His research ineress are mainly wihin he areas of sysem idenificaion and machine learning. He is currenly a Ph.D. suden a Linköping Universiy. ACKNOWLEDGMENT The auhors would like o hank he reviewers for heir useful commens and consrucive suggesions, and he associae edior for his ime handling he paper. The firs auhor would also like o hank Umu Orguner for his coninuous suppor and helpful discussions. This work was suppored by he Sraegic Research Cener MOVIII, funded by he Swedish Foundaion for Sraegic Research, SSF, and CADICS, a Linnaeus cener funded by he Swedish Research Council. Lennar Ljung received his PhD in Auomaic Conrol from Lund Insiue of Technology in Since 1976 he is Professor of he chair of Auomaic Conrol in Linköping, Sweden, and is currenly Direcor of he Sraeic Research Cener Modeling, Visualizaion and Informaion Inegraion (MOVIII). He has held visiing posiions a Sanford and MIT and has wrien several books on Sysem Idenificaion and Esimaion. He is an IEEE Fellow, an IFAC Fellow and an IFAC Advisor as well as a member of he Royal Swedish Academy of Sciences (KVA), a member of he Royal Swedish Academy of Engineering Sciences (IVA), an Honorary Member of he Hungarian Academy of Engineering and a Foreign Associae of he US Naional Academy of Engineering (NAE). He has received honorary docoraes from he Balic Sae Technical Universiy in S Peersburg, from Uppsala Universiy, Sweden, from he Technical Universiy of Troyes, France, from he Caholic Universiy of Leuven, Belgium and from Helsinki Universiy of Technology, Finland. In 2002 he received he Quazza Medal from IFAC, in 2003 he recieved he Hendrik W. Bode Lecure Prize from he IEEE Conrol Sysems Sociey, and he was he recepien of he IEEE Conrol Sysems Award for 2007.