F2E5216/TS1002 Adaptive Filtering and Change Detection. Likelihood Ratio based Change Detection Tests. Gaussian Case. Recursive Formulation

Size: px

Start display at page:

Download "F2E5216/TS1002 Adaptive Filtering and Change Detection. Likelihood Ratio based Change Detection Tests. Gaussian Case. Recursive Formulation"

Philippa Richard
5 years ago
Views:

1 Adapive Filering and Change Deecion Fredrik Gusafsson (LiTH and Bo Wahlberg (KTH Likelihood Raio based Change Deecion Tess Hypohesis es: H : no jump H 1 (k, ν : a jump of magniude ν a ime k. Lecure 8 Filer Banks for Sae Changes Explici modeling of addiive change: GLR and MLR Muliple models: pruning, merging and off-line algorihms Likelihood raio: In previous noaion, g (k = p(yk p(y k+1 p(y g (k is jus a normalized version of he likelihood. g (k is a disance measure beween H and H 1 (k. ν = θ 1 when θ =is assumed. Lecure 8, 25 1 Lecure 8, 25 2 or in he negaive logarihm log g (k }{{} ḡ (k Recursive Formulaion g (k = p(yk p(yk+1 p(y = g 1 (k p(y y 1 k+1 p(y y 1 = log g 1 (k +( log p(y }{{} y 1 k+1 +logp(y y 1 }{{} ḡ 1 (k s (k Fis he general sopping rule framework. Gaussian Case The jump ν can be ML esimaed (he generalized likelihood raio es or marginalized (he marginalized likelihood raio es g GLR (k = g MLR (k = ˆν 2 (k H R/( k ˆν 2 (k R/( k H 1 h log(2πr H H 1 The noise variance R is assumed known. Remark 1: I is he produc Rh ha deermines he performance of GLR. Remark 2: There is no hreshold o design in MLR (implicily given by R. Lecure 8, 25 3 Lecure 8, 25 4

2 Implemenaion Aspecs All <k<are involved in he es. Approximaion 1: Consider only change imes in a sliding window L<k<. Approximaion 2: Consider only one change ime k = L (Brand s GLR. Off-line algorihm: 1. Forward filer compues p(y k, k. 2. Backward filer compues p(yk+1 N, k. 3. MLR combines hese as p(yk p(yk+1 N p(y N. Daa Models Explici modeling of addiive pulse change (Ch. 9 and 11: x +1 = A x + B u, u + B v, v + δ k B θ ν y = C x + e + D u, u + δ k D θ, ν. Sep changes are modeled by changing noaion δ σ (sep funcion. Muliple models wih mode parameer δ, usually or 1 in Ch. 1, or Markov chain in jump Markov models x +1 = A (δx + B u, (δu + B v, (δv y = C (δx + D u, (δu + e v N(m v, (δ,q (δ e N(m e, (δ,r (δ. Lecure 8, 25 5 Lecure 8, 25 6 A Direc Approach Assume sep changes. Augmened sae space model ( ( ( x +1 = x +1 A B = θ, x + θ +1 I ( ( + B v, v I δ (k 1 ν y = x = P = ( C D θ, x + e + D u, u ( x ( P B u, u Adapive Filer or Whieness Tes Approach Disregards explici use of δ k changes. Parameer (change esimaor: ˆθ +1 = ˆθ 1 + K θ (y C ˆx 1 D θ,ˆθ 1 D u, u, K = ( K x K θ ( P xx, P = P θx Adapive filering wih sae noise covariance Q = ( Q Q θ Whieness based residual es, where Q θ when a change is deeced. P xθ P θθ. is momenarily increased Lecure 8, 25 7 Lecure 8, 25 8

3 Muliple-Model Approach Run N mached filers (sandard KF o each hypohesis H 1 (k. Compare likelihoods (or likelihood raios compued from ε (k and S (k. u y u y High gain (Q θ = δ k αi Filers No gain (Q θ = Filer ˆx (k,p (k ˆx,P Hyp. es ˆk Idea of GLR Kalman filer mached o H ˆx,K (gain, ε,s =Cov(ε Kalman filer mached o H 1 (k ˆx (k, ε (k, ϕ (k, μ (k Idenificaion under H 1 (k ε (k =ϕ T (kν(k+e Compensaion under H 1 (k ˆx (k ˆx + μ (kν(k. Noe: linear regression for change magniude! Need: one KF and RLS filers ˆν(k Firs: updae equaions for ε (k and μ (k. Lecure 8, 25 9 Lecure 8, 25 1 GLR Lemma Linear model influence of change linear posulae ˆx (k = ˆx + μ (kν ε (k = ε + ϕ T (kν. Lemma Updae recursion ϕ T +1(k = ( C +1 A i A μ (k i=k μ +1 (k = A μ (k+k +1 ϕ T +1(k, iniialized by μ k (k =and ϕ k (k =. GLR Algorihm Main filer: Kalman filer assuming no jump. Filer bank: Regressors ϕ (k and he LS quaniies R (k = ϕ i(ks 1 i ϕ T i (k and f (k = ϕ i(ks 1 i ε i for each k, 1 k. GLR Tes: A ime = N, he es saisic is given by l N (k, ˆν(k = f T N (kr 1 N (kf N(k. A jump candidae is given by ˆk =argmaxl N (k, ˆν(k. I is acceped if l N (ˆk, ˆν(ˆk >h Idenificaion: ˆν N (ˆk =R 1 N (ˆkf N (ˆk. Lecure 8, Lecure 8, 25 12

4 Commens on GLR The sysem is (ofen no persisenly excied. Tha is, ϕ decays o zero. Inuiively, his means ha he KF compensaes iself, making idenificaion of ν unnecessary afer a while. Tes saisic χ 2 disribued. Regressors pre-compuable, decay raher fas o zero for many sysems and depend only on k for ime-invarian sysems. Efficien implemenaions migh exis. RLS beer o use marix inversion of R N (k no needed: l (k, ˆν(k = f T (kˆν (k, MLR versus GLR In GLR, he hreshold is sensiive o incorrecly specified noise scalings (which does no affec he KF. R = λr, P = λp, Q = λq l N (k =l N (k/λ H H 1 h. In MLR, here is no hreshold. The noise scaling can be incorporaed as a nuissance parameer. Complexiy. GLR requires N 2 filer updaes. Sliding window approximaion requires NL filer updaes. Two-filer MLR requires 2N filer updaes. Lecure 8, Lecure 8, Muliple Models x +1 = A (δx + B u, (δu + B v, (δv y = C (δx + D u, (δu + e v N(m v, (δ,q (δ e N(m e, (δ,r (δ. Discree parameer δ is he mode, or discree sae, of he sysem. 1. δ has S possible oucomes. Mosly S =2considered. 2. δ has S Markov saes wih ransiion marix Π (jump Markov models, hidden Markov model (HMM. Difficul on-line. EM-algorihm or Baum-Welch mehod off-line. Example 1: Q(δ =(1+9δQ can be used o model addiive sae changes implicily. Example 2: A(δ can be used o model differen urn raes in arge racking, wih (x 1, ẋ 1,x 2, ẋ 2 T as saes. Formulaion incorporaes: change deecion, segmenaion, model srucure selecion, equalizaion, blind equalizaion, ouliers and missing daa! Lecure 8, Lecure 8, 25 16

5 Basic Sraegy 1. Condiional Kalman filer given he mode sequence gives ˆx (δ, P (δ. 2. Compue he poserior probabiliy of he mode sequence p(δ y. 3. There are S differen sequences δ, labelled δ (i, i =1, 2,..., S. Theorem of oal probabiliy gives he Gaussian mixure: p(x y 1 S = S p(δ (i y N (ˆx (δ (i,p (δ (i. p(δ (i y Approximaions p(x y 1 S = S p(δ (i y N (ˆx (δ (i,p (δ (i. p(δ (i y 4. On-line: Merging (imm Add overlapping disribuions N (ˆx (δ (i,p (δ (i Pruning sequences (deecm Remove componens wih small coefficiens p(δ (i y 5. Off-line: numerical approaches based on he EM algorihm and MCMC mehods (mcmc, gibbs. Lecure 8, Lecure 8, Pruning versus Merging i =1 i =2 i =3 i =4 i =5 i =6 i =7 i =8 Pruning: cu off branches. Merging: represen several branches by one. A Merging Formula The bes approximaion of a sum of L Gaussian disribuions p(x = α(in(ˆx j,p j α N(ˆx, P, where α = α(i, P = 1 α ˆx = 1 α α(iˆx(i α(i ( P (i+(ˆx(i ˆx(ˆx(i ˆx T Second erm: spread of he mean. Lecure 8, Lecure 8, 25 2

6 Generalized Pseudo Bayesian GPB Merging Sraegy i =1 GPB(n: n is he size of sliding i =2 1 memory (n =sandard i =3 1 GPB(: merge all sequences i =4 ( GPB(1: merge sequences i =5 1 i =6 (1,3,5,7 and (2,4,6,8. 1 GPB(2: merge sequences (1,5, 1 i =7 (2,6, (3,7 and (4,8. 1 i =8 IMM Ineracing Muliple Model (IMM by Bar-Shalom and Li. Essenially as GPB, bu merging afer ime updae, raher han afer measuremen updae. Lecure 8, Lecure 8, Summary: Sae Deecion Exercises: 41, 42 (should be (8.1 in 2-ediion, 43. Abrup sae changes can be deeced and isolaed wih eiher: Likelihood raio (MLR, GLR hypohesis es, using he saisical approach. Muliple models (IMM,GPB Nex Time Pariy space change deecion (deerminisic approach Lecure 8, Lecure 8, 25 24

Two Popular Bayesian Estimators: Particle and Kalman Filters. McGill COMP 765 Sept 14 th, 2017

Two Popular Bayesian Estimators: Particle and Kalman Filters. McGill COMP 765 Sept 14 th, 2017 Two Popular Bayesian Esimaors: Paricle and Kalman Filers McGill COMP 765 Sep 14 h, 2017 1 1 1, dx x Bel x u x P x z P Recall: Bayes Filers,,,,,,, 1 1 1 1 u z u x P u z u x z P Bayes z = observaion u =