Jianfeng Feng. DCSP-15: Wiener Filter and Kalman Filter* Department of Computer Science Warwick Univ.

Transcription

1 DCSP-15: Wiener Filter and Kalman Filter* Jianfeng Feng Department of Computer Science Warwick Univ., UK

2 Father of cybernetics Norbert Wiener: Harvard awarded Wiener a PhD, when he was 17.

3 Example The RGB format stores three color values, R, G and B, for each pixel. RGB = imread( bush.png'); size(rgb) ans = imshow(rgb)

4 Example

5 Setup Recorded signal x(n,m) = s(n,m) +x(n,m) True Signal noise for example, an image of 1500 X1200

6 Setup RGB = imread( bush.png'); I = rgb2gray(rgb); J = imnoise(i,'gaussian',0,0.0 05); figure, imshow(i), figure, imshow(j)

7 Setup To find a constant a(m,n) such that E ( a(m,n) x(m,n) s(m,n) ) 2 as small as possible y(m,n) = a(m,n) x(m,n) Different from the filter before, a(m,n) depends on (m,n) : Adaptive filter

8 Setup E( ax s) E( a x 2 axs s ) a Ex 2aExs Es The quanttity above is minimized if the derivative of it with respect to a is zero 2aEx 2-2Exs = 0 a = Exs E(s +x)s = = Es 2 = s 2 - Ex 2 Ex 2 Es 2 + Ex 2 Es 2 + Ex 2 s 2 (optimal Wiener gain)

9 Setup Without loss of generality, we assume that Es=0 Ex=0 for simplicity of formulation. S, x are independent

10 Algorithm The filtered output is given by y(n 1,n 2 ) = m(n 1,n 2 ) + s 2 (n 1,n 2 ) - Ex 2 (x(n s 2 (n 1,n 2 ) 1,n 2 ) - m(n 1,n 2 )) Note that the coefficient a depends on the position s is the local variance, Ex 2 is the global variance (noise)

11 Algorithm 1 NM s 1 n, n x( n, n ) x 1 2 ( n, n ) NM n, n where summation is over an area of N and M 3X 3 area (N=3, M=3)

12 Algorithm in Matlab wiener2 lowpass - filters an intensity image that has been degraded by constant power additive noise. wiener2 uses a pixelwise adaptive Wiener method based on statistics estimated from a local neighborhood of each pixel.

13 Example RGB = imread( bush.png'); I = rgb2gray(rgb); J = imnoise(i,'gaussian',0, 0.005); K = wiener2(j,[5 5]); figure, imshow(j), figure, imshow(k)

14 Further issues Large M and N reduce noise, but blur the photo small M and N will not affect noise optimal size should be used N=M=50 N=M=2

15 Further issues A more general version of Wiener: Kalman filter Kaman filter is widely used in many areas (still a very hot research topic) As an introductory course, we only give you a taste of its flavour

16 Kalman Filter

17 Kalman Filter: find out x k Have two variables state variable x which is not observable x k = F k x k-1 + B k u k + w k B k Input u k X k-1 Output x k Observable variable z which we can observe, but is contaminated by noise z k = H k x k +v k where w k ~ N(0,Q k ) and v k ~ N(0,R k ) are noise, F k and H k are constants.

18 Kalman Filter Predict For given x k-1 k-1 and p K-1 K-1 Predicted (a priori) state estimate x k k-1 = F k-1 x k-1 k-1 +B k u k Predicted (a priori) estimate covariance p k k-1 = F k-1 p k-1 k-1 F K-1 +Q K Update measurement residual y k = z k H k X k k-1 measurement residual covariance S k = H k P k k-1 H K +R k optimal Kalman gain K k = P k k-1 H k (S K ) -1 updated (a posteriori) state estimate X k k = x k k-1 + K k y k updated (a posteriori) estimate covariance p K K = (I-K k H k ) P k k-1

19 Kalman Filter clear all close all N=1000; q=1; r=1; f=0.1; h=0.2; x(1)=0.5; xsignal(1)=0.5; xhat=zeros(1,n+1); phat=zeros(1,n+1); xhat(1)=x(1); phat(1)=x(1); p(1)=1; z(1)=0.1; y(1)=1; for i=1:n xsignal(i+1)=f*xsignal(i)+randn(1,1)*sqrt(q)+1*cos(2* z(i+1)=h*xsignal(i)+randn(1,1)*sqrt(r); end for i=1:n x(i+1)=f*xhat(i)+1*cos(2*pi*0.01*i); p(i+1)=f*phat(i)*f+q; y(i+1)=z(i+1)-h*x(i+1); s(i+1)=h*p(i+1)*h+r; k(i+1)=p(i+1)*h/s(i+1); atemp=x(i+1)+k*y(i+1); xhat(1,i+1)=atemp(1,i+1); atemp=(1-k*h)*p(i+1); phat(i+1)=atemp(1,i+1); clear atemp end plot(xsignal); hold on plot(xhat,'r') plot(z,'g') plot(xhat,'r') It is one of the most useful filters in the literature, as shown below We do not have time to go further, but do read around

20 Applications Attitude and Heading Reference Systems Autopilot Battery state of charge (SoC) estimation [1][2] Brain-computer interface Chaotic signals Tracking and Vertex Fitting of charged particles in Particle Detectors [35] Tracking of objects in computer vision Dynamic positioning Economics, in particular macroeconomics, time series, and econometrics Inertial guidance system Orbit Determination Power system state estimation Radar tracker Satellite navigation systems Seismology [3] Sensorless control of AC motor variable-frequency drives Simultaneous localization and mapping Speech enhancement Weather forecasting Navigation system 3D modeling Structural health monitoring Human sensorimotor processing[36] Flight control unit of an Airbus A340 modern weather forecasting.

21 DCSP-Revision Jianfeng Feng Department of Computer Science Warwick Univ., UK

22 Fourier Thm. Any signal x(t) of period T can be represented as the sum of a set of cosinusoidal and sinusoidal waves of different frequencies and phases x(t) = A 0 + ì ï ï ï ï í ï ï ï ï îï å n=1 A n cos(nwt)+ A 0 =<1, x(t) >= 1 T T /2 ò -T /2 å n=1 B n sin(nwt) x(t)dt A n =< cos(nwt), x(t) >= 2 T B n =< sin(nwt), x(t) >= 2 T w = 2p T T /2 ò -T /2 T /2 ò -T /2 x(t) cos(nwt) dt x(t)sin(nwt) dt

23 Example I X(F) 2/p = Time domain Frequency domain

24 FT in complex exponential If the periodic signal is replace with an aperiodic signal (general case) FT is given by ì ïx(f) = FT(x) = í ïx(t) = FT -1 î (X) = ò - ò - x(t)exp(-2p jft)dt X(F)exp(2p jft)df Is that deadly simple?

25 Information Defined the information contained in an outcome x i in x={x 1, x 2,,x n } Entropy I(x i ) = - log 2 p(x i ) H(X)= S i P(x i ) I(x i ) = - S i P(x i ) log 2 P(x i )

26 Shannon's first theorem An instantaneous code can be found that encodes a source of entropy H(X) with an average number L s (average length) such that L s >= H(X)

27 Huffman coding code length L s = 0.729* *3* *5* * 5 = (remember entropy is 1.4)

28 Definition and properties: The DTFT gives the frequency representation of a discrete time sequence with infinite length. ì X(w) = DTFT(x) = å x[n]exp(- jwn) ï n=- í x[n] = IDTFT(X) = 1 p ï ò X(w)exp( jwn)dw î ï 2p -p X(w): frequency domain x [n]: time domain

29 DFT III Definition: The discrete Fourier transform (DFT) and its own inverse DFT (IDFT) associated with a vector X = ( X[0], X[1],, X[N-1] ) to a vector x = ( x[0], x[1],, x[n-1] ) of N data points, is given by ì N-1 æ X[k] = DFT{x[n]} = x[n]expç- j 2pk è N n ö ï å ø n=0 í x[n] = IDFT{X[k]} = 1 N-1 ï æ X[k]expç j 2pk N è N n ö ï å î ø k=0

30 Mysterious sound Fig. 2 clear all close all sampling_rate=100;%hz omega=30; %signal frequecy Hz N=20000; %total number of samples for i=1:n x_sound(i)=cos(2*pi*omega*i/sampling_rate); %sign x(i)=x_sound(i)+2*randn(1,1); %signal+noise axis(i)=sampling_rate*i/n; % for psd time(i)=i/sampling_rate; % for time trace end subplot(1,2,1) plot(time,x); %signal + noise, time trace xlabel('second') ylabel('x') subplot(1,2,2) plot(axis,abs(fft(x)),'r'); % magnitude of signal xlabel('hz'); ylabel('amplitude') sound(x_sound) Simple Matlab program is left on slides

31 Spectrogram t=0:0.001:2; % 2 1kHz sample rate y=chirp(t,100,1,200,'q'); % 100Hz, cross 200Hz at t=1sec spectrogram(y,128,120,128,1e3); % Display the spectrogram title('quadratic Chirp: start at 100Hz and cross 200Hz at t=1sec'); sound(y) y=chirp(t,100,1,200,'q'); % Fs=5000; filename = 'h.wav'; audiowrite(filename,y,fs); F r e q u e n c y Time (sec)

32 DFT for image DFT for x N 2-1 N X[k 1,k 2 ] = å 1-1 å x[n 1, n 2 ]exp(- 2p jk 1 n 1 ) exp(- 2p jk 2 n 2 ) n 2 =0 n 1 =0 N 1 N 2 IDFT for X x[n 1, n 2 ] = 1 N 2-1 N å 1-1 å X[k 1, k 2 ]exp( 2p jk 1 n 1 ) exp( 2p jk 2 n 2 ) N 1 N 2 k2 =0 k 1 =0 N 1 N 2

33 Example I clear all close all figure(1) I = imread('peppers.png'); imshow(i); K=rgb2gray(I); figure(2) F = fft2(k); figure; imagesc(100*log(1+abs(fftshift(f)))); colormap(gray); title('magnitude spectrum'); figure; imagesc(angle(f)); colormap(gray) Original picture Gray scale picture

34 Example I

35 simple filter design Notice two facts: The signal has a frequency spectrum within the interval 0 to F s /2 khz. The disturbance is at frequency F 0 (1.5) khz. Design and implement a filter that rejects the disturbance without affecting the signal excessively. We will follow three steps:

36 Step 1: Frequency domain specifications Reject the signal at the frequency of the disturbance. Ideally, we would like to have the following frequency response: where w 0 =2p (F 0 /F s ) = p/4 radians, the digital frequency of the disturbance.

37 Step 2: Determine poles and zeros We need to place two zeros on the unit circle z 1 =exp(jw 0 )=exp(jp/4) and z 2 =exp(-jw 0 )=exp(-jp/4) z 2 H(z) = k(z - z 1 )(z - z 2 ) = k[z 2 - (z 1 + z 2 )z + z 1 z 2 ] = k[z 2 - (1.414)z +1] p/4

38 Step 2: Determine poles and zeros We need to place two zeros on the unit circle z 1 =exp(jw 0 )=exp(jp/4) and z 2 =exp(-jw 0 )=exp(-jp/4) z 2 H(z) = k(z - z 1 )(z - z 2 ) = k[z 2 - (z 1 + z 2 )z + z 1 z 2 ] = k[z 2 - (1.414)z +1] If we choose, say K=1, the frequency response is shown in the figure.

39 PSD and Phase As we can see, it rejects the desired frequency As expected, but greatly distorts the signal

40 Result

41 Recursive Filter A better choice would be to select the poles close to the zeros, within the unit circle, for stability. For example, let the poles be p 1 = r exp(jw 0 ) and p 2 = r exp(-jw 0 ). With r =0.95, for example we obtain the transfer function H(z) = k (z - z 1)(z - z 2 ) (z - p 1 )(z - p 2 )

42 Recursive Filter H(z) = k (z - z 1)(z - z 2 ) (z - p 1 )(z - p 2 ) = k (z - p 1 +e)(z - z 2 ) (z - p 1 )(z - p 2 ) æ = kç1+ è e ö (z - z 2) (z - p 1 ) ø(z - p 2 ) e= z 1 - p 1 = (1-r) z 1 ~ 0. So we have H(w 0 ) = 0, but close to 1 everywhere else

43 Recursive Filter We choose the overall gain k so that H(z) z=1 =1. This yields the frequency response function as shown in the figures below. H(z) = k (z - z 1 )(z - z 2 ) (z - p 1 )(z - p 2 ) = z z +1 z z

44 Step 3: Determine the difference equation in the time domain From the transfer function, the difference equation is determined by inspection: y(n)= x(n) x(n-1) x(n-2) y(n-1) y(n-2)

45 Step 3: difference equation Easily implemented as a recursion in a high-level language. The final signal y(n) with the frequency spectrum shown in the following Fig., We notice the absence of the disturbance.

46 Step 3: Outcomes

47 Matched filter Define a i = s N-i (reversing the order) y(n) = a 0 x(n) + a 1 x(n-1) + + a N x(n-n) So when the signal arrives we have y(n) = S N x(n) + S N-1 x(n-1) + + S 1 x(n-n) = S N S N + S N-1 S N S 1 S 1

48 Wiener Filter The filtered output is given by y(n 1,n 2 ) = m(n 1,n 2 ) + s 2 (n 1,n 2 ) - Ex 2 (x(n s 2 (n 1,n 2 ) 1,n 2 ) - m(n 1,n 2 )) Note that the coefficient a depends on the position s is the local variance, Ex 2 is the global variance (noise)

49 past papers before the exam

50 Next week No lecture next Monday Assignment help Tuesday (same time and venue as lecture) Revision class (second week, next term)