Adaptive Linear Filtering Using Interior Point. Optimization Techniques. Lecturer: Tom Luo

Size: px

Start display at page:

Download "Adaptive Linear Filtering Using Interior Point. Optimization Techniques. Lecturer: Tom Luo"

Ann Underwood
5 years ago
Views:

1 Adaptive Linear Filtering Using Interior Point Optimization Techniques Lecturer: Tom Luo

2 Overview A. Interior Point Least Squares (IPLS) Filtering Introduction to IPLS Recursive update of IPLS Convergence/transient analysis of IPLS B. Applications System identification Beamforming Channel equalization in a CDMA forward link 1

3 Interior Point Optimization for Optimal Linear Filtering A discrete-time linear system can be described by yi = x T i w + vi, i = 1, 2,... Using input output pairs {xi, yi} the linear least-squares problem is then to estimate a filter w that minimizes the mean-squared error Fn(w) = 1 n n ( ) 2 yi x T i w = 1 n yt n y n 2w T n p xy(n) + w T Rxx(n)w, (1) i=1 where yn = [y1, y2,..., yn] T, pxy(n) = 1 n n i=1 x iyi, Rxx(n) = 1 n n i=1 x ix T i. Note: pxy(n) and Rxx(n) are both recursively updatable with per-sample complexity of O(M 2 ). The optimum linear filter then satisfies Fn(w) = 0, or Rxx(n)w pxy(n) = 0. 2

4 One Motivation: Transient Convergence The RLS algorithm estimates (see e.g., Sayed and Kailath 96) w rls n := [ δ n I + R xx(n)] 1 pxy(n), where δ n I is a regularization term to improve conditioning. Problem: The regularization depends entirely on the constant δ. If, for example, SNR is underestimated = slower asymptotic convergence SNR is overestimated = bad transient behaviour Remedy: wn := [αni + Rxx(n)] 1 pxy(n), αn adjusted adaptively 3

5 The Analytic Center Approach Formulate a convex feasibility problem at each iteration. w is a feasible filter only if it is contained in Ωn = {w R M Fn(w) τn, w 2 R 2 }, (2) 1 st constraint: minimize the mean-squared error Fn(w). 2 nd constraint: make Ωn a bounded region. The analytic center w a n of Ω n is the minimizer of φn(w) = log(τn Fn(w)) log(r 2 w 2 ) which can be found by solving φn(w) = 0 ( Fn sn(w n a ) + 2wa n s tn(w n a ) = 0, therefore wa n = n(w a n ) ) 1 tn(w n a )I + R xx(n) pxy(n). where sn(w) := τn Fn(w) and tn(w) := R 2 w a n 2. 4

6 Definition of τn We are concerned with the behaviour of αn = sn/tn. 1. The goal is to make αn Fn(w). Fn(w) = 2(Rxx(n)w pxy(n)) Thus, if Fn(w) is large = need αn large for regularization. Fn(w) is small = need only a small αn. 2. Define sn := β R 2 Fn(w a n 1 ) R/ 2: a normalization constant required to show asymptotic convergence β: also required to show convergence 3. The definition of τn follows simply from τn = Fn(w a n 1 ) + s n 5

7 6 w(1) w(1) w(1) w(2) 0 w(2) 0 w(2) w(1) w(1) w(1) w(2) 0 w(2) 0 w(2) Stanford University EE392o Z.Q. Luo An Example

8 Asymptotic Convergence Analysis Condition 1. (Bounded Autocorrelation matrix) There exist n0 > 0, σ1 > 0, σ2 > 0 such that σ1i 1 n xix T i σ2i, n n0. n i=1 Condition 2. (Bounded Outputs) There exists a fixed ρy such that for all n > n0 there holds 1 n y 2 i n ρ y. i=1 The left inequality in Condition 1 is known as weak persistent excitation condition. Theorem 1. Let the sequence of estimates {wn, n = 1, 2, 3,...} be generated by the IPLS algorithm. Then Fn(wn) = 2 Rxx(n)wn pxy(n) = O (1/n). 7

9 Transient Convergence Analysis Condition 1. (Bounded Autocorrelation matrix) There exist n0 > 0, σ1 > 0, σ2 > 0 such that σ1i 1 n xix T i σ2i, n n0. n i=1 Condition 2. (No Noise) The system is free from measurement noise, i.e., yi = x T i w, i = 1, 2,.... We assume that the data has no statistical fluctuations. Convergence then implies the phasing out of effects of initialization and thus is dictated entirely by the transient behaviour of the algorithm. Theorem 2. Let the sequence of estimates {wn, n M} be generated by the IPLS algorithm. If the observations are free of noise, then wn w = O(R 1 ) wn 1 w. 8

10 Transient Analysis a simple example Example. yi = w xi + vi, with xi = w = 1, i = 1, 2,... RLS Assuming no statistical averaging (e.g., no noise), Rxx(n) = E(xx T ) = 1, pxy(n) = E(xy) = 1, n The RLS estimator then reduces to w rls n = (1 + δ/n) 1. IPLS Now, Fn(w) = (w 1) 2, and Fn(w) = 2(w 1). Evaluating τn, the condition φn(w a n ) = 0 for the analytic center becomes, 2(w a n 1) a βr 2 w n 1 a 1 + (wa n 1 1)2 (w n a + 2w n 1)2 R 2 (w = 0 n a )2 which implies w a n 1 = O(R 1 ) w a n 1 1, i.e., exponential decay of the transient error. 9

11 Direct Comparison of RLS, IPLS Property RLS IPLS Asymptotic Convergence O(1/n) O(1/n) Computational Complexity O(M 2 ) O(M 2.2 ) Transient Convergence O(1/n) O(R n ) Robustness to Initialization no yes Additional constraints req. new algorithm easily accommodated Numerical Stability (in constrained case) problems occur when λ is small (λ : forgetting factor) stable even at small values of λ, and in cases of limited precision calculations Sliding window implementation can be accommodated easily accommodated 10

12 The Interior Point Least Squares (IPLS) algorithm 1. We don t need the exact analytic center of Ωn, an approximate center is sufficient. 2. Such an approximate center is found by taking just a single Newton iteration in the minimization of φn(w). wn := wn 1 ( 2 φn(wn 1)) 1 φn(wn 1), (3) To compute (3) we need φn(w) = F n sn(w) + 2w tn(w), (4) 2 φn(w) = ( F n) T Fn s 2 n (w) + 2 Fn sn(w) + 4wwT 2I + t 2 n (w) tn(w) (5) where Fn = 2pxy(n) + 2Rxx(n)w and 2 Fn = 2Rxx(n). 3. To compute the Newton direction, an O(M 2.2 ) recursive update procedure has been devised (using the work of Powell, 1997). 11

13 Interior Point Least Squares (IPLS) Algorithm Step 1: Initialization. Let β, R be given. Set w0 = 0, pxy(0) = 0, Rxx(0) = 0, F0(0) = 0. Step 2: Updating. For n 1, acquire new data xn, yn. Then recursively update pxy(n) = n 1 n p xy(n 1) + 1 n x nyn, Rxx(n) = n 1 n R xx(n 1) + 1 n x nx T n. Update 2 Fn(wn 1) and Fn(wn 1) sn(wn 1) and tn(wn 1) ( 2 φn(wn 1)) 1 φn(wn 1) using the update procedure (or using (4), (5)) Step 3: Recentering. The new center of Ωn is obtained by taking just one Newton iteration starting at wn 1: wn := wn 1 ( 2 φn(wn 1)) 1 φn(wn 1) Set n := n + 1, and return to Step 2. 12

14 Summary Contributions provided a new look at (recursive) adaptive filtering first application of interior point optimization to a dynamic problem Features of IPLS converges asymptotically at the rate O(1/n) exhibits fast transient convergence, and is robust to initialization easily accommodates additional linear or convex quadratic constraints, and is numerically stable O(M 2.2 ) complexity 13

15 Application: System Identification Performance Measure εip(n) = wn w 2 and εrls(n) = w rls n w 2 Sources (i) White Gaussian noise, (ii) White Gaussian noise filtered through H(z) = 1 + 2z 1 + 3z 2 ( z z 2 )( z 1 ). SNRs (i) SNR1 = 40dB, (ii) SNR2 = 10dB Nominal Parameter Settings RLS λ = 1, δ = 10 4 IPLS β = 2, R = 1000 Experiment 1 w R 20, w(i) [ 1, +1], 500 independent Monte Carlo trials Experiment 2 Comparing sliding window versions of RLS (Liu & He 95) and IPLS: w R 10, Tl = 15 14

16 System Identification: Experiment 1 IPAF RLS IPAF RLS Iteration k Iteration k (a) SNR = 40dB, Source: White Gaussian Noise (b) SNR = 40dB, Source: Correlated IPAF RLS IPAF RLS Iteration k Iteration k (c) SNR = 10dB, Source: White Gaussian Noise (d) SNR = 10dB, Source: Correlated 15

17 System Identification: Experiment SW RLS SW IPAF Iteration k Figure 1: Comparison of sliding-window versions of IPLS and RLS when channel characteristics change abruptly (at iteration 100). 16

18 Application: Minimum Variance Beamforming Sensor Array Adjustable Weights Target Signal Interferer Σ Output y(n) Adaptive Filtering Algorithm Steering Vectors 17

19 Minimum Variance Beamforming By adaptively adjusting the tap weights hi(n) the beamformer must 1. Steering Capability: protect the target signal c H (θ)h(n) = 1, n, θ = θ1, θ2,... c H (θ) = [1, e jθ,..., e j(m 1)θ ], where M : number of tap weights hi, θi : Electrical Angle determined by the direction of the target i with respect to the first sensor 2. Minimize the effects of the interferers i.e., minimize the Output Power E( y 2 ) of the beamformer This beamforming problem can be cast in the framework of a Constrained Adaptive Estimation Problem. 18

20 Beamforming: Constrained Adaptive Estimation minimize Fn := 1 n n λ n i d(i) x T i h 2, i=1 subject to C T h = f, h R M, (6) λ: forgetting factor d( ): desired response x T i : vector input sequence h: vector of tap weights C, f: define the linear constraints on h In the Minimum Variance Beamforming problem the reference signal d( ) is zero. During the adaption process we assume that no target is present. The rows of C correspond to steering vector constraints. 19

21 Beamforming: Numerical Simulation Input: (interference at 0.3, and 0.7) x(n) = sin(0.3nπ) + sin(0.325nπ) + sin(0.7nπ) + b(n). b(n): white Gaussian noise at 40dB. Constraints: (desired response at freq. 0.2 and 0.5) C T = 1 cos(0.2π)... cos((m 1)0.2π) 1 cos(0.5π)... cos((m 1)0.5π) 0 sin(0.2π)... sin((m 1)0.2π) 0 sin(0.5π)... sin((m 1)0.5π) 1, f =

22 21 LCMV µ = 0.1 LCFLS λ = 0.99, Eo = 0.1 IPLS λ = 0.99, ε = 0.01, R = 100, β = 2 Precision 4 digits for LCFLS and IPLS Figure 2: (a) Freq. response at Iteration 4000, (b) Mean-squared error in h(n) Normalized frequency Time (Iteration) LCMV LC FLS IPM1 Optimum Filter 10 1 abs H(f) [db] Norm[ h opt h(n) ] LCMV LC FLS IPM Stanford University EE392o Z.Q. Luo Beamforming: Numerical Simulation

23 Application: Channel Equalization in a CDMA Downlink u (k) 1 Code Filters L C (z) 1 u (k) 2 L C (z) 2 Σ Channel noise H (z) Σ chip rate sampler Received Signal y(k) u (k) K L C (z) K Figure 3: Discrete-time model of CDMA downlink 22

24 Equalizer/Decoder Structure code matched filter Received signal y(k) FFF -1 C 1 (z ) r(k) x(k) L Σ ^ u 1 (k) x(k) FBF Figure 4: Code Matched Filter Chip rate DFE 23

25 24 Static Channel Fading Channel sampled at a random instant. Symbol Index LOS Path First multi path Second Multi Path Individual path amplitudes Fading Channel LOS component is 5 db higher than 2 multipath components, fading rate fd = 0.005, delay spread: 6Tc Sources QPSK with uniform probabilities for each symbol (i.i.d.) Stanford University EE392o Z.Q. Luo CDMA Downlink: System Description

26 25 Message Signal N = 200, NT = 10, CL = 16, Users = 1 Algorithms λ = 1.0, δ = R = 10 4, β = 2 Equalizer Mff = 14, Mfb = 2, delay = 1, pfr = SNR SNR SQRT RLS IPLS BER Prob(packet arrival) SQRT RLS IPLS Stanford University EE392o Z.Q. Luo Experiment 1: Static Channel, Single user

27 26 Message Signal N = 200, NT = 10, CL = 16, Users = 4 Algorithms λ = 1.0, δ = R = 10 4, β = 2 Equalizer Mff = 14, Mfb = 2, delay = 1, pfr = SNR SNR SQRT RLS IPLS BER 10 2 Prob(packet arrival) SQRT RLS IPLS Stanford University EE392o Z.Q. Luo Experiment 1: Static Channel, 4 Users

28 27 Message Signal N = 200, SNR = 12dB, CL = 16, Users = 1 Algorithms λ = 1.0, δ = R = 10 4, β = 2 Equalizer Mff = 14, Mfb = 2, delay = 1, pfr = N T N T 0.1 SQRT RLS IPLS BER Prob(packet arrival) SQRT RLS IPLS Stanford University EE392o Z.Q. Luo Experiment 1: Dependence on Training Length

29 28 Message Signal N = 200, NT = 2/10, CL = 16, Users = 1 Algorithms λ = 0.85, δ = R = 10 4, β = 2 Equalizer Mff = 14, Mfb = 2, delay = 1, pfr = 10 2 SNR SNR SQRT RLS, N =2 T SQRT RLS, N T =10 IPLS, N =2 T IPLS, N =10 T BER 10 2 Prob(packet arrival) 0.5 SQRT RLS, N T =2 SQRT RLS, N T =10 IPLS, N T =2 IPLS, N T = Stanford University EE392o Z.Q. Luo Experiment 2: Time-Varying Channel, Single user

30 29 Message Signal N = 200, NT = 2/10, CL = 16, Users = 4 Algorithms λ = 0.85, δ = R = 10 4, β = 2 Equalizer Mff = 14, Mfb = 2, delay = 1, pfr = 10 2 SNR SNR SQRT RLS, N T =10 IPLS, N =10 T BER Prob(packet arrival) SQRT RLS, N =10 T IPLS, N =10 T Stanford University EE392o Z.Q. Luo Experiment 2: Time-Varying Channel, 4 users

31 30 Message Signal N = 200, SNR = 16dB, CL = 16, Users = 1 Algorithms λ = 0.85, δ = R = 10 4, β = 2 Equalizer Mff = 14, Mfb = 2, delay = 1, pfr = N T N T 0.2 SQRT RLS IPLS BER Prob(packet arrival) SQRT RLS IPLS Stanford University EE392o Z.Q. Luo Experiment 2: Dependence on Training Length

32 Conclusions Transient convergence of IPLS is O(1/R n ), n M Superior transient convergence to RLS even when n < M Gain of using IPLS over the RLS algorithm can range from 5-6 db to well over 10 db 31

Lecture 7: Wireless Channels and Diversity Advanced Digital Communications (EQ2410) 1

Lecture 7: Wireless Channels and Diversity Advanced Digital Communications (EQ2410) 1 Wireless : Wireless Advanced Digital Communications (EQ2410) 1 Thursday, Feb. 11, 2016 10:00-12:00, B24 1 Textbook: U. Madhow, Fundamentals of Digital Communications, 2008 1 / 15 Wireless Lecture 1-6 Equalization