Growing Window Recursive Quadratic Optimization with Variable Regularization

Transcription

1 49th IEEE Conference on Decision and Control December 15-17, Hilton Atlanta Hotel, Atlanta, GA, USA Growing Window Recursive Quadratic Optimization with Variable Regularization Asad A. Ali 1, Jesse B. Hoagg 2, Magnus Mossberg 3 and Dennis S. Bernstein 4 Abstract We present a growing-window variableregularization recursive least squares GW-VR-RLS algorithm. Standard recursive least squares RLS uses a time-invariant regularization. More specifically, the inverse of the initial covariance matrix in classical RLS can be viewed as a regularization term, which weights the difference between the next state estimate and the initial state estimate. The present paper allows for time-varying in the weighting as well as what is being weighted. This extension can be used to modulate the speed of convergence of the estimates versus the magnitude of transient estimation errors. Furthermore, the regularization term can weight the difference between the next state estimate and a time-varying vector of parameters rather than the initial state estimate as is required in standard RLS. I. INTRODUCTION Recursive least squares RLS is widely used in signal processing, identification, estimation, and control [1], [2], [3], [4], [5], [6], [7], [8]. Under ideal conditions, that is, nonnoisy measurements and persistency of the data, RLS is guaranteed to converge to the minimizer of a quadratic function [5], [6]. In practice, the accuracy of the estimates and the rate of convergence depend on the level of noise and persistency of the data. The goal of the present paper is to extend standard RLS in two ways. First, in standard RLS, the positive-definite initialization of the covariance matrix serves as the weighting of a regularization term within the context of a quadratic optimization. Until at leastnmeasurements are available, this regularization term compensates for the lac of persistency in order to obtain a unique minimizer. Traditionally, the regularization weighting is fixed for all steps of the recursion. In the present wor, we derive a growing-window variable-regularization RLS GW-VR-RLS algorithm, where the weighting of the regularization term changes at each step. As a special case, the regularization can be decreased in magnitude or ran as the ran of the covariance matrix increases, and can be removed entirely when no longer needed. This ability is not available in standard RLS where the regularization term is weighted by the inverse of the initial covariance at every step. A second extension presented in this paper also involves the regularization term. Specifically, the regularization term in standard RLS weights the difference between the next state estimate and the initial state. In the present paper, the regularization term weights the difference between the next state estimate and an arbitrarily chosen time-varying vector of parameters. As a special case, the time-varying vector can be the current state estimate, and thus the regularization term weights the difference between the next state estimate and the current state estimate. This formulation allows us to modulate the rate at which the current estimate changes from step to step. For these extensions, we derive GW-VR-RLS update equations. While standard RLS entails the update of the state estimate and the covariance matrix, GW-VR-RLS entails the update of an additional symmetric matrix of dimension n n to allow for the variable regularization. Thus, GW-VR- RLS entails some additional computational burden relative to classical RLS. II. PROBLEM FORMULATION AND GW-VR-RLS ALGORITHM For all i, let A i R n n, b i,α i R n, and R i R n n, where A i and R i are positive semidefinite, define A, b, and assume that, for all, A i + R and A i+r +1 are positive definite. Hence R and R 1 are positive definite. For, define the regularized quadratic cost J x x T A i x+b T i x +x α T R x α, 1 where x R n. The minimizer x of 1 is given by x 1 2 A i +R 1 b i 2R α Note that x α is the minimizer of J x. To rewrite 2 recursively, define P A i +R 1. 2, 3 1 Graduate student, Department of Aerospace Engineering, The University of Michigan, Ann Arbor, MI , asadali@umich.edu. 2 Assistant Professor, Department of Mechanical Engineering, The University of Kentucy, Lexington, KY 6-3, jhoagg@engr.uy.edu. 3 Associate Professor, Department of Physics and Electrical Engineering, Karlstad University, Sweden, Magnus.Mossberg@au.se 4 Professor, Department of Aerospace Engineering, The University of Michigan, Ann Arbor, MI , dsbaero@umich.edu. where the inverse exists by assumption, so that x 1 2 P b i 2R α //$26. IEEE 496

2 Using 4, it follows that +1 x P +1 b i 2R +1 α P +1 b i +b +1 2R +1 α P +1 2P 1 x +2R α +b +1 2R +1 α +1 P +1 P 1 +1 x +R +1 x R x +A +1 x +R α b +1 R +1 α +1 x P +1 A +1 x +R +1 R x +R α R +1 α b Next, it follows from 3 that +1 A i +A +1 +R +1 P 1 Consider the decomposition P 1 +R +1 R +A +1. A +1 ψ +1 ψ T +1, 6 where ψ +1 R n n +1 and n +1 rana+1. Consequently, where P +1 P 1 +R +1 R +ψ +1 ψ T +1 1 Q ψ +1ψ T +1 1, 7 Q +1 A i +R +1 1 P 1 +R +1 R 1, 8 where the inverse exists by assumption. Note that Next, using the matrix inversion lemma P +1 Q X +UCV 1 X 1 X 1 U C 1 +VX 1 U 1 VX 1 with X Q 1 +1, U ψ +1, C I, and V ψ+1 T, it follows from 7 that P +1 Q +1 I n ψ +1 I n+1 +ψ+1q T +1 ψ +1 1 ψ+1 T Q Next, consider the decomposition R +1 R φ +1 S +1 φ T +1, 12 where φ +1 R n m +1, m +1 ranr+1 R, and S +1 R m +1 m +1 is a matrix of the form ±1 S +1 ± ±1 497 Therefore, 8 can be expressed as Q +1 P 1 +φ +1 S +1 φ T Letting X P 1, U φ +1, C S +1, and V φ T +1 it follows from and 14 that Q +1 P P φ +1 S +1 +φ T +1 P φ +1 1 φ T +1 P P I n φ +1 S +1 +φ T +1P φ +1 1 φ T +1P. 15 Therefore, for, the recursive regularized quadratic cost minimizer of 1 is given by 5, 11, and 15, that is, Q +1 P I n φ +1 S +1 +φ T +1P φ +1 1 φ T +1P, 16 P +1 Q +1 I n ψ +1 I n+1 +ψ+1 T Q +1ψ +1 1 ψ T +1Q +1, 17 x +1 x P +1 A +1 x +R +1 R x +R α R +1 α b +1, 18 where x α, P R 1, ψ +1 is given by 6, and φ +1 is given by 12. A. Standard RLS III. SPECIALIZATIONS Consider the special case R R and α x. Then the quadratic cost J x x T A i x+b T i x +x x T R x x 19 is minimized by x +1 x P +1 A +1 x b +1, P +1 P I n ψ +1 I n+1 +ψ+1p T ψ +1 1 ψ+1p T, 21 where P R 1. Since the recursive update for Q +1 given by 16 simplifies to Q +1 P +1, standard RLS does not require the update of Q +1. B. Standard RLS with α x 1 and R R Consider the special case R R and α x 1. Then the quadratic cost J x x T A i x+b T i x +x x 1 T R x x 1 is minimized by P +1 P I n+1 ψ +1 I n+1 +ψ+1p T ψ +1 1 ψ+1p T, x +1 x P +1 A +1 x +P 1 x 1 x b +1, where P R 1. Note that the update for P does not require Q.

3 C. Standard RLS with forgetting factor Let < λ 1, and consider the modified cost J x λ i x T Ā i x+ b T i x +x x T λ R x x, where for i, Ā i ψ i ψt i. Next, it follows that J x λ λ i x T Ā i x+ b T i x +x x T R x x λ x T A i x+b T i x +x x T R x x, where A i λ i Ā i, b i λ i bi, and R R. Therefore, J x λ J x, where J x is given by the traditional RLS quadratic cost 19. Minimizing J x is equivalent to minimizing J x. In this case, the minimizer of J is given by and 21; however, the minimizer x is expressed in terms of A +1 and b +1 rather than Ā+1 and b +1. Substituting A +1 λ 1 Ā +1, b +1 λ 1 b +1, and ψ +1 λ +1/2 ψ +1 into and 21 yields P +1 P I n λ 1 ψ+1 I n+1 +λ 1 ψt +1 P ψ+1 1 ψt +1 P, x +1 x P +1 λ 1 Ā +1 x λ 1 b+1. Next, for i, define P i λ i P i, and it follows that the minimizer of J is given by P +1 λ 1 P I n ψ +1 λi n+1 + ψ T +1 P ψ+1 1 ψt +1 P, x +1 x P +1 Ā+1 x b +1, where P R 1. D. Standard RLS with α x 1 and forgetting factor Let < λ 1, and consider the modified cost J x λ i x T Ā i x+ b T i x +x x 1 T λ R x x 1, where for i, Ā i ψ i ψt i. Next, it follows that J x λ λ i x T Ā i x+ b T i x +x x 1 T R x x 1 λ x T A i x+b T i x +x x 1 T R x x 1, 22 where A i λ i Ā i, b i λ i bi, and R R. Combining the steps in Section III-C and Section III-D, it follows that the minimizer of J is given by 498 P +1 λ 1 P I n ψ +1 λi n+1 + ψ T +1 P ψ+1 1 ψt +1 P, x +1 x P +1 Ā+1 x +λ +1 P 1 x 1 x b +1, where P R 1. IV. SETUP FOR NUMERICAL SIMULATIONS For all, let x,opt R n, ψ R n, where its i th entry ψ,i is generated from a zero mean, unit variance Gaussian distribution. The entries of ψ are independent. Define β ψ T x,opt. Let l be the number of data points. Define σ ψ,i 1 l ψ 2 l,i 1, l σ β 1 l 1 l 1 β 2 l x T,opt x,opt. Next, for i 1,...,n, let N,i R, and M R be generated from zero-mean Gaussian distributions with variances σ 2 N,i and σ2 M, respectively, where σ N,i and σ M are determined from the signal-to-noise ratio SNR. More specifically, for i 1,...,n, σ ψ,i σ β SNR ψ,i, and SNR β, σ N,i σ M where, for i 1,...,n, σ N,i 1 K K 1 M2. 1 K K 1 N2,i and σ M Finally, for, define A ψ + N ψ + N T and b 2β + M ψ +N, where N is the noise in ψ and M is the noise in β. Define z 1 [ ] T, z 2 [ ] T. Unless otherwise specified, for all, x,opt z 1, α x, and x 7 1. Define the performance x,opt x ε. x,opt V. NUMERICAL SIMULATIONS WITH NOISELESS DATA We now investigate the effect of R, α and λ on GW- VR-RLS. Furthermore, in this section, A and b contain no noise, specifically, for all, N 7 1 and M. A. Effect of R We begin by testing the effect of R on convergence of ε when R is constant. In the following example, we test GW-VR-RLS for three values of R. Specifically, for all, R I 7 7, R.1I 7 7 or R.1I 7 7. In all three cases, for all, A and b are the same. For this example, Figure 1 shows that smaller values of R yield faster convergence of ε to zero. This effect occurs because decreasing R reduces the magnitude of the regularization

4 ε % R I 7 7 R.1I 7 7 R.1I Time step Fig. 1. Effect of R on convergence of x to x,opt. For this example, smaller values of R yield faster convergence of ε to zero. term in the cost function 1. Next, we let R be constant and positive definite until 1 A i has full ran, then we let R. { More specifically,.1i R 7 7, if ran 1 A i < n,, if ran 1 A 23 i n. For R given by 23, if there is no noise in the data, then x may converge to x,opt in finite time. In particular, if there exists a positive integer N such that N 1 A i has full ran, then, for all N, x x,opt. Figure 2 shows that ε converges to zero in finite time when R is given by 23. In this case for all 7, 1 A i has full ran. Thus, for all 8, x x,opt. ε % Time step Fig. 2. Effect of R on convergence of x to x,opt. In this example, 7 A i has full ran. Therefore, for 8, R and x x,opt. Next, we choose the smallest R such that A i is positive definite. More specifically, we conduct the singular value decomposition USU T 1 A i, where U R n n, S R n n and has the form [ ] S Γ m m m n m, n m m n m n m whereγ R m m contains themnon-zero singular values of 1 A i. Note that the singular value decomposition has the form USU T because 1 A i is symmetric [9, Corollary 5.4.5]. Next, define [ Ŝ ] m m m n m, n m m ǫi n m n m where ǫ. Finally, R, R UŜUT, if ran 1 A i < n,, if ran 1 A i n, 24 In the following example we compare GW-VR-RLS with R I 3 3 and R given by 24 with ǫ 1. In both cases, for all, A and b are the same. For this example, Figure 3 shows that setting R given by 24 with ǫ 1 yields faster convergence of ε to zero than setting R I 7 7. ε % Time step Fig. 3. Effect of R on convergence of x to x,opt. The solid line denotes ε with R given by 24 and the dashed line denotes ε with R I 7 7. For this example, setting R given by 24 with ǫ 1 yields faster convergence of ε to zero than setting R I 7 7. B. Effect of α Figure 4 compares GW-VR-RLS with α x 1 and α x, where, for all, R I 7 7. For this example, setting α x 1 yields faster convergence of ε to zero than setting α x. ε % α x 1 α x Time step Fig. 4. Effect of one step regularization on convergence of x to x,opt. For this example, setting α x 1 yields faster convergence of ε to zero than setting α x C. Effect of Forgetting Factor In this section, we examine standard RLS with forgetting factor as described in Section III-C. In the following example, we consider three values of λ, specifically λ1, λ.995 or λ.9. For all, R.1I 7 7 and { z1, x,opt z 2, > For this example, Figure 5 shows that, for, the forgetting factor has negligible impact on the behavior of ε. For >, smaller values of λ yield faster convergence of ε to zero. 499

5 ε % λ 1 λ.995 λ.9 Time step Fig. 5. Effect of forgetting factor on convergence of x to x,opt. For, the forgetting factor has negligible impact on the behavior of ε. For >, x,opt x,opt, and a smaller value of λ yields faster convergence of x to x,opt. VI. NUMERICAL SIMULATIONS WITH NOISY DATA We now investigate the effect of R, α, and λ on GW- VR-RLS when the data have noise. More specifically, for all, M and N,i are generated from zero mean Gaussian distributions with variances depending on SNR ψ,i and SNR β, respectively. Figure 6 shows the effect of noise on standard RLS for different SNR values. In this example, a smaller value of SNR yields a larger asymptotic value of ε. In the next example, we examine the convergence of ε for standard RLS when ψ and β have constant bias. We consider three cases of constant bias, specifically, for all, N and M.2, N and M or N 7 1 and M.2. For this example, Figure 7 shows that bias increases the asymptotic value of ε. Furthermore, bias in β yields a higher asymptotic value of ε than an equal percent of bias in ψ. ε % Time step SNR ψ,i SNR β SNR ψ,i SNR β SNR ψ,i SNR β 5 Fig. 6. Effect of noise on standard RLS. In this example, smaller values of SNR yield larger asymptotic values of ε. A. Effect of R In this section, we examine the effect of R where R is constant. In the following example, we test GW-VR-RLS for three different values of R. Specifically, for all, R I 7 7, R.1I 7 7 or R.1I 7 7. Furthermore, SNR ψ,i SNR β 5 and, for all, A and b are the same. For this example, Figure 8 shows that smaller values of R can result in larger pea values of ε. Recall that, Figure 1 showed that smaller values of R can yield faster convergence of ε to zero. However, if the data have noise, then Figure 8 shows that the transient response ε % N.2I 7 1, M.2 N.2I 7 1, M N 7 1, M.2 Time step Fig. 7. Effect of bias on standard RLS. For this example, bias increases the asymptotic value of ε. Furthermore, bias in β yields a higher asymptotic value of ε than an equal percent of bias in ψ. of ε can be worse for smaller values of R. As the SNR increases, Figure 8 converges to Figure 1. ε % R I 7 7 R.1I 7 7 R.1I 7 7 Time step Fig. 8. Effect of R on convergence of x to x,opt. For this example, smaller values of R can result in larger pea values of ε. B. Effect of α Figure 9 compares GW-VR-RLS with α x 1 and α x, where, SNR ψ,i SNR β 5 and for all R.1I 7 7. For this example, Figure 9 shows that the transient response of ε can be worse for α x 1 than it is for α x. Figure 4 showed that setting α x 1 can yield faster convergence of ε to zero than setting α x. However, if the data have noise, then Figure 9 shows that the transient response of ε can be worse with α x 1 than it is with α x. As the SNR increases, Figure 9 converges to Figure 4. Next, we compare GW-VR-RLS for different choices of α. More specifically, we let α L ν where L ν { x 1, < ν, x ν, > ν, where ν is a positive integer. In the following example, we test GW-VR-RLS for three different ν. Specifically, ν 1, ν 5, ν. In all cases, for all, A and b are the same, R I 7 7 and SNR β SNR ψ,i 5. For this example, Figure shows that larger values of ν can yield better transient performance of ε. Next, we let α W ρ where x, 1, W ρ i1 x i, 1 < ρ, 1 ρ ρ i1 x i, > ρ,

6 ε % α x 1 α x Time step Fig. 9. Effect of α on convergence x to x,opt. For this example, this figure shows that the transient response of ε can be worse for α x 1 than it is for α x. ε % α L ν, ν 1 α L ν, ν 5 α L ν, ν Time step Fig.. Convergence of x to x,opt. For this example, larger values of ν yield better transient performance of ε. where ρ is a positive integer. In the following example, we test GW-VR-RLS for three different values ofρ. Specifically, ρ 1, ρ 5, ρ. In all cases, for all, A and b are the same, R I 7 7 and SNR β SNR ψ,i 5. For this example, Figure 11 shows that larger values of ρ can yield better transient performance of ε than smaller values of ρ. ε % α W ρ, ρ 1 α W ρ, ρ 5 α W ρ, ρ Time step Fig. 11. Convergence of x to x,opt. In this example, larger values of ρ yield better transient performance of ε than smaller values of ρ. C. Effect of Forgetting Factor In this section, we examine standard RLS with forgetting factor. In the following example, we test RLS for three values of λ, specifically λ1, λ.95 or λ.9. Let SNR ψ,i SNR β 5, and, for all, R.1I 7 7. For this example, Figure 12 shows that smaller values of λ yield larger asymptotic value of ε. Next, we let x,opt { z1,, z 2, >, For this example, Figure 13 shows that, for, smaller values ofλyield larger asymptotic values ofε. For >, x,opt x,opt, and a smaller value of λ yields faster convergence of ε to its asymptotic value. ε % 1 1 λ 1 λ.95 λ.9 Time step Fig. 12. Effect of forgetting factor on convergence of x to x,opt. For this example smaller values of λ yield larger asymptotic values of ε. ε % λ 1 λ.95 λ.9 Time step Fig. 13. Effect of forgetting factor on convergence of x to x,opt. For, smaller values of λ yield larger asymptotic values of ε. For >, x,opt x,opt, and a smaller value of λ yields faster convergence of ε to its asymptotic value. VII. CONCLUSIONS In this paper, we presented a growing-window variableregularization recursive least squares GW-VR-RLS algorithm. This algorithm allows for a time-varying regularization term in the RLS cost function. More specifically, GW-VR- RLS allows us to vary both the weighting in the regularization as well as what is being weighted, that is, the regularization term can weight the difference between the next state estimate and a time-varying vector of parameters rather than the initial state estimate. Future wor will include an investigation of the convergence properties. REFERENCES [1] A. H. Sayed, Adaptive Filters. Hoboen, New Jersey: John Wiley and Sons, Inc., 8. [2] J. N. Juang, Applied System Identification. Prentice-Hall, [3] L. Ljung and T. Söderström, Theory and practice of Recursive Identification. Cambridge, MA: The MIT Press, [4] L. Ljung, System Identification: Theory for the User, 2nd ed. Upper Saddle River, NJ: Prentice-Hall, [5] K. J. Åström and B. Wittenmar, Adaptive Control, 2nd ed. Addison- Wesley, [6] G. C. Goodwin and K. S. Sin, Adaptive Filtering, Prediction, and Control. Prentice Hall, [7] G. Tao, Adaptive Control Design and Analysis. Wiley, 3. [8] P. Ioannou and J. Sun, Robust Adaptive Control. Prentice Hall, [9] D. S. Bernstein, Matrix Mathematics, 2nd ed. Princeton University Press, 9. 1