An Iteration-Domain Filter for Controlling Transient Growth in Iterative Learning Control

21 American Control Conference Marriott Waterfront, Baltimore, MD, USA June 3-July 2, 21 WeC14.1 An Iteration-Domain Filter for Controlling Transient Growth in Iterative Learning Control Qing Liu and Douglas A. Bristow Abstract Transient growth is a problem in Iterative Learning Control (ILC) in which the tracking error temporarily grows very large during the learning process, before converging to a small value. While some ILC algorithms can guarantee monotonic convergence, there are limitations when the model is uncertain. This paper presents a new algorithm to reduce the transient growth in ILC. An domain filter, which can be applied to any linear ILC system, is proposed. The filter slows the learning process, in a controlled manner, to limit transient growth. Fundamental results relating the learning process convergence rate to explicit bounds on the transient growth are presented. Two examples that demonstrate the effectiveness of the method are presented: one in SISO design and one in network design. I I. INTRODUCTION terative learning control (ILC) is an approach to improve the tracking performance of a system that operates repetitively [1, 2]. The basic idea of the learning process is to use the control and error signal in the previous (s) to generate control signal for current system. Since it was first formulated in 1978 [3], ILC has been applied to a variety of applications. In these applications, model uncertainty at large frequencies requires some type of lowpass filtering to prevent transient growth. The lowpass filtering undesirably limits the learning bandwidth. Simple examples show that transient growth in exponentially stable ILC systems can mimic the response of an unstable system [1]. That is, the error can grow exponentially for several s before it begins to exponentially converge. Therefore, in application it is critical that the ILC be designed in such a way as to safely control transient growth. In previous works, transient growth is controlled using robust monotonically convergent algorithms like norm-optimal [4] and time domain method [5], but at the cost of asymptotic performance. Control of transient growth is even more challenging in networked systems, where centralized control methods such as norm Qing Liu is with the Department of Mechanical and Aerospace Engineering, Missouri University of Science and Technology, Rolla, MO 6549 USA Douglas A. Bristow is with the Department of Mechanical and Aerospace Engineering, Missouri University of Science and Technology, Rolla, MO 6549 USA (corresponding author, dbristow@mst.edu) optimal and frequency domain may not apply well. Therefore, other methods of controlling transient growth are necessary. In this paper, we will present an domain filter that can be applied to any linear ILC algorithm to control transient growth. The filter contains a tuning variable that provides a tradeoff between growth reduction and convergence rate. In section II, we summarize the transient growth problem in ILC. In section III, we present our domain filter and prove an explicit relationship between the filter s convergence rate and the transient growth. A tuning process is presented, which is useful in practice when the system is not well modeled or the system scale is large to easily analyze. In section IV, a P-type ILC system and a master-slave system are presented to demonstrate the effectiveness of the domain filter. Finally in section V, we describe the future works. II. ILC TRANSIENT GROWTH BACKGROUND A discrete-time, linear time invariant system has the form 1, (1) where, is the time during a given. Define 1, 2,,, and, 1,, 1, where means the th (operation) of learning. Let 1, 2,, be the vector of desired outputs. Using the lifted setting [6], the system can be written as, (2) where is the Toeplitz matrix [7] of the plant:, (3) where,,, is the impulse response of the system. There are a number of ILC algorithms that can be designed for the system (1). The reader is referred to [1,2] for a categorization and listing. Here, we take a general approach and assume that any linear, first-order ILC algorithm is used. Such an algorithm can always be written as, (4) where and are NxN learning gains, and the error. Assuming, we have that, and the closed-loop equation for the error can be written as 1. (5) 978-1-4244-7425-7/1/$26. 21 AACC 239

When, 1 1, where. The convergence dynamics can be obtained by writing (5) as. (6) Clearly the system is convergent if 1, for 1,,, where is the i th eigenvalue of. From equation (6),. Thus, we say that the system is monotonically convergent if 1 [1], because the error approaches the asymptotic solution monotonically. If the system is not monotonically convergent, there will be a transient growth, and a tight bound on the magnitude of the growth is difficult to calculate. While, it is well known that, where, is a (non unique) matrix of the eigenvectors of, and is the spectral radius of, or the largest eigenvalue of [8]. In the existing ILC design method, the convergence of system is not hard to achieve, but the transient growth is usually a problem. We will illustrate this in the following two examples. After developing our new approach in Section III, we return to these examples in Section IV to demonstrate its effectiveness. P-type ILC system performance Consider the P-type ILC algorithm [6], which is the simplest method, as an example. Then 1, where is a constant. From (6). (7) Clearly the system is exponentially convergent if 1 1. RMS error 4 2-2 -4-6 -8 large transient growth convergence speed λ -1 2 4 6 8 Figure 1: Large transient in a stable learning control system.. Here we use,.. 1,.6, 4. The eigenvalues are all located at.4, well inside the unit disk, but 4 orders of magnitude growth are observed in the transient. In fact, convergence of this system is quite robust. Using only the sign and magnitude of, we can pick a sufficiently small (or ) to provide robust convergence. Although convergence depends only on the eigenvalues, and thus only on and, the transient growth bound depends on the full system. Thus, as shown in figure 1, it is easy to construct an example where eigenvalues are small but growth is large. Although the system is convergent and robust to dynamic variations in P, it is not acceptable in practice due to the large transient growth. Master-slave networked systems In some applications such as UAV [9], robotic systems [1] and manufacturing [11], coordinated, or networked, systems are used. The master-slave networked system shown in figure 2 is a simple type of networked system. Here the first system tracks and system 1 tracks system. The connection among systems is decentralized, which is especially useful if the network contains a large number of systems. u 1 j u 1( j 1) e1( j 1 ) u 2 j P 1 P 2 e 1 j u 2( j 1) e2 ( j 1 ) e 2 j y d y 1j y 1 j Figure 2: A master-slave networked system y 2 j When the number of systems is large, it is difficult to use any existing ILC method to control. Particularly in the case of lifted-systems based approaches, it may be difficult to construct the nnxnn lifted-system. Here we assume that all systems are identical,, n is the number of systems in the group. Use same ILC learning law for each system,, and, and further assume that L u and L e are selected such that 1 where. By selecting L u and L e in this manner, each system is capable of monotonically tracking its input as a stand-alone system. However, as we will show, when the systems are interconnected, monotonicity is lost. 24

For the networked system we find that the error is given by, Thus, where 1 1 1. (8) 1 1,,,,,,,,,,,,,,,,. Clearly, convergence of the networked system is inherited from the individual sub-systems, because the eigenvalues of are simply the eigenvalues of in the sub-systems. However, 1 does not imply 1. Therefore, monotonicity of the networked system is not inherited from the individual sub-systems. Developing a method that extends monotonicity of the individual systems to monotonicity of the networked system is a primary motivator for this work. Remark: In the above example, it may be possible to leverage the Toeplitz structure of to design decentralized controllers and. However, a simple extension of the above to the case where each system is unique would increase the complexity of such an approach. From the above examples, it is not hard to see that transient growth of convergent systems is a big problem. Our approach can solve the transient problem without changing final performance by slowing the system down. III. ITERATION DOMAIN FILTER Assume we are given the learning gains and, which can be designed by any ILC algorithm, like P-type, PD-type, time-varying method and norm-optimal design. Assume also that the ILC system is known to be stable ( 1) but not monotonically convergent ( 1). Define the scalar sequence as a monotonically increasing filter with the following properties, lim 1. (1) Consider the modified learning algorithm where the prefilter is added to (4) as,, (11) (9) Our modified learning algorithm can be interpreted as a forgetting-factor algorithm, where the forgetting factor is aproaches zero with increasing. The closed-loop error with the modified learning algorithm can be written as 1 1, (12) where,. For every,,1,2, we define the f j - steady-state error as, 1. (13) Note that always exists because the eigenvalues of are smaller than 1. The f j -steady-state error is the error that would be obtained if f j were frozen for all future s. That is, is the steady-state error when. For the -varying filter, the error contains two parts, the -steady-state error and the transient error in learning process, as shown in figure 3. The transient error is the source of transient growth problem described in section II. To isolate the two parts, the transient error is defined in a special way. (14) RM Steady state error System error Transient error Figure 3: System error, steady-state error and transient error We can derive the propagation of as follows, 1 1, (15) where. The -steady-state error is a smooth and bounded function, so there exists an L such that, where is the Lipschitz constant of [12]. 241

Define as the maximum rate of increase of as. The main contribution of this work is the following theorem, which proves that the transient growth is proportional to, the rate of increase of. Thus, the transient error can be controlled with appropriate selection of. Theorem: Given the system (2), the modified ILC algorithm (11). If 1 and, then there exists a learning rate,1, for all, such that the transient error is bounded by, for all j. Proof: From (15), there is. Since 1, there is. Furthermore, since 1, so there exists K> and 1 such that. Using the Lipschitz bound on,. Then,. Thus, the transient error bound is proportional to. It remains to show that there exists an,1 to satisfy the bound. Choosing such that. (16) Then,, which completes the proof. Remark: From the above proof, it is clear that when P is known one can explicitly calculate to achieve the desired bound on transient growth. However, if the system is not well known or has a large size (for example, the large scale coordinated systems), calculation of an appropriate calculate is challenging. For such systems, a tuning process is always practical. Because always exists for convergent systems, it is always possible to tune through a trial and error process to achieve the desired behavior. Remark: It is notable from (equation 16) that is inversely proportional to K. Since, K is a measure of the transient growth of the system. Therefore, the larger the system s transient growth, the smaller one must choose, and thus the slower the convergence. Clearly, gives the fastest rate of increase for, of course, should not exceed one, so we use min,1. When, 1, the steady state error 1 1, which is same as the final system error without the filter from equation (5). The final performance will not be influenced by the filter. From the analysis above, we can see that the modified learning algorithm (11) will decrease the transient growth, while maintaining the same asymptotic performance as the unmodified learning algorithm (4). The tradeoff is slower convergence. IV. SIMULATED RESULTS In this part, we will present two examples to demonstrate the effectiveness of the domain filter. Example 1: SISO Tracking First we use a second order discrete time system [5]. (17).. The reference is given in figure 4. y d 2 1.5 1.5.1.2.3.4 t Figure 4: reference To demonstrate our filter, we use a P-type ILC design, which is known to result in transient growth. While other ILC designs such as norm-optimal or model inverse will result in monotonic convergence, we can treat this example as a highly uncertain system, such that those approaches will not be benefitial. Let 1,.6, then.6. This is the same system we examined in Section II. The system is convergent since.4, but it is not monotonically convergent since 1.3967 1. We use the filter. In this case, we determine that 1, 2,.4, so according to (16) there is 31. Clearly this value of is impractically slow. However, (16) is a conservative condition, as are estimates of K. We find that the tuning process will be more practical in this case. Several choices of, 1,.1,.1, are shown in figure 5. Note that algorithm (11) with 1 is identical to algorithm (4), and thus this is the nominal P-type ILC. The results are shown in table 1 and figure 6. Table 1: Transient error with different Transient error 1 51.1 149.1 14.5 242

f j RMS (transient error) 1.5 α=.1 α=.1 5 1 15 Figure 5: Filter 1 1 1 1-1 α=.1 α=.1 5 1 15 Figure 6: for ILC system in example 1, with filter, when is smaller, the transient error is smaller, and the convergent speed is slower. As expected, we see a large transient growth for 1, with decreasing growth, but slower convergence as is decreased. To further reduce transient growth much smaller values of are needed. Alternatively, we consider other forms of the filter that also satisfy. Beginning with the results in figure 6, we see that the transient error appears to grow suddenly when is close to, or at, one. Therefore, rather than continuing to select smaller values of, in the algorithm, it may be more advantageous to select a algorithm with a slower transition to 1. Therefore an exponential filter 11 is used. Here the largest increase rate in occurs from j= to j=1, so. Several choices of, 1,.2,.1, are selected and the filter and simulation results are shown in table 2, figure 7and figure 8. Clearly the exponential filter is more effective at reducing the transient growth. Although it is slower than the linear filter, as evident by the asymptotic convergence rates, the improved reduction in transient growth at early s makes it overall the more practical option. Table 2: Transient error with exponential filter Transient error 1 1 51.2 1.8.71.1 1.9.22 f j RMS(transient error) 1.5 α=.2 α=.1 5 1 15 Figure 7: Filter 11 1 1 1 1-1 5 1 15 Figure 8: for ILC system in example 1, with filter 1 1, when is smaller, the transient error is smaller, and the convergent speed is slower. These results above shows that the proposed varying filter reduces transient error, while the trade off is slower convergent speed. Furthermore, the tuning approach appears to work quite well. Clearly, however, the best -varying filter is not necessarily the fastest one. Example 2: Master-slave Networked System Next we examine the master-slave networked system described in Section II. Let α=.2 α=.1..., (18) and use PD-type ILC [1] for each system, 1,.75.3. (19) The master system tracks the reference in Figure 4. According to (9), the convergence is provided by.251. For each individual system, there is.6882, which indicates the monotonic convergence of each system. But 1 cannot be ensured even with 1, because will be changed every time when a new slave system is attached. Thus the system is convergent but not necessarily monotonically convergent. In Figure 9 we show that the networked system is not monotonic for the nominal learning algorithm (4). In fact, the transient growth increases with each additional system. The largest growth in the networked system is 1, which 243

appears on the 2 th system. Clearly adding more systems will result in increased growth. RMS error 1 1 1 5 1 1-5 1-1 2 4 6 8 Figure 9: Transient growth for the networked system using the nominal learning algorithm Since it shows that the exponential filter works better in Example 1, we apply an exponential filter 11 on the system. Using the modified learning algorithm, (11), we tune until the transient growth is effectively removed. Figure 1 shows the series of the error with.1. RMS error 1 1 1 5 1 1-5 1-1 increasing transient growth with increasing # of systems master 5th sys 1th sys 15th sys 2th sys blue: master sys green: 1th sys red: 2th sys α=.1 2 4 6 8 Figure 1: Transient growth for the networked system using the modified learning algorithm V. CONCLUSION In this paper, our work is focused on transient growth problem in ILC system(s). First we reviewed the transient growth problem, and showed that although the convergence of the ILC system is not hard to provide with existing algorithms, when the system model is uncertain or too complicated such as in a network system, a suitable transient growth bounded is challenging. We proposed an domain filter to control transient error. Analysis shows that such a filter can always be found to bound the transient error to low magnitude. With the filter applied on a stable ILC system designed by any algorithm, it is possible to trade convergent speed for lower transient growth. Two examples were used to demonstrate the utility of this method in SISO design and in a master-slave network design. REFERENCES [1] D. Bristow, M. Tharayil and A. Alleyne, A Servey of Iterative Learning Control, IEEE Control Systems Magazine, vol. 26, no. 3, 26, pp 96-114. [2] H. Ahn, Y. Chen and K. Moore, Iterative Learning Control: Brief Survey and Categorization, IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews, v 37, n 6, Nov. 27, p 199-1121 [3] M. Uchiyama, Formation of high-speed motion pattern of a mechanical arm by trial, Trans. Soc. Instrument Contr. Engineers, vol. 14, no. 6, pp. 76 712, 1978. [4] N.Amann, D.H.Owens and E. Rogers, Iterative learning control for discrete-time systemswith exponential rate of convergence, IEEE Proc-Control Theory Appl., Vol. 143, No. 2, March 1996 [5] K. Moore, Y. Chen, and V. Bahl, Monotonically convergent iterative learning control for linear discrete-time systems, Automatica, vol. 41, issue 9, Sep. 25, pp 1529-1537. [6] M. Phan and R. Longman, A mathematical theory of learning control for linear discrete multivariable systems, in Proc. of the AIAA/AAS Astrodynamics Specialist Conference, 1988 [7] T. Kailath, Linear Systems. Englewood Cliffs, NJ : Prentice-Hall, 198. [8] L. N. Trefethen and M. Embree, Spectra And Pseudospectra. Princeton, NJ: Princeton University Press, 25 [9] J. How, B. Bethke, A. Frank, D. Dale and J. Vian, Real-Time Indoor Autonomous Vehicle Test Enviroment, IEEE Control Systems Magazine, vol. 28, no. 2, Apr. 28, pp 51-64 [1] C. Belta and V. Kumar, Abstract and Control for Groups of Robots, IEEE Trans. Robotics, vol. 2, no. 5, 24, pp 865-875 [11] K. Yoram, Cross-coupled Biaxial Computer Control for Manufacturing Systems, Journal of Dynamic Systems, Measurement and Control, Transactions of the ASME, v 12, n 4, Dec. 198, p 265-272. [12] Hassan K. Khalil, Nonlinear Systems, Upper Saddle River, NJ: Prentice Hall, 1996 Just as with the nominal algorithm, adding more systems will result in the appearance, and increased size, of transient growth. However, with the modified algorithm it is a simple process of re-tuning the parameter to achieve the desired transient behavior. 244