Iteration-Domain Robust Iterative Learning Control

Transcription

1 Iteration-Domain Robust Control Iteration-Domain Robust Control Presenter: Contributors: Kevin L. Moore G.A. Dobelman Distinguished Chair and Professor of Engineering Division of Engineering Colorado School of Mines egweb.mines.edu/faculty/kmoore/ Hyo-Sung Ahn ETRI, Korea YangQuan Chen Utah State University presented at Tsinghua University Beijing, China 3 July

2 Iteration-Domain Robust Control Colorado School of Mines Located in Golden, Colorado, USA 10 miles West of Denver CSM sits in the foothills of the Rocky Mountains CSM has about 300 faculty and 4000 students CSM is a public research institution devoted to engineering and applied science, especially: Discovery and recovery of resources Conversion of resources to materials and energy Utilization in advanced processes and products Economic and social systems necessary to ensure prudent and provident use of resources in a sustainable global society 2

3 Iteration-Domain Robust Control Outline Overview of ILC Interval ILC Problem 1: Analysis of Maximum Allowable Perturbation Problem 2: Design for Maximum Allowable Perturbation Problem 3: Convergence Analysis Problem 4: Design for Convergence (with Interval Model Conversion) Iteration-Domain H ILC Design 3

4 Iteration-Domain Robust Control Systems that Execute the Same Trajectory Repetitively Step 1: Robot at rest, waiting for workpiece. Step 2: Workpiece moved into position. Step 3: Robot moves to desired location Step 4: Robot returns to rest and waits for next workpiece. 4

5 Iteration-Domain Robust Control Errors are Repeated When Trajectories are Repeated A typical joint angle trajectory for the example might look like this: Each time the system is operated it will see the same overshoot, rise time, settling time, and steady-state error. learning control attempts to improve the transient response by adjusting the input to the plant during future system operation based on the errors observed during past operation. 5

6 Iteration-Domain Robust Control Control Standard iterative learning control scheme: u k System y k Memory Memory Memory u k+1 y d A typical ILC algorithm has the form: u k+1 (t) = u k (t) + γe k (t + 1). Standard ILC assumptions include: Stable dynamics or some kind of Lipschitz condition. System returns to the same initial conditions at the start of each trial. Each trial has the same length. 6

7 Iteration-Domain Robust Control The Supervector Framework of ILC Consider an SISO, LTI discrete-time plant with relative degree m: Y (z) = H(z)U(z) = (h m z m + h m+1 z (m+1) + h m+2 z (m+2) + )U(z) By lifting along the time axis, for each trial k define: U k Y k Y d = [u k (0), u k (1),, u k (N 1)] T = [y k (m), y k (m + 1),, y k (m + N 1)] T = [y d (m), y d (m + 2),, y d (m + N 1)] T Thus the linear plant can be described by Y k = H p U k where: h h 2 h H p = h 3 h 2 h h N h N 1 h N 2... h 1 The lower triangular matrix H p is formed using the system s Markov parameters. Notice the non-causal shift ahead in forming the vectors U k and Y k. 7

8 Iteration-Domain Robust Control The Supervector Framework of ILC (cont.) For the linear, time-varying case, suppose we have the plant given by: x k (t + 1) = A(t)x k (t) + B(t)u k (t) y k (t) = C(t)x k (t) + D(t)u k (t) Then the same notation again results in Y k = H p U k, where now: H p = h m, h m+1,0 h m, h m+2,0 h m+1,1 h m, h m+n 1,0 h m+n 2,1 h m+n 3,2... h m,n 1 The lifting operation over a finite interval allows us to: Represent our dynamical system in R 1 into a static system in R N. 8

9 Iteration-Domain Robust Control The Update Law Using Supervector Notation Suppose we have a simple Arimoto-style ILC update equation with a constant gain γ: In our R 1 representation, we write: u k+1 (t) = u k (t) + γe k (t + 1) In our R N representation, we write: U k+1 = U k + ΓE k where Γ = γ γ γ γ 9

10 Iteration-Domain Robust Control The Update Law Using Supervector Notation (cont.) Suppose we filter with an LTI filter during the ILC update: In our R 1 representation we would have the form: u k+1 (t) = u k (t) + L(z)e k (t + 1) In our R N representation we would have the form: U k+1 = U k + LE k where L is a Topelitz matrix of the Markov parameters of L(z), given, in the case of a causal, LTI update law, by: L = L m L m+1 L m L m+2 L m+1 L m L m+n 1 L m+n 2 L m+n 3... L m 10

11 Iteration-Domain Robust Control The Update Law Using Supervector Notation (cont.) We may similarly consider time-varying and noncausal filters in the ILC update law: U k+1 = U k + LE k A causal (in time), time-varying filter in the ILC update law might look like, for example: n 1, n 2,0 n 1, L = n 3,0 n 2,1 n 1, n N,0 n N 1,1 n N 2,2... n 1,N 1 A non-causal (in time), time-invariant averaging filter in the ILC update law might look like, for example: K K K K K K L = K K K K K 11

12 Iteration-Domain Robust Control The Update Law Using Supervector Notation (cont.) The supervector notation can also be applied to other ILC update schemes. For example: The Q-filter often introduced for stability (along the iteration domain) has the R 1 representation: u k+1 (t) = Q(z)(u k (t) + L(z)e k (t + 1)) The equivalent R N representation is: U k+1 = Q(U k + LE k ) where Q is a Toeplitz matrix formed using the Markov parameters of the filter Q(z). 12

13 Iteration-Domain Robust Control The ILC Design Problem The design of an ILC controller can be thought of as the selection of the matrix L: For a causal ILC updating law, L will be in lower-triangular Toeplitz form. For a noncausal ILC updating law, L will be in upper-triangular Toeplitz form. For the popular zero-phase learning filter, L will be in a symmetrical band diagonal form. L can also be fully populated. Motivated by these comments, we will refer to the causal and non-causal elements of a general matrix Γ as follows: Γ = γ 11 γ 12 γ γ 1N γ 21 γ 22 γ 23 noncausal γ 2N γ 31 γ 32 γ γ 3N. causal..... γ N1 γ N2 γ N3... γ NN The diagonal elements of Γ are referred to as Arimoto gains. 13

14 Iteration-Domain Robust Control w-transform: the z-operator in the Iteration Domain Introduce a new shift variable, w, with the property that, for each fixed integer t: w 1 u k (t) = u k 1 (t) For a scalar x k (t), combining the lifting operation to get the supervector X k with the shift operation gives what we call the w-transform of x k (t), which we denote by X(w) Then the ILC update algorithm: u k+1 (t) = u k (t) + L(z)e k (t + 1) which, using our supervector notation, can be written as U k+1 = U k + LE k can also be written as: wu(w) = U(w) + LE(w) where U(w) and E(w) are the w-transforms of U k and E k, respectively. Note that we can also write this as where. E(w) = C(w)U(w) C(w) = 1 (w 1) L 14

15 Iteration-Domain Robust Control ILC as a MIMO Control System The term C(w) = 1 (w 1) L is effectively the controller of the system (in the repetition domain). This can be depicted as: 15

16 Iteration-Domain Robust Control Higher-Order ILC in the Iteration Domain We can use these ideas to develop more general expressions ILC algorithms. For example, a higher-order ILC algorithm could have the form: which corresponds to: u k+1 (t) = k 1 u k (t) + k 2 u k 1 (t) + γe k (t + 1) C(w) = γw w 2 k 1 w k 2 Next we show how to extend these notions to develop an algebraic (matrix fraction) description of the ILC problem. 16

17 Iteration-Domain Robust Control A Matrix Fraction Formulation Suppose we consider a more general ILC update equation given by (for relative degree m = 1): u k+1 (t) = D n (z)u k (t) + D n 1 (z)u k 1 (t) + + D 1 (z)u k n+1 (t) + D 0 (z)u k n (t) +N n (z)e k (t N n 1 (z)e k 1 (t N 1 (z)e k n+1 (t + 1) + N 0 (z)e k n (t + 1) which has the supervector expression U k+1 = D n U k + D n 1 U k D 1 U k n+1 + D 0 U k n +N n E k + N n 1 E k N 1 E k n+1 + N 0 E k n Aside: note that there are a couple of variations on the theme that people sometimes consider: U k+1 = U k + LE k+1 U k+1 = U k + L 1 E k + L 0 E k+1 These can be accomodated by adding a term N n+1 E k+1 in the expression above, resulting in the so-called current iteration feedback, or CITE. 17

18 Iteration-Domain Robust Control A Matrix Fraction Formulation (cont.) Applying the shift variable w we get: D c (w)u(w) = N c (w)e(w) where D c (w) = Iw n+1 D n 1 w n D 1 w D 0 N c (w) = N n w n + N n 1 w n N 1 w + N 0 This can be written in a matrix fraction as U(w) = C(w)E(w) where: C(w) = 1 D c (w)n c (w) Thus, through the addition of higher-order terms in the update algorithm, the ILC problem has been converted from a static multivariable representation to a dynamic (in the repetition domain) multivariable representation. Note that we will always get a linear, time-invariant system like this, even if the actual plant is time-varying. Also, because D c (w) is of degree n + 1 and N c (w) is of degree n, we have relative degree one in the repetition-domain, unless some of the gain matrices are set to zero. 18

19 Iteration-Domain Robust Control ILC Convergence via Repetition-Domain Poles From the figure we see that in the repetition-domain the closed-loop dynamics are defined by: G cl (w) = H p [I + C(w)H p ] 1 C(w) = H p [ D c (w) + N c (w)h p ] 1 N c (w) Thus the ILC algorithm will converge (i.e., E k a constant) if G cl is stable. Determining the stability of this feedback system may not be trivial: It is a multivariable feedback system of dimension N, where N could be very large. But, the problem is simplified due to the fact that the plant H p is a constant, lower-triangular matrix. 19

20 Iteration-Domain Robust Control Repetition-Domain Internal Model Principle Because Y d is a constant and our plant is type zero (e.g., H p is a constant matrix), the internal model principle applied in the repetition domain requires that C(w) should have an integrator effect to cause E k 0. Thus, we modify the ILC update algorithm as: U k+1 = (I D n 1 )U k + (D n 1 D n 2 )U k 1 + +(D 2 D 1 )U k n+2 + (D 1 D 0 )U k n+1 + D 0 U k n +N n E k + N n 1 E k N 1 E k n+1 + N 0 E k n Taking the w-transform of the ILC update equation, combining terms, and simplifying gives: where (w 1)D c (w)u(w) = N c (w)e(w) D c (w) = w n + D n 1 w n D 1 w + D 0 N c (w) = N n w n + N n 1 w n N 1 w + N 0 20

21 Iteration-Domain Robust Control Repetition-Domain Internal Model Principle (cont.) This can also be written in a matrix fraction as: but where we now have: U(w) = C(w)E(w) C(w) = (w 1) 1 D 1 c (w)n c (w) For this update law the repetition-domain closed-loop dynamics become: ( G cl (w) = H I + I (w 1) C(w)H ) 1 I (w 1) C(w), = H[(w 1)D c (w) + N c (w)h] 1 N c (w) Thus, we now have an integrator in the feedback loop (a discrete integrator, in the repetition domain) and, applying the final value theorem to G cl, we get E k 0 as long as the ILC algorithm converges (i.e., as long as G cl is stable). 21

22 Iteration-Domain Robust Control Review learning control (ILC) is a control strategy for systems that execute the same trajectory, motion, or operation over and over. Consider the following single-input, single-output (SISO) 2-dimensional plant: x k (t + 1) = Ax k (t) + Bu k (t) y k (t) = Cx k (t) where t represents the discrete time point along the time axis (assumed finite) and the k represents the iteration trial number along the iteration axis. Taking z-transforms in time and defining G(z) = C(zI A) 1 B + D, the plant can be written as Y (z) = G(z)U(z) = (h m z m + h m+1 z (m+1) + h m+2 z (m+2) + )U(z), where m is the relative degree of the system, z 1 is the standard delay operator in time, and the parameters h i are the standard Markov parameters of the system G(z). Further, introduce the input update equation where L(z) is the learning filter. u k+1 (t) = u k (t) + L(z)(y d (t + m) y k (t + m)), 22

23 Iteration-Domain Robust Control Review (cont.) Define the lifted super-vectors as: U k = (u k (0), u k (1),, u k (N 1)), Y k = (y k (m), y k (m + 1),, y k (N 1 + m)), Y d = (y d (m), y d (m + 1),, y d (N 1 + m)), = Y d Y k = (E k (m), E k (m + 1),, E k (N 1 + m)) E k This gives the relationships Y k = HU k and U k+1 = U k + ΓE k, where H and Γ are lower-triangular Toeplitz matrices representing the system and the learning filter L(z), respectively. The error propagation equation becomes E k+1 = (I HΓ)E k,where h m γ 11 γ 12 γ γ 1N h m+1 h m γ H = h m+2 h m+1 h m ; Γ = 21 γ 22 γ 23 noncausal γ 2N γ 31 γ 32 γ γ 3N. causal h m+n 1 h m+n 2 h m+n 3... h m γ N1 γ N2 γ N3... γ NN Adding iteration-varying reference, disturbance, and noise signals, the possibility of an iterationvarying plant, and uncertainty in the plant model gives the complete ILC framework. 23

24 Iteration-Domain Robust Control Complete Framework D (w) N(w) Y d (w) - E(w) C ( w ILC ) U (w) 1 ( w 1) H p (w) Y (w) C CITE (w) ΔH(w) The resulting design problem is to determine the learning algorithm so the closed-loop system in the iteration domain is optimally convergent (asymptotically or monotonically) along the iteration axis in an appropriate norm topology, subject to specific assumptions on the noises, disturbances, and uncertainty in the system. This gives rise to a categorization of problems to consider. In the remainder of this presentation we consider robust learning controller design for two types of uncertainty problems: interval ILC and iteration-domain H ILC. 24

25 Iteration-Domain Robust Control Categorization of Problems Y d D N C H Y d (z) 0 0 Γ(w 1) 1 H p Classical Arimoto algorithm Y d (z) 0 0 Γ(w 1) 1 H p (w) Owens multipass problem Y d (z) D(w) 0 C(w) H p General (higher-order) problem Y d (w) D(w) 0 C(w) H p General (higher-order) problem Y d (w) D(w) N(w) C(w) H p (w) + (w) Most general problem Y d (z) D(z) 0 C(w) H(w) + (w) Frequency-domain uncertainty Y d (z) 0 w(t), v(t) Γ(w 1) 1 H p Least quadratic ILC Y d (z) 0 w(t), v(t) C(w) H p Stochastic ILC (general) Y d (z) 0 0 Γ(w 1) 1 H I Interval ILC Y d (z) D(z) w(t), v(t) C(w) H(z) + H(z) Time-domain H problem Y d (z) D(w) w(k, t), v(k, t) C(w) H(w) + H(w) Iteration-domain H ILC Y d (z) 0 w(k, t), v(k, t) Γ(k) H p + H(k) Iteration-varying uncertainty and control Y d (z) 0 H Γ Hp Intermittent measurement problem

26 Iteration-Domain Robust Control Interval ILC Prehistory of automatic control Primitive period Classical control Modern control Classic control Nonlinear control Estimation Robust control Optimal control Adaptive control Intelligent control H_inf Interval Fuzzy Neural Net ILC... Interval ILC 26

27 Iteration-Domain Robust Control Interval Definitions A scalar a is called an interval parameter if it lies between two extreme boundaries according to a [a, a], where a is the minimum value of a and a is the maximum value of a. The superscript I is used to represent an interval variable, e.g., a a I := [a, a] denotes an interval scalar. An interval matrix (A I ) is defined as: { [ ] A I = A : A = a ij [a ij, a ij ] }, i, j = 1,, n, where a ij is the maximum extreme value of the i th row and j th column element of the uncertain plant, and a ij is the minimum extreme value of the i th row and j th column element of the uncertain plant. The upper bound matrix (A) is a matrix whose elements are a ij. The lower bound matrix (A) is a matrix whose elements are a ij. The vertex matrices (A v ) are defined by: { [ ] } A v = A : A = a ij {a ij, a ij }, i, j = 1,, n If the Markov parameters of a system are intervals such as: h i [h i, h i ], then the system is said to have interval uncertainties. The interval Markov matrix of such a system is denoted as H I and its interval vertex Markov matrices are denoted as H v. 27

28 Iteration-Domain Robust Control Iteration loop + Interval ILC Yr - C Y H I Iteration loop The interval ILC problem is concerned with the analysis and design of the ILC system when the system to be controlled is subjected to interval uncertainties. In this section we consider this problem, assuming: No disturbances and no noise. First-order ILC, where C(w) = Γ. 28

29 Iteration-Domain Robust Control Interval ILC Problems We consider four problems: 1. Given a Markov matrix H and a gain matrix Γ, how much perturbation H can the system tolerate and still be stable? This analysis question was addressed at the 2005 ACC. 2. Given a nominal Markov matrix H 0, find Γ to maximize the perturbation H that can be allowed while still allowing convergence. This design question was addressed at the 2005 ACC. 3. Given an interval Markov matrix H I and a gain matrix Γ, what are the stability and convergence properties of the closed-loop system? This analysis question was addressed at the 2005 IFAC. 4. Given an interval Markov matrix H I, design Γ, so as to achieve desired stability and convergence properties of the closed-loop system. This design question was addressed at the 2005 ISIC and in a TAC paper. An intermediate step in this problem is to solve for [h i, h i ] given the interval plant A I 29

30 Iteration-Domain Robust Control Reminder: ILC Stability/Convergence Conditions When Arimoto-like gains and purely causal gains are used, the asymptotic stability condition is defined as: 1 γ ii h 1 < 1, i = 1,, n, where h 1 is the first non-zero Markov parameter. When non-causal gains are used, the asymptotic stability condition is defined as: ρ(i HΓ) < 1 where ρ is the spectral radius of (I HΓ), and H is the Markov matrix. Monotonic convergence is defined in an appropriate norm topology as follows: If I HΓ 1 < 1, then E k is monotonically convergent to zero in the l 1 -norm topology. If I HΓ < 1, then E k is monotonically convergent to zero in the l -norm topology. 30

31 Iteration-Domain Robust Control Outline Overview of ILC Interval ILC Problem 1: Analysis of Maximum Allowable Perturbation Problem 2: Design for Maximum Allowable Perturbation Problem 3: Convergence Analysis Problem 4: Design for Convergence (with Interval Model Conversion) Iteration-Domain H ILC Design 31

32 Iteration-Domain Robust Control Problem Setup Assume the plant is Y k = HU K Assume first-order ILC with an update equation U k+1 = U k + ΓE k This gives the following evolution of the error vector in iterations: E k+1 = (I HΓ)E k, where E k = Y d Y k and Γ is the learning gain matrix defined as Γ = {γ ij }, i, j = 1,, n. The ILC algorithm is called Arimoto-like when the ILC gains are γ ij = 0, i j and γ ij = γ, i = j. The gains γ ij are called causal ILC gains for i > j and non-causal ILC gains for i < j. If the gains do not exhibit Toeplitz-like symmetry we call the learning algorithm time-varying. We also refer to the band size of Γ. For example, If band size is 1, only a diagonal line composed of Arimoto-like gains is used. If band size is 2, then one causal diagonal line and one noncausal diagonal line are used in addition to Arimoto-like gains. 32

33 Iteration-Domain Robust Control Some Definitions The interval radius matrix ( H r ) is defined as H r = H H 2 For convenience, the symbols T and T I are introduced, where T I HΓ, T I I H I Γ Then, the following notation is defined: T = I HΓ (I H I Γ) = (H I H)Γ = HΓ, where T is the interval uncertainty of the iterative learning control system, and H is the interval uncertainty of the nominal Markov matrix. Also define [ ] [ ] T I 0 (I H s = I Γ) T 0 (I HΓ) (I H I ; T Γ) 0 s = T. (I HΓ) 0 We introduce the symbol, to represent the bigger norm value between a matrix and its transpose: where denotes any kind of matrix norm. T max{ ( T ) T, T }, 33

34 Iteration-Domain Robust Control Asymptotic Stability Condition of Interval ILC Theorem 1 Given Γ designed for the nominal plant H, if there exists a symmetric, positive definite matrix P that satisfies the constraint (I HΓ) T P (I HΓ) P = I, then the maximum allowable interval uncertainty (AIU), H, for which (I (H ± H)Γ) is guaranteed to have asymptotic stability is bigger than H that satisfies I HΓ + I HΓ P H. Γ Corollary 1 If Γ = H 1, the maximum AIU of the interval ILC is 1 Γ = 1 H 1. 34

35 Iteration-Domain Robust Control Monotonic Convergence Condition of Interval ILC Theorem 2 Given Γ designed for the nominal plant H, if there exists a symmetric, positive definite matrix P s that satisfies the constraint T T s P s T s P s = I 2n 2n then the maximum AIU, H, for which (I (H ± H)Γ) is guaranteed to have monotonic stability is bigger than H that satisfies T s 2 + T s P s 2 H k, Γ k where k is 1 or. Corollary 2 The 2-norm based AIU is calculated as: T s 2 + T s P s 2 H 2 Γ k where k = 1 or. 35

36 Iteration-Domain Robust Control Optimization (Problem 2) The purpose of optimization is to maximize H asym and H mono by designing Γ for either asymptotic stability or monotonic convergence. To find the optimal Γ that allows more interval uncertainties in terms of asymptotic stability, the following optimization scheme is suggested: max Γ H asym s.t. (I HΓ) T P (I HΓ) P = I. The same optimization idea can be used for increasing the uncertainty interval of the system in terms of monotonic convergence. It is designed as: max Γ H mono s.t. T T s P s T s P s = I 2n 2n. 36

37 Iteration-Domain Robust Control Simulation Illustration The following discrete system is considered in this presentation: x k+1 = x k y k = [ ] x k, which has three poles ( j0.62, 0.62 j0.62, and 0.50) and two finite zeros (0.65 and 0.71). We assume a zero initial condition. The simulation test is performed with the following reference sinusoidal signal: Y d = sin(8.0j/n), where n = 10 and j = 1,, n. The band size is fixed at 3, and learning gains are determined by optimization. Since the gains of each band are not fixed at the same value, the ILC algorithm is considered to be linear, time-varying, and non-causal. The uniformly distributed random number generator of MATLAB was used to make interval uncertainties in Markov parameters according to: h i = h i + δ h i w, where w [ 1, 1] is a uniformly distributed random number; and δ is tuned to limit the interval amount (in matrix 2 norm). 37

38 Iteration-Domain Robust Control Test Results 1. Asymptotic stability test: tests were performed using the ILC learning gain matrix designed from optimization with asym = 0.737: 2. Monotone convergence test: tests were performed using the ILC learning gain matrix designed from optimization mono = : In following figures: Left figures show the interval amount of random plants in matrix 2-norms First row corresponds to perturbations less that asym Perturbation size is bigger on second row and biggest on third row. Right figures show ILC performance corresponding to left figures Circle-marked lines are the l 2 -norm errors vs. ILC iteration number corresponding to the plant with the maximum matrix 2-norm Diamond-marked lines are l 2 -norm errors vs. ILC iteration number corresponding to the plant with the minimum matrix 2-norm, among random plants The figures confirm the sufficiency of the proposed optimization approach. 38

39 Iteration-Domain Robust Control Test Results-1: AS Maximum l 2 error among random plants Minimum l 2 error among random plants H l 2 norm errors Random plant Trial Fig.1.1.a Fig.1.1.b Maximum l 2 error among random plants Minimum l 2 error among random plants H l 2 norm errors Random plant Trial Fig.1.2.a Fig.1.2.b Maximum l 2 error among random plants Minimum l 2 error among random plants H l 2 norm errors Random plant Trial Fig.1.3.a Fig.1.3.b Fig.1. Asymptotical stability test: Tests were performed using the ILC learning gain matrix designed from optimization- 1. Left figures show the interval amount of random plants in matrix 2-norms. Right figures show ILC performance corresponding to left figures. Circle marked lines are maximum l_2 norm errors and diamond marked lines are 39

40 Iteration-Domain Robust Control Test Results-2: MC Maximum l 2 error among random plants Minimum l 2 error among random plants H l 2 norm errors Random plant Trial Fig.2.1.a Fig.2.1.b Maximum l 2 error among random plants Minimum l 2 error among random plants H l 2 norm errors Random plant Trial Fig.2.2.a Fig.2.2.b Maximum l 2 error among random plants Minimum l 2 error among random plants H l 2 norm errors Random plant Trial Fig.2.3.a Fig.2.3.b Fig.2. Monotone convergence test: Tests were performed using the ILC learning gain matrix designed from optimization-2. Left figures show the interval amount of random plants in matrix 2-norms. Right figures show ILC performance corresponding to left figures. Circle marked lines are maximum l_2 norm errors and diamond marked lines are minimum l_2 norm errors among random plants. 40

41 Iteration-Domain Robust Control Conclusion In this section we have considered the interval ILC problem. We calculated bounds on the the maximum allowable uncertainty in the plant Markov parameters for both asymptotic stability and monotonic convergence. These bounds were then used to design the ILC learning gain matrix to maximize the asymptotic and monotonic stability radii of the nominal plant. Simulation results illustrated the ideas. This approach, though conservative, provides an effective scheme for designing a robust ILC system. 41

42 Iteration-Domain Robust Control Outline Overview of ILC Interval ILC Problem 1: Analysis of Maximum Allowable Perturbation Problem 2: Design for Maximum Allowable Perturbation Problem 3: Convergence Analysis Problem 4: Design for Convergence (with Interval Model Conversion) Iteration-Domain H ILC Design 42

43 Iteration-Domain Robust Control Research Motivation In the ILC literature, robust design of the learning gain matrix has been considered using standard techniques such as H -ILC, LQ-ILC, optimal-ilc, etc. However, much of the literature approaches this problem from the perspective of the time-domain axis. In this presentation we will study the stability of the ILC problem when the plant Markov parameters are subject to interval uncertainty, formulating the problem in the iteration-domain using the supervector notation. In the robust control literature there are numerous results related to Hurwitz stability for interval matrixes and Schur stability. Kharitonov s theorem has also been very popular for interval matrix stability analysis. Here, we exploit these works to develop an analysis method for checking the convergence properties of the interval ILC problem. Similar to the Kharitonov vertex polynomial method, it will be shown that the extreme values of the interval Markov parameters provide a sufficient condition for checking the monotonic convergence of interval ILC systems. 43

44 Iteration-Domain Robust Control Asymptotic Stability for Interval ILC For interval ILC asymptotic stability conditions, existing results can be adopted: Lemma 1 With a given interval matrix A I, the spectral radius of A I is bounded by the maximum value of the spectral radii of vertex matrices A v. (Mau-Hsiang Shih, Yung-Yih Lur, and Chin-Tzong Pang, An ineqality for the spectral radius of an interval matrix, Linear Algebra and Its Applications, 274:2736, 1998; H. S. Han and J. G. Lee, Necessary and sufficient conditions for stability of time varying discrete interval matrices, Int. J. Control, 59: , 1994.) Lemma 2 Let the interval matrix be given as A A I A. If β = max{ρ(ms 1 ), ρ(ms 2 )} < 1, where MS 1 = a ij if i = j and MS 1 = max{ a ij, a ij } if i j; MS 2 = a ij if i = j and MS 2 = min{ a ij, a ij } if i j, then the interval matrix A I is Schur stable. (J. J. D. Delgado-Romero, R. S. Gonzalez-Garza, J. A. Rojas-Estrada, G. Acosta-Villarreal, and F. Delgado-Romero, Some very simple Hurwitz and Schur stability tests for interval matrices, In Proceedings of the 30th Conference on Decision and Control, pages , Kobe, Japan, December 1996.) These lead to the following result: Theorem 3 Let the first Markov parameter h 1 be an interval parameter given by h I 1 [h 1, h 1 ] and let Arimoto-like/causal ILC gains be used in Γ. Then the interval ILC system, E k+1 = (I H I Γ)E k, is asymptotically stable if max{ 1 γ ii h 1, 1 γ ii h 1 } < 1, i = 1,, n. 44

45 Iteration-Domain Robust Control Asymptotic Stability for Interval ILC (cont.) Now consider the case of a general Γ. In I H I Γ, the interval matrix is H I. So, the lower bound and the upper bound of I H I Γ should be re-calculated: For convenience, let T = HΓ, calculated as: t ij = i h k γ (i+1 k)j, i, j = 1,, n where t ij are elements of T and γ (i+1 k)j are ILC learning gains. Similarly define T I = H I Γ and also define P = I T and P I = I T I. k=1 The lower and upper bounds of P I, i.e., P and P, can be calculated easily from the structure of T I. Then, using the lower and upper bounds of P I, it can be shown from Lemma 1 that the maximum spectral radius of I H I Γ occurs at one of vertex matrices, P v, of P I. 45

46 Iteration-Domain Robust Control Monotone Convergence Condition for Interval ILC For monotonic convergence of interval ILC, the following lemmas are developed first. Lemma 3 Let x I [x, x] be an interval parameter. Then for y = γ 11 x I + γ 12 + γ 21 x I + γ 22, γ 11, γ 12, γ 21, γ 22 R, the max{y} occurs at a vertex point of x (i.e., x v {x, x}). Lemma 4 Let x I [x, x] be an interval parameter. Then for y = γ 11 x I + γ 12 + γ 21 x I + γ γ n1 x I + γ n2, γ i1, γ i2 R, i = 1,, n, the max{y} occurs at one of vertex points of x v (i.e., x v {x, x}). Lemma 5 Let x j [x j, x j ], j = 1,, m be interval parameters (for convenience we omit the superscript I and v). Then for y = (γ 1 11x 1 + γ 1 12) + + (γ m 11x m + γ m 12) + + (γ 1 n1x 1 + γ 1 n2) + + (γ m n1x m + γ m n2), γ j i1, γ j i2 R, i = 1,, n, j = 1,, m, the max{y} occurs at the vertices of x j. 46

47 Iteration-Domain Robust Control Monotone Convergence Condition for Interval ILC (cont.) Based on the lemmas of the preceding slides, the main results for monotone ILC conditions with interval uncertainty can be obtained, as summarized in the following theorems: Theorem 4 Given interval Markov parameters h I i [h i, h i ], the interval ILC system is monotonically convergent in the l -norm topology if max{ I H v Γ } < 1, where H v are vertex Markov matrices of the interval plant. Theorem 5 Given interval Markov parameters h I i [h i, h i ], the following equality is true: max{ I H I Γ 1 } = max{ I H v Γ 1 }, where 1 is a matrix 1-norm, which is defined as: A 1 = max j=1,,n n i=1 A ij. 47

48 Iteration-Domain Robust Control Example Let us consider a single-input, single-output system given as: A = ; B = 1 0 ; C = [ ] which has first and second Markov parameters given as h 1 = CB = 1 and h 2 = CAB = It is assumed that there are interval uncertainties in h 1 and h 2 given as h I 1 [0.9, 1.1]; and h I 2 [ 0.366, 0.166]. For Case-1 we suppose Arimoto-like ILC (only diagonal terms). For Case-2 we use the inverse of the nominal (without interval) Markov matrix. To increase the robustness, in the Case-1 test, we used two different γ 1 : the dot-dashed lines are results with γ 1 = 1 h 1, and the solid lines are results with γ 1 = 1 h 1. 48

49 Iteration-Domain Robust Control Norms of Case vertex points Norms Interval range of h Interval range of h

50 Iteration-Domain Robust Control Interval range of h Norms of Case Interval range of h vertex points 0.3 Norms Interval range of h Interval range of h

51 Iteration-Domain Robust Control Performance Test of Case-1 Absolute errors Maximum error with h 1 =1.1 Minimum error with h 1 =1.1 Average error with h 1 =1.1 Maximum error with h 1 =1.0 Minimum error with h 1 =1.0 Average error with h 1 = Trial number 0.25 Maximum error Minimum error 51

52 0.05 Iteration-Domain Robust Control Performance Trial Test number of Case Maximum error Minimum error Average error Absolute errors Trial number 52

53 Iteration-Domain Robust Control Conclusions We have presented a stability analysis of the ILC problem when the plant Markov parameters are subject to interval uncertainty. It was shown that checking just the vertex Markov matrices of an interval plant is enough to determine the asymptotic stability and the monotonic convergence properties of the interval ILC system. This is a powerful result from a computational perspective. Next we consider the design problem: given an interval Markov matrix H I, find Γ so as to guarantee that the monotonic and asymptotic convergence conditions are satisfied. 53

54 Iteration-Domain Robust Control Outline Overview of ILC Interval ILC Problem 1: Analysis of Maximum Allowable Perturbation Problem 2: Design for Maximum Allowable Perturbation Problem 3: Convergence Analysis Problem 4: Design for Convergence (with Interval Model Conversion) Iteration-Domain H ILC Design 54

55 Iteration-Domain Robust Control Overview In this section we present an approach to: Design the learning gain matrix using the super-vector framework to guarantee the monotone convergence of the output response on the iteration axis For linear plants in which the A-matrix is an interval matrix. To do this, we must convert the interval uncertainty in the A-matrix into an uncertainty in the system Markov parameters. This is called interval model conversion. Then, we introduce the idea of interval matrix eigenpair bounds. From the interval matrix eigenpair bounds we can then deduce monotonic convergence conditions, which can then be used for robust design. 55

56 Iteration-Domain Robust Control Interval Model Conversion Consider the nominal single-input, single-output discrete-time system model given in state space form: x(t + 1) = Ax(t) + Bu(t) y(t) = Cx(t), where A, B, and C are the matrices describing the system in the state space; and x(t), u(t), and y(t) are the state, input, and output variables, respectively. The system Markov parameters are calculated by h k = CA k 1 B. These Markov parameters form the Markov matrix, denoted H, in the usual way. Suppose that A is subject to interval model uncertainty according to A [A, A] and denote this interval matrix as A I. Given A, B, and C, in particular A [A, A], the interval model conversion is defined as finding h k and h k such that h k [h k, h k ], where the underline ( ) represents the minimum value of ( ) and the overline ( ) represents the maximum value of ( ). 56

57 Iteration-Domain Robust Control Basic Definitions The n n interval matrix set A I is defined as { } A I = A = [a ij : a ij [a ij, a ij ]] The nominal matrix A 0 is defined to be [ A 0 = a 0 ij : a 0 ij = a ] ij + a ij 2 where a ij and a ij are the lower and upper bounds associated with the elements a ij of A I. The interval perturbation matrix set A I is defined as { ]} A I = A = [ a ij : a ij = a ij a 0 ij = a 0 ij a ij where a ij, a ij, and a 0 ij are the lower bounds, upper bounds, and nominal values associated with the elements a ij of A I, respectively. Note that we can also write A I = { A : A = A A 0, A A I} 57

58 Iteration-Domain Robust Control Definitions (cont.) and Assumptions The vertex matrix set A v is a subset of the interval matrix set A I that is defined as: { } A v = A A I : a ij {a ij, a ij } Next, to proceed we make two assumptions. The first is a technical assumption. Although we conjecture that this assumption can be relaxed, it is easiest to proceed with the following requirement. The second is more practical, as will be described in a later remark. Assumption 1 Any matrix A A I is diagonable and can be decomposed as: A = XΛX 1, with Λ = diag(λ i ), where λ i are the eigenvalues of A. Assumption 2 Any matrix A A I is Schur stable. 58

59 Iteration-Domain Robust Control Power of Interval Matrix Throughout this presentation, we call X the left eigenvector matrix, Λ the eigenvalue matrix, and Y := X 1 the right eigenvector matrix. We also define the matrix sets (might not be interval matrices) Λ A = { Λ : A = XΛX 1, A A I} X A = { X : A = XΛX 1, A A I} Y A = { Y = X 1 : X X A } The set of the power k of the interval matrix A is defined as (A I ) k = { A k : A A I}. The set of the generalized power k of the interval matrix set is defined as A I k = { A = XΛ k Y : Λ Λ A, X X A, Y Y A }. Note that clearly (A I ) k A I k. 59

60 Iteration-Domain Robust Control Remarks Remark From a computational perspective, it should be noticed that the boundary of the set (A I ) k can be estimated by multiplying the interval matrices using interval calculation software such as Intlab [1], but the result will be quite conservative as k increases and it requires huge amount of computation. Further, notice that the exact boundary of (A I ) k cannot be calculated mathematically or analytically. Remark On the other hand, since (A I ) k A I k, we can estimate the boundary of the set (A I ) k by estimating the boundary of the set A I k, which is also easily done in Intlab, but which can also be done analytically, as we show below. Specifically, since the boundary of A I k can be estimated by using three different sets: Λ A, X A, and Y A, the lower and upper boundary of (A I ) k can be subsequently estimated. Remark From the fact that the boundary of A I k is estimated from Λ A, X A, and Y A, we observe that if the maximum eigenvalue of A A I is less than 1, for all A k A I k, A k will converge to zero as k because Λ k, Λ Λ A, converges to zero as k. However, if the maximum eigenvalue is bigger than 1, any A k A I k will diverge, hence the Markov parameters become bigger and our bound on the the uncertain interval boundary of h k become bigger as k increases. For this reason we assumed A is stable. MAIN POINT: We can contain our original interval system inside a bigger interval system according to: (A I ) k A I k. Therefore, the remaining work is to estimate the boundaries of Λ A, X A, and Y A and from these estimates to compute bounds on A k. 60

61 Iteration-Domain Robust Control Interval Matrix Eigenpair Bounds and Bounds of Markov Parameters First-order perturbation theory can be used to find an analytical bounds of Λ A, X A, and Y A from A I using two lemmas: Lemma 6 The lower and upper boundary matrices of Λ A are estimated from vertex matrices of A I in real part and imaginary part separately. Lemma 7 The lower and upper boundary matrices of X A and Y A are estimated from vertex matrices of A I in real part and imaginary part separately. Then, the interval boundaries of A k, where A A I, can be bounded using the inequality: (A I ) k A I k, which provides the following relationship: where A k (A I ) k and A k A I k. A k A k A k A k A k Then, the interval boundaries of Markov parameters (ex: h k+1 = CA k B) can be estimated such as: h k+1 = CA k B; h k+1 = CA k B where C, B are constant vectors describing the system (1) and A k are the interval matrices which are lower-bounded and upper-bounded by A k A k A k. 61

62 Iteration-Domain Robust Control Monotone Convergence Condition Assuming that we calculated h I k [h k, h k ] from A I, let us discuss the stability of interval ILC system. The interval Markov matrix is expressed as: h I H I = h I 2 h I h I n h I n 1 h I 1 where n is the total number of the discrete time points in time domain. Let H 0 be the nominal Markov matrix without interval uncertainty, calculated as: [ ] h I i + h I i H 0 =, i = 1,, n 2 The following symbols T 0 and T I are used to represent super-vector ILC: T 0 := I n n H 0 Γ, and T I := I n n H I Γ, where I n n is an n n identity matrix, and Γ is a learning gain matrix. 62

63 Iteration-Domain Robust Control Cont. Make the following substitutions: s 1 ij s 1 ij s 2 ij := a ij if i = j := max{ a ij, a ij } if i j := a ij if i = j s 2 ij := min{ a ij, a ij } if i j, where s 1 ij is i th row and j th column element of matrix S 1 ; s 2 ij is an element of matrix S 2, and a I ij is an element of general interval matrix A I. Lemma 8 Let an interval matrix be given element-wise as A A A, A A I. If β = max{ρ(s 1 ), ρ(s 2 )} < 1, where ρ is the spectral radius, then the interval matrix set A I is Schur stable [2]. The monotonic convergence in 2-norm topology can be checked by Lemma 8 after small modifications. Consider the following error vector update law: E k+1 = (I H I Γ)E k, where the singular value of (I H I Γ) = T I is calculated as: σ ( T I) = ρ [ (T I ) T T I] [ 0 (T I ] ) = ρ T T I 0 63

64 Iteration-Domain Robust Control Cont. So, if the following interval matrix is defined: H I = [ 0 (T I ] ) T T I, 0 the monotonic convergence property of the interval ILC system is then checked by analyzing the Schur stability of H I in the 2-norm topology using the Lemmas above. Monotonic convergence conditions in 1 and norm topologies can also be developed: Lemma 9 Given h I i [h i, h i ], the interval ILC system is monotonically convergent, if I H v Γ k < 1, where k is 1 or ; and H v are vertex Markov matrices. 64

65 Iteration-Domain Robust Control Design of Interval ILC Suggestion 1 Let h I ij be the i th row and j th column elements of H I. If we define a matrix M whose elements are given as: m ij = max{h ij, h ij } then, we solve the following optimization problem to design Γ: min Γ ρ(m) s.t. h k [ h k, h k ] Remark We explain why the matrix M is introduced. Let us define s 1 ij (S 1 = {s 1 ij}) and s 2 ij (S 2 = {s 2 ij}) such as: s 1 ij := h ij if i = j; s 1 ij := max{ h ij, h ij } if i j; s 2 ij := h ij if i = j; s 2 ij := min{ h ij, h ij } if i j Then, since matrix H I is symmetric, S 1 = S 2 ; hence using the fact that ρ(s 1 ) = ρ(s 2 ) and diagonal terms of H I are all zeros, we only check the spectral radius of the matrix composed of the off-diagonal terms of S 1, which is denoted as M above. Therefore, Suggestion 1 is reasonable. 65

66 Iteration-Domain Robust Control Cont. If Lemma 9 is used, another optimization scheme is suggested without constraint: Suggestion 2 If k is 1 or, the following optimization is straightforward. min Γ I H v Γ k. Remark The optimization is to minimize I H v Γ k using Γ, where Γ is a band-fixed learning gain matrix. In a small band size, it is possible that there might not exist an optimization solution such that I H v Γ k < 1. In this case, the band size should be increased until the optimization algorithm finds Γ such that I H v Γ k < 1. Remark Depending on the interval ILC system, the optimization scheme suggested above may not find the optimization solution even with the fully populated learning gain matrix. In this case, the following control update law could be used: U k+1 = Q(U k + ΓE k ), where Q is a time-invariant diagonal matrix. Then, since the error vector is updated by the following formula: E k+1 = Q(I HΓ)E k + (I Q)Y d it is easy to make Q(I HΓ) < 1 by Q and Γ. However, the remaining term (I Q)Y d makes a non-zero steady-state error. This is a trade-off. 66

67 Iteration-Domain Robust Control Example Let us assume that the following simple discrete servo system was given from a continuous system: x 1 (k + 1) = a 11 x 1 (k) + a 12 x 2 (k) + b 1 u(k) x 2 (k + 1) = a 21 x 1 (k) + a 22 x 2 (k) + b 2 u(k) y(k) = c 1 x 1 (t) + c 2 x 2 (k), where b 1 = 2, b 2 = 0.5, c 1 = 1, c 2 = 0, interval parameters are bounded as: 0.74 a , 0.53 a , 0.95 a , and 0.19 a , and u is the control force. For this system we have A 0 and A I given by: A 0 = [ ] ; A = A = [ ] where A is the matrix composed of the absolute values of A element-wise. 67

68 Iteration-Domain Robust Control Results From the suggested method From Intlab Interval matrix method Norm based method Interval boundaries Error norms l 2 norm random test l 1 norm Markov parameters Trial number Left: Calculated interval uncertain boundaries of Markov parameters. Right: ILC convergence test. 68

69 Iteration-Domain Robust Control Conclusion We have shown how to design a robust iterative learning controller for plants subject to interval uncertainty in their A-matrix. The interval uncertainty of the system plant was converted into the super-vector iterative learning control system (i.e., into an interval Markov matrix). Optimization schemes were suggested based on an interval matrix stability analysis method and a norm-based method. In the case of the norm-based method, the ILC learning gain matrix guaranteed monotonic convergence of the uncertain ILC system with zero steady-state error. However, the interval matrix method only guaranteed monotonic convergence with non-zero steady error. From these results, we conclude that the norm-based method is less conservative than the interval matrix method. However, the norm based method requires much more computational time than the interval matrix method. 69

70 Iteration-Domain Robust Control Experimental Test: Setup Software Terminal board Power module Rotary motion SRV02 70

71 Iteration-Domain Robust Control Impulse Responses and Desired Trajectory Interval ranges Speed (radians/s) Markov parameters Time (seconds)

72 Interv Iteration-Domain Robust Control 2 Speed Markov parameters Achieved results Time (seconds) Y d (t) - Y k (t) Y d - Y k Time (seconds) Trial number 72

73 Iteration-Domain Robust Control Iteration-Domain H ILC Prehistory of automatic control Primitive period Classical control Modern control Classic control Nonlinear control Estimation Robust control Optimal control Adaptive control Intelligent control H_inf Interval Fuzzy Neural Net ILC... H_inf ILC 73

74 Iteration-Domain Robust Control H ILC Yr - D Do C H + H + Y Iteration loop + N For the most part, existing work has focused on ILC design for performance improvement, assuming the plant is iteration-invariant and with external disturbances treated along the time axis. To date, iteration axis robustness has not been treated in a systematic way. A new framework is suggested for robust ILC design assuming both time-varying model uncertainty and iteration-varying external disturbances. Using the super-vector approach, we can easily incorporate iteration-varying disturbances and the system can be analyzed using discrete (iteration axis) frequency domain techniques. 74

75 Iteration-Domain Robust Control Review: Higher-Order ILC Higher-order update: U k+1 = (I D n 1 )U k + (D n 1 D n 2 )U k 1 + +(D 2 D 1 )U k n+2 + (D 1 D 0 )U k n+1 + D 0 U k n +N n E k + N n 1 E k N 1 E k n+1 + N 0 E k n Taking the w-transform of the ILC update equation, combining terms, and simplifying gives (w 1)D c (w)u(w) = N c (w)e(w) where D c (w) = w n + D n 1 w n D 1 w + D 0, and N c (w) = N n w n + N n 1 w n N 1 w + N 0 This can also be written in a matrix fraction as U(w) = C(w)E(w), where C(w) = (w 1) 1 Dc 1 (w)n c (w). For this update law the repetition-domain closed-loop dynamics become: ( ) 1 I G cl (w) = H I + (w 1) C(w)H I (w 1) C(w), = H[(w 1)D c (w) + N c (w)h] 1 N c (w). 75

76 Iteration-Domain Robust Control Higher-order ILC as a MIMO control problem Figure 1-(a) below depicts the general higher-order ILC problem as a MIMO control problem with the plant H. This figure highlights the fact that the ILC process (1) inherently is a relative degree one process; and (2) should have an integrator in order to converge to zero steady-state error. However, it can also be noted that the controller C(w) has relative degree zero. convenient to consider the reformulation shown in Figure 1-(b). This makes it Yd(w) + - E(w) C(w) (w-1) U(w) H Y(w) (a) D (w) I D (w) O Yd(w) - + E(w) C(w) U(w) + H+ H(w) (w-1) + Y(w) (b) Figure 1 76

77 Iteration-Domain Robust Control Problem Formulation The integrator has been grouped with the plant, so that we now define a new plant H p (w) = (w 1) 1 H. We suppose that H is subject to a perturbation such as H = H 0 + H(w) where H(w) represents iteration varying uncertainty in the plant model. The plant is disturbed by a plant input disturbance d I and a plant output disturbance d o. These disturbances and plant perturbation models lead to a standard H problem. Specifically, the design problem for the uncertain ILC system can be formulated as: Problem: Given H p (w) = (w 1) 1 H and Y d (w), find C(w) in Figure 1-(b) such that E k 2 is minimum from the l 2 -bounded disturbances d I and d o and the closed-loop system is stable over all H = H 0 + H(w), with H(w) < ɛ H. 77

78 Iteration-Domain Robust Control Problem Formulation (cont.) Definition In super-vector ILC, the l 2 norm is defined along the iteration axis (i.e., in the w domain). To distinguish this concept from the discrete-time domain, the iteration domain l 2 norm is written as: 2 w and denoted l 2 w. For example, we replace E k 2 in the problem statement by E k 2 w. In the same way, the iteration domain H norm is written as: w Notice that the problem set-up above indicates that the higher order ILC system can be synthesized from the H framework. That is, the minimization problem of E(w) 2 w is translated as the minimization problem of E k 2 w T EW w = sup W k l 2 w W k 2 w such that T EW w < γ (1) where W = [d I, d o ] T, E k 2 w = k= k=0 E(k)T E(k) and W k 2 w = k= W k=0 (k)t W (k). When γ is not fixed, this is an optimal H ILC problem and when γ is fixed this a the sub-optimal H ILC. Note: we assume E k and E(k) have the same meaning. Both represent the iteration trial number. For convenience, we use both symbols. 78