Selected Topics in Iterative Learning Control Research

Transcription

1 Selected Topics in Iterative Learning Control Research Electronics and Computer Science University of Southampton Southampton, SO17 1BJ, UK

2 Contents Constrained ILC Point-to-Point ILC Robust ILC with Experimental Verification A Brief Visit to Nonlinear Model ILC

3 Introduction However, most of the currently known ILC algorithms on assume that the system is unconstrained systems, e.g., no limits on actuator demands. It is, however, possible that such limits exist in some applications. In this section the ILC design problem with general convex input constraints is considered and two algorithms are developed to solve the problem with well defined convergence properties based on a successive projection method.

4 ILC Design with General Convex Input Constraints Consider the discrete linear time-invariant system y = Gu + d, where G is a linear operator. The system is required to track a given signal r(t) defined on a finite duration [0, N] repeatedly under convex input constraints. In practice, the input constraint set Ω could, for example, be of the following forms Ω = {u H : u(t) M(t)} Ω = {u H : λ(t) u(t) µ(t)} Ω = {u H : 0 u(t)}

5 ILC Design with General Convex Input Constraints Successive Projection An iterative algorithm to find a point in the intersection of two closed, convex sets S 1, S 2. k 1 k 3 S 1 k 0 S 1 S 2 k 2 S 2

6 ILC Design with General Convex Input Constraints Successive Projection An iterative algorithm to find a point in the intersection of two closed, convex sets S 1, S 2. Iterative Learning Control To find a point (0, u ) in the intersection of k 1 k 3 S 1 S 1 = {(e, u) : e = r Gu} S 2 = {(e, u) : e = 0} k 0 S 1 S 2 k 2 S 2

7 Successive Projection Theorem Let K 1 H and K 2 H be two closed convex sets in a real Hilbert space H. Define K j = K 1 (resp.k 2 ) when j is odd (resp. even). Then, given the initial guess k 0 H, the sequence {k j } j 0 k j k j 1 = min k K j k k j 1, j 1 is uniquely defined for each k 0 H and, continuously gets arbitrarily close to every point in K 1 K 2 and when K 1 K 2 is empty, the algorithm converges in the sense that k j+1 k j d(k 1, K 2 ) defining the minimum distance d(k 1, K 2 ) between the two sets K 1 and K 2.

8 Interpretation of Constrained ILC Problem The constrained ILC problem is to find the intersection of the following closed, convex sets in H = R N R N : S 1 = {(e, u) H : e = r Gu} S 2 = {(e, u) H : e = 0} under the constraint S 3 = {(e, u) H : u Ω}. This problem is equivalent to finding the intersection of two closed, convex sets K 1 and K 2 K 1 = S 1 S 3 K 2 = S 2

9 Interpretation of Constrained ILC Problem The constrained ILC problem is to find the intersection of the following closed, convex sets in H = R N R N : S 1 = {(e, u) H : e = r Gu} S 2 = {(e, u) H : e = 0} under the constraint S 3 = {(e, u) H : u Ω}. This problem is equivalent to finding the intersection of two closed, convex sets K 1 and K 2 K 1 = S 1 S 3 K 2 = S 2 K 1 = S 1 K 2 = S 2 S 3 Computational algorithm can be obtained by successive projection.

10 Constrained ILC Algorithm 1 Taking K 1 = S 1 S 3 and K 2 = S 2 results in: u k+1 = arg min u Ω { e 2 Q + u u k 2 R } e = r Gu r 3 r 1 (0, u ) r 2 (k 1 ) r 0 (k 0 ) e = 0

11 Convergence of Constrained ILC Algorithm 1 Theorem The Constrained Algorithm 1 achieves monotonic convergence in tracking error norm, i.e., e k+1 e k, k = 0, 1, to the point that is uniquely defined by u s = arg min u Ω e 2 The algorithm has desired convergence properties but considerable computational effort is required to solve the constrained Quadratic Programming (QP) problem. Various solutions are available, e.g., iterative algorithms, and receding horizon methods.

12 Constrained ILC Algorithm 2 Taking K 1 = S 1 and K 2 = S 2 S 3 results in: } NOILC : ũ k+1 = arg min { e 2 u Q + u u k 2 R Clipping : u k+1 = arg min u ũ k+1 u Ω e = r Gu r 1 1 Original Input Clipping Result r 3 (0, u ) r 2(k 1) r 0(k 0) e = 0 u(t) Time

13 Convergence of Constrained ILC Algorithm 2 Theorem The Constrained Algorithm 2 achieves monotonic convergence with respect to performance index J k = Ee k 2 Q + F e k 2 R to the point that is uniquely defined by u { s = arg min Ee 2 Q + F e 2 } R u Ω where E = I G ( G T QG + R ) 1 G T Q, F = ( G T QG + R ) 1 G T Q Compared with the best solution in terms of tracking error norm u s = arg min u Ω e 2, this algorithm provides an approximate solution using simple computations!

14 Numerical Example 1 Example Consider the following second-order plant model s 4 G(s) = s 2 + 5s + 6, which is sampled using a zero-order hold and a sampling time of 0.1 sec. The trial duration is 20 sec and the actuator constraint is u(t) 1, t = 0, 1, 1 Input signal u ref (t) Reference signal r(t) uref (t) and r(t) Time

15 10 1 NOILC Algorithm 1 Algorithm lg ek Trials Comparison of convergence for Example 1.

16 Numerical Example 2 Example Consider the same plant model as Example 1 s 4 G(s) = s 2 + 5s + 6. but with actuator constraint u(t) 0.8, t = 0, 1,. 1 Input signal u ref (t) Reference signal r(t) uref (t) and r(t) Time

17 Algorithm 1 Algorithm 2 Constrained Optimal lg ek Trials Comparison of convergence for Example 2.

18 Conclusions Two algorithms for constrained ILC have been developed that can be computed using successive projection that have well defined convergence properties. Both algorithms can solve the constrained ILC problem but the computational complexity and convergence properties do differ. The ideas and results can apply more widely and, in particular, to the case of continuous-time systems with no change in the abstract form of the algorithm or results.

19 References D. H. Owens and R. P. Jones. Iterative solution of constrained differential/algebraic systems, International Journal of Control, 27(6): , B. Chu and D. H. Owens. Accelerated Norm-Optimal iterative learning control algorithms using successive projection, International Journal of Control, 82(8): , 2009.

20 Point-to-Point ILC

21 Background In this presentation we concentrate on point-to-point movement operations This is motivated by 1. Industrial robotic movement tasks 2. Recent successful use of ILC in stroke rehabilitation here electrical stimulation must extend the patient s arm, but the exact path taken is not significant, rather the final position reached. The ILC framework allows accuracy to be gained by learning from experience over previous trials There are few ILC algorithms in which the repeated operation consists of a more general objective than the tracking of a static pre-defined reference. Those which exist are usually application-specific, such as gas arc welding.

22 ILC... or they 1. do not involve updating the reference which is static throughout, and 2. involve the derivation of explicit algorithms rather than a technique which can be applied to an existing ILC scheme Point-to-point motion control usually involves generation of a reference, then design of a controller to track this signal.

23 Motivation: Stroke Rehabilitation Patient s arm must move through predetermined points though application of electrical stimulation. Points prescribed by physio, inter-point trajectory unimportant Can build up complex movements through selection of points

24 Motivation: Stroke Rehabilitation In this context high value is placed on a control structure which can be used with both standard tracking trajectories and point-to-point tasks use of existing ILC laws which have already been developed and tested with patients adding robustness to these laws

25 3D Rehabilitation Robot

26 ILC A SISO LTI system is considered, and given, on trial k, by x k (t + 1) = Ax k (t) + Bu k (t) y k (t) = Cx k (t) The tracking task is only defined at times 0 and T when the output must equal 0 and r N respectively for all k, where T denotes the task length. To achieve the tracking task, we consider an ILC algorithm of the form u k+1 = u k + Ke k (1) where K is a linear operator which may be non-causal, and e k = r y k where r is the reference. At the end of the k th trial the reference is allowed to change, and is replaced with r k+1, leading to e k = r k+1 y k (2) to produce u k+1 = u k + Ke k (3)

27 ILC The time domain relationships which then arise are y k+1 = y k + GKe k r k+1 y k+1 = r k+1 y k GKe k e k+1 = (I GK) (r k+1 y k ) where G is the plant operator, and I is the identity operator. The monotonic convergence criterion is e k+1 2 = (I GK) (r k+1 y k ) 2 < e k 2 A sufficient condition for monotonic convergence is therefore I GK 2 < 1 (4) together with which equates to r k+1 y k 2 e k 2 = r k+1 y k 2 r k y k 2 1 r k+1 y k 2 r k y k 2 (5)

28 ILC The first condition, (4), is the standard monotonic convergence criterion for a static reference The second condition is r k+1 y k 2 = r k + e k 2 e k 2 (6) where r k = r k+1 r k r k must therefore be chosen from a suitable set of functions with end-points equal to zero for k 2. We consider the set of harmonic sinewaves, since this leads to simplification in how r k is chosen This choice is equivalent to taking the DFT of r k, with components, R k,i, and requiring that they are all real, i.e. Im{ R k,i } = 0 for i = {0, 1,... N 1}

29 Approach formulation Taking the inner product of the updated error and applying Parseval s theorem e k 2 = r k + e k 2 = r k 2 + e k 2 + 2Re r k, e k = 1 N 1 E k,i N 1 R k,i ( R k,i + 2Re{E k,i }) N N i=0 i=0 The differential with respect to R k is r k + e k 2 = 2 N 1 R k,i + 2 R k N N which is minimized using i=0 N 1 i=0 Re{E k,i } R k,i = αre{e k,i } for i = 0, 1... N 1 (7) with α = 1. However, we will consider α (0, 1].

30 Approach formulation cont d The corresponding updated cost on trial k is given by r k + e k 2 = 1 N since 0 < α(2 α) 0 N 1 i=0 E k,i 2 α(2 α) 1 N < e k 2 iff i s.t Re{E k,i } = 0 Note that the reference is being updated using N 1 i=0 (Re{E k,i }) 2 r k+1 = r k + r k (8) which we chose to speed-up learning of the final trajectory. It is always possible to ensure r k + e k 2 e k 2 since N 1 r k + e k 2 e k 2 = α(2 α) 1 (Re{E k,i }) 2 N i=0 < 0 iff i s.t Re{E k,i } = 0 (9)

31 Approach formulation cont d The error norm is given by e k+1 2 = ( r k + e k ) 2 (I GK) e k e k ( r k + e k ) 2 (I GK) e k 2 (10) e k in which the division, and multiplication,, are executed component-wise. The norm error ratio is e k+1 2 e k 2 N 1 ( 1 Ek,i 2 α(2 α) (Re{E k,i }) 2) 1 G ik i 2 N E k,i 2 (11) i=0 = 1 N 1 ( 1 α(2 α) cos 2 ( E k,i ) ) 1 G ik i 2 N i=0

32 Approach formulation cont d We have introduced the multiplier the i th frequency component. 1 α(2 α) cos 2 ( E k,i ) 1 on This multiplier relaxes the monotonic convergence criterion given by (4). From (11) a sufficient condition to produce trial-to-trial error reduction is now ( 1 α(2 α) cos 2 ( E k,i ) ) 1 G ik i 2 < 1 (12) for each frequency, i, so that 1 G ik i < 1 1 α(2 α) cos 2 ( E k,i ) (13) Increasing α reduces the denominator and so provides additional robustness with respect to the constantly modified reference.

33 System Convergence Properties Substitute R k,i = αre{e k,i } into the frequency-transformed ILC algorithm (3) (with E k,i = R k+1,i Y k,i = R k+1,i G iu k,i ). The resulting system is Ȳ k+1,i Ŷ k+1,i ˆR k+1,i = Ξ(i) Ȳ k,i Ŷ k,i ˆR k,i Re{G ik i} R 0,i + Im{G ik i} R 0,i 0 where ˆ and denote Re{ } and Im{ } respectively. (14) Ξ(i) is given by 1 Re{G ik i} (1 α)im{g ik i} (1 α)im{g ik i} Im{G ik i} 1 (1 α)re{g ik i} (1 α)re{g ik i} (15) 0 α 1 α and R k,i = R 0,i.

34 System convergence and robustness The eigenvalues of Ξ(i) are λ 1 = 1 and ( λ 2,3 = 1 α ) ( α ) 2 (1 Re{G ik i}) ± (1 Re{GiK i}) 2 (1 α) (Im{G ik i}) (16) Movement of eigenvalues (16) due to learning factor, α.

35 Proof of convergence Theorem 1. The system (14) converges asymptotically to a constant vector as k. Proof. Let D = diag{λ 1, λ 2, λ 3}, and V contain the corresponding eigenvectors such that V DV 1 represents the matrix diagonalization of Ξ(i). It is straightforward to show that V 1 ˆR k+1,i Re{G ik i} Im{G ik i} 0 ˆR k,i = 0 v 1 v 2 (17) where v 1, v 2 may be complex (and if so occur as a conjugate pair). Then (14) can be rewritten as Ȳ k+1,i Ȳ k,i 0 Ŷ k+1,i = D Ŷ k,i + v 1 (18) v 2 with [ Ȳ k,i, Ŷ ] T ] T k,i, ˆRk,i = V 1 [Ȳk,i Ŷ k,i ˆRk,i.

36 Proof of convergence As k, this converges asymptotically to [ ] T Ȳ 0,i, v 1/(1 λ 2), v 2/(1 λ 3) (19) and it follows that (14) converges asymptotically to Ȳ,i Ȳ 0,i Ŷ,i v = V 1 1 λ 2 (20) v ˆR 2,i 1 λ 3 Here the converged values are given by Ȳ,i = R 0,i (21) Ŷ,i = ξ α + (1 α) Re{G ik i} sec 2 ( G ik i) (22) ˆR,i = Ŷ,i (23) ξ = α (Ŷ0,i + ( Ȳ 0,i R ) ) 0,i tan( GiK i) + (1 α) ˆR 0,iRe{G ik i} sec 2 ( G ik i) (24)

37 Error System Since the plant output and reference both converge asymptotically, the error. Let the error now be redefined to be with respect to the final converged reference value, so that Ê k,i = ˆR,i Ŷk,i and Ē k,i = R,i Ȳk,i. This then gives Ê,i = ˆR,i Ŷ,i = 0 and Ē,i = R,i Ȳ,i = 0. The resulting dynamic error system is Ē k+1,i Êk+1,i = Ξ(i) ˆR k+1,i Ē k,i Ê k,i ˆR k,i (25) with the eigenvalues λ 1 = 1, and λ 2,3 given by (16) governing the asymptotic convergence of the error components to zero, and reference to a fixed value.

38 System Robustness Properties The system robustness can be examined by determining the range of plant uncertainty which may exist such that (14) remains asymptotically stable. Assume the frequency-wise multiplicative plant uncertainty G i = G 0,iM i i = 0, 1,... N 1 (26) where G 0,i is the nominal plant. Insert in the expression for the eigenvalues (16), and find the bound on the region of uncertainty which leads to their lying within the unit circle by setting ( 1 α ) (1 Re{G 0,iK i} 2 ˆM i + Im{G 0,iK i} M ) i ± ς = 1 (27) with ς = ( α ) 2 ( 1 Re{G 0,iK i} 2 ˆM i + Im{G 0,iK i} M ) 2 i ( (1 α) Re{G 0,iK i} M i + Im{G 0,iK i} ˆM ) 2 i (28) This prescribes a region of C in which the uncertainty must lie.

39 System Robustness Properties Hence the solution is the intersection of three sub-regions ( ˆM i Re{G0,iKi} ) 2 ( G 0,iK i 2 + M i + Im{G0,iKi} G 0,iK i 2 1 = (1 α) G 0,iK i 2 ( ˆM i + αre{g0,iki} ) 2 ( 2 G 0,iK i 2 + M i αim{g0,iki} ) 2 (1 α) 2 G 0,iK i 2 (1 α) α 2 = 4 G 0,iK i 2 (1 α) 2 ( ) 2 ( (3α 4) Re{G0,iKi} ˆM i + 2 G 0,iK i 2 + M i (1 α) α 2 = 4 G 0,iK i 2 (1 α) 2 and each represents a circle. ) 2 ) 2 (3α 4) Im{G0,iKi} 2 G 0,iK i 2 (1 α) (29) (30) (31)

40 System Robustness Properties The solution defines the region in which the plant uncertainty is bounded for any given 0 α 1. The transition between the cases α = 0 and α = 1 is shown for the values Re{G 0,iK i} = 0.6, Im{G 0,iK i} = 0.2, and the uncertainty region 30 ˆM i 30, 20 M i 20. Uncertainty region in which the eigenvalues λ 2,3 given by (16) lie in unit circle, with Re{G i K i } = 0.6, Im{G i K i } = 0.2, for a) α = 0.3, b) α = 0.9, c) α = 0.97 and d) α = 0.99.

41 Multiple point-to-point movements Let the reference be specified at a fixed number, M, of sample instants given by I 1, I 2,..., I M with I M = N. Let the prescribed values of the output at these instants be J 1, J 2,..., J M, with J M = r N. It is proposed to choose the initial reference, u, to satisfy the optimisation minimise u 2 2 subject to ΦGu = Γ (32) where the M N matrix Φ is defined by { 1 i = 1, 2... M, p = I i Φ i,p = 0 otherwise and (33) Γ = [ J 1 J 2 J M ] T (34)

42 Multiple point-to-point example Assume T = 10 and I 1 = 300, I 2 = 650, and I 3 = T/T s = 1000 with J 1 = 7, J 2 = 5 and J 3 = 10. Since I 3 > I 2 > I 1, Φ is of the form Φ = (35) so that the term ΦG appearing in (32) is given by ΦG = CA 299 B CAB CB 0 0 CA 649 B CAB CB 0 0 (36) CA 999 B CAB CB and Γ = [ ] T Having performed the optimisation, the corresponding initial reference r 1 = Gu is shown in the results section. (37)

43 Multiple point-to-point tracking cont d Since the reference satisfies the desired point-to-point movement constraints, it can be updated by adding segments which are zero at samples I 1, I 2,... I M. Therefore the single point-to-point update approach can be applied to each inter-point segment of the reference. The update for the p th interval is accordingly given by r p,k+1 = r p,k + r p,k p = 1, 2,... M (38) where r p,k = [ r k (I p 1 + 1)... r k (I ] T p) with I 0 = 1. Likewise, the other signals are given by (39) y p,k = [ y k (I p 1 + 1)... y k (I p) ] T e p,k = [ e k (I p 1 + 1)... e k (I p) ] T r p,k = [ r k (I p 1 + 1)... r k (I p) ] T (40) (41) (42) Since the p th segment has I p I p 1 elements, the reference update (7) is replaced by R p,k,i = αêp,k,i i = 0, 1,... I p I p 1 1 (43)

44 Multiple point-to-point tracking cont d Theorem 2. Application of the ILC control law (3), in combination with (2), (8), and the update law (43), for an M segment point-to-point task, yields a system whose eigenvalues encompass those of the M individual segments. Proof. The i th frequency component of the p th segment, E p,k,i, is associated with the system (25), and its eigenvalues govern the convergence of the error in that segment. To find the contribution of each of the M segments to the overall error, e k, the i th frequency component of e k is written as N 1 E k,i = e k e j2πni/n = n=0 M = p=1 M = p=1 I p I p 1 1 n=0 e j2πi p 1i/N I M p 1 e p,k e j2πni/n p=1 n=i p 1 e p,k e j2π(n+i p 1)i/N I p I p 1 1 n=0 M = e j2πip 1i/N E p,k,i(i p I p 1)/N p=1 e p,k e j2πni/n (44)

45 Multiple point-to-point tracking continued Here the DFT components of the p th segment are given by E p,k,i = I p I p 1 1 n=0 e p,k e j2πni/(ip I p 1) (46) so each frequency component of e k is a sum of frequencies from the M segments. The i th component of e k+1 is e j2πi T 0i/N E 1,k+1,i(I1 I 0 )/N e j2πi 1i/N E 2,k+1,i(I2 I 1 )/N E k+1,i =.. e j2πi M 1i/N E M,k+1,i(I M I M 1)/N (47) Each segment s component is governed by (25), and is given by Ē p,k+1,l Ē p,k,l [ ] Ē Êp,k+1,l = Ξ(l) Êp,k,l p,k+1,l, E p,k+1,l = Ê p,k+1,l (48) ˆR p,k+1,l with l = i(i p I p 1)/N. ˆR p,k,l ˆR p,k+1,l

46 Multiple point-to-point tracking cont d Therefore (47) becomes E k+1,i = e j2πi 0i/N e j2πi 1i/N. e j2πi M 1i/N T ΦX k+1 X k+1 = Λ(i)X k (49) with X k+1 R 3M containing all the error and reference components, and {[ ] [ ] [ ]} Φ = diag,, Λ(i) = diag{ξ(i (I 1 I 0) /N), Ξ(i (I 2 I 1) /N),... Ξ(i (I M I M 1) /N)} (50) As Λ(i) is block diagonal, the eigenvalues of this system are λ (Λ(i)) = { λ (Ξ(i (I 1 I 0) /N)), λ (Ξ(i (I 2 I 1) /N)),... λ (Ξ(i (I M I M 1) /N)) } where λ (W ) represents the set of all eigenvalues of W. (51)

47 Hence the multiple point-to-point system inherits the eigenvalues and associated asymptotic convergence and robustness properties of the single point-to-point systems which govern the behavior of each segment. For the case α = 1 the bound on the error, e k, is r k + e k 2 = 1 N = 1 N < 1 N M r p,k + e p,k 2 ( ) M N 1 (I p I p 1) (Im{E p,k,i }) 2 p=1 i=0 ( ) M N 1 (I p I p 1) E p,k,i 2 p=1 p=1 i=0 < e k 2 iff i, p s.t Ê p,k,i 0 (52)

48 Experimental results From the non-minimum phase test facility. Transfer-function G(s) = (4 s) s s s s (53) The adjoint ILC algorithm is selected as a well known member of the class considered, and is given in discrete form by u k+1 (z) = u k (z) + βg (z)e k (z) (54) where G (z) is the adjoint of the plant model used.

49 Experimental results An attractive feature of the adjoint law is that, with a sufficiently small positive scalar multiplier, β, it is guaranteed to satisfy the condition for monotonic convergence over all frequencies, and hence ensure a satisfactory transient response. The static reference monotonic convergence criterion corresponding to (4) in this case is I βgg 2 < 1 (55) The adjoint ILC algorithm is one of the methods used in the stroke rehabilitation programme described in the introduction. This algorithm has many interesting extensions...! In the following experimental tests, a sampling period of T s = 0.01 sec has been used.

50 Experimental results: reference and output evolution We start with a single point-to-point trajectory (M = 1, T = 5.12s, r N = 12rad I 1 = 512, J 1 = r N ) Figure 1: Objective-driven ILC using the adjoint algorithm with β = 0.7, α = 1, showing a) reference evolution, b) plant output and c) error norm e k.

51 Experimental results: error norm and total point-to-point error Figure 2:Point-to-point error for objective-driven ILC using the adjoint algorithm with β = 0.7, α = 1.

52 Experimental results: reference and output evolution Move onto the same multiple point-to-point trajectory as previously defined Figure 3: Multiple objective-driven ILC using the adjoint algorithm with β = 0.6 and α = 1, showing a) reference evolution, and b) plant output.

53 Experimental results: error norm and total point-to-point error The total error at the three points comprising the point-to-point movement is given by 3 p=1 Jp y k(i p) Figure 4: Multiple objective-driven ILC using the adjoint algorithm with β = 0.6 for a range of α, showing a) error norm e k, and b) total point-to-point error.

54 Experimental results: summary Improvement in tracking of the points (I p, J p) is evident. Use of the proposed method results in faster convergence to the desired points, as well as increased accuracy in attaining them, compared with using a static reference. The results also indicate that the relaxation factor provides additional robustness since less fluctuation is present in the error when higher values of α are used. There is also no significant reduction in convergence speed associated with values of α = 0.5 and α = 0.75 compared with α = 1.

55 Conclusions A novel method of applying ILC to point-to-point movements has been developed and is applicable to a broad class of linear ILC algorithms. The approach is based upon updating the reference between successive trials, and its convergence and robustness properties have been examined both theoretically and experimentally. The technique s ability to vary the reference from trial to trial distinguishes it from traditional point-to-point motion strategies such as input Shaping. This also separates it from almost all previous applications of ILC to point-to-point motion control, which also assume a static reference. Moreover, whilst also providing the ability to learn from experience gained over previous trials of the task, the proposed scheme also has the benefit of being suitable for application to a broad class of existing ILC laws.

56 Future work The approach will be applied to current research in the area of stroke rehabilitation, to assist patients in the performance of reaching tasks with their impaired arms using electrical stimulation. Extend the technique to allow variation in the time-points at which the movement attains the prescribed output values. Extension of the approach for application to non-linear systems will then be formulated, with the addition of other constraints which govern the manner in which the task is performed.

57 References C. T. Freeman, Z. Cai, E. Rogers, C. T. Freeman, P. L. Lewin Iterative Learning Control for Multiple Point-to-Point Tracking Application IEEE Transactions on Control Systems Technology, 19(3): , C. T. Freeman Constrained point-to-point iterative learning control with experimental verification Control Engineering Practice, 20(5): , 2012.

58 Robust ILC with Experimental Verification

59 Adjoint Robustness SISO plant model in lifted form (CB 0) y k = G eu k (56) where G e = CB CAB CB CA N 1 Γ CA N 2 Γ CA N 3 b... 0 (57)

60 Analysis Standard steepest-descent algorithm cost function Use J(u k+1 ) = e k+1 2, e k+1 = r G eu k+1 (58) u k+1 = u k + ɛ k+1 δ k+1 (59) where ɛ k+1 is a scaling factor and δ k+1 is the vector that determines the direction of the update vector. Resulting tracking error J(u k+1 ) = J(u k + ɛ k+1 δ k+1 ) = e k+1 2 = e k 2 2ɛ k+1 δ T k+1g T e k + ɛ 2 k+1δ T k+1g T e G eδ k+1 (60) or e k+1 2 e k 2 = 2ɛ k+1 δ T k+1g T e e k + ɛ 2 k+1δ T k+1g T e G eδ k+1 (61)

61 Analysis For monotonic convergence, the right-hand side in (61) must be negative. One way to achieve this is to take δ k+1 = G T e e k, resulting in the control law u k+1 = u k + ɛ k+1 G T e e k (62) and the difference e k+1 2 e k becomes e k+1 2 e k 2 = 2ɛ k+1 G T e e k 2 + ɛ 2 k+1 G eg T e e k 2 (63) and because the negative term ɛ k+1 G T e e k 2 is of O(ɛ) and the positive term ɛ 2 k+1 G eg T e e k 2 is of O(ɛ 2 ), by using a sufficiently small positive ɛ k+1 the right-hand side of (63) can be made negative (note that G e was assumed to be invertible), resulting in monotonic convergence. In order to automate the selection process for ɛ k+1, it was suggested that ɛ k+1 is taken to be the solution of the following optimization problem

62 Analysis The optimal ɛ k+1 is given by and hence ɛ k+1 = arg min ɛ k+1 R J(u k + ɛg T e e k ) (64) ɛ k+1 = GT e e k 2 G eg T e e k 2 (65) e k+1 2 e k 2 = GT e e k 4 (66) G eg T e e k 2 where the right-hand side is negative, implying monotonic convergence to zero tracking error. If if the original plant is positive-definite in the sense that G e + G T e is a positive-definite matrix, the simple control law u k+1 (t) = u k (t) + γe k (t + 1) (67) will result in convergence if γ is selected to be sufficiently small. In order to automate the selection of γ a parameter-optimal ILC algorithm can be used.

63 Analysis Assume that the true plant contains multiplicative uncertainty, which is modeled with the equation G e = G ou. In this equation G o is the nominal model (i.e. an estimate of the true plant), and U reflects the multiplicative uncertainty (i.e. modeling errors), and G o is used instead of G e in the update law, i.e., u k+1 = u k + ɛ k+1 G T o e k (68) In the presence of uncertainty e k+1 2 e k 2 becomes e k+1 2 e k 2 = 2ɛ k+1 e T k G ou T G T o e k +ɛ 2 k+1e T k G og T e G eg T o e k (69)

64 Analysis Lemma Suppose that U + U T is a positive-definite matrix. If e k = 0 there exists an ɛ k+1 > 0 such that e k+1 2 e k 2 < 0 In the standard steepest-descent algorithm, ɛ k+1 is given by ɛ k+1 = GT o e k 2 G og T o e k 2 (70) and hence there is no clear mechanism to modify ɛ k+1 so that it would be sufficiently small in the above result. u k+1 = u k + ɛ k+1 G T e e k (71) where ɛ k+1 is selected to be the solution of the optimisation problem min ɛk+1 R J(ɛ k+1 ) J(ɛ k+1 ) := e k wɛ 2 k+1 (72) where w R, w > 0.

65 Analysis The cost function J(ɛ k+1 ) in (72) reflects two design objectives. The first term in J(ɛ k+1 ) reflects the objective that the tracking error should be small during each trial. The second term, on the other hand, tries to keep the magnitude of ɛ k+1 small, possibly resulting in a more cautious and robust algorithm when compared to the standard steepest-descent algorithm. The optimal solution is ɛ k+1 = G T e e k 2 w + G eg T e e k 2 (73)

66 Analysis Lemma If w R, w > 0 then e k+1 e k,where equality holds if and only if ɛ k+1 = 0. Furthermore, lim k e k = 0 and lim k ɛ k = 0 (74) demonstrating monotonic convergence to zero tracking error. Lemma Assume that U + U T is positive-definite and w is chosen so that w > 1 G T o 2 G og T e 2 e σ min(g oug T o ) (75) where σ min(g oug T o ) is the smallest singular value of the positive-definite matrix G oug T o. In this case, the sequence of tracking errors satisfies e k+1 < e k when e k 0.

67 Analysis Note that the estimate for w can be very conservative, because the term e T k G oug T o e k is estimated in terms of the smallest singular value of G oug T o. Furthermore, if w is selected to be excessively large in magnitude, this can have a undesirable effect on the convergence speed, because a large w will result in a small ɛ k+1, implying that u k+1 u k in such a case. Consequently this result should be taken as an existence result, and in practice w can be selected by resorting to a trial and improvement approach. Note that the sufficient value of w decreases as e k 2 decreases. This opens up the potential to reduce w with each successive iteration k and a reduced w will possibly result in an increase in the convergence speed. A natural choice to exploit this would be w = w 1 e k 2, however, as e k 2 approaches zero dangerously high inputs could be applied to the plant. A simple remedy would be to choose w = w 0 + w 1 e k 2.

68 Analysis The next result shows that in addition to monotonic convergence, U T + U > 0 also implies that lim k e k = 0 if w is selected to be sufficiently large. Lemma Under the assumptions of the previous result. the algorithm converges monotonically to zero tracking error. For monotonic convergence to zero tracking error it is required that the multiplicative uncertainty U has to be positive in the sense that U T + U is a positive-definite matrix. However, in most applications the nominal matrix model G o is obtained by truncating the lifted transfer function zg o(z), where G o(z) is the transfer function of the nominal plant model when time-axis is infinite. Therefore it is important to understand this positivity result in terms of these transfer-function models.

69 Analysis Using the transfer-function descriptions of the plant, the uncertainty model is zg(z) = U(z)G o(z) or U(z) = G(z) 1 G o(z) (76) Furthermore, it can be shown that if U(z) is a positive-real system (or equivalently, its Nyquist diagram lies strictly in the right-half plane,), the truncated system (matrix) U is positive, i.e., U T + U is a positive-definite matrix. The Nyquist-diagram condition, on the other hand, is equivalent to the condition that phase of the uncertainty model U(z) has to lie inside ±90 degrees, demonstrating a good degree of robustness.

70 Experiments Using the gantry robot, specific features of the algorithm have been investigated, including: learning performance compared to both a PIE controller and a simple P-type ILC controller robustness to plant modeling error robustness to initial state error The dimensions of the G o matrices is determined by the sampling frequency and the time period of one iteration. For the robust optimal algorithm, the selected sampling frequency is very low at only 100 Hz. This choice is justified by a number of factors: In velocity control mode, the linear motor amplifiers operate with their own closed-loop control. The input to the amplifiers is therefore simply a setpoint adjustment and is not directly required to achieve a stable system.

71 Experiments A low sample frequency minimizes the sizes of the G o matrices implying that less memory and computation time are required to generate the next input to the plant. The transfer-function models for the axes were reduced to a second order transfer=function with similar low frequency gain, but none of the high order dynamics. The first trial was set at zero input for all sample intervals, implying that the algorithm must learn the correct input from nothing. G x(s) = 11 s(s + 200) (77) G y(s) = G z(s) = 17 s(s + 300) 10 s(s + 400) (78) (79)

72 Experiments The log mise results in the next figure show that the simplification of the model has little effect on the convergence rate and minimum error, the algorithm also remains stable. The difference is most noticeable for the X-axis where the performance of the low order model is noticeably worse that of the high order model. The X-axis has the most significant high frequency dynamics and so is most affected by the simplification process. Stability theory suggests that a model of the form 1/s is adequate to meet the stability conditions for any plant which has a phase between 0 and -180 degrees. One simple way of testing this would be to use a model of the form 1/s during the controller design process, then implement this controller in practice. The model 1/s satisfies the requirements for phase, but the gain is also likely to have a significant effect on stability. It is therefore more logical to consider a model β/s where β is the model gain.

73

74 Experiments G 0(s) = β (80) s This model has been implemented using a selection of values for β to investigate the effect of the model gain on stability and convergence. The values used were 1.00, 0.10, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04, 0.03, 0.02, Comparing the data for β equal to 1.00, 0.05 and 0.03, confirms that with a large value for β the rate of learning is slower. As β is reduced, the learning steadily becomes faster until an optimum β is reached. Increasing β any further then rapidly reduces the learning rate and very quickly the system becomes unstable. The best values of β for each axis were found to be 0.05 for the X and Y -axes and 0.03 for the Z-axis. Generating Bode plots for 0.05/s and 0.03/s and comparing these plots to the high order models for each axis reveals why these values for β are best. They generate a model gain which is equal to the plant gain at low frequency.

75 Experiments Set β to a large value (if any form of plant model is available ensure that β generates a gain which is greater than the gain of the model at low frequency). Operate the system and ensure that stability is achieved. Reduce β. Continue steps 2 and 3 until optimal convergence is achieved or until the system begins to destabilize. If the system begins to destabilize, increase β slightly. β is set. The results and conclusions of further experimental investigations on the performance of this algorithm are given in the cited reference.

76 Trial 4000, reference and output.

77 References J. D. Ratcliffe, J. J. Hätönen, P. L. Lewin, E. Rogers and D. H. Owens Robustness analysis of an adjoint optimal iterative learning controller with experimental verification International Journal of Robust and Nonlinear Control, 18(10): , D. H. Owens and E. Rogers (Guest Editors) Robust Iterative Learning Control: Theory and Experimental Verification International Journal of Robust and Nonlinear Control, 18(10), 2008.

78 A Brief Visit to Nonlinear Model ILC

79 Nonlinear ILC Nonlinear ILC has received a very large amount of attention in the literature. Arguably far too much attention on trial-to-trial error convergence proofs, all based on the same idea just different assumptions. In this section the background to methods that have gone beyond convergence proof are briefly considered. The application of one such method Newton ILC is critical to the stroke rehabilitation research covered in the next part of the course. Nonlinear systems can, in general terms, be split into two groups those that are affine in the control and those that are not. Those that are affine in the control are assumed to be in the following form

80 Nonlinear ILC ẋ(t) = f(x(t)) + B(x(t))u(t) y(t) = g(x(t)) (81) where x is the state vector, u is the input and y is the output. A special case is M r(x)ẍ C r(x, ẋ)ẋ g r(x) d r(x, ẋ) = τ (82) where for robotics the vectors x, ẍ, ẋ are the positions, velocities and accelerations of the link, τ is the torque input, M r(x) is the symmetric positive-definite matrix of link inertias, C r(x, ẋ) is the Coriolis and centripetal acceleration matrix, g r(x) is the gravitational force vector and d r(x, ẋ) is the friction torque vector. ILC for affine nonlinear systems uses a wide variety of algorithms but a critical common assumption is that the nonlinear system is smooth.

81 Nonlinear ILC This last requirement is often expressed as a global Lipshitz assumption on each of the functions in (81). These take the form f(x 1) f(x 2) f 0 x 1 x 2 B(x 1) B(x 2) b 0 x 1 x 2 g(x 1) g(x 2) g 0 x 1 x 2 (83) The constants f 0, b 0 and g 0 are used in a contraction mapping setting to obtain (sufficient) conditions for trial-to-trial error convergence and ILC law design. Nonaffine systems have the form ẋ(t) = f(x(t)) + B(x(t), u(t)) y(t) = g(x(t)) (84)

82 Nonlinear ILC ILC has also been extended to nonlinear discrete-time systems. Obtaining discrete-time models for nonlinear systems is sometimes non-trivial but, for example, trial-to-trial error convergence proofs are simpler and the final design is directly compatible with digital implementation. One other way is to study and design ILC for nonlinear systems is to treat the nonlinearities as perturbations to a linearized system model.

83 Newton ILC Use the full model in the computation of the next trial input. State-space model x k (t + 1) = f(x k (t), v k (t)) q u,k (t) = h(x k (t)) where t [0, 1, 2,..., N 1] is the sample number, x k (t) is the state vector, and N = T/T s + 1 with T s the sampling frequency. Introduce the vectors v k = [v k (0) T, v k (1) T,..., v k (N 1) T ] T q u,k = [q u,k (0) T, q u,k (1) T,..., q u,k (N 1) T ] T (85)

84 Newton ILC Also the reference vector q u = [q u(0) T, q u(1) T,..., q u(n 1) T ] T (86) The Newton method based ILC update takes the form where e k = q u q u,k is the tracking error. v k+1 = v k + g (v k ) 1 e k (87) The term g (v k ) is equivalent to the system linearization around v k, with the system q u = g (v k )ṽ corresponding to the following linear time-varying, denoted LTV, system

85 Newton ILC with x(t + 1) = A(t) x(t) + B(t)ṽ(t) q u(t) = C(t) x(t) + D(t)ṽ(t) t = 0, 1,..., N 1 ( ) f A(t) = x ( ) h C(t) = x v k (t),x k (t) ( ) f, B(t) = v ( k ) h, D(t) = v k (t),x k (t) v k v k (t),x k (t) v k (t),x k (t) (88)

86 Newton ILC The term g (v k ) 1 in (87) is computationally expensive and may be singular or contain excessive amplitudes and high frequencies. To overcome this difficulty, introduce e k = g (v k ) v k+1 (89) and then v k+1 = v k+1 v k equals the input that forces the LTV system (88) to track the error e k. This is itself an ILC problem and can be solved in between experimental trials using any ILC algorithm that converges globally. In this article norm optimal ILC is considered, with the input and output on trial j denoted by e k,j and v k+1,j, respectively.

87 Newton ILC On trial j + 1, the trade-off between minimizing the tracking error, e k e k,j, and the change in control input, v k+1,j+1 v k+1,j, is represented by the cost function N 1 J j+1 = (e k e k,j )(t) T Q(e k e k,j )(t)+ t=0 N 1 ( v k+1,j+1 v k+1,j )(t) T R( v k+1,j+1 v k+1,j )(t) t=0 The ILC computation is stopped after 100 trials or after the error reaches a preset threshold. The input obtained, v k+1,j, is then used to approximate v k+1 in (87) to generate the control input for the next trial. Actual results from clinical trials with stroke patients where Newton ILC is used are given in the last section of this course.