On the History, Accomplishments, and Future of the Iterative Learning Control Paradigm

Transcription

1 Control Paradigm On the History, Accomplishments, and Future of the Control Paradigm Presenter: Contributors: Kevin L. Moore G.A. Dobelman Distinguished Chair and Professor of Engineering Division of Engineering Colorado School of Mines egweb.mines.edu/faculty/kmoore/ YangQuan Chen Utah State University Hyo-Sung Ahn ETRI, Korea presented at the 2006 International Symposium on Advanced Robotics and Machine Intelligence Beijing, China, 10 October 2006 in honor of Professor Suguru Arimoto s Achievements in Academic Research and Education 1

2 Control Paradigm Introduction Outline Control System Design: Motivation for ILC Control: The Basic Idea On the History and Accomplishments of ILC Professor Arimoto s ILC Contributions An ILC Framework for the Past, Present, and Future General Operator-Theoretic Framework The Supervector Notation for Discrete-Time ILC The w-transform: z-operator Along the Repetition Axis ILC as a MIMO Control System: Repetition-Domain Poles and Internal Model Principle The Complete Framework: Iteration-Domain Uncertainty Concluding Remarks: Future Research Vistas in ILC 2

3 Control Paradigm Control Design Problem Reference Error Input System to be Output controlled Given: Find: Such that: System to be controlled. (using feedback). 1) Closed-loop system is stable. 2) Steady-state error is acceptable. 3) Transient response is acceptable. 3

4 Control Paradigm Motivation for the Problem of Control Transient response design is hard: 2.5 1) Robustness is always an issue: - Modelling uncertainty. - Parameter variations. - Disturbances. 2) Lack of theory (design uncertainty): Relation between pole/zero locations and transient response. - Relation between Q/R weighting matrices in optimal control and transient response. - Nonlinear systems. Many systems of interest in applications are operated in a repetitive fashion. Control (ILC) is a methodology that tries to address the problem of transient response performance for systems that operate repetitively

5 Control Paradigm Systems that Execute the Same Trajectory Repetitively Step 1: Robot at rest, waiting for workpiece. Step 2: Workpiece moved into position. Step 3: Robot moves to desired location Step 4: Robot returns to rest and waits for next workpiece. 5

6 Control Paradigm Errors are Repeated When Trajectories are Repeated A typical joint angle trajectory for the example might look like this: Each time the system is operated it will see the same overshoot, rise time, settling time, and steady-state error. learning control attempts to improve the transient response by adjusting the input to the plant during future system operation based on the errors observed during past operation. 6

7 Control Paradigm Control Standard iterative learning control scheme: u k System y k Memory Memory Memory u k+1 y d A typical ILC algorithm has the form: u k+1 (t) = u k (t) + γe k (t + 1). Standard ILC assumptions include: Stable dynamics or some kind of Lipschitz condition. System returns to the same initial conditions at the start of each trial. Each trial has the same length. 7

8 Control Paradigm Example 1 Consider the plant: We wish to force the system to follow a signal y d : y(t + 1) =.7y(t).012y(t 1) + u(t) y(0) = 2 y(1) = 2 8

9 Control Paradigm Example 1 (cont.) Use the following ILC procedure: 1. Let 2. Run the system 3. Compute 4. Let 5. Iterate u 0 (t) = y d (t) e 0 (t) = y d (t) y 0 (t) u 1 (t) = u 0 (t) + 0.5e 0 (t + 1) Each iteration shows an improvement in tracking performance (plot shows desired and actual output on first, 5th, and 10th trials and input on 10th trial). 9

10 Control Paradigm Example 1 (cont.) 10

11 Control Paradigm Example 2 Consider a simple two-link manipulator modelled by: where A(x k )ẍ k + B(x k, ẋ k )ẋ k + C(x k ) = u k x(t) = (θ 1 (t), θ 2 (t)) ( T ) cos θ cos θ A(x) = cos θ ( ).135 sin θ2 0 B(x, ẋ) =.27 sin θ 2.135(sin θ 2 ) ( θ 2 ) sin θ sin(θ C(x) = 1 + θ 2 ) sin(θ 1 + θ 2 ) u k (t) = vector of torques applied to the joints g m 1 θ 2 l 1 l 2 m 2 θ 1 l 1 = l 2 = 0.3m m 1 = 3.0kg m 2 = 1.5kg 11

12 Control Paradigm Example 2 (cont.) Define the vectors: y k y d = (x T k, ẋ T k, ẍ T k ) T = (x T d, ẋ T d, ẍ T d ) T The learning controller is defined by: u k = r k α k Γy k + C(x d (0)) r k+1 α k+1 = r k + α k Γe k = α k + γ e k m Γ is a fixed feedback gain matrix that has been made time-varying through the multiplication by the gain α k. r k can be described as a time-varying reference input. r k (t) and adaptation of α k are effectively the ILC part of the algorithm. With this algorithm we have combined conventional feedback with iterative learning control. 12

13 Control Paradigm Example 2 (cont.) 13

14 Control Paradigm Introduction Outline Control System Design: Motivation for ILC Control: The Basic Idea On the History and Accomplishments of ILC Professor Arimoto s ILC Contributions An ILC Framework for the Past, Present, and Future General Operator-Theoretic Framework The Supervector Notation for Discrete-Time ILC The w-transform: z-operator Along the Repetition Axis ILC as a MIMO Control System: Repetition-Domain Poles and Internal Model Principle The Complete Framework: Iteration-Domain Uncertainty Concluding Remarks: Future Research Vistas in ILC 14

15 Control Paradigm Control Engineering - History and ILC Prehistory of automatic control Primitive period Classical control Modern control Classic control Nonlinear control Estimation Robust control Optimal control Adaptive control Intelligent control H_inf Interval Fuzzy Neural Net ILC... 15

16 Control Paradigm On the History and Accomplishments of ILC United States Patent 3,555,252 Control of Actuators in Control Systems, filed 1967, awarded 1971, learned characteristics of actuators and used this knowledge to correct command signals. J. B. Edwards. Stability problems in the control of linear multipass processes. Proc. IEE, 121(11): , M. Uchiyama. Formulation of high-speed motion pattern of a mechanical arm by trial. Trans. SICE (Soc. Instrum. Contr. Eng.), 14(6): (in Japanese), S. Arimoto, S. Kawamura, and F. Miyazaki. Bettering operation of robots by learning. J. of Robotic Systems, 1(2): ,

17 Control Paradigm Edwards, Proc. IEE (1974) 17

18 On the History, Accomplishments, and Future of the Control Paradigm First ILC paper- in Japanese (1978) 18

19 Control Paradigm First ILC paper- in English (1984) 19

20 Control Paradigm ILC Research History ILC has a well-established research history: More than 1000 papers: 90 Number of publication Journal papers Conference papers Year At least four monographs Over 20 Ph.D dissertations But, ILC is still relatively young in ieeexplore.ieee.org, using the search term (in all fields): learning control generates 465 papers Robust control generates 10,075 papers Adaptive control generates 11,654 papers 20

21 Control Paradigm Robots Robots ILC Research History (cont.) Rotary systems Rotary systems Process control Miscellaneous Process control Bio-applications Semiconductors Miscellaneous Bio-applications Actuators Power systems Semiconductors Actuators Power systems Miscellaneous Typical By application areas Structure Update rule Robust Optimal Mechanical nonlinearity Adaptive Neural ILC for control By theoretical areas. 21

22 Control Paradigm Selected ILC Industrial Applications ILC patents in hard disk drive servo: YangQuan Chen s US6,437,936 Repeatable runout compensation using a learning algorithm with scheduled parameters. YangQuan Chen s US6,563,663 Repeatable runout compensation using iterative learning control in a disc storage system. Robotics: Michael Norrlöf s patent on ABB robots. US Path correction for an industrial robot. 22

23 Control Paradigm Introduction Outline Control System Design: Motivation for ILC Control: The Basic Idea On the History and Accomplishments of ILC Professor Arimoto s ILC Contributions An ILC Framework for the Past, Present, and Future General Operator-Theoretic Framework The Supervector Notation for Discrete-Time ILC The w-transform: z-operator Along the Repetition Axis ILC as a MIMO Control System: Repetition-Domain Poles and Internal Model Principle The Complete Framework: Iteration-Domain Uncertainty Concluding Remarks: Future Research Vistas in ILC 23

24 Control Paradigm ILC Problem Formulation Standard iterative learning control scheme: u k System y k Memory Memory Memory u k+1 y d Goal: Find a learning control algorithm u k+1 (t) = f L (previous information) so that for all t [0, t f ] lim y k(t) = y d (t) k 24

25 Control Paradigm Professor Arimoto s Early Contributions Professor Arimoto s Six Postulates of ILC: P1. Every trial (pass, cycle, batch, iteration, repetition) ends in a fixed time of duration T > 0. P2. A desired output y d (t) is given a priori over [0, T ]. P3. Repetition of the initial setting is satisfied, that is, the initial state x k (0) of the objective system can be set the same at the beginning of each iteration: x k (0) = x 0, for k = 1, 2,. P4. Invariance of the system dynamics is ensured throughout these repeated iterations. P5. Every output y k (t) can be measured and therefore the tracking error signal, e k (t) = y d (t) y k (t), can be utilized in the construction of the next input u k+1 (t). P6. The system dynamics are invertible, that is, for a given desired output y d (t) with a piecewise continuous derivative, there exists a unique input u d (t) that drives the system to produce the output y d (t). Professor Arimoto proposed a learning control algorithm of the form: Convergence is assured if I CBΓ i < 1. u k+1 (t) = u k (t) + Γė k (t) Professor Arimoto also considered more general algorithms of the form: u k+1 = u k + Φe k + Γė k + Ψ e k dt 25

26 Control Paradigm Professor Arimoto s Papers and Citations Based my comments on 45 papers from my personal file and on IEEE Xplore and Web of Science Citation searches. But, my data is not exhaustive! Categorize ILC contributions as (roughly chronologically) Foundations Robotics Foundations Mobile Robots Robustness Higher-Level Under Geometric Constraints Gripping and Multiple Manipulators Passivity and Impendance In particular, the article Bettering Operation of Robots by, Journal of Robotic Systems, Vol.1, No. 2, , 1984, is seminal (thanks to Dr. YangQuan Chen of Utah State University, USA, for compiling this information): 474 Citations noted by Google Scholar 357 citation noted by Wed of Science 26

27 Control Paradigm When Where Statistics compiled by Prof. YangQuan Chen of Utah State University, USA 27

28 Control Paradigm Who Statistics compiled by Prof. YangQuan Chen of Utah State University, USA 28

29 Control Paradigm Foundations Foundations Can mechanical robots learn by themselves 2nd Int. Symp on Robotics Res, Aug learning control for robotic systems IECMO, Oct Bettering operation of dynamic systems by learning: a new control theory for servomechanism or mechatronics systems, CDC, Dec Bettering operation of robots by learning Journal of Robotic Systems, Mathematical theory of learning with applications to robot control 4th Yale Workshop on Applications of Adaptive Systems Theory, May control theory for dynamic systems CDC, Dec

30 Control Paradigm Foundations Robotics Foundations Robotics Foundations Hybrid position/force control of robot manipulators based on learning method Int Conf on Adv Robotics, Sept Applications of learning method for dynamic control of robotic manipulators CDC, Dec Convergence, stability, and robustness of learning control schemes for robot manipulators, Elsevier book chapter, control scheme for a class of robot systems with elasticity CDC Intelligent control of robot motion based on learning method, IEEE ISIC, Realization of robot motion based on a learning method IEEE Trans. SMC, Jan

31 Control Paradigm Foundations Robotics Foundations Mobile Robotics Mobile Robots and Others Visual Control of Autonomous Mobile Robot Based on Self-Organizing Model for Pattern Journal of Robotic Systems, Oct Self-organizing model of learning and its application to eyesight system of robot IEEE Conf on AI for Industrial App, May A navigation scheme with learning for a mobile robot among multiple moving obstacles IROS, Jul

32 Control Paradigm Foundations Robotics Foundations Robustness Robustness Mobile Robotics Robustness of learning control for robot manipulators ICRA, May Robustness of P-type learning control with a forgetting factor for robotic motions CDC, Dec Selective learning with a forgetting factor for robotic motion control ICRA, Apr Experimental studies on robustness of a learning method with a forgetting factor for robotic motion control ICAR, Jun

33 Control Paradigm Foundations Robotics Foundations Mobile Robotics Higher-Level for Skill Refinement in Robotic Systems IEICE Trans on Fund. Of ECCS, Feb for skill refinement IROS, Nov Strategy generation and skill acquisition for Robustness automated robotic assembly task IEEE ISIC, Aug Task Strategies in Robotic Assembly Systems Robotica, Sep 1992 Higher-Level

34 Control Paradigm Foundations Robotics Foundations Mobile Robotics Robustness Higher-Level Under Geometric Constraints

35 Control Paradigm Under Geometric Constraints Foundations control for robot tasks under geometric endpoint constraints ICRA, May control for geometrically constrained robot manipulators IROS, Jul 1993 Robotics Foundations Robustness Control for Robot Motion under Geometric End-Point Constraint Robotica, Mar control for robot tasks under geometric constraints ICRA, May 5. control for robot tasks under geometric endpoint constraints IEEE Trans. Robotics/Automation, June A learning control method for coordination of multiple manipulators holding a geometrically constrained object Adv Robotics, 1999 Under Geometric Constraints

36 Control Paradigm Foundations Robotics Foundations Mobile Robotics Robustness Higher-Level Gripping and Multiple Manipulators Under Geometric Constraints

37 Control Paradigm Multiple Manipulators and Hands Foundations 1. Coordinated learning control for multiple manipulators holding an object rigidly ICRA, May and adaptive controls for coordination of Robustness multiple manipulators without knowing physical parameters Robotics of an object ICRA, Apr Foundations and adaptive controls for communication of multiple manipulators holding a geometrically constrained object IROS, Sep Control of physical interaction Higher-Level between a deformable finger-tip and a rigid object IEEE SMC Conf., Oct Principle Mobile of superposition in design of feedback control signals for dexterous multi-fingered robot Robotics hands 4th Korea-Russia Int Sym on Sci/Tech, June Gripping and Multiple Manipulators

38 Control Paradigm Foundations Robotics Foundations Mobile Robotics Robustness Higher-Level Passivity and Impedance Gripping and Multiple Manipulators Under Geometric Constraints

39 Control Paradigm Passivity and Impedance Passivity of Robot Dynamics Implies Capability of Motor Foundations Program Lecture Notes in Control and Information Sciences, Passivity and Learnability for Mechanical Systems A Control Theory for Skill Refinement IEICE Trans on Fund. Of ECCS, May Ability of motion learning comes from passivity and dissipativity of its dynamics ASCC, July 1997 Robotics CDC, Dec 1998 Foundations May Robustness System structure rendering iterative learning convergent 5. of robot tasks via impedance matching ICRA, 6. Equivalence of learnability to output-dissipativity and application for control of nonlinear mechanical systems IEEE Trans. SMC, Oct 1999 Higher-Level 7. learning of impedance control IROS, Oct of robot tasks on the basis of passivity and impedance concepts Robotics/Autonomous Systems, Aug Mobile Robotics learning of impedance control from the viewpoint of passivity IJC, Jul Equivalence relations between learnability, outputdissipativity, and strict positive realness IJC, Nov Learnability and adaptability from the viewpoint of passivity analysis Intell Auto/Soft Comp, Mar 2002 Passivity and Impedance

40 Control Paradigm Foundations Robotics Foundations Mobile Robotics Robustness Higher-Level Passivity and Impedance Gripping and Multiple Manipulators Under Geometric Constraints

41 Control Paradigm Introduction Outline Control System Design: Motivation for ILC Control: The Basic Idea On the History and Accomplishments of ILC Professor Arimoto s ILC Contributions An ILC Framework for the Past, Present, and Future General Operator-Theoretic Framework The Supervector Notation for Discrete-Time ILC The w-transform: z-operator Along the Repetition Axis ILC as a MIMO Control System: Repetition-Domain Poles and Internal Model Principle The Complete Framework: Iteration-Domain Uncertainty Concluding Remarks: Future Research Vistas in ILC 41

42 Control Paradigm LTI ILC Convergence Conditions Theorem: For the plant y k = T s u k, the linear time-invariant learning control algorithm converges to a fixed point u (t) given by with a final error defined on the interval (t 0, t f ) if Observation: u k+1 = T u u k + T e (y d y k ) u (t) = (I T u + T e T s ) 1 T e y d (t) e (t) = lim k (y k y d ) = (I T s (I T u + T e T s ) 1 T e )y d (t) If T u = I then e (t) = 0 for all t [t o, t f ]. Otherwise the error will be non-zero. T u T e T s i < 1 42

43 Control Paradigm LTI Control - Nature of the Solution Question: Given T s, how do we pick T u and T e to make the final error e (t) as small as possible, for the general linear ILC algorithm: u k+1 (t) = T u u k (t) + T e (y d (t) y k (t)) Answer: Let T n solve the problem: min T n (I T s T n )y d It turns out that we can specify T u and T e in terms of T n and the resulting learning controller converges to an optimal system input given by: u (t) = T ny d (t) Conclusion:The essential effect of a properly designed learning controller is to produce the output of the best possible inverse of the system in the direction of y d. 43

44 Control Paradigm The Supervector Framework of ILC Consider an SISO, LTI discrete-time plant with relative degree m: Y (z) = H(z)U(z) = (h m z m + h m+1 z (m+1) + h m+2 z (m+2) + )U(z) By lifting along the time axis, for each trial k define: U k Y k Y d = [u k (0), u k (1),, u k (N 1)] T = [y k (m), y k (m + 1),, y k (m + N 1)] T = [y d (m), y d (m + 2),, y d (m + N 1)] T Thus the linear plant can be described by Y k = H p U k where: h h 2 h H p = h 3 h 2 h h N h N 1 h N 2... h 1 The lower triangular matrix H p is formed using the system s Markov parameters. Notice the non-causal shift ahead in forming the vectors U k and Y k. 44

45 Control Paradigm The Supervector Framework of ILC (cont.) For the linear, time-varying case, suppose we have the plant given by: x k (t + 1) = A(t)x k (t) + B(t)u k (t) y k (t) = C(t)x k (t) + D(t)u k (t) Then the same notation again results in Y k = H p U k, where now: H p = h m, h m+1,0 h m, h m+2,0 h m+1,1 h m, h m+n 1,0 h m+n 2,1 h m+n 3,2... h m,n 1 The lifting operation over a finite interval allows us to: Represent our dynamical system in R 1 into a static system in R N. 45

46 Control Paradigm The Update Law Using Supervector Notation Suppose we have a simple Arimoto-style ILC update equation with a constant gain γ: In our R 1 representation, we write: u k+1 (t) = u k (t) + γe k (t + 1) In our R N representation, we write: U k+1 = U k + ΓE k where Γ = γ γ γ γ 46

47 Control Paradigm The Update Law Using Supervector Notation (cont.) Suppose we filter with an LTI filter during the ILC update: In our R 1 representation we would have the form: u k+1 (t) = u k (t) + L(z)e k (t + 1) In our R N representation we would have the form: U k+1 = U k + LE k where L is a Topelitz matrix of the Markov parameters of L(z), given, in the case of a causal, LTI update law, by: L = L m L m+1 L m L m+2 L m+1 L m L m+n 1 L m+n 2 L m+n 3... L m 47

48 Control Paradigm The Update Law Using Supervector Notation (cont.) We may similarly consider time-varying and noncausal filters in the ILC update law: U k+1 = U k + LE k A causal (in time), time-varying filter in the ILC update law might look like, for example: n 1, n 2,0 n 1, L = n 3,0 n 2,1 n 1, n N,0 n N 1,1 n N 2,2... n 1,N 1 A non-causal (in time), time-invariant averaging filter in the ILC update law might look like, for example: K K K K K K L = K K K K K 48

49 Control Paradigm The Update Law Using Supervector Notation (cont.) The supervector notation can also be applied to other ILC update schemes. For example: The Q-filter often introduced for stability (along the iteration domain) has the R 1 representation: u k+1 (t) = Q(z)(u k (t) + L(z)e k (t + 1)) The equivalent R N representation is: U k+1 = Q(U k + LE k ) where Q is a Toeplitz matrix formed using the Markov parameters of the filter Q(z). 49

50 Control Paradigm The ILC Design Problem The design of an ILC controller can be thought of as the selection of the matrix L: For a causal ILC updating law, L will be in lower-triangular Toeplitz form. For a noncausal ILC updating law, L will be in upper-triangular Toeplitz form. For the popular zero-phase learning filter, L will be in a symmetrical band diagonal form. L can also be fully populated. Motivated by these comments, we will refer to the causal and non-causal elements of a general matrix Γ as follows: Γ = γ 11 γ 12 γ γ 1N γ 21 γ 22 γ 23 noncausal γ 2N γ 31 γ 32 γ γ 3N. causal..... γ N1 γ N2 γ N3... γ NN The diagonal elements of Γ are referred to as Arimoto gains. 50

51 Control Paradigm w-transform: the z-operator in the Iteration Domain Introduce a new shift variable, w, with the property that, for each fixed integer t: w 1 u k (t) = u k 1 (t) For a scalar x k (t), combining the lifting operation to get the supervector X k with the shift operation gives what we call the w-transform of x k (t), which we denote by X(w) Then the ILC update algorithm: u k+1 (t) = u k (t) + L(z)e k (t + 1) which, using our supervector notation, can be written as U k+1 = U k + LE k can also be written as: wu(w) = U(w) + LE(w) where U(w) and E(w) are the w-transforms of U k and E k, respectively. Note that we can also write this as where E(w) = C(w)U(w) C(w) = 1 (w 1) L 51

52 Control Paradigm Higher-Order ILC in the Iteration Domain We can use these ideas to develop more general expressions ILC algorithms. For example, a higher-order ILC algorithm could have the form: which corresponds to: u k+1 (t) = k 1 u k (t) + k 2 u k 1 (t) + γe k (t + 1) C(w) = γw w 2 k 1 w k 2 52

53 Control Paradigm The term ILC as a MIMO Control System C(w) = 1 (w 1) L or C(w) = γw w 2 k 1 w k 2 is effectively the controller of the system (in the repetition domain). This can be depicted as: Next we show how to extend these notions to develop an algebraic (matrix fraction) description of the ILC problem. 53

54 Control Paradigm A Matrix Fraction Formulation Suppose we consider a more general ILC update equation given by (for relative degree m = 1): u k+1 (t) = D n (z)u k (t) + D n 1 (z)u k 1 (t) + + D 1 (z)u k n+1 (t) + D 0 (z)u k n (t) +N n (z)e k (t N n 1 (z)e k 1 (t N 1 (z)e k n+1 (t + 1) + N 0 (z)e k n (t + 1) which has the supervector expression U k+1 = D n U k + D n 1 U k D 1 U k n+1 + D 0 U k n +N n E k + N n 1 E k N 1 E k n+1 + N 0 E k n Aside: note that there are a couple of variations on the theme that people sometimes consider: U k+1 = U k + LE k+1 U k+1 = U k + L 1 E k + L 0 E k+1 These can be accomodated by adding a term N n+1 E k+1 in the expression above, resulting in the so-called current iteration feedback, or CITE. 54

55 Control Paradigm A Matrix Fraction Formulation (cont.) Applying the shift variable w we get: D c (w)u(w) = N c (w)e(w) where D c (w) = Iw n+1 D n 1 w n D 1 w D 0 N c (w) = N n w n + N n 1 w n N 1 w + N 0 This can be written in a matrix fraction as U(w) = C(w)E(w) where: C(w) = 1 D c (w)n c (w) Thus, through the addition of higher-order terms in the update algorithm, the ILC problem has been converted from a static multivariable representation to a dynamic (in the repetition domain) multivariable representation. Note that we will always get a linear, time-invariant system like this, even if the actual plant is time-varying. Also, because D c (w) is of degree n + 1 and N c (w) is of degree n, we have relative degree one in the repetition-domain, unless some of the gain matrices are set to zero. 55

56 Control Paradigm ILC Convergence via Repetition-Domain Poles From the figure we see that in the repetition-domain the closed-loop dynamics are defined by: G cl (w) = H p [I + C(w)H p ] 1 C(w) = H p [ D c (w) + N c (w)h p ] 1 N c (w) Thus the ILC algorithm will converge (i.e., E k a constant) if G cl is stable. Determining the stability of this feedback system may not be trivial: It is a multivariable feedback system of dimension N, where N could be very large. But, the problem is simplified due to the fact that the plant H p is a constant, lower-triangular matrix. 56

57 Control Paradigm Repetition-Domain Internal Model Principle Because Y d is a constant and our plant is type zero (e.g., H p is a constant matrix), the internal model principle applied in the repetition domain requires that C(w) should have an integrator effect to cause E k 0. Thus, we modify the ILC update algorithm as: U k+1 = (I D n 1 )U k + (D n 1 D n 2 )U k 1 + +(D 2 D 1 )U k n+2 + (D 1 D 0 )U k n+1 + D 0 U k n +N n E k + N n 1 E k N 1 E k n+1 + N 0 E k n Taking the w-transform of the ILC update equation, combining terms, and simplifying gives: where (w 1)D c (w)u(w) = N c (w)e(w) D c (w) = w n + D n 1 w n D 1 w + D 0 N c (w) = N n w n + N n 1 w n N 1 w + N 0 57

58 Control Paradigm Repetition-Domain Internal Model Principle (cont.) This can also be written in a matrix fraction as: but where we now have: U(w) = C(w)E(w) C(w) = (w 1) 1 D 1 c (w)n c (w) For this update law the repetition-domain closed-loop dynamics become: ( G cl (w) = H I + I (w 1) C(w)H ) 1 I (w 1) C(w), = H[(w 1)D c (w) + N c (w)h] 1 N c (w) Thus, we now have an integrator in the feedback loop (a discrete integrator, in the repetition domain) and, applying the final value theorem to G cl, we get E k 0 as long as the ILC algorithm converges (i.e., as long as G cl is stable). 58

59 Control Paradigm Iteration-Varying Uncertainty In ILC, it is assumed that desired trajectory y d (t) and external disturbance are invariant with respect to iterations. When these assumptions are not valid, conventional integral-type, first-order ILC will no longer work well. In such a case, ILC schemes that are higher-order along the iteration direction will help. Example: consider a stable plant H a (z) = z 0.8 (z 0.55)(z 0.75) Let the plant be subject to an additive output disturbance d(k, t) = 0.01( 1) k 1 This is an iteration-varying, alternating disturbance. If the iteration number k is odd, the disturbance is a positive constant in iteration k while when k is even, the disturbance jumps to a negative constant. In the simulation, we wish to track a ramp up and down on a finite interval. 59

60 Control Paradigm Example: First-Order ILC output signals desired output output at iter. #60 2 norm of the input signal time (sec.) Iteration number The Root Mean Square Error Iteration number h(t) N h 1 =1 and sum hj j=2 = t u k+1 (t) = u k (t) + γe k (t + 1), γ = 0.9 C(w) = 1 (w 1) L 60

61 Control Paradigm Example: Second-Order, Internal Model ILC 1 5 output signals desired output output at iter. #60 2 norm of the input signal time (sec.) Iteration number The Root Mean Square Error Iteration number h(t) N h 1 =1 and sum hj j=2 = t u k+1 (t) = u k 1 (t) + γe k 1 (t + 1) with γ = 0.9 C(w) = 1 (w 2 1) L 61

62 Control Paradigm A Complete Design Framework We have presented several important facts about ILC: The supervector notation lets us write the ILC system as a matrix fraction, introducing an algebraic framework. In this framework we are able to discuss convergence in terms of pole in the iteration-domain. In this framework we can consider rejection of iteration-dependent disturbances and noise as well as the tracking of iteration-dependent reference signals (by virtue of the internal model principle). In the same line of thought, we can next introduce the idea of iteration-varying models. 62

63 Control Paradigm Iteration-Varying s Can view the classic multi-pass (Owens and Edwards) and linear repetitive systems (Owens and Rogers) as a generalization of the static MIMO system Y k = H p U k into a dynamic (in iteration) MIMO system, so that Y k+1 = A 0 Y k + B 0 U k becomes H(w) = (wi A 0 ) 1 B 0 Introduce iteration-varying plant uncertainty, so the static MIMO system Y k = H p U k becomes a dynamic (in iteration) and uncertain MIMO system, such as or or or H p = H 0 (I + H) H p [H, H] H p = H 0 + H(w) H p (w) = H 0 (w)(i + H(w)) etc. 63

64 Control Paradigm Complete Framework D (w) N(w) Y d (w) - E(w) C ( w ILC ) U (w) 1 ( w 1) H p (w) Y (w) C CITE (w) ΔH(w) Y d (w), D(w) and N(w) describe, respectively, the iteration-varying reference, disturbance, and noise signals. H p (w) denotes the (possibly iteration varying) plant. H p (w) represents the uncertainty in plant model, which may also be iteration-dependent. C ILC (w) denotes the ILC update law. C CITE (w) denotes any current iteration feedback that might be employed. The term 1 (w 1) denotes the natural one-iteration delay inherent in ILC. 64

65 Control Paradigm Introduction Outline Control System Design: Motivation for ILC Control: The Basic Idea On the History and Accomplishments of ILC Professor Arimoto s ILC Contributions An ILC Framework for the Past, Present, and Future General Operator-Theoretic Framework The Supervector Notation for Discrete-Time ILC The w-transform: z-operator Along the Repetition Axis ILC as a MIMO Control System: Repetition-Domain Poles and Internal Model Principle The Complete Framework: Iteration-Domain Uncertainty Concluding Remarks: Future Research Vistas in ILC 65

66 Control Paradigm Categorization of Problems Y d D N C H Y d (z) 0 0 Γ(w 1) 1 H p Classical Arimoto algorithm Y d (z) 0 0 Γ(w 1) 1 H p (w) Owens multipass problem Y d (z) D(w) 0 C(w) H p General (higher-order) problem Y d (w) D(w) 0 C(w) H p General (higher-order) problem Y d (w) D(w) N(w) C(w) H p (w) + (w) Most general problem Y d (z) D(z) 0 C(w) H(w) + (w) Frequency-domain uncertainty Y d (z) 0 w(t), v(t) Γ(w 1) 1 H p Least quadratic ILC Y d (z) 0 w(t), v(t) C(w) H p Stochastic ILC (general) Y d (z) 0 0 Γ(w 1) 1 H I Interval ILC Y d (z) D(z) w(t), v(t) C(w) H(z) + H(z) Time-domain H problem Y d (z) D(w) w(k, t), v(k, t) C(w) H(w) + H(w) Iteration-domain H ILC Y d (z) 0 w(k, t), v(k, t) Γ(k) H p + H(k) Iteration-varying uncertainty and control Y d (z) 0 H Γ Hp Intermittent measurement problem

67 Control Paradigm Concluding Remarks: Future Research Vistas in ILC Thanks to the pioneering contributions of Professor Arimoto and the later researchers inspired by his work, ILC is now a relatively-matured, inherently- robust, less model-based, tolerant-to-slightnonlinearities technique. Significant body of literature. Significant industrial applications. Transient and monotonic convergence issues have not been well-handled but are now gaining attention. Applications to PDE (partial differential equation) systems not well understood. ILC for fractional order dynamic systems (polymer/piezo/silicon gel etc) is an interesting new area. ILC for large-scale, uncertain, spatial-temporal, interconnected systems will be increasingly important. ILC in the network control systems (NCS) setting (telepresence/tele-training) could provide benefits. But, in the case of NCS, we face the problem of intermittent ILC (intermittent sensing, intermittent actuation, intermittent learning updating) Nonlinear updating laws may not be necessary if one considers linear ILC (iteration-domain) together with nonlinear feedback control (time-domain). But, a systematic theory of ILC for nonlinear systems is still an open question. 67

68 Control Paradigm Concluding Remarks: Future Research Vistas in ILC Repetitive control must obey the waterbed effect. ILC may not have such a requirement in the time domain, due to the resetting operation, but ILC must obey the waterbed effect in the iteration domain. Research remains to understand this issue. ILC with nonuniform sampling and asynchronous ILC have not been completely addressed in the literature. Joint time-frequency domain ILC techniques, such as wavelets, TFR, even fractional order Fourier transformation are other areas for future research. Cooperative ILC with over-populated (or densely distributed) sensors and actuators, possibly networked, each with dynamic neighbors under uncertain communication topologies is a question that can arise in the area of (mobile) sensor/actuator networks. 68

69 Vision-Based Spatial ILC COLORADO SCHOOL OF MINES Target Path y Laser Pointer Camera x Image Capture ILC Algorithm Motor Control (two computers) Kevin L. Moore, Colorado School of Mines, July 2006

70 COLORADO SCHOOL OF MINES Gimbal Motion Trial 1 Gimbal Motion Trial 4 Kevin L. Moore, Colorado School of Mines, July 2006

71 ILC for a Diffusion Process COLORADO SCHOOL OF MINES Center Pivot Irrigator Control Assume can bury a sensor at some prescribed depth at regular intervals Adjust water application rate based on the sensor readings between cycles Sensor Kevin L. Moore, Colorado School of Mines, July 2006

72 ILC for a Diffusion Process (cont.) COLORADO SCHOOL OF MINES ILC/RC-like update equation Kevin L. Moore, Colorado School of Mines, July 2006

73 Kevin L. Moore, Colorado School of Mines, July 2006 COLORADO SCHOOL OF MINES

74 Control Paradigm Thank You! to Professor Arimoto Foundations Robotics Foundations Mobile Robotics Robustness Higher-Level Passivity and Impedance Gripping and Multiple Manipulators Under Geometric Constraints

75 Control Paradigm To Learn More about ILC K. L. Moore, M. Dahleh, and S. P. Bhattacharyya. results. J. of Robotic Systems, 9(5): , learning control: a survey and new K. L. Moore. learning control for deterministic systems Springer-Verlag Series on Advances in Industrial Control, Springer-Verlag, London, January K. L. Moore. learning control - an expository overview. Applied & Computational Controls, Signal Processing, and Circuits, 1(1): , D. A. Bristow, M. Tharayil, and A. G. Alleyne, A Survey of Control. Control Systems Magazine, 26(3): , IEEE H. S. Ahn, Y. Q. Chen, and K. L. Moore. learning control: brief survey and categorization IEEE Trans. on Systems, Man, and Cybernetics, Accepted to appear. 70