Enhancing a Model-Free Adaptive Controller through Evolutionary Computation

Transcription

1

2 Enhancing a Model-Free Adaptive Controller through Evolutionary Computation Anthony Clark, Philip McKinley, and Xiaobo Tan Michigan State University, East Lansing, USA

3 Aquatic Robots Practical uses autonomous mobile sensors biological studies (elicit natural behaviors)

4 Research platform Simple physical design (relatively) few actuators Nonlinear environment changing currents Complex dynamics flexible fins

5 Focus on Control We d like controllers to: 1. match oscillating frequency with material properties 2. handle changes in the environment 3. handle changes to the robotic device 4. unknown conditions? We do not want to account for these by hand Leads us to adaptive control

6 Adaptive Control Model-based require a precise model perform parameter identification Data-driven model-free input / output data

7 Model-based Adaptive Control Reference Model y p _ e r Controller u y + System Adaptive Law θ

8 Model-based Adaptive Control Reference Model y p _ e r Controller u y + System Reference model: based on complex system dynamics Adaptive Law make simplifying assumptions θ

9 Model-free Adaptive Control r e u + y x MFA System + _ Controller + d For gray-box situations partial / incomplete information known about the system

10 Model-free Adaptive Control d r e MFA u x + y System Controller +_ +

11 Model-free Adaptive Control What do we gain? 1. do not have to create a dynamic model 2. adapts to changing internal dynamics 3. adapts to noisy environment 4. adapts to varying high-level control input What are the drawbacks? 1. less precise 2. still need to specify a number of parameters ANN topology, learning rate, gain values, error bounds, activation timing, network bias values

12 This Study Exploit EC to Enhance an MFAC evolve MFAC parameters controlling a robotic fish adapt to: changing fin flexibilities changing fin length changing control demands

13 MFAC vs. Neural Plasticity Plastic neural networks will generally learn (or transition to) a new behavior merge high-level logic and low-level control Adaptive controllers regulate a control signal behaviors are still determined at a higher level

14 Adaptive Neural Network Network Activation feed-forward network propagated error sigmoid activation Network Update minimize error

15 Adaptive Neural Network

16 Simulation Task Swim at a given speeds Optimize MFAC parameters Adapt to: different control signals changing fin flexibilities changing fin lengths Evaluation simulate for 60 seconds mean absolute error

17 Baseline Experiment r e u y + _ MFA Controller System

18 Baseline Experiment r e u y + _ MFA Controller System

19 Baseline Experiment r e u y + _ MFA Controller System

20 Baseline Experiment r e u y + _ MFA Controller System

21 Baseline Experiment Output of the MFAC regulates speed of the robotic fish frequency of oscillation r e u y + _ MFA Controller System

22 Baseline Experiment

23 Differential Evolution Evolutionary algorithm for real-valued problems Evolved parameters neural network size learning rate upper and lower error bounds controller gain controller update timing

24 Single Evaluation Experiment 20 overfit Speed (cm/s) 7.5 y r e Time (s)

25 Multiple Evaluations Trial Flexibility Length sim1 100 % 100 % sim2 200 % 100 % sim3 50 % 100 % sim4 100 % 110 % sim5 200 % 110 % sim6 50 % 110 % sim7 100 % 90 % sim8 200 % 90 % sim9 50 % 90 %

26 Multiple Evaluations Experiment 14 Speed (cm/s) 5 0 y r e Time (s)

27 Goals of the Study We want to adapt to: changing fin flexibilities changing fin length changing control signal dynamics any combination of the above changes

28 Fin Length Damaged Fin (60%) 4.5 cm 9 Evaluations (best replicate) : 80% length Nominal 7.6 cm Attached Debris (137%) 10.4 cm

29 Control and Flexibility Different speeds Different accelerations Different decelerations Flexibility of 150% compared to the nominal value

30 Simultaneous Changes 9 Evaluations (best replicate) : 80% length, 120% flexibility 14 Speed (cm/s) 5 0 y r e Time (s)

31 Extended Multiple Evaluations Trial Flexibility Length sim1 100 % 100 % sim2 200 % à 1000 % 100 % sim3 50 % à 10 % 100 % sim4 100 % 110 % à 200 % sim5 200 % à 1000 % 110 % à 200 % sim6 50 % à 10 % 110 % à 200 % sim7 100 % 90 % à 67 % sim8 200 % à 1000 % 90 % à 67 % sim9 50 % à 10 % 90 % à 67 %

32 Increase Simulation Ranges 20 9 Evaluations, wide range (best replicate) Speed (cm/s) 7.5 y r e Time (s)

33 When Adaptation Breaks-Down

34 When Adaptation Breaks-Down

35 Key Points 1. Attained adaptability to varying parameters for the robotic fish 2. Performance was easily better than expert chosen values 1. Envelope of adaptability for evolution (tested values) for operation (range of adaptability)

36 Ongoing Work 1. Integrate with highlevel control self-modeling takes over when adaptation fails 1. Multiple-input, Multiple-output regulate speed and direction 1. Physical testing perform adaptation online

37 The authors gratefully acknowledge the contributions and feedback on the work provided by: Jared Moore, Jianxun Wang, and the BEACON Center at Michigan State University. This work was supported in part by National Science Foundation grants IIS , CCF , CNS , CNS , and DBI , and by a grant from Michigan State University.

38 References [Wang 2012] : Dynamic modeling of robotic fish with a flexible caudal fin. In Proceedings of the ASME th Annual Dynamic Systems and Control Conference, joint with the JSME th Motion and Vibration Conference, Ft. Lauderdale, Florida, USA, October [Clark 2012] : Evolutionary design and experimental validation of a flexible caudal fin for robotic fish. In Proceedings of the Thirteenth International Conference on the Synthesis and Simulation of Living Systems, pages , East Lansing, Michigan, USA, July [Rose 2013] : Just Keep Swimming: Accounting for Uncertainty in Self- Modeling Aquatic Robots In Proceedings of the 6th International Workshop on Evolutionary and Reinforcement Learning for Autonomous Robot Systems, Taormina, Italy, September 2013

39 Uncertainty in Robotics Materials materials changing with temperature flexibility changing due to water absorption Hardware motors becoming less efficient Environment transitioning from smooth to rough terrain

40 Address Uncertainty (1) Mimicking biology biomimetic / bioinspired design soft / flexible materials evolutionary design evolutionary robotics and optimization evolving / learning behaviors artificial neural networks (ANNs) central pattern generators (CPGs)

41 Address Uncertainty (2) Complex (feedback) control strategies robust control handle a static range of uncertainty robust to a noisy environment adaptive control adapting to varying parameters explicitly changes the controller s dynamics

42 Our Research Autonomous behaviors Feedback motor control Biomimetic robots High-level Control Low-level Control Robotic System Sensor Feedback

43 Our Research Neural Networks Adaptive Control Robotic Fish State Machine Digital Muscles Wheeled Robot High-level Control Low-level Control Robotic System

44 Our Research Neural Networks Adaptive Control Robotic Fish State Machine Digital Muscles Wheeled Robot High-level Control Low-level Control Robotic System