Comparing Robot Controllers Through System Identification

Transcription

1 Comparing Robot Controllers Through System Identification Ulrich Nehmzow 1,OtarAkanyeti 1, Roberto Iglesias 2, Theocharis Kyriacou 1, and S.A. Billings 3 1 Department of Computer Science, University of Essex, UK 2 Electronics and Computer Science, University of Santiago de Compostela, Spain 3 Dept. of Automatic Control and Systems Engineering, University of Sheffield, UK Abstract. In mobile robotics, it is common to find different control programs designed to achieve a particular robot task. It is often necessary to compare the performance of such controllers. So far this is usually done qualitatively, because of a lack of quantitative behaviour analysis methods. In this paper we present a novel approach to compare robot control codes quantitatively, based on system identification. Using the NARMAX system identification process, we translate the original behaviour into a transparent, analysable mathematical model of the original behaviour. We then use statistical methods and sensitivity analysis to compare models quantitatively. We demonstrate our approach by comparing two different robot control programs, which were designed to drive a Magellan Pro robot through door-like openings. 1 Introduction In mobile robotics it is common to find different control programs, developed for the same behaviour, typically developed through an empirical trial-and-error process of iterative refinement. It is often interesting to compare and analyse such different controllers for the same behaviour. However, because of a lack of quantitative descriptors of behaviour such analyses are usually qualitative. As the number of controllers developed for identical tasks increases, the need for fair comparisons based on quantitative measures also becomes more important, and to find quantitative answers to questions like What are the advantages and disadvantages of each controller?, Which one is more stable and efficient?, Will they work in completely different environments or crash even with slight modifications? will enhance our understanding of robot-environment interaction. Realistic mobile robot control programs tend to be so complex that their direct analysis is impossible the meaning of thousands of lines of code is not clear to a human observer. In this paper we therefore propose the alternative approach of identifying the behaviour in question, using transparent polynomial models S. Nolfi et al. (Eds.): SAB 2006, LNAI 4095, pp , c Springer-Verlag Berlin Heidelberg 2006

2 Comparing Robot Controllers Through System Identification 841 obtained through NARMAX system identification [1], to ascertain the validity of the model, and to analyse the model instead of the original behaviour, both qualitatively and quantitatively. 2 Controller Identification Motivation and Related Work. The main benefits of system identification (i.e. mathematical modelling) are 1. Efficiency: The obtained models are polynomials. They are fast in execution and occupy little space in memory. 2. Transparency: The transparent model structure simplifies mathematical analysis 3. Portability: Polynomial models are universal, in the sense that they can be quickly incorporated into any robot programming language. Our models are obtained based on NARMAX (Nonlinear Auto-Regressive Moving Average model with exogenous inputs) system identification [1]. This identification process has already been applied in modelling sensor-motor couplings of robot controllers [2, 4]. In [5] we demonstrate how easy it is to exchange the generated models between different robot platforms. After the identification of the controllers it is relatively straightforward to evaluate the models and to compare them quantitatively. The work presented in [3] gives examples of the characterisation of models of robot behaviour. The system identification process can be divided into two stages: First the robot is driven by the original robot controller which we wish to analyse. While the robot is moving, we log sensor and motor information to model the relationship between the robot s sensor perception and motor responses. In the second stage, a nonlinear polynomial NARMAX model (see below) is estimated. This model relates input sensor values to output actuator signals and subsequently can be used to control the robot. NARMAX Modelling. The NARMAX modelling approach is a parameter estimation methodology for identifying the important model terms and associated parameters of unknown nonlinear dynamic systems. For multiple input, single output noiseless systems this model takes the form: y(n) =f(u 1(n),u 1(n 1),u 1(n 2),,u 1(n N u),u 1(n) 2,u 1(n 1) 2, u 1(n 2) 2,,u 1(n N u) 2,,u 1(n) l,u 1(n 1) l,u 1(n 2) l,, u 1(n N u) l,u 2(n),u 2(n 1),u 2(n 2),,u 2(n N u),u 2(n) 2, u 2(n 1) 2,u 2(n 2) 2,,u 2(n N u) 2,,u 2(n) l,u 2(n 1) l, u 2(n 2) l,,u 2(n N u) l,,u d (n),u d (n 1),u d (n 2),, u d (n N u),u d (n) 2,u d (n 1) 2,u d (n 2) 2,,u d (n N u) 2,, u d (n) l,u d (n 1) l,u d (n 2) l,,u d (n N u) l,y(n 1),y(n 2),, y(n N y),y(n 1) 2,y(n 2) 2,,y(n N y) 2,,y(n 1) l, y(n 2) l,,y(n N y) l )

3 842 U. Nehmzow et al. where y(n) and u (n) are the sampled output and input signals at time n respectively, Ny and Nu are the regression orders of the output and input respectively and d is the input dimension. f () is a non-linear function, this is typically taken to be a polynomial or wavelet multi-resolution expansion of the arguments. The degree l of the polynomial is the highest sum of powers in any of its terms. The NARMAX methodology breaks the modelling problem into the following steps: i) structure detection, ii) parameter estimation, iii) model validation, iv) prediction and v) analysis. Logged data is first split into two sets (usually of equal size). The first, the estimation data set, is used to determine model structure and parameters. The remaining data set, the validation data set, is subsequently used to validate the model. 3 Experimental Scenario: Door Traversal The example presented in this paper demonstrates how system identification can be used to model sensor-motor couplings of two different robot controllers designed to achieve the same particular task: the episodic task of door traversal, where each episode comprises the movement of the robot from a starting position to a final position, traversing a door-like opening en route. In this paper, we compare two different controllers for this task. Model 1 is a hard-wired laser controller which maps the laser perception of the robot onto action, using a behaviour based approach. For model 2, the robot was driven manually by a human operator. While these two original behaviours were being executed, we logged all sensory perceptions, motor responses and the robot s precise position, using a camera logging system, every 160 ms. 300cm 300cm 60cm 60cm 80cm 80cm (c) Fig. 1. The Magellan Pro robot used in the experiments. Training environments for the Hard-Wired Laser Controller (model 1, centre) and Manual Control (model 2, right). Environment has extra walls on the wings of the door-like opening. The initial positions of the robot were within the shaded area indicated in each figure; door openings were twice the robot s diameter (80cm).

4 Comparing Robot Controllers Through System Identification 843 Each controller was then modelled through non-linear polynomial functions (NARMAX), so that the resulting two models identify the two robot controllers by relating robot perception to action. To test the validity of the two models, we then let the models themselves control the robot. The models of the two different robot controllers were deliberately trained in different environments (figure 1 (b,c)), with different translational velocities and different sensor information, to simulate the natural differences which might exist between different implementations of the same task. All experiments were conducted with a Magellan Pro robot (figure 1 ) in the Robotics Arena at Essex University. 3.1 Model 1: Hardwired Laser Control The first control program used a hardwired, behaviour-based control strategy that mapped laser perception to steering speed of the robot. The translational velocity of robot was kept constant at 0.2 m/s, the laser perception of the robot was encoded in 36 sectors by taking the median value of every 5 degree interval over a semi-circle. The model of the robot s angular velocity ω, obtained through NARMAX system identification, contains 84 terms and is given in table 1. Table 1. Model 1: NARMAX model of the angular velocity ω as a function of laser perception for the hard-wired laser controller. d 1,,d 29 are laser bins, coarse coded by taking the median value over each 5-degree interval. ω = d 1(t) d 6(t) d 7(t) d 8(t) d 9(t) d 10(t) d 11(t) d 12(t) d 13(t) d 14(t) d 15(t) d 16(t) d 20(t) d 21(t) d 22(t) d d 24(t) d 25(t) d 26(t) d 27(t) d 28(t) d 29(t) d 30(t) d 31(t) d 32(t) d 2 1(t) d 2 6(t) d 2 7(t) d 2 8(t) d 2 13(t) d 2 15(t) d 2 16(t) d 2 17(t) d 2 19(t) d 2 20(t) d 2 21(t) d 2 22(t) d 2 23(t) d 2 24(t) d 2 25(t) d 2 27(t) d 2 28(t) d 2 31(t) d 2 35(t) d 2 36(t) d 1(t) d 7(t) d 1(t) d 18(t) d 1(t) d 20(t) d 1(t) d 22(t) d 1(t) d 23(t) d 1(t) d 25(t) d 1(t) d 32(t) d 1(t) d 33(t) d 1(t) d 34(t) d 1(t) d 36(t) d 7(t) d 16(t) d 8(t) d 10(t) d 9(t) d 14(t) d 9(t) d 17(t) d 9(t) d 18(t) d 9(t) d 28(t) d 10(t) d 13(t) d 11(t) d 16(t) d 11(t) d 18(t) d 12(t) d 15(t) d 12(t) d 16(t) d 12(t) d 18(t) d 13(t) d 15(t) d 13(t) d 16(t) d 14(t) d 16(t) d 14(t) d 23(t) d 14(t) d 36(t) d 15(t) d 21(t) d 16(t) d 19(t) d 16(t) d 25(t) d 17(t) d 28(t) d 18(t) d 31(t) d 20(t) d 21(t) d 20(t) d 28(t) d 20(t) d 31(t) d 21(t) d 26(t) d 21(t) d 30(t) d 25(t) d 29(t)

5 844 U. Nehmzow et al. Qualitative Model Evaluation. Figure 2 shows the trajectories of the robot under the control of the original hard-wired laser controller and its NARMAX model respectively. In both cases the robot was started from 36 different initial positions, and completed the task successfully in each case. Fig. 2. Robot trajectories under control by the original Hard-wired Laser Controller and under control by its model, model 1. Note that side walls of the environment can not be seen in the figures because they were outside the field of view of the camera. 3.2 Model 2: Manual Control The second control strategy used was to drive the robot through the opening manually. Here, the translational velocity of robot was kept constant at 0.07 m/s. For sensor information (i.e. model input), the values delivered by the laser scanner were averaged in twelve sectors of 15 degrees each, to obtain a twelve dimensional vector of laser distances. These laser bins as well as the 16 sonar sensor values were inverted before they were used to obtain the model, so that large readings indicate close-by objects. The identified model of the angular velocity ω, which contained 35 terms, is given in table 2. Qualitative Model Evaluation. Figure 3 shows the trajectories of the robot under the control of human operator and its NARMAX model respectively. In both cases the robot was started with 41 different initial positions, and passed through the door-like gap successfully each time. 4 Comparison of the Two Models Figures 2 and 3 show that on a qualitative level both models are good representations of the original. Small differences in trajectory between original and model-controlled behaviour we attribute to natural fluctuations in robotenvironment interaction.

6 Comparing Robot Controllers Through System Identification 845 Table 2. Model 2: NARMAX model of the angular velocity ω as a function of sensor perception for the door traversal behaviour under manual control. s 10,,s 16 are the inverted and normalised sonar readings (s i =(1/s i 0.25)/19.75), while d 1,,d 6 are the inverted and normalised laser bins d i =(1/d i 0.12)/ Taken from [4]. ω(t) = d 1(t) d 2(t) d 3(t) d 4(t) d 5(t) d 6(t) s 13(t) s 15(t) d 2 1 (t) d 2 2 (t) s 2 15 (t) s 2 16 (t) d 1(t) d 3(t) d 1(t) d 5(t) d 1(t) d 6(t) d 1(t) s 11(t) d 1(t) s 16(t) d 2(t) d 3(t) d 2(t) d 5(t) d 2(t) s 11(t) d 3(t) s 12(t) d 3(t) s 15(t) d 4(t) d 6(t) d 4(t) s 12(t) d 4(t) s 15(t) d 5(t) s 11(t) d 5(t) s 15(t) d 5(t) s 16(t) d 6(t) s 12(t) d 6(t) s 13(t)) d 6(t) s 14(t) s 11(t) s 14(t) s 12(t) s 15(t) s 14(t) s 16(t) Fig. 3. Robot Trajectories under under manual control and under control of its model, model 2. Taken from [4]. It is interesting to see that Model 1 has significantly more terms (83) than Model 2 (35), and therefore requires more memory and computing power resources. Immediately obvious is also that model 2 is simpler, in that it only uses information from sensors found on the right side of the robot. In the following, we were interested in investigating the differences between the models of the two door-traversal behaviours further. Testing and Evaluation Setup. We first tested the controllers in four different environments (figure 4) to reveal differences in the behaviour of the robot when controlled by the two models. To minimise the influence of constant errors, model 1 and model 2 were selected randomly to control the robot. In the first scenario (figure 4 (1)), the robot was driven by Model 1 in the training environment of Model 2. After this, the two controllers were tested in a completely different environment, where two door-like openings with equal

7 846 U. Nehmzow et al. 80cm 80cm 80cm 80cm (1) (2) (3) (4) Fig. 4. Four different test environments used to compare the two models qualitatively. 1) Single-door test scenario, 2) Double-door test scenario, 3) Completely-enclosed test scenario, 4) Enclosure with a single door. The initial positions of the robot are within the shaded area indicated in each figure. All openings found in the environment had a width of two robot diameters (80cm). width were presented at the same time (figure 4 (2)). This demonstrated how each opening interfered with the robot s behaviour. It also revealed the dominant sensors. In the third and fourth scenarios (figure 4 (3) and (4)) we tried to measure the effect of a gap found in an environment. We assumed that in this application openings would be the dominant environmental factors in robot-environment interaction. Therefore the models were tested first in a completely closed environment, subsequently we introduced a gap to observe the resulting change in behaviour. 4.1 Qualitative Comparison As can be seen in figure 5, when we remove the side walls from the environment, the robot under the control of Model 1 has some problems in centreing itself while passing through the gap. We realised that side walls make the robot turn sharper while approaching to the door-like opening, which made it easier for robot to centre itself while passing through the gap. For the double-door test scenario, it is interesting to see how two gaps interfere with each other for the behaviour of robot under the control of Model 1. If the robot was started from the middle region between two gaps, it generally collided with the wall, without turning towards any of the gaps. If the robot pointed at one of the gaps initially, it was attracted towards it, but the other gap still effected the behaviour and the robot could not centre itself enough to pass through the gap (figure 6 ). When we look at the trajectories of robot under the control of Model 2 (figure 6 ), we observe that the two gaps do not interfere with each other. This is a predictable outcome, as Model 2 uses sensors only from the right side of the robot. Therefore, if the robot detects the gap found on the right side, it

8 Comparing Robot Controllers Through System Identification 847 Fig. 5. Robot trajectories under the control of Model 1 in the training environment of Model 1, and in the training environment of Model 2. The arrows in both figures indicate how much the robot deviated from the centre point of the opening while passing through it. Note that side walls of the environment in (A) can not be seen in the figure because of the limited range of camera view. Fig. 6. Robot Trajectories in the double door test scenario under the control of Model 1, and controlled by Model 2. passes through it. If it doesn t, it automatically turns to the left, finds the other gap and traverses it. In the completely-enclosed environment, Model 1 turned the robot constantly to the left. Figure 7 shows a sample trajectory of robot under the control of Model 1. When we introduced a gap to the environment, we observed that the gap has the dominating effect on the controller behaviour. Although the model can not pass through the gap successfully in every trial, we still observe a general tendency of robot towards the gap (figure 7 ). Finally, Model 2 showed very unstable characteristics in the last two environments. When we look at the trajectories closely we observe oscillations in

9 848 U. Nehmzow et al. (c) (d) Fig. 7. Robot trajectories under control of Model 1 in the Completely enclosed environment and when an opening is introduced to the enclosed environment. Robot trajectories under control of Model 2 in the Completely enclosed environment (c), and when an opening is introduced to the enclosed environment (d). the robot s behaviour, especially when the robot is close to corners. The overall model generally looks like a right wall follower when the robot is far from the corners. As it comes closer, the variation in angular velocity of the robot increases and it bumps into the corners (figure 7 (c)). We can also see that for Model 2, the influence of the gap on the robot behaviour is not as big as the influence on Model 1 (figure 7 (d)). 4.2 Quantitative Comparison When we look at the behaviours of the robot qualitatively in different test environments, we see that the responses of the two models differ. We also observe that both models do not produce a general door-traversal behaviour, but that they fail in environments that differ considerably from the training environment. In this section, we extended our work by comparing the two models quantitatively, based on a hardware in the loop simulation process. Hardware in the Loop Simulation. In the first quantitative analysis we wanted to see if the two models produce similar outputs for the same inputs. We

10 Comparing Robot Controllers Through System Identification 849 Fig. 8. Trajectory along which sensor data for hardware in the loop simulation was taken, and steering velocity graphs of both models for the given trajectory. There is no significant correlation between the two model s angular velocities (r S=0.078, not sig.,p>5%). Table 3. Spearman rank correlation coefficients between model 1 and model 2 responses to identical real world input data Environment r S Stat. Sig not sig. (p>0.05) sig. (p<0.05) sig. (p<0.05) not sig. (p>0.05) therefore fed the two models with same sensory perception of the robot, taken from the real test environments, and then compared the corresponding outputs of the models quantitatively in order to see if there is a significant correlation between the responses of the two models. For each of the four test environments, we therefore first drove the robot manually to traverse the door from different initial positions and logged the sensory perception of the robot at every 250ms. We then fed identical logged sensor data to the two models individually, and computed the resulting angular velocity vectors. As an example, figure 8 shows the steering velocity graphs which the robot would attain along the sample trajectory given in figure 8). To assess the agreement between both model responses to identical input vectors, we computed the Spearman rank correlation coefficient r S [6] between the two model outputs for each test environment. The results are given in table 3, even in cases where there is a statistically significant correlation coefficient they are very low, indicating quantitatively that the two models generate different robot behaviour. Stability and Efficiency Analysis. The fact that the controller is known as a transparent polynomial simplifies stability and efficiency analysis, as follows.

11 850 U. Nehmzow et al. Efficiency. The efficiency of the controllers can be evaluated by measuring their execution time on the robot and memory space required to represent the model. Obviously, a polynomial consisting of fewer terms will execute faster and occupy less memory space. It therefore becomes immediately obvious that Model 1, obtained through the hardwired laser-guided door traversal, is considerably less efficient (83 model terms) than Model 2, obtained through manual training (35 model terms). BIBO stability. A controller is considered stable if it produces bounded output for a bounded input. With today s robot programming techniques, the only way to determine if a controller is stable or not is to check the response of the controller for the entire input space and make sure that the output never goes to ±. Typically, robot programmers try to address this issue by writing extra safety routines in the control code which constrain the controller s output within an acceptable interval, without formally analysing for controller stability. With system identification it is possible to analyse the stability of the controllers without searching the entire input space. The identified polynomials presented in this paper can be classified as zero order open loop controllers, because they don t incorporate any feedback from the output (open loop control), and don t use any lags related to past input values (no difference equations in the polynomial). For this kind of polynomials it has been shown that their responses are bounded for bounded inputs [7], which proves the stability of the controllers. 5 Summary and Conclusion It is sometimes useful in mobile robotics to be able to describe and analyse the behaviour of a mobile robot quantitatively, for example when one is interested in establishing the differences between different robot controllers. In the experiments presented in this paper, we compared two controllers that accomplish the same task (door traversal), one implemented by a hardwired, traditional control program and the other through manual control. In order to compare these two behaviours, we identified both behaviours by determining transparent, analysable non-linear polynomial mappings between sensor perception and motor response, and subsequently comparing the two models with each other. Such transfer to a common descriptor of behaviour has a number of advantages. Once transparent mathematical models of sensor motor behaviour are available, they can be analysed quantitatively. In the experiments presented in this paper, we used a hardware-in-the-loop simulation to demonstrate that the hardwired and the manual control strategy differ from each other noticeably. We furthermore evaluated the controllers in terms of efficiency and stability, again revealing considerable differences between the two models in terms of efficiency. Future Work. Further analysis methods, not demonstrated here, include analysing the models spectra, detailed stability analysis, or sensitivity analy-

12 Comparing Robot Controllers Through System Identification 851 sis [8, 9, 10]. Such detailed mathematical analysis of the polynomial models is part of ongoing research at the universities of Sheffield and Essex. Acknowledgements The authors thank Hugo Vieira Neto for his contributions to the preparation of this paper. The authors also thank the following institutions for their support: The RobotMODIC project is supported by the Engineering and Physical Sciences Research Council under grant GR/S30955/01. Roberto Iglesias is supported through the research grants PGIDIT04TIC206011PR, TIC C04-03 and TIN References [1] Korenberg, M., Billings, S. A., Liu, Y. P., and Mcllroy, P. J., Orthogonal Parameter Estimation Algorithm for Non-Linear Stochastic Systems, Int. J. Control, 1998, vol 48(1): [2] Iglesias, R., Kyriacou, T., Nehmzow, U., and Billings, S. A., Task Identification and Characterisation in Mobile Robotics, Proc. Towards Autonomous Robotic Systems, TAROS, Essex [3] Iglesias, R., Kyriacou, T., Nehmzow, U., and Billings, S. A., Modelling and characterisation of a mobile robot s operation, Proc. CAEPIA 2005, 11 th Conference of the Spanish association for Artificial Intelligence, Santiago De Compostela [4] Iglesias, R., Nehmzow, U., Kyriacou, T., and Billings, S. A., Programming Through System Identification and Training, Proc. European Conference on Mobile Robotics, Ancona [5] Kyriacou, T., Nehmzow, U., Iglesias, R., and Billings, S. A., Cross-Platform Programming Through System Identification, Proc. Towards Autonomous Robotic Systems, TAROS, London [6] Barnard, C., Gilbert, F., and McGregor, P., Asking questions in Biology, key Skills for Practical Assessments and Project Work, Pearson Education Limited, [7] P.C. Dorf and R.H. Bishop, Modern Control Systems; Addison Wesley 1995 [8] Max D. Morris, Factorial sampling plans for preliminary computational experiments, Technometrics, v.33 n.2, p , May [9] A. Saltelli, K. Chan and E.M. Scott, Sensitivity Analysis, John Wiley 2000 [10] I.M. Sobol, Sensitivity Estimates for Nonlinear Mathematical Models, J.Mathematical Modelling and Computational Experiment, Vol 1, pp , 1993.