Visual Servoing via Nonlinear Predictive Control

Transcription

1 Chapter 2 Visal Seroing ia Nonlinear Predictie Gillame Allibert, Estelle Cortial, and François Chamette Abstract In this chapter, image-based isal seroing is addressed ia nonlinear model predictie control. The isal seroing task is formlated into a nonlinear optimization problem in the image plane. The proposed approach, named isal predictie control, can easily and explicitly take into accont 2D and 3D constraints. Frthermore, the image prediction oer a finite prediction horizon plays a crcial role for large displacements. This image prediction is obtained thanks to a model. The choice of this model is discssed. A nonlinear global model and a local model based on the interaction matrix are considered. Adantages and drawbacks of both models are pointed ot. Finally, simlations obtained with a 6 degrees of freedom (DOF) free-flying camera highlight the capabilities and the efficiency of the proposed approach by a comparison with the classical image-based isal seroing. 2.1 Introdction Visal seroing has lead to many fritfl researches oer the last decades. In regard to the kind of feedback information considered, one can distingish three main approaches: image-based isal seroing (IBVS) where the feedback is defined in the image plane, position-based isal seroing (PBVS) where the feedback is composed of 3D data sch as the robotic system pose, and the 2-1/2D isal seroing where the feedback combines both 2D and 3D data. Frther details abot the different approaches can be fond in [5, 6, 15]. Here, we focs or interest on IBVS strategy. The IBVS task consists in determining the control inpt applied to the robotic system so that a set of isal featres designed from image measrements Gillame Allibert and Estelle Cortial Institt PRISME, Polytech Orleans, 8 re Leonard de Vinci, 4572 Orleans, France, {gillame.allibert,estelle.cortial}@ni-orleans.fr François Chamette INRIA, Camps de Bealie, 3542 Rennes, France, francois.chamette@irisa.fr 393

2 394 Gillame Allibert, Estelle Cortial, and François Chamette reaches a desired static reference or follows a desired dynamic reference. Althogh IBVS approach is robst to modeling errors, seeral drawbacks can be mentioned when the isal featres are not correctly chosen. Besides the classical problem of local minima and singlarities in the interaction matrix [4], the constraint handling is a tricky problem in IBVS. For instance, the 2D constraint, also named isibility constraint, has to garantee that the image measrements stay into the camera field of iew. Of corse, if the isibility of the target is no longer ensred then the control algorithm is interrpted. The 3D constraints sch as workspace limits hae to make sre that the robot achiees admissible motions in its workspace all along the task. Among the nmeros works which hae inestigated this critical isse, three points of iew exist. The first one consists in designing adeqate isal featres. In [17] for instance, the athors hae shown that the system behaior explicitly depends on the kind of featres. Conseqently, lines, spheres, circles, cylinders bt also moments may be sed and combined to obtain good decopling and linearizing properties, implicitly ensring the constraint handling. The control law is generally a decopled exponential decreasing law. Another way to deal with constraint handling is to combine path-planning and trajectory tracking [7, 16, 19, 24]. When it is sccessfl, this soltion allows ensring both an optimal trajectory of the camera in the Cartesian space and the isibility of the featres. Path-planning ia linear matrix ineqality (LMI) optimization has recently been proposed in [7] to flfill 2D and 3D constraints. In the third approach, the effort is done on the control law design. The isal featres considered are generally basic, namely point-like featres. Adanced control laws sch as optimal control [14, 22], adaptie control [21], LMIs [9, 1] and predictie control [2, 3, 12, 13, 23] hae been reported in the literatre. In [12, 13], a predictie controller is sed for motion compensation in target tracking applications. The prediction of the target motion is sed to reject pertrbation in order to cancel tracking errors. In [23], the predictie controller is sed from ltrasond images for a medical application. The strategy proposed in this chapter exploits nonlinear model predictie control for isal seroing tasks. The IBVS objectie is formlated as soling on-line a nonlinear optimization problem expressed in the image plane [2, 3]. This strategy, named isal predictie control (VPC), offers two adantages. First, 2D and 3D constraints sch as isibility constraints, mechanical constraints and workspace limits can be easily taken into accont in the optimization problem. Secondly, the image prediction oer a finite horizon plays a crcial role for difficlt configrations. The image prediction is based on the knowledge of a model. It can be a nonlinear global model combining the robot model and the camera one. The image prediction can also be obtained thanks to a linear model sing the interaction matrix. The choice of the model is addressed and discssed in the seqel. The interest of the image prediction is pointed ot throgh many simlations describing difficlt configrations for a free-flying perspectie camera. The chapter is organized as follows. In Section 2.2, the context of the stdy is stated and the principle of VPC is presented. The control strctre and the mathematical formlation are addressed. Then, in Section 2.3, the choice of the image prediction model is discssed. In Section 2.4, nmeros simlations on a 6 DOF

3 2 Visal Seroing ia Nonlinear Predictie 395 free-flying camera illstrate the comparison of the different approaches: classical IBVS, predictie control laws with local and global model. Difficlt configrations sch as large displacements to achiee are tested nder constraints. Finally, conclsions are gien in the last section. 2.2 Predictie for Constrained IBVS The aim of isal seroing is to reglate to zero an error e(t) between the crrent featres s(t) and the reference featres s. In IBVS, the featres are expressed in the image. The relationship between the camera elocity τ(t) and the time ariation of the isal featres ṡ(t) is gien by the interaction matrix noted L s. Ths, specifying a decopled exponential decay law for the error e(t), we obtain the control inpt to be applied to the camera: τ(t) = λ L s + e(t) with λ>, (2.1) where L s + is the approximate psedo-inerse matrix of Ls. The classical IBVS is ery easy to implement bt its weak points are the constraint handling and its possible bad behaior for large displacements to achiee as already mentioned in Section 2.1. The control objectie of IBVS can also be formlated into an optimization problem. The goal is to minimize an image error and to take into accont constraints. When a model of the system is aailable, control predictie strategies are well-adapted to deal with this kind of problem. The extension of predictie strategy to isal seroing tasks is detailed below Visal Predictie All predictie strategies are based on for common points: a reference trajectory, a model of the dynamic process, a cost fnction and a soling optimization method. The keystone of the predictie approach is the model sed to predict the process behaior oer the ftre horizon. Its choice will impact on the tracking accracy and on the comptational time. In VPC case, the process considered is generally composed of the robotic system and the camera. For instance, the robotic system can be a nonholonomic mobile robot [2], a drone or a robot arm. The camera system can be a perspectie or catadioptric camera [3] whateer its configration with respect to the robot, that is on board or remote. The model sed is then a global model describing the process. The model inpts are the control ariables of the robotic system. The model otpt are the isal featres. The model is sed to predict the ales of the featres oer a prediction horizon in regard to the control ariables and to satisfy the constraint handling. Before discssing the choice of the model, we first introdce the control strctre and then state the mathematical formlation of VPC.

4 396 Gillame Allibert, Estelle Cortial, and François Chamette Internal Model Strctre The control strctre considered is the well-known internal model control (IMC) strctre [2] (see Fig. 2.1). The process block contains the robotic system and the s * (k) + (k) _ s d (k) + _ Optimization Model s m (k) ler U(k) Process Model s(k) s m (k) _ + Fig. 2.1 IMC Strctre. camera. The inpt U is the robotic control ariable and the otpt s is the crrent ale of the isal featres. For IBVS, the reference s is expressed in the image plane, as the isal featres, and can be static or dynamic. The error signal ε represents all modeling errors and distrbances between the crrent featres and the ales predicted from the model of the system: ε(k) = s(k) s m (k). (2.2) The sal controller is replaced, in the predictie approach, by an optimization algorithm. The latter minimizes the difference between a desired trajectory s d and the predicted model otpt s m. Indeed, according to Fig. 2.1, we can write (k is the crrent iteration): s d (k) = s (k) ε(k), s d (k) = s (k) (s(k) s m (k)), (2.3) s d (k) s m (k) = s (k) s(k). Conseqently, the tracking of the reference featres s by the process otpt s is ths eqialent to the tracking of the desired featres s d by the model otpt s m. The model predicts the behaior of the featres oer a finite prediction horizon N p. The difference s d (k) s m (k) between the desired featres and the predicted model featres is sed to define the cost fnction J to be minimized with respect to a control seqence Ũ. Only the first component U(k) of the optimal control seqence is really applied to the process. At the next sampling time, de to distrbances and model mismatches, the measrements are pdated, the finite horizon moes one step forward and the procedre starts again.

5 2 Visal Seroing ia Nonlinear Predictie Mathematical Formlation The cost fnction J is defined as a qadratic fnction of the error to be minimized. De to the IMC strctre, the mathematical formlation of VPC strategy can be written in discrete-time as: min J(U) (2.4) Ũ R m Np with: J(U) = k+np j=k+1 [s d ( j) s m ( j)] T Q( j)[s d ( j) s m ( j)] (2.5) sbject to: s d ( j) = s ( j) ε( j), (2.6) { x( j) = f (x( j 1),U( j 1)) (2.7) s m ( j) = h(x( j)). The ariables x R n, U R m and s m R p are respectiely the state, the inpt and the otpt of the model. We will see, in the next section, that the state can be differently chosen in regard to the prediction model sed and in regard to the constraints to be handled. The first nonlinear eqation of (2.7) describes the dynamics of the system where x( j) represents the predicted state at time j, j [k + 1;k + N p ]. For j = k + 1, the predicted state s m is initialized with the system state s at time k which garantees the feedback of the IMC strctre. Moreoer, in case of modeling errors and distrbances, a second feedback is ensred by the error signal ε( j) which modifies the reference trajectory accordingly. The second eqation of (2.7) is the otpt eqation. To compte s d ( j), j [k + 1;k + N p ], we need to compte the error ε( j) defined in (2.2). This error depends on s m ( j) that is aailable bt also on s( j) that is nknown oer the prediction horizon. Conseqently, the error ε( j) is assmed to be constant oer the prediction horizon: ε( j) = ε(k) = s(k) s m (k), j [k + 1;k + N p ]. (2.8) Finally, Ũ = {U(k),U(k + 1),...,U(k + N c ),...,U(k + N p 1)} is the optimal control seqence. From U(k + N c + 1) to U(k + N p 1), the control inpt is constant and eqal to U(k + N c )wheren c is the control horizon. The weighted matrix Q( j) isa symmetric definite positie matrix. One of the main adantages of VPC is the capability to explicitly handle constraints in the optimization problem. Three kinds of constraints are distingished: constraints on the state of the robotic system. It can typically be a mechanical constraint sch as workspace limit when the state represents the camera pose for instance, x min x(k) x max ; (2.9) 2D constraints also named isibility constraints to ensre that the isal featres stay in the image plane or to represent forbidden areas in the image. The latter

6 398 Gillame Allibert, Estelle Cortial, and François Chamette can be ery sefl to deal with obstacle aoidance or image occlsion, s min s m (k) s max ; (2.1) control constraints sch as actator limitations in amplitde or elocity, U min U(k) U max. (2.11) These constraints are added to the problem (2.4) which becomes a nonlinear constrained optimization problem: min J(U) (2.12) Ũ K where K is the constraint domain defined by: { C(U) (2.13) Ceq(U) =. The constraints (2.9), (2.1) and (2.11) can be formlated by nonlinear fnctions C(U) andceq(u) [8]. Nmeros constrained optimization rotines are aailable in software libraries to sole this kind of problem: projected gradient methods, penalty methods, etc. In or case, a seqential qadratic program (SQP) is sed and more precisely, the fnction fmincon from Matlab optimization toolbox. The setting parameters of the predictie approach are the prediction horizon (N p ), the control horizon (N c ) and the weighted matrix (Q( j)): the prediction horizon is chosen in order to satisfy a compromise between scheme stability (long horizon) and nmerical feasibility in term of comptational time reqirement (short horizon); the control inpt is sally kept constant oer the prediction horizon, which corresponds to a control horizon eqal to 1. A N c > 1 can be sefl for stabilization task of nonholonomic mobile robot for instance [1]; the matrix Q( j) is often the identity matrix bt it can also be a time-arying matrix sefl for stabilizing the system. If Q(k + 1) = I and Q(k + l) = l [2; N p ], the cost fnction J is then similar to the standard criterion of IBVS. It is also eqialent to hae a prediction horizon eqal to Model of Image Prediction Here we focs on the model sed to predict the image eoltion. We consider a 6 DOF free-flying perspectie camera obsering fixed point featres. A 3D point with coordinates P = (X,Y,Z) in the camera frame is projected in the image plane as a 2D point with coordinates s = (,). The sampling period is T e and the control inpt U is the camera elocity noted τ = (T x,t y,t z,w x,w y,w z ). The role of the model is to predict, oer the horizon N p, the eoltion of the isal featres in regard to the camera elocity. The principle of the image prediction is

7 2 Visal Seroing ia Nonlinear Predictie 399 depicted in Fig To perform this image prediction, two kinds of model can Desired image Crrent image Model inpt Camera elocity Model otpt: Predicted featres oer Np k k k+1 k+2 k+3 k+1 k+1 k+2 k+3 k+4 k+2 Time k+2 k+3 k+4 k+5 Fig. 2.2 Principle of image prediction (N p = 3, N c = 2). be considered: a nonlinear global model and a local model based on the interaction matrix. The identification of the model, described aboeby (2.7),is discssed with respect to both cases in the next section Nonlinear Global Model The control inpt of the free-flying process is the camera elocity τ applied to the camera. Here, the state of the system can be the camera pose in the target frame: x = (P x, P y, P z,θ x,θ y,θ z ). The dynamic eqation can be approximated by 1 : x(k + 1) = x(k) + Teτ(k) = f (x(k),τ(k)). (2.14) The otpt is the isal featres expressed in the image plane noted s m. In the case of a perspectie camera, the otpt eqation for one point-like featre in normalized coordinates can be written as: s m (k) = ( (k) (k) ) = ( ) X(k)/Z(k) = g(x(k),y(k),z(k)), (2.15) Y(k)/Z(k) where (X,Y,Z,1) Rc are the point coordinates in the camera frame. The rigid transformation between the camera frame and the target frame, noted l(x), can easily be dedced from the knowledge of the camera pose x(k). If the point coordinates are known in the target frame, (X,Y,Z,1) Rt, then the point coordinates in the camera frame, (X,Y,Z,1) Rc are gien by: 1 The exponential map cold be also sed to better describe the camera motion.

8 4 Gillame Allibert, Estelle Cortial, and François Chamette X Y Z 1 R c = ( ) X R(x) T(x) Y Z 1 R t = l(x(k)). (2.16) Finally, we obtain: s m (k) = g l(x(k)) = h(x(k)). (2.17) Eqations 2.7 are now completely identified with (2.14) and (2.17). This dynamic model combines 2D and 3D data and so it is appropriate to deal with 2D and/or 3D constraints. The constraints are respectiely expressed on the states and/or the otpts of the prediction model and are easily added to the optimization problem. The nonlinear global model has a large alidity domain and ths it can be sed for large displacements. Neertheless, the prediction oer the prediction horizon can be time consming. Moreoer, this model reqires 3D data that are the pose of the target in the initial camera frame, as well as the target model. To redce the 3D knowledge, a soltion can be the linearization of the model based on the interaction matrix Local Model Based on the Interaction Matrix For a point-like featre s expressed in normalized coordinates sch that = X/Z and = Y/Z, the interaction matrix related to s is gien by [5]: [ 1 L s = Z Z (1 + 2 ] ) Z 1 Z 1 +. (2.18) 2 The ale Z is the depth of the 3D point expressed in the camera frame. The relationship between the camera elocity τ and the time ariation of the isal featres ṡ is gien by: ṡ(t) = L s (t)τ(t). (2.19) In [11], this dynamic eqation is soled to reconstrct the image data in case of occlsion. Here, with a simple first order approximation, we obtain: s(k + 1) = s(k) + T e L s (k)τ(k). (2.2) To aoid the estimation of the depth parameter at each iteration, its ale Z gien or measred at the reference position can be sed. Conseqently, the interaction matrix (2.18) becomes L s and depends only on the crrent measre of the isal featres. By considering here the isal featres s as the state x, we obtain the set of eqations describing the process dynamics and otpts (2.7): { x(k + 1) = x(k) + Te L s (k)τ(k) = f (x(k),τ(k)) (2.21) s m (k) = x(k) = h(x(k)).

9 2 Visal Seroing ia Nonlinear Predictie 41 This approximated local model does not reqire 3D data bt only the approximate ale of Z. 2D constraints can be taken into accont since the model states and otpts are the isal featres. On the other hand, no information is aailable on the camera pose and so 3D constraints can not be directly handled. For doing that, as for the nonlinear model, it wold be necessary to reconstrct the initial camera pose by sing the knowledge of the 3D model target. That is of corse easily possible bt has not been considered in this chapter. Finally, for large displacements, a problem can be mentioned as we will see on simlations: the linear and depth approximations may be too coarse and can lead to control law failres. 2.4 Simlation Reslts For all presented simlations, the sampling time T e is eqal to 4 ms. This choice allows considering real-time application with an sal camera (25 fps). The control task consists in positioning a perspectie free-flying camera with respect to a target composed of for points. These for points form a sqare of 2 cm in length in Cartesian space. The reference image is obtained when the target pose expressed in the camera frame (R C ) is eqal to P T/C = (,,.5,,,) (see Fig. 2.3), where the first three components are the translation expressed in meters and the last three components are the roll, pitch and yaw angles expressed in radians. The coordinates of the for points in the reference image are: s = ( d1, d1, d2, d2, d3, d3, d4, d4 ) = (.2,.2,.2,.2,.2,.2,.2,.2) (see Fig. 2.4). 3D plane Z R C X C YC Z C RT X T.2 Z T.3 YT.5 Y.5.5 X Fig D desired postre. Fig D reference image. Different simlations illstrate the performance of the VPC strategy. Besides the choice of the model sed to predict the point positions, VPC reqires to set three parameters, the prediction horizon N p, the control horizon N c and the weighted matrix Q( j): the choice of the prediction horizon is crcial. The system behaior and the conergence speed depend on the prediction horizon. The ale of N p is discssed below;

10 42 Gillame Allibert, Estelle Cortial, and François Chamette the control horizon is kept constant and eqal to 1 (N c = 1). Only one control is calclated oer the prediction horizon; the weighted matrix Q( j) is either the identity matrix Q( j) = I 8 8 j, constant oer the prediction horizon, or a time-arying matrix Q( j) = 2Q( j 1) with Q(1) = I 8 8. In this last case, this matrix weights the error at each sampling instant more and more oer the prediction horizon and so, stresses the error at the end of the horizon N p which corresponds to the final objectie. In stabilization task, this time-arying matrix can be compared to the terminal constraint sed in the classical predictie control strategy. Howeer it is less restrictie for the optimization algorithm. In the seqel, the time-arying matrix is noted Q TV. The VPC simlation reslts are compared with the classical IBVS approaches based on the exponential control law (2.1) where L s can be chosen as: L s = L (s(t),z(t)), noted L c : the interaction matrix is pdated at each iteration; L s = L (s(t),z ), noted L p : the depth compted or measred at the reference position noted Z is sed. The interaction matrix aries only throgh the crrent measre of the isal featres; L s = L (s,z ), noted L d : the interaction matrix is constant and corresponds to its ale at the reference position; L s = 1 2 ( L (s,z ) + L (s(t),z(t)) ), noted L m : the interaction matrix is the mean of the constant and crrent interaction matrices. In order to compare the VPC approach with the classical IBVS, no constraint on the control inpt is considered in the first part. Then, mechanical and isibility constraints will be taken in consideration with the VPC approach. In all cases, the control inpts are normalized if needed. The bonds are.25 m/s for the translation speed and.25 rad/s for the rotation speed Case 1: Pre Rotation arond the Optical Axis In case 1, the reqired camera motion is a pre rotation of π 2 radians arond the optical axis. De to the lack of space and since it is a classical case, all simlation reslts are not presented here bt all are discssed Classical IBVS For the classical IBVS, the following reslts are obtained: with L c, the trajectories in the image plane are pre straight lines as expected [5]. The camera motion is then a combination of a backward translation and a rotation with respect to the camera optical axis (retreat problem). De to this ndesired retreat, the camera might reach the limit of the workspace;

11 2 Visal Seroing ia Nonlinear Predictie 43 with L p, the reslts are approximatiely the reslts obtained in the first case (retreat problem), see Fig The isal featre trajectories tend to straight lines; with L d, the camera moes toward the target and simltaneosly rotates arond the optical axis (adance problem) [5]. De to this ndesired forward motion, some featres can go ot the camera field of iew dring the camera motion; with L m, the camera motion is a pre rotation [18]. No additional motion is indced along the optical axis and the isal featre trajectories are circlar (pixels) ε 1 ε 2 ε 3 ε Fig. 2.5 Case 1: Classical IBVS with L p. (pixels) ε 1 ε 2 ε 3 ε ε T x ε T y ε T z ε W x ε W y ε W z VPC with a Local Model (VPC LM ) The following simlations are obtained with the VPC strategy sing a local model based on the interaction matrix L p. The comparison with the classical IBVS is done for different N p ales (N p = 1,1,2) and different weighted matrices (Q( j) = I or Q TV ). For N p = 1, the reslts are similar to the classical IBVS with L p since the model sed to predict the next image is exactly the same (see Fig. 2.6). The only difference is the behaior of the control law, decreasing exponentially with IBVS. For N p = 1 (see Fig. 2.7) or N p = 2 (see Fig. 2.8), the trajectories in the image plane become circlar. Indeed, the only constant control oer N p which minimizes the cost fnction is a pre rotation. Ths the translation motion along the optical axis decreases with the increase of N p ale.

12 44 Gillame Allibert, Estelle Cortial, and François Chamette The time-arying matrix Q TV accentates the decopling control by giing importance at the end of N p which corresponds to the final objectie (see Fig. 2.9). It seems to be eqialent to the behaior obtained with L m which takes into accont the desired position. For a π rotation arond the optical axis, the classical IBVS with L c, L p or L d fails as well as VPC LM with Q( j) = I and whateer N p.onthe other hand, VPC LM achiees the satisfying motion with Q TV and N p 2 (see Fig. 2.1). To illstrate the capability of isibility constraint handling, the isal featres are constrained to stay in a window defined by the following ineqalities: [ min=.22 min=.22 ] s m ( j) [ max =.22 max =.22 ]. (2.22) In that case, VPC LM satisfies both isibility constraint and control task (see Fig. 2.11). A translation along the optical axis is then indced to ensre that the isal featres do not get ot the camera field of iew (pixels) ε 1 ε 2 ε 3 ε Fig. 2.6 Case 1: VPC LM with N p = 1, Q( j) = I. (pixels) ε 1 ε 2 ε 3 ε ε T x ε T y ε T z ε W x ε W y ε W z VPC with a Nonlinear Global Model (VPC GM ) The preios reslts obtained with VPC LM are improed with VPC GM since no linearization is done. For instance, with VPC GM and N p = 1, the image plane tra-

13 2 Visal Seroing ia Nonlinear Predictie (pixels) ε 1 ε 2 ε 3 ε (pixels) ε 1 ε 2 ε 3 ε ε T x ε T y ε T z ε W x ε W y ε W z Fig. 2.7 Case 1: VPC LM with N p = 1, Q( j) = I (pixels) ε 1 ε 2 ε 3 ε (pixels) ε 1 ε 2 ε 3 ε ε T x ε T y ε T z ε W x ε W y ε W z Fig. 2.8 Case 1: VPC LM with N p = 2, Q( j) = I. jectories are circlar as the ones obtained with L m. Here, we focs on 3D constraint handling. Added to the isibility constraint, we limit the camera workspace along

14 46 Gillame Allibert, Estelle Cortial, and François Chamette (pixels) ε 1 ε 2 ε 3 ε (pixels) ε 1 ε 2 ε 3 ε ε T x ε T y ε T z ε W x ε W y ε W z Fig. 2.9 Case 1: VPC LM with N p = 1, Q( j) = Q TV (pixels).5.5 ε 1 ε 2 ε 3 ε (pixels) ε 1 ε 2 ε 3 ε ε T x ε T y ε T z ε W x ε W y ε W z Fig. 2.1 π rotation arond optical axis: VPC LM with N p = 2, Q( j) = Q TV. the Z c axis. As can be seen on Fig. 2.12, VPC GM conerges nder both constraints by sing the other camera DOF. If no admissible trajectory ensring isibility con-

15 2 Visal Seroing ia Nonlinear Predictie (pixels) ε 1 ε 2 ε 3 ε (pixels) ε 1 ε 2 ε 3 ε ε T x ε T y ε T z ε W x ε W y ε W z Fig Case 1: VPC LM with N p = 1, Q( j) = I and isibility constraint. straints and 3D constraints (sch as.5 < X c <.5,.5 < Y c <.5, Z c <.6m) exists, VPC GM stops at the position minimizing the constrained cost fnction (see Fig. 2.13) Case 2: Large Displacement In case 2, the initial target pose expressed in the camera frame is gien by P T/C = (.4,.3,.35,1,.98,.43). The classical IBVS with L p or L d does not conerge for sch a large displacement. Indeed, dring the motion, the camera reaches the object plane where Z =. The same reslt is obtained by the VPC LM whateer N p de to the too coarse approximations. Howeer, the conergence is obtained with the weighted time-arying matrix Q TV.VPC GM always conerges een if Q( j) = I (see Fig. 2.15). The trajectories in the image plane are ery similar to the ones obtained with the classical IBVS with L m (see Fig. 2.14). These good reslts are still kept een if isibility constraints (.29< <.29,.4 < <.4) are considered (see Fig. 2.16).

16 48 Gillame Allibert, Estelle Cortial, and François Chamette (pixels) ε 1 ε 2 ε 3 ε (pixels) ε 1 ε 2 ε 3 ε Z T/C Meters Withot 3D constraint With 3D constraint Fig Case 1: VPC GM with N p = 1, Q( j) = I, isibility constraint and Z T/C <.6 m (pixels) ε 1 ε 2 ε 3 ε (pixels) ε 1 ε 2 ε 3 ε ε T x ε T y ε T z ε W x ε W y ε W z Fig Case 1: VPC GM with N p = 1, Q( j) = I, isibility and strong 3D constraints sch that there is no soltion to the optimization problem.

17 2 Visal Seroing ia Nonlinear Predictie (pixels) (pixels) ε.2 1 ε 2.1 ε 3 ε ε 1.4 ε 2.2 ε 3 ε ε T x ε T.8 y ε T z.6 ε W x.4 ε W y ε W.2 z Fig Case 2: Classical IBVS with L m (pixels) (pixels) ε.2 1 ε 2.1 ε 3 ε ε 1.4 ε 2.2 ε 3 ε ε T x ε T.8 y ε T z.6 ε W x.4 ε W y ε W.2 z Fig Case 2: VPC GM with N p = 1, Q( j) = I. 2.5 Conclsions In this chapter, we hae shown that an alternatie approach of IBVS can be the VPC strategy. The isal seroing task is then formlated into a nonlinear optimiza-

18 41 Gillame Allibert, Estelle Cortial, and François Chamette ε T x ε T y ε T z ε W x ε W y ε W z (pixels) (pixels) ε.2 1 ε 2.1 ε 3 ε ε 1.4 ε 2.2 ε 3 ε Fig Case 2: VPC GM with N p = 1, Q( j) = I and isibility constraint. tion problem oer a prediction horizon. The adantage of this formlation is the capability of easily dealing with isibility constraints and 3D constraints. The optimization procedre can be compared to an on-line implicit and optimal constrained path-planning of featres in the image plane. The choice of the image prediction model has been discssed. The approximated local model can be less efficient than the global model for difficlt configrations bt no 3D data are reqired. On the other hand, if 3D data are aailable, VPC GM gies satisfying reslts for any initial configration and motion to achiee. The VPC setting parameters, i.e., the prediction horizon and the weighted matrix, play a crcial role in terms of camera and isal featre trajectories. Simlation reslts highlight the efficiency of VPC. Finally, this strategy is ery flexible and can be sed whateer the robotic system (mobile robot or robot arm) and the camera (perspectie or catadioptric). References [1] Allibert G, Cortial E, Toré Y (26) Visal predictie control. In: IFAC Workshop on Nonlinear Predictie for Fast Systems, Grenoble, France [2] Allibert G, Cortial E, Toré Y (28) Real-time isal predictie controller for image-based trajectory tracking of mobile robot. In: 17th IFAC World Congress, Seol, Korea [3] Allibert G, Cortial E, Toré Y (28) Visal predictie control for manip-

19 2 Visal Seroing ia Nonlinear Predictie 411 lators with catadioptric camera. In: IEEE Int. Conf. on Robotics and Atomation, Pasadena, USA [4] Chamette F (1998) Potential problems of stability and conergence in imagebased and position-based isal seroing. In: Kriegman D, Hager G, Morse A (eds) The Conflence of Vision and, LNCIS Series, No 237, Springer- Verlag, pp [5] Chamette F, Htchinson S (26) Visal sero control, part i: Basic approaches. IEEE Robotics and Atomation Magazine 13(4):82 9 [6] Chamette F, Htchinson S (27) Visal sero control, part ii: Adanced approaches. IEEE Robotics and Atomation Magazine 14(1): [7] Chesi G (29) Visal seroing path planning ia homogeneos forms and lmi optimizations. IEEE Trans Robotics and Atomation 25(2): [8] Chong E, Zak S H (21) An Introdction to Optimization. John Wiley & Sons Inc, 2nd Edition [9] Danès P, Bellot D (26) Towards an lmi approach to mlticriteria isal seroing in robotics. Eropean Jornal of 12(1):86 11 [1] Drola S, Danès P, Cotinho D, Cordesses M (29) Rational systems and matrix ineqalities to the mlticriteria analysis of isal seros. In: IEEE Int. Conf. on Robotics and Atomation, Kobe, Japan [11] Folio D, Cadenat V (28) Dealing with isal featres loss dring a isionbased task for a mobile robot. International Jornal of Optomechatronics 2(3): [12] Gangloff J, De Mathelin M (22) Visal seroing of a 6 dof maniplator for nknown 3-d profile following. IEEE Trans Robotics and Atomation 18(4): [13] Ginhox R, Gangloff J, De Mathelin M, Soler M, Sanchez L (25) Actie filtering of physiological motion in robotized srgery sing predictie control. IEEE Trans Robotics and Atomation 21(1):67 79 [14] Hashimoto K, Kimra H (1993) Visal Seroing, ol 7, K. Hashimoto, Ed. (Robotics and Atomated Systems). Singapore:World Scientific, chap LQ optimal and nonlinear approaches to isal seroing, pp [15] Htchinson S, Hager GD, Corke P (October 1996) A ttorial on isal sero control. IEEE Trans Robotics and Atomation 12(5): [16] Kazemi M, Gpta, K, Mehrandezh M (29) Global path planning for robst isal seroing in complex enironments. In: IEEE Int. Conf. on Robotics and Atomation, Kobe, Japan [17] Mahony R, Corke P, Chamette, F (22) Choice of image featres for depthaxis control in image-based isal sero control. In: IEEE/RSJ Int. Conf. on Intelligent RObots and Systems, Lasanne, Switzerland [18] Malis E (24) Improing ision-based control sing efficient second-order minimization techniqes. In: IEEE Int. Conf. on Robotics and Atomation, New Orleans, LA, USA [19] Mezoar Y, Chamette F (23) Optimal camera trajectory with image-based control. Int Jornal of Robotics Research 22(1): [2] Morari M, Zafirio E (1983) Robst. Dnod

20 412 Gillame Allibert, Estelle Cortial, and François Chamette [21] Nasisi O, Carelli R (23) Adaptie sero isal robot control. Robotics and Atonomos Systems 43(1):51 78 [22] Papanikolopolos N, Khosla P, Kanade T (Febrary 1993) Visal tracking of a moing target by a camera monted on a robot: A combination of ision and control. IEEE Trans Robotics and Atomation 9(1):14 35 [23] Saée M, Poignet P, Dombre E, Cortial E (26) Image based isal seroing throgh nonlinear model predictie control. In: 45th IEEE CDC, San Diego, CA, pp [24] Schramm F, Morel G (26) Ensring isibility in calibration-free path planning for image-based isal seroing. IEEE Trans Robotics and Atomation 22(4):