Control o industrial robots Control with eternal sensors Pro. Paolo Roo (paolo.roo@polimi.it) Politenio di Milano Dipartimento di Elettronia, Inormazione e Bioingegneria
Introdution So ar we have assumed that the robot is ontrolled in a ree environment However the robot has to interat with the environment either to manipulate it, or to avoid to ollide with it, or to interat with other devies or humans In order to eetively interat with the environment, the robot must be endowed with eternal (eteroeptive) sensors, whih make the robot ontroller aware o the situation Common sensors used to measure suh interation are ore sensors and vision sensors In the ollowing we will study with some detail the ontrol systems based on ore measurements. We will also give a short outline on vision based ontrol systems Control o industrial robots Control with eternal sensors P. Roo [2]
Interation with the environment A robot ommonly interats with the working environment, manipulating objets and perorming operations on suraes (think o grinding, deburring, part assembly). In modern robotis, situations are ommon where the robot physially interats with the operator (think o telemanipulation or rehabilitation). Control o industrial robots Control with eternal sensors P. Roo [3]
Passive ontrol o ompliane A irst way to address the interation with the environment is to endow the manipulator with devies that ailitate the eeution o the task in a passive way: RCC: Remote Center o Compliane The RCC is used in assembly tasks (peg-in-a-hole) It is typially plaed between the robot s wrist and the gripper The RCC lets the gripper assembly move in the plane perpendiular to the peg s ais. This allows the peg to rotate With the RCC, the ores generated by any misalignment an be ompensated in a passive way Control o industrial robots Control with eternal sensors P. Roo [4]
Ative ontrol o the interation During more general interation tasks, the use o a purely positional ontrol strategy (the same adopted in ree motion) may lead to problems due to positioning errors and unertainties in task planning, related to a inomplete knowledge o the environment. Eamples where an ative ontrol o the interation is desirable inlude: mehanial mahining (deburring, surae inishing, polishing, assembly,...) tele-manipulation ooperation o multi-manipulator systems deterous robot hands physial human-robot interation Eperiments in ore/position ontrol with a rigid surae at POLIMI. For an ative ontrol o interation it is neessary to use devies or ore (and moment) measurement. Control o industrial robots Control with eternal sensors P. Roo [5]
Fore sensors Fore/torque sensors are devies that return the measurements o three omponents o ore and three omponents o moment with respet to a loal rame. Control o industrial robots Control with eternal sensors P. Roo [6]
Internal behaviour Sensors are based on strain gauges, i.e. devies that an measure a strain. The strain indues a variation in the resistane o a iruit. Strain gauges are suitably mounted, in suh a way as to return inormation rom whih three omponents o ore and moment an be omputed. Control o industrial robots Control with eternal sensors P. Roo [7]
Taonomy Fore based interation ontrol strategies an be divided into two ategories: Impedane/admittane ontrol The goal is to assign a presribed dynami relation between interation ores and position errors We want that the manipulator, in ase o unplanned interations, behaves like a generalized mass-spring-damper system Hybrid position and ore ontrol We separate diretions whih are only position ontrolled rom diretions whih are only ore/torque ontrolled In the diretions onstrained by the environment we want that a ore/torque with a speiied value be established Furthermore we will talk about: Epliit ontrol The ore (or the impedane) is diretly assigned, ating on the ontrol variables (joint torques) Impliit ontrol The ore (or the impedane) is indiretly assigned, ating on the set-points o the position ontrol loops Control o industrial robots Control with eternal sensors P. Roo [8]
Mehanial impedane The low o power among two physial systems an always be deined as the produt o two onjugate quantities: a (generalized) eort and a (generalized) low In the eletrial ramework, the low is the urrent, while the eort is the voltage. In the mehanial ramework, instead, the low is the (linear or angular) veloity, the eort is the ore (or the moment) The mehanial impedane is then deined as dynamial relation that is established between ore and veloity (or displaement) or a mehanial system. The admittane is just the reiproal o the impedane The impedane ontrol aims at making the manipulator, position ontrolled and in interation with the environment, assume a desired mehanial impedane, like a generalized mass-spring-damper system Control o industrial robots Control with eternal sensors P. Roo [9]
Mehanial impedane An eample o impedane/admittane ontrol or physial human-robot interation: Control o industrial robots Control with eternal sensors P. Roo [10]
A 1 d.o.. system In order to understand how an impedane ontroller operates, let us irst onsider the simple eample o a body with mass M interating with the environment, whih eerts a ore : u M The system an be modeled with the blok diagram: u + 1 M.. 1 s. 1 s The model o the interation with the environment is not speiied Control o industrial robots Control with eternal sensors P. Roo [11]
A 1 d.o.. system Let us onsider an inverse dynamis motion ontrol law: u = My with: y = && + K ~ & + K d D P ~ where: ~ = d The losed loop dynamis are obtained: M ~ && + D ~ & d + Kd ~ = where: D = MK, K = d D d MK P The equation establishes a mehanial impedane relation between the ore ating on the system and the positional error. However the inertial omponent o the impedane (M) annot be arbitrarily assigned. Control o industrial robots Control with eternal sensors P. Roo [12]
A 1 d.o.. system Suppose now that the ore is measured and let: u = My + with: y = ( ) M && + D ~ + K ~ 1 & d d d M d.. d d This orresponds to the blok diagram: The losed loop dynamis are:. d d + + D d K d + 1 + + M d + y M + + u 1 M.. 1 s. 1 s M ~ & d + Dd ~ & + Kd ~ = It is an impedane relation where all the terms an be arbitrarily assigned. Control o industrial robots Control with eternal sensors P. Roo [13]
Impliit impedane ontrol (admittane ontrol) When a positional ontrol is already applied to the mehanial system, it might be onvenient to lose the impedane ontrol outside the position ontrol. Consider again the ase o single d.o.. system, position ontrolled: d R u M The system an be modeled with the blok diagram: + u + d R(s) G(s) ( ) 1 G s = 2 Ms Control o industrial robots Control with eternal sensors P. Roo [14]
Impliit impedane ontrol (admittane ontrol) Consider the ollowing blok diagram, where the ore applied by the environment is measured and iltered through a system with transer untion G d (s), to give a variation o the reerene d : d + + R(s) u + G(s) G d (s) impedane ontroller Elaborating the blok diagram: = G s 1+ R s ( ) ( ) G( s) + ( G ( s) ) d d ( ) G( s) ( ) G( s) R s 1+ R s Also alled admittane ontroller (it aepts a ore and delivers a displaement) Control o industrial robots Control with eternal sensors P. Roo [15]
Impliit impedane ontrol (admittane ontrol) In a band o requenies well inside the bandwidth o the position ontrol system, we an assume the ore disturbane well rejeted and the position reerene orretly ollowed. Thus: d Gd ( s) = G ( s) d 1~ I we impose: G d ( s) = M d s 2 1 + D d s + K d M ~ && d + Dd ~ & + Kd ~ = we assign to the manipulator the desired impedane behaviour. The advantage o this sheme is that the position ontrol is kept unhanged: the impedane ontrol modiies the set-points o the position ontrol, whih is usually the only openness allowed by an industrial motion ontrol system. Control o industrial robots Control with eternal sensors P. Roo [16]
Statis o the manipulator Beore applying the onepts o impedane ontrol to a omplete manipulator, we need to inlude the eet o a ore applied at the end-eetor in the model. Suppose then that a ore is applied at the end-eetor. What are the joint torques that keep the system in equilibrium???? Control o industrial robots Control with eternal sensors P. Roo [17]
Systems o ores As a matter o at a system o ores may at at the end-eetor. How an this system be redued? From lassial mehanis One ore applied in a point (the resultant) One torque It ollows that we will always be able to represent a system o ores with two vetors: A ore vetor applied in a point O (the resultant) A moment vetor (the moment o the original system o ores with respet to the ulrum O) Control o industrial robots Control with eternal sensors P. Roo [18]
Statis o the manipulator Let us then onsider a manipulator subjet to a set o ores at the end eetor, having a resultant and moment, w.r.t. the origin o the end eetor rame, µ. We then deine vetor: h = µ Let us all τ the vetor o torques (or ores) applied to the joints. The virtual work o the ores/torques at the joints is: T δw τ = τ δq The virtual work o the ores at the end-eetor is: T T δw = δp + µ ωdt = J µ h where δq is the elementary displaement o the joint oordinates. T P T T ( q) δq + J ( q) δq = h J( q) δq where δp is the virtual displaement o the origin o the end-eetor rame, onsistent with the dierential kinematis o the robot, and ω is the angular veloity o the end eetor rame. O Control o industrial robots Control with eternal sensors P. Roo [19]
Statis o the manipulator T δw τ = τ δq T δw = J µ h P T T ( q) δq + J ( q) δq = h J( q) δq, O J ( q) = J J P O ( q) ( ) q position Jaobian orientation Jaobian Based on the priniple o virtual work the manipulator is in stati equilibrium i and only i: δw τ = δw h τ = ( q)h J T Notie that the same matri, the Jaobian matri, whih deines the dierential kinematis o the manipulator, is also involved in the statis o the system (kinetostati duality). Control o industrial robots Control with eternal sensors P. Roo [20]
Inverse dynamis ontrol The dynami model in interation with the environment is then given by the equation: ( )& T + C( q, q& ) q& + g( q) = τ J ( q)h B q q where h is the vetor o the ontat ores (and moments) applied by the end eetor on the environment. Let us onsider now the adoption o an inverse dynamis ontrol law: τ ( q) y + C( q, q& ) q g( q) = B & + Substituting we obtain: q&& = y B 1 T ( q) J ( q)h nonlinear oupling term Control o industrial robots Control with eternal sensors P. Roo [21]
Inverse dynamis ontrol Assume now or y the epression introdued in the inverse dynami ontrol in the operational spae: y By substituting the epression or y we obtain the equation o the dynamis in losed loop: && ~ where = d, while: B A 1 A = J ( ) q& ( q) & + K ~ & + K ~ J& ( q, q& ) & d T 1 ( q) J ( q) B( q) J ( q) D ~ K P ~ 1 K D ~ + + = B A = A A P ( q) h A is the inertia matri in the operational spae and: h A = T T A ( ) h, J = T A ( ) J A A We have obtained an impedane relation whih is oupled and only partially assignable: a ore applied along a Cartesian diretion yields motion in other diretions as well. T J h = J generalized ores perorming work on ẋ Control o industrial robots Control with eternal sensors P. Roo [22] T A h A
Impedane ontrol with ore measurement I ore measurements are available, we an set: τ T ( ) y + C( q, q& ) q + g( q) + J ( q)h = B q & and: y = J 1 A ( ) q& h 1 ( q) M M & + D ~ & + K ~ M J& ( q, q& ) d d d d d d A A where M d, D d, K d are diagonal, positive deinite matries. We obtain: M ~ && + D ~ & + K ~ = d d d h A i.e. a ompletely deoupled system. Control o industrial robots Control with eternal sensors P. Roo [23]
Impedane ontrol The equation: M ~ && + D ~ & + K ~ = d d d h A deines a mehanial impedane relation between the ores/moments deined at the right hand side o the equations and the position/orientation errors in the operational spae. This impedane has the orm o a mehanial system with mass matri M d, damping D d, and stiness K d. The dynami behaviour along eah diretion o the operational spae an thus be speiied: a ore or a moment along a ertain Cartesian diretion yields a motion in suh diretion, whih is related to the ore and moment by a ully ontrollable dynamis. But what do we eatly mean or impedane in the operational spae? Control o industrial robots Control with eternal sensors P. Roo [24]
Translational impedane The irst three degrees o reedom in the operational spae are the position oordinate o the end-eetor. In this ase the deinition o the impedane is rather intuitive: p d : desired position p e : atual position p = p d p e The impedane relation is: M p&& + D p& + K p = p p p We thus dupliate on eah degree o reedom what we have already seen or the single mass. Control o industrial robots Control with eternal sensors P. Roo [25]
Rotational impedane The remaining three degrees o reedom in the operational spae onsist in a minimal representation (Euler angles) o the end-eetor orientation. In this ase the deinition o the impedane is less intuitive: φ d : desired orientation φ e : atual orientation φ = φ d φ e The impedane relation is: M && φ + D φ& + K φ = T φ φ φ Τ ( φ )µ e Due to the presene o matri T, this relation is subjet to representation singularities. Better deinition o the rotational impedane have been proposed in the literature. Control o industrial robots Control with eternal sensors P. Roo [26]
Natural onstraints and artiiial onstraints An alternative way to perorm an interation task is to assign reerene values o the ores and o the positions, onsistent with the geometry o the environment. Some diretions an be in at subjeted to onstraints on the position, other ones to onstraints on the ores that an be applied. In general, it is not possible to impose both the ore, and the position, along the same diretion. We make the ollowing distintion: Natural onstraints: Artiiial onstraints: they are imposed by the environment along eah degree o reedom o the task; they depend on the geometry o the task. onstraints imposed by the ontrol system, related to the task eeution strategy. For task desription we reer to a onstraint rame, y, z. Control o industrial robots Control with eternal sensors P. Roo [27]
Sliding on a lat surae Natural Constraints p& ω ω z y y µ z Artiiial Constraints µ µ z p& p& ω y y z This piture is taken rom the tetbook: B. Siiliano, L. Siavio, L. Villani, G. Oriolo: Robotis: Modelling, Planning and Control, 3rd Ed. Springer, 2009 Control o industrial robots Control with eternal sensors P. Roo [28]
Insertion o a peg in a hole Natural Constraints p& p& ω ω y y z Artiiial Constraints µ µ y p& y z This piture is taken rom the tetbook: µ z. Usually we set p z 0 and all the other artiiial onstraints to zero. ω z B. Siiliano, L. Siavio, L. Villani, G. Oriolo: Robotis: Modelling, Planning and Control, 3rd Ed. Springer, 2009 Control o industrial robots Control with eternal sensors P. Roo [29]
Rotation o a rank Oriented like the rank Natural Constraints p& p& ω z Artiiial Constraints µ z Idle knob ω y y µ z µ p& ω y y z This piture is taken rom the tetbook: B. Siiliano, L. Siavio, L. Villani, G. Oriolo: Robotis: Modelling, Planning and Control, 3rd Ed. Springer, 2009 Normally artiiial onstraints on ores and moments are assumed to be zero. Control o industrial robots Control with eternal sensors P. Roo [30]
Seletion matri We assume that the onstraints an always be epressed along mutually orthogonal diretions o rame C. Thus on eah diretion either a natural onstraint on position, or a natural onstraint on ore, eists. In order to speiy the onstraints we use a seletion matri Σ diagonal and with null or unitary elements: Natural onstraints Artiiial onstraints n ( I ) h h Σv = v Σ = ( I Σ) v = v Σh = h a 0 0 Σ = 0 0 0 0 n a 00 0 0 0 0 0 01 0 0 0 The seletion matri etrats diretions where a natural onstraint in veloity applies and thus an artiiial onstraint in ore an be enored 0 0 0 0 0 0 01 0 0 01 0 0 00 0 0 Control o industrial robots Control with eternal sensors P. Roo [31]
Hybrid ore/position ontrol Hybrid ore/position ontrol is a ontrol strategy that uses the artiiial onstraints to speiy the goals o the ontrol system, in terms o both ore and position. The 6 diretions o the operational spae are divided into position/veloity ontrolled diretions and ore/moment ontrolled diretions: Σ h A R h Ad v d Σ I Σ INTERACTION CONTROL MOTION CONTROL y F y v + + y y u h A R NONLINEAR MANIPULATOR COMPENSATIONS AND ENVIRONMENT v I Σ v R Control o industrial robots Control with eternal sensors P. Roo [32]
Soures o inonsisteny Hybrid position/ore ontrol is based on a nominal model o the interation. Inonsisteny may however our in the measurements, due e.g. to: 1. rition at the ontat (a ore is deteted in a nominally ree diretion) 2. ompliane in the robot struture and/or at the ontat (a displaement is deteted in a diretion whih is nominally onstrained in motion) 3. unertainty in the environment geometry at the ontat The irst two soures o inonsisteny are automatially iltered out through the seletion matries. The third soure o inonsisteny an be mitigated by real time estimation proess. Control o industrial robots Control with eternal sensors P. Roo [33]
A 1 d.o.. system In order to have a better idea on how the ontrol system operates along the ore ontrolled diretions, let s onsider the simple eample o a body o mass M in interation with the environment: u M We assume a purely elasti model o the environment. Blok diagram: u + G(s) K u G (s) G K ( s) = 2 Ms + K Control o industrial robots Control with eternal sensors P. Roo [34]
A 1 d.o.. system u G (s) G K ( s) = 2 Ms + K The system presents a unitary gain and resonant, high requeny (or a sti ontat) dynamis. Notie that = u or K. The best way to ontrol this system is through an integral ontroller: + u d R (s) G (s) R ( s ) = K s i This way the bandwidth o the ore ontrol is diretly assigned with the integrator, leaving outside the bandwidth the high requeny dynamis o the interation with the environment. Control o industrial robots Control with eternal sensors P. Roo [35]
Impliit ore ontrol As with impedane ontrol, when a position ontrol is already ative, it might be onvenient to lose the ore ontrol around the position ontrol loop. Consider or simpliity only the ase o a, position ontrolled, single degree o reedom: d R u M The system an be modelled through the ollowing blok diagram: d + R(s) u + G(s) K Control o industrial robots Control with eternal sensors P. Roo [36]
Impliit ore ontrol Under the assumption o an ininitely sti ontat (K ) we obtain =0 and u: d u R(s) 1 I a PID position ontroller is used: R ( s) we an adopt or the ore ontrol the sheme: d + R (s) C(s) d The position ontrol loop opens! = 2 KD s + KPs + K s u R(s) 1 I with: ( ) =, R ( s) C s K D s 2 1 + K P s + K I = K p + K s i The advantage is that the position ontrol is let unhanged. Control o industrial robots Control with eternal sensors P. Roo [37]
Industrial appliation o ore ontrol ABB RobotWare Mahining Fore Control (mahining o blades): Control o industrial robots Control with eternal sensors P. Roo [38]
Visual measurements Artiiial vision devies are useul sensors or robotis beause they mimi the human sense o sight and allow to gather inormation rom the environment without ontat. Nowadays several roboti ontrollers integrate vision systems. The typial use o vision in industrial robotis is to detet an objet in the robot s sene, whose position (and orientation) is then used or online path planning in order to drive the robot to the identiied objet. Online re-planning o the path an also be perormed when the vision system detets some unepeted hange in the path the robot is supposed to ollow (or eample a orner in a ontouring task). Alternatively, visual measurements an be used in a real time eedbak loop in order to improve position ontrol o the end eetor: this is the onept o visual servoing. Control o industrial robots Control with eternal sensors P. Roo [39]
An eample o path re-planning Following a orner during a ontouring task: View rom an eternal amera View rom the onboard amera Control o industrial robots Control with eternal sensors P. Roo [40]
The hardware amera video data rame grabber industrial PC zoom lens Control o industrial robots Control with eternal sensors P. Roo [41]
The amera The amera is a devie that an measure the intensity o light, onentrated by a lens on a plane, the image plane, whih ontains a matri o piels (CCD: Charge Coupled Devie). Piels then transorm the light energy in eletrial energy. The amera perorms a 2D projetion o the sene. This projetion entails a loss o depth inormation: eah point in the image plane orresponds to a ray in the 3D spae. In order to determine the 3D oordinates o a point orresponding to a 2D point in the plane additional inormation is needed: multiple views with a single amera multiple ameras knowledge o harateristi relations between relevant points o the ramed objets Control o industrial robots Control with eternal sensors P. Roo [42]
Perspetive projetion A point with oordinates (X, Y, Z) in the amera rame is projeted into the ollowing point in the image plane, epressed in piels: ξ = y = α λ Z X Y λ: oal length α: sale ator in piel/m Other methods eist to represent the projetion on the 2D plane (saled orthographi projetion, aine projetion) Control o industrial robots Control with eternal sensors P. Roo [43]
Image eatures In artiiial vision we denote with image eature any harateristis that an be etrated rom an image (e.g. an edge or a orner). We then deine a parameter o an image eature a quantity, epressed by a real numeri value, whih an be omputed rom one or more image eatures. Parameters o an image eature an be gathered in a vetor: ξ = [ξ 1, ξ 2, ξ k ]. Eamples o parameters o image eatures: point oordinates length and orientation o a line onneting two points entroids and higher order moments parameters o an ellipse Control o industrial robots Control with eternal sensors P. Roo [44]
Calibration The amera has to be alibrated beore usage in a roboti vision system: Internal alibration: Determination o the intrinsi parameters o the amera (like the oal length λ) as well as o some additional distortion parameters due to lens imperetions and misalignments in the optial system Eternal alibration: Determination o the etrinsi parameters o the amera like the position and the orientation o the amera with respet to a reerene rame Control o industrial robots Control with eternal sensors P. Roo [45]
Camera oniguration The irst deision to be made when setting up a vision-based ontrol system is where to plae the amera. The amera an be: mounted in a ied loation in the workspae (eye-to-hand oniguration) so that it an observe the manipulator and any objets to be manipulated attahed to the robot above the wrist (eye-in-hand oniguration) Eye-To-Hand Eye-In-Hand Control o industrial robots Control with eternal sensors P. Roo [46]
Eye-to-hand oniguration Advantages the ield o view does not hange as the manipulator moves the geometri relationship between the amera and the workspae is ied and an be alibrated oline Eye-To-Hand Disadvantages as the manipulator moves through the workspae it an olude the amera s ield o view Control o industrial robots Control with eternal sensors P. Roo [47]
Eye-in-hand oniguration Advantages the amera an observe the motion o the end eetor at a ied resolution and without olusion as the manipulator moves through the workspae Eye-In-Hand Disadvantages the geometri relationship between the amera and the workspae hanges as the manipulator moves the ield o view an hange drmatially or even small motion o the manipulator Control o industrial robots Control with eternal sensors P. Roo [48]
Control arhitetures: lassiiation Vision ontrol systems an be lassiied based on various riteria. A irst lassiiation is based on the ollowing question: Is the ontrol struture hierarhial, with the vision system providing set-points as input to robot s joint-level ontroller, or does the visual ontroller diretly ompute the joint-level inputs? In the irst ase: In the seond ase: dynami look and move diret visual servoing Advantages o the dynami look and move approah: the redued sampling rate o the visual signal does not ompromise the overall perormane o the position ontrol system in several industrial robot ontrollers it is only allowed to operate at the position setpoints level the robot an be seen as an ideal positioner in the Cartesian spae, thus simpliying the design o the vision ontrol system Control o industrial robots Control with eternal sensors P. Roo [49]
Control arhitetures: lassiiation A seond lassiiation is based on the ollowing question: Is the error signal deined in 3D (task spae) oordinates or diretly in terms o image eatures? In the irst ase : In the seond ase : position based ontrol image based ontrol Position based ontrol: vision data are used to build a partial 3D representation o the world pose estimation algorithms are omputationally intensive (a real-time implementation is required) and sensitive to errors in amera alibration Image based ontrol: uses the image data diretly to ontrol the robot motion an error untion is deined in terms o quantities that an be diretly measured in an image, and a ontrol law is onstruted that maps this error diretly to robot motion Control o industrial robots Control with eternal sensors P. Roo [50]
Typial ontrol arhitetures Dynami position-based look-and-move d + Cartesian ontrol law Joint ontrollers Atuators Camera Pose estimation Image eature etration Video Dynami image-based look-and-move + Control law in the image eature spae Joint ontrollers Atuators Camera Image eature etration Video Control o industrial robots Control with eternal sensors P. Roo [51]
Typial ontrol arhitetures Position-based visual servoing d + Cartesian ontrol law Atuators Camera Pose estimation Image eature etration Video Image-based visual servoing + Control law in the image eature spae Atuators Camera Image eature etration Video Control o industrial robots Control with eternal sensors P. Roo [52]