A Comparatie Study of Vision-Based ateral Control Strategies for Autonomous Highway Driing J. Košecká, R. Blasi, C. J. Taylor and J. Malik Department of Electrical Engineering and Computer Sciences Uniersity of California at Berkeley Berkeley, CA 97, USA. email: janak,blasirs,camillo,malik@cs.berkeley.edu 1 Abstract This paper will present the results of a comparatie study of a set of ision-based control strategies that hae been applied to the problem of steering an autonomous ehicle along a highway. The aim of this work has been to further our understanding of the characteristics of arious control laws that could be applied to this problem with a iew to making informed design decisions. The control strategies that we explored include a lead lag control law, a fullstate linear controller and input-output linearizing control law. Each of these control strategies was implemented and tested on our experimental ehicle, a Honda Accord X, both with and without a curature feedforward component. ntroduction With the increasing speeds of modern microprocesors it has become eer more common for computer ision algorithms to find application in real-time control tasks. n particular, the problem of steering an autonomous ehicle along a highway using the output from one or more ideo cameras mounted inside the ehicle has been a popular target for researchers around the world and a number of groups hae demonstrated impressie results on this control task. Dickmanns et. al. [?] deeloped systems that droe autonomously on the German Autobahn as early as 195. The Nalab project at CMU has produced a number of successful isually guided autonomous ehicle systems. Other research groups include Ozguner et. al. at Ohio State [11], Broggi et al at the Uniersita di Parma, - at the National nstitute of Standards and ockheed-martin. The goal of our research efforts in this field has been to understand the fundamental characteristics of this ision based control problem and to use this knowledge to design better control strategies. n [7] we presented an analysis of the problem of ision-based lateral control and inestigated the effects of changing arious important system parameters like the ehicle elocity, the lookahead range of the ision sensor and the processing delay associated with the perception and control system. We also described a static feedback strategy that enabled us to perform the lateral control task at highway speeds. We were able to erify the accuracy and efficacy of our modelling and control techniques on our experimental ehicle platform, a Honda Accord X. n this paper we present the results of a series of experiments that were designed to proide a systematic comparison of a number of control strategies. The aim of this work has been to further our understanding of the characteristics of arious control laws that could be applied to this problem with a iew to making informed design decisions. The control strategies that we explored include a lead lag control law, a full-state linear controller and input-output linearizing control law. Each of these control strategies was implemented and tested both with and without a curature feedforward component. Section of this paper presents the basic equations that we hae used to model the dynamics of our ehicle and our sensing system. Section 3 describes the design of the obserer that we use to estimate the states of our system and the curature of the roadway. Section describes the arious control strategies that we implemented on our experimental platform and section 5 presents the results of the experiments that we carried out with these controllers. Section contains the conclusions that we hae drawn from these experiments. 3 Modeling The dynamics of a passenger ehicle can be described by a detailed -DOF nonlinear model [1]. Since it is possible to decouple the longitudinal and lateral dynamics, a linearized model of the lateral ehicle dynamics is used for controller design. The linearized model of the ehicle re- 1
tains only lateral and yaw dynamics, assumes small steering angles and a linear tire model, and is parameterized by the current longitudinal elocity. Coupling the two front wheels and two rear wheels together, the resulting bicycle model (Figure 1) is described by the following ariables and parameters: y CG δ f y x y ψ x linear elocity ector ( x, y), x denotes speed f r f side slip angles of the front and rear tires ehicle yaw angle within a fixed inertial frame front wheel steering angle commanded steering angle m total mass of the ehicle l f l r total inertia ehicle around center of graity (CG) distance of the front and rear axles from the CG l distance between the front and the rear axle l f + l r c f c r cornering stiffness of the front and rear tires. F r l r y y l f F f δ f α f β x x R ref Figure : The ision system estimates the offset from the centerline y and the angle between the road tangent and heading of the ehicle " at some lookahead distance. " the angle between the tangent to the road and the ehicle orientation denotes the lookahead distance of the ision system. The equations capturing the eolution of these measurements due to the motion of the car and changes in the road geometry are: _y = x " ; y ; _ () _" = x K ; _ (3) We can combine the ehicle lateral dynamics and the ision dynamics into a single dynamical system of the form: ε _x = A x + B u + E w y = C x Figure 1: The motion of the ehicle is characterized by its elocity = ( x y) expressed in the ehicle s inertial frame of reference and its yaw rate _. The forces acting on the front and rear wheels are F f and F r, respectiely. The lateral dynamics equations are obtained by computing the net lateral force and torque acting on the ehicle following Newton-Euler equations [] and choosing _ and y, as state ariables. The state equations hae the following form: _ y 3 5 = ; a1 m x ;m x +a m x a 3 x ; a x 3 7 5 y _ 3 5 + b 1 b 3 5 f (1) where a 1 = c f + c r, a = c r l r ; c f l f, a 3 = ;l f c f + l r c r, a = lf c f + lr c r, b 1 = cf m and b lf cf =. The additional measurements proided by the ision system (see Figure ) are: y the offset from the centerline at the lookahead, with the state ector x =[ y _ y " ] T, the output y = [ _ y " ] T and control input u = f. The road curature K enters the model as an exogenous disturbance signal w = K. 3.1 Analysis The block diagram of the oerall system following the state equations is shown in Figure 3. The transfer function V 1 (s) between the steering angle f and offset at the lookahead y has the following form: V 1 (s) = 1 s as + bs + c ds + es + f where the numerator is a function of both speed and lookahead distance and the denominator is parameterized by the speed of the car. V 1 (s) can be rewritten according to Figure 3 by singling out the ehicle dynamics in terms of y CG and followed by the integrating action 1=s : () V 1 (s) = 1 s (G(s) +G (s)) (5)
K y = C x () C(s) δ A(s) δ f + ψ.. ψ. - G (s) 1/s G(s). y 1/s y + +. ε 1/s D(s) + 1/s D(s) ε y where x =[y " K ] T, y =[y " ] T. Note that the state ector x includes the road curature K. This differential equation can be conerted to discrete time in the usual manner by assuming that the yaw rate, _, is constant oer the sampling interal T. Figure 3: The block diagram of the oerall system with the two outputs proided by the ision system. where G(s) and G (s) are transfer functions between steering angle and lateral acceleration and yaw acceleration respectiely. The actuator A(s) is modeled as a low pass filter of the commanded steering angle and a pure time delay element D(s) =e ;Tds represents the latency T d of the ision subsystem. n our system T d =:57s. The transfer function C(s) corresponds to the controller to be designed. More detailed analysis of how the behaior of this dynamic system changes as a function of important system parameters like, lookahead distance, processing delay and ehicle elocity can be found in [7]. Vision System The ision-based lane tracking system used in our experiments is an improed ersion of the one presented in [1]. This system takes its input from a single forward-looking CCD ideo camera. t extracts potential lane markers from the input using a template-based scheme. t then finds the best linear fits to the left and right lane markers oer a certain lookahead range through a ariant of the Hough transform. From these measurements we can compute an estimate for the lateral position and orientation of the ehicle with respect to the roadway at a particular lookahead distance,. The ision system is implemented on an array of TMS3C digital signal processors which are hosted on the bus of an ntel-based industrial computer. The system processes images from the ideo camera at a rate of 3 frames per second. 5 Obserer Design n order to estimate the curature of the roadway we hae chosen to implement an obserer based on a slightly simplified ersion of the systems state equations as shown in Equation ( ). More specifically, in these equations we hae chosen to neglect the ehicles lateral elocity, y. _x = A ( x )x + B _ x(k +1)=( x )x(k) + _ (7) Equation (7) allows us to predict how the state of the system will eole between sampling interals. Measurements are obtained from two sources: the ision system proides us with measurements of y and ", while the on-board fiber optic gyro proides us with measurements of the yaw rate of the ehicle, _. Our use of the yaw rate sensor measurements is analogous to the way in which information from the proprioceptie system is used in animate ision. The measurement ector y is used to update an estimate for the state of the system ^x as shown in the following equation: ^x + (k) =^x ; (k) +(y (k) ; C ^x ; (k)) () where ^x ; (k) and ^x + (k) denote the state estimate before and after the sensor update respectiely. The gain matrix can be chosen in a number of ways [], depending on the assumptions one makes about the aailability of noise statistics and the criterion one chooses to optimize. n our case the resulting gain matrix was computed as the steady state optimal gain matrix which minimizes estimation error, using the funtion dlqe aailable in Matlab. The coariances of the both the process and measurement noise were computed from the collected output data while closing the loop using output feedback lead-lag controler. Controllers The goal of all of the control schemes presented in the sequel is to regulate the offset at the lookahead, y, to zero. Passenger comfort is another important design criterion and this is typically expressed in terms of jerk, corresponding to the rate of change of acceleration. For a comfortable ride no frequency aboe.1-.5 Hz should be amplified in the path to lateral acceleration [5]. Additional road following criteria can be specified in terms of maximal allowable offset y max as a response to the step change in curature as well as bandwidth requirements on the transfer function F (s) = y(s) K (s).
.1 ead-lag Control Preious analysis reealed that up to 15 m/s the lookahead one can guarantee satisfactory damping of the closed loop poles of V 1 (s) and compensate for the delay using simple unity feedback control with proportional gain in the forward loop. As the elocity increases the transient response is affected more by the poor damping of the poles of V 1 (s) introducing additional phase lag around the.1- Hz. Since further increasing the lookahead does not improe the damping, gain compensation only cannot achiee satisfactory performance. The natural choice for obtaining an additional phase lead in the frequency range.1- Hz would be to introduce some deriatie action. n order to keep the bandwidth low an additional lag term is necessary. One satisfactory lead-lag controller has the following form: :9s +:1 C(s) = :5s +1:5s + where C(s) is a lead network in series with a single pole. The aboe controller was designed for a elocity of 3 m/s (1 km/h, 5 mph), a lookahead of 15 m and ms delay. The resulting closed loop system has a bandwidth of.5 Hz with a phase lead of 5 at the crossoer frequency. A discretized ersion of the aboe controller taking into account the 3 ms sampling time of the ision system hae been used in our experiments. Since increasing the speed has a destabilizing effect on V 1 (s), designing the controller for the highest intended speed guarantees stability at lower speeds and achiees satisfactory ride quality. n order to tighten the tracking performance at lower speeds indiidual controllers can be designed for arious speed ranges and gain scheduling techniques used to interpolate between them.. Full State Feedback With the aailability of the state information through the obseration process we explored the possibility of using the full state feedback control, using pole placement method. For good step response and bandwith requirements the poles from origin were moed to a conjugate pair with damping ratio =.77 and natural frequency about! n =.99 rad/s. The location of the ehicle dynamics poles was compensated by increased lookahead at higher elocities and remained unchnaged by pole placements methods..3 nput-output inearization nput-ouput linearization technique is typically used for linearization of nonlinear systems by state feedback and its (9) theoretical background can be found in []. The application of this technique to the bicycle model isn t strictly speaking linearization by state feedback, since the bicycle model is already linear. Nonetheless, the feedback rule is applied to render the model longitudinal-elocity independent. n this case the feedback law has a zero cancelling effect instead of linearizing one. Gien the bicycle model in the form The control law u = _x = f(x) + g(x)u (1) 1 g 1 f h(x) (; f h(x) +u ) (11) where i g denotes the i-th ie deriatie along g. For our particular example the control law becomes: u = a (u ; ( a3 ; a1 m ) y ; ( ;a ; a V m ) _ with constants a =1=(;b ;b 1 ) and a 1 a a 3 a b 1 b as defined after Equation??. Employing this control law yields a second order equation y = u. Now with two poles at the origin and the other two poles unobserable but well behaed we used the original lead-lag controller which gae us a complete control oer the placement of the systems poles.. Feedforward Control The steady state behaior of the system during perfect tracking of a cure with radius R ref, is characterized by particular alues of _ ref yref and ref. By setting the [_ y _y _" ] T to, the steering angle ref can be obtained from state equations and becomes: ref = K ref (l ; (l f c f ; l r c r )x m ) : (1) c r c f l This feedforward control component can be added to any of the control schemes that hae been described. The feedforward control law essentially proides information about the disturbance ahead of the car and improes the transient behaior of the system when encountering changes in curature. The effectieness of the feedforward term depends on the quality of the curature estimates. We discussed the curature estimation process as part of the obserer design in section 5. 7 Experimental Results Conclusions The strategy behind the design of the lead-lag and full state feedback controlers was based on the obseration that )
x 1 Offset at lookahead x 1 Offset at lookahead 1 1 x 1 Offset at lookahead x 1 Offset at lookahead 1 1 1 1 1 1 1 Velocity 1 Velocity 9 9 1 Velocity 1 Velocity 7 7 9 9 7 7 5 5 5 5 3 3 1 1 3 3 1 1.5.5...5..5..75.7.75.7.75.7.75.7.5..5..5.5.55.55.. x 1 3 Curature estimate 1 Feedforward component s time.55.55 1 1 Figure : The plots of the left side of this figure depict the performace of a lead-lag controler, while trackin an oal consisting of two straight segments and two cured segments with the radius of curature about 1 m. The spikes both in the offset and lateral acceleration profiles during the cured sections (the sections where the offset is larger) correspond to the lane change maneuers performed by the ehicle. The transitions between the straight and cured segments are smooth without noticable oershoot. The plots on the right hand side depict full state feedback controller. While in the straight line sections the performance of the two is comparable, in the cured sections at high elocities the tracking error increases. n this case the control was perfomed using purely feedback term. Curature (1/m).... Feedforward (ticks) 1 35 Figure 5: The plots on the left hand side and right hand side demonstrate the effect of the feedforward control term on the oerall tracking performance, while using input-ouput linearized controller. The offset during the cured sections was essentially eliminated (see plots on the right). The row of pots depicts the feedforward term, which was computed from the curature estimates (left) proided by the obserer. The offset exihibits slight oershoot until the curature estimate conerges.
the dominant effect on systems behaior is caused by the two poles at the origin, while the ehicle dynamics poles are well behaed as long as the lookahead is large enough or en extra deriatie control action is proided. This allowed us to design controllers for the highest intended operating elocity, which would operate satisfactorily in the whole range of lower elocities. Howeer taking this approach one has to sacrifice some performance criteria at lower elocities. Acknowledgment. This research has been supported by Honda R&D North America nc., Honda R&D Company imited, Japan, PATH MOU57 and MUR program DAAH-9-1-31. References [11] Ü. Özgüner, K. A. Ünyelioglu, and C. Hatipoğlu. radar reflectie tape. Somehwere. [1] H. Peng. Vehicle ateral Control for Highway Automation. PhD thesis, Department of Mechanical Engineering, Uniersity of California, Berkeley, 199. [13] D. Rai and M. Herman. A non-reconstruction approach for road following. n SPE proceedings on ntelligent Robots and Computer Vision, pages 1, 1991. [1] C. J. Taylor, J. Malik, and J. Weber. A real-time approach to stereopsis and lane-finding. n Proceedings of the 199 EEE ntelligent Vehicles Symposium, pages 7 13, Seikei Uniersity, Tokyo, Japan, September 19-199. [1] R. S. Blasi. A study of lateral controllers for the stereo drie project. Master s thesis, Department of Computer Science, Uniersity of California at Berkeley, 1997. [] E. D. Dickmans and B. D. Mysliwetz. Recursie 3-D road and relatie ego-state estimation. EEE Transactions on PAM, 1():199 13, February 199. [3] B. Espiau, F. Chaumette, and P. Ries. A new approach to isual seroing in robotics. EEE Transactions on Robotics and Automation, (3):313 3, June 199. [] Arthur Gelb et al. Applied optimal estimation. MT Press, 199. [5] J. Guldner, H.-S. Tan, and S. Patwarddhan. Analysis of automated steering control for highway ehicles with look-down lateral reference systems. Vehicle System Dynamics (to appear), 199. [] Alberto sidori. Nonlinear Control Systems. Springer Verlag, 199. [7] J. Košecká, R. Blasi, C.J. Taylor, and J. Malik. Visionbased lateral control of ehicles. n Proc. ntelligent Transportation Systems Conference, Boston, 1997. [] J. Košecká. Vision-based lateral control of ehicles:look-ahead and delay issues. nternal Memo, Department of EECS, Uniersity of California Berkeley, 1997. [9] M. F. and and D. N. ee. Where we look when we steer? Nature, 39(3), June 199. [1] Y. Ma, J. Košecká, and S. Sastry. Vision guided naigation for a nonholonomic mobile robot. n submitted to CDC 9, 1997.