Tyre-road friction potential estimation by data fusion

Transcription

1 Tyre-road friction potential estimation by data fusion Bayesian approach to road type classification Mr. Ari J. Tuononen, Helsinki University of Technology, Espoo Mr. Jouni Hartikainen, Helsinki University of Technology, Espoo Abstract Several ADAS (Advanced Driver Assistance Systems) need information about the maximum available friction force between tyre and road. This can be estimated from the output of vehicle dynamics sensors, but the problem is that the friction potentials vary considerably between different road types. One possibility is to use an enviromental sensor to estimate the current road type, and hence to allow the usage of vehicle dynamics based friction estimates on all driving situations. In this article we propose a Bayesian method for sequentially classifying the road surface measurements. The approach is based on casting the problem into a multiple target tracking framework, in which the tracked targets are the measurement regions of different road types. Simple heuristic dynamic models are proposed for the measurement regions, which allow to estimate the data association indicators together with the target states efficiently with a Rao- Blackwellized particle filter. The proposed framework allows also to model clutter measurements, and to define dynamic models for the data association variables. Preliminary classification results of real road surface measurements are presented for the proposed classification method. The results show that the classification results for overlapping road types can be improved by defining suitable dynamic models for data associations. We also illustrate with a simple simulated example how the surface classification results can be combined with vehicle dynamics based friction potential estimates. 1. Introduction Several ADAS (Advanced Driver Assistance Systems) applications, such as Automatic Emergency Braking, Adaptive Cruise Control (ACC) and Collision Mitigation Systems, need information about the maximum available friction force between tyre and road. For example, on icy road, information about the friction potential reduces 60-80% crash energy when implemented into Collision Mitigation System [1].

2 An EC-funded ( ) studied sensor fusion based on environmental sensors, vehicle dynamics sensors and tyre sensors. Helsinki University of Technology continues friction estimation research towards more product friendly formulation, which means adaptability, robustness and cutting down the number of cumbersome parameters. The number of sensors has also been reduced to existing vehicle sensors and an environmental sensor. The friction estimation is studied intensively in literature. First, one has to separate tyre force estimation (used friction) from the friction potential estimation, where the latter means estimation of maximum friction coefficient. However, in friction potential estimation, the accurate estimation of used friction is beneficial. Fig. 1 clarifies why the low-slip approach [2] does not perform adequately in practise. The slip stiffness depends on many other factors as well than maximum friction coefficient. Actually, the low-slip -method can overestimate friction potential very dangerously on wet conditions, because slip stiffness may increase compared to dry conditions. In addition, rough ice can result rather high slip stiffness values [3] and creates a challenge to estimate friction potential as well. Recently, a brush model based friction estimator was developed without assumption of generic relation between slip stiffness and friction coefficient, but unfortunately required friction utilization for the estimation is relatively high [4]. Actually, those friction levels are mostly achieved in limit braking or during hard cornering. Fig. 1: Some possible tyre force characteristics Fundamental problems when estimating friction potential are:

3 Vehicle dynamics based estimates are only available when μ used > ~30 %-100% of μ max [4,5] o In practice: cornering, braking or accelerating are needed o The estimation is not possible in normal highway driving situations Environmental sensors can classify the road surface [6,7], but not estimate friction potential o tyre friction coefficient e.g. on ice can be and on tarmac (roughly), thus information about road surface is not enough o continuous signal from road surface The Fig. 2 shows how the information is always available from the road surface monitoring sensor. Thus, it is natural that the final estimate is based on road classification, even if the signal does not directly include any information about possible friction conditions. Meanwhile, the slip stiffness (cornering stiffness & longitudinal slip stiffness) is rather often available during longitudinal or lateral excitation. However, as explained in Fig. 1, this is not enough to derive maximum friction coefficient. Finally, in some driving conditions, the maximum friction coefficient is available, but too rarely to be feasible independently. Fig. 2: Periodic nature of available information relevant to friction estimation Due to the fact that vehicle dynamic based estimates are not available on every time instance, classical data fusion by weighting signals according to their variance does not produce sufficient performance. Meanwhile, the heuristic information about the signals can be exploited to evaluate validity of a prior estimate of friction potential. The developed estimator will exploit longitudinal and lateral vehicle dynamics excitation to estimate slip stiffness and friction potential for the current road type. Simultaneously, road surface is characterized with the environmental sensor and in situations where no reliable vehicle dynamics based estimates are available, previously estimated value for the friction potential of the associated road type is used. We discuss a statistical (Bayesian) formulation for the problem of associating the road surface measurements to different road types. The

4 key advantage of the formulation is that it allows the properties of the road types and measurement devices to vary across time. We also briefly discuss extensions, in which the number of different road types can also vary across time. Hence, this article shortly reviews the problematic of vehicle dynamics based estimation and concentrates on how the road type can be associated to certain frictional conditions. 2. Estimation of friction potential A block diagram of the estimator is shown in Fig. 3. The required hardware is typical sensor setup in ESC-equipped vehicle enhanced with an environmental sensor, which can classify different road conditions relevant to tyre-road friction. However, it is not necessary to be able to judge if the road is wet or icy as long as they are separated in classification. Required hardware Road Surface Characterization Road surface classification Data fusion - Friction estimate associated to certain road classification Continuous estimates: C α μ lat C κ μ long Lateral dynamics estimates Cα and μlat Vehicle dynamics sensors (ESC equipped vehicles) Parameter identification: Ground speed vx Relation of front and rear axle dynamic rolling radius Tyre vertical load estimator Tyre longitudinal force estimator Classification of driving condition Longitudinal dynamics estimates Cκ and μlong Fig. 3: The block diagram of the estimator 2.1. Vehicle dynamics estimate The both longitudinal and lateral response of the vehicle can be implemented into friction estimation purposes. Because the friction potential estimate is rarely available, it is reasonable to exploit both methods as shown in the block diagram. This article discuss only longitudinal dynamics and its problematic. The typical friction potential estimator assumes that longitudinal slip stiffness can be exploited in evaluating friction potential (maximum friction coefficient). Similar assumption is often drawn between cornering stiffness and friction potential for lateral grip. However, wet

5 tarmac can produce greater slip stiffness than dry tarmac, probably because of cooling effect of water, which makes tread rubber stiffer [4]. Fig. 4 shows Atyre force relation to longitudinal slip as it is available from the vehicle sensor during acceleration. The longitudinal slip at the front axle reads: 1 (1) where ω is corresponding rotational velocity of the wheel (subindex: Front, Left, Rear, Right) The formulation is naturally is possible only for front wheel drive vehicles. In addition, compensation is needed for wheel load and tyre pressure dependency of dynamic rolling radius. The normalised longitudinal force can be expressed:,,, (2) when and wheel inertia are minor. The driveshaft torque is M ds and r f,w is wheel radius. The wheel load of the front axle wheels is approximated from the load transfer:,., (3) where m is mass of the vehicle, h is height of the centre of gravity, l is axle length and is longitudinal acceleration, which can be obtained from the sensor or differentiated from the wheel speed measurement of rear axle as done in this paper. The Fig. 4 shows the difficulty to evaluate friction coefficient based on purely vehicle dynamics. First, there is no major difference between wet and dry tarmac instead of area ( ~ 0.07). Another problem seems to be shifting of the longitudinal force values to the left when F x, norm > This phenomenon comes from the dynamic rolling radius changes due to load transfer. Thus, the front axle rotational velocity is decreased due to increased r dyn. Similarly, the rear axle rotational velocity is increased due to the decreased r dyn and this results underestimation of κ during load transfer. It can be also seen that the measurements are rather easily clustered, if the driver accelerates at constant torque.

6 F x,norm [-] WET TARMAC DRY TARMAC κ [-] A Fig. 4: Tyre characteristics for wet and dry road vehicle measurements If these exceptions are known and can be described to the algorithms, the slip based friction potential estimation can be accurate when adequate utilization is available. In addition, it is clear that very slippery conditions can be detected also with rather low friction utilization. The estimator will take into account these constraints and data fusion is able to extend operating area of the vehicle dynamics estimate Recursive Bayesian road type classification The RoadEye [7] is a near infrared spectroscopy based sensor, which measures three wavelengths of light backscattered from road surface. In principle, the wavelength measurements for different road types fall into different regions such that it is possible to classify different road types from the observed wavelengths. In practice, however, these regions can be overlapping and can even change dynamically due to changes in road surfaces as well as in properties of vehicle and measurement device. The measurements can also be corrupted with outliers (that is, clutter), which can potentially lead to serious misclassification results. In this article we take a Bayesian approach for sequentially classifying the road type measurements. Our approach allows to simultaneously estimate the measurement regions of each road type and to solve the road type classification problem. The approach is based on casting the estimation problem into a multiple target tracking framework, in which the known or unknown number of targets are the different road types and their states are the measurement models, which describe how the RoadEye measurements are distributed for individual road types. The proposed classification method is based on Rao-Blackwellized

7 Monte Carlo Data Association (RBMCDA) algorithm [8,9], which is reviewed briefly in the next section Rao-Blackwellized Monte Carlo Data Association By formulating the models for the road types in a certain way the estimation problem can be performed efficiently with RBMCDA algorithm, which itself is based on a Rao-Blackwellized particle filters [10,11]. In this context Rao-Blackwellization means that the states of the targets can be estimated in closed form with a standard optimal filter (that is, with Kalman filter [12]), while the data association indicators are estimated with Monte Carlo sampling. To achieve this, the filtering model must specified in the following form: The dynamic models for each target 1,, are assumed to be discrete time linear Gaussian state space models of form,,,,,,,, (4) where, is the state,, the dynamic state transition matrix and, the process noise covariance matrix of target on time step. In similar spirit a linear Gaussian form is assumed for the measurements,,,,,,, (5) where, is the measurement on time step, a stacked vector of the states of all targets on time step,, the measurement model matrix and, the measurement noise covariance matrix of target on time step. The variable is a data association indicator variable, which has the value 0 clutter and for targets 1,. The clutter measurements can have any distribution, which is independent of states, that is,, 0 0. (6) The dynamics of the data association variables can be any order Markov model of form,,. (7) If the data association indicators were known on each time step we could solve the filtering distributions : of states in closed form with Kalman filter, given the measurement sequence :,, and matrices,,,,, and,. Since this is not the case we need to jointly estimate the variables together with states,. This can be done with Rao- Blackwellized particle filter, which uses weighted particles to represent to the joint

8 distribution of data association variables and state values. What makes this approach efficient is that the states, can be estimated with Kalman filter while data associations are estimated with a sequential importance sampling (SIR) algorithm [11]. This means that each particle 1,,N on time step represents a hypothesis for a data association sequence,,, and the state trajectories for the targets are independent Gaussians, that is, :, :,,,,, (8) where the state means, and covariances, are computed with Kalman filter given the estimated data association sequence. Given the importance weights,, of the particles, which are computed during the SIR algorithm, the joint distribution of data associations and states on time step is, :,,,,. (9) To avoid the degeneracy of particles a resampling [13] procedure is performed when the effective number of particles is too low. However, as the optimal importance distribution for the SIR algorithm can be formed in closed form, resampling need not to be done very often in practice. The number of particles can also be relatively low (say, 5-10) as state variables are estimated analytically. More details about algorithm can be found in [9]. A free Matlab toolbox implementing the RBCMDA algorithms is also available online [14] Models for road types In the context of road type classification we define the target state vectors to be the expected values of the measurement regions, where the three dimensional RoadEye measurements are distributed for different road types. The forms of the measurement regions are represented by covariance matrices, such that the measurement models are Gaussians,,,,,. (10) The covariances, can have fixed a priori values (that is,, ), or they can be estimated dynamically along with the state vector, with adaptive filters, which we discuss briefly in next section.

9 To allow some variability to the expected values of the measurement regions a simple Gaussian random walk model is assumed for the state dynamics:,,,,,,. (11) The process noise covariances, are assumed to have the same value for each road type and time step, that is,,. We also assume that is diagonal with equal diagonal elements. Since we don't have any a priori information for the form of clutter measurements we use a uniform distribution, 0 1, (12) where is the volume of the three dimensional measurement space. The target data association priors are formulated such that after associating a measurement to a target the time t a to next association is distributed according to a Gamma distribution,. (13) This means that if the last association to target was at time, the prior probability for associating th measurement to target on time step is given by,,,. (14) The parameters of the Gamma distribution was set to 1,2 to give a monotonically decreasing probability for associating to a target as the time from the previous association increases. This is reasonable assumption, since in real driving conditions the changes of road surfaces occur only rarely Estimation of measurement noise covariances The classical of way estimating dynamic systems in cases where a priori noise statistics are unknown is to use an adaptive filter, which estimates the covariance statistics along with the state vector. The traditional approaches to adaptive filtering can be divided into Bayesian, Maximum-Likelihood, correlation and covariance-matching methods [15]. In this work, we use a Bayesian adaptive filter presented by Särkkä and Nummenmaa [16], which employs Variational Bayesian (VB) methods for performing approximate inference for the joint distribution of state and measurement noise covariance. The approach assumes a simple heuristic dynamic model for the noise parameters, which retains the same functional

10 form for the probability densities (that is, inverse-gamma) of parameters on time step, and employs a factorized free form distribution for approximating the true posterior distribution. The result is a fast recursive algorithm, in which the state and noise statistics are estimated with a fixed-point iteration of Kalman filter equations. In the original paper [16] the measurement noises were assumed to be independent, that is, is a diagonal matrix. This is clearly too simple assumption as there are correlations in measurements, as can be seen from Fig. 5. Fortunately, the independence assumption can be avoided by replacing the marginal inverse-gamma distributions with a joint inverse- Wishart distribution, for which a similar kind of heuristic dynamics as well as VB inference algorithm can be derived. The details of the derivation are going to presented in the working paper [17] Data fusion The final aim in the estimation is the friction potential for each road type 1,,, which we estimate with Kalman filter after estimating the data association variable on each time step. To allow time-variability for the friction potentials we assume a Gaussian random walk model for their dynamics:,,,,,. (15) The measurements of the friction potentials are obtained from the vehicle dynamics such that the measurement model for the Kalman filter can specified as py µ, µ,c iny µ, µ,,r,. (16) As the estimates from the vehicle dynamics are not available on each time instance, the Kalman filter update step is performed only on instances when an estimate is available, otherwise only the prediction step is performed. The Kalman filter updates are actually performed for each particle separately, and posterior distribution of the friction potential of road type is a mixture of Gaussians N,µ,µ m,,p, pµ, y : w Nµ,. (17) where the mixing proportions are given by the importance weights. This is illustrated in Figure 8. A point estimate for the friction potential can be obtained either as a weighted sum of each particles mean or as the mean of the particle with the highest weight.

11 3. Results To demonstrate the proposed approach we constructed a test data set, which consisted of real RoadEye measurements of dry tarmac, wet tarmac, snow and ice. The measurements were not actually from the same real measurement cases, as they were concatenated together artificially to construct a driving sequence, which is shown in the topmost panel of Figure 6. The friction potentials were not estimated from real vehicle dynamics, instead each road type was given a fixed value, for which a sequence of measurement were simulated according to the model described in the previous section. The left panel of Figure 5 shows the measurements of test data set, and the estimates of the measurement models for each road type are shown in the right panel. It can be seen that measurements of ice and snow separates clearly form each other, but dry and wet tarmac are slightly overlapping. The model estimates are reasonable, even though the sizes of the measurement regions are slightly overestimated. The Figure 6 shows the classification results on the test data set with both uniform and gamma road type priors. Clearly without any prior assumptions about the data association probabilities the method is not able to classify dry and wet tarmac very reliably, but when assuming that it is very probable that the next measurement comes from a road type, for which a measurement was associated recently, classification reliability can be approved significantly. With uniform prior the number of misclassifications for the highest weighted particle on each time step was 359 and for the particle with the highest weight on the last time step the number was 445. With Gamma prior the corresponding misclassification counts were 129 and 53. The Figure 7 shows friction potential estimation results for the wet tarmac on different time steps with the constructed data set. On time steps 1,2200 the estimates wander around the value of dry tarmac 1.25 due to random misclassifications, but on time steps after that the estimates begin to converge toward the right value ( 0.75). In Figure 8 the estimated probability distributions of friction potential are shown on three different time step, which illustrates that the estimation method provides uncertainty estimates in addition to point estimates.

12 Fig. 5: The real RoadEye measurements are shown in left panel and model estimates in the right panel. Black dots denote dry tarmac, red dots wet tarmac, blue dots snow and magenta dots ice. It can be seen that regions of dry and wet tarmac are overlapping while ice and snow can be separated easily. Fig. 6: Classification results on the test data set. Black line show the real road type (1 for dry tarmac, 2 for wet tarmac, 3 for snow and 4 for ice) on different time steps. Blue line is the classified road type of the particle with the highest weight on each time step with the uniform data association prior and red line is results for the particle with the highest weight on the last time step. Green and magenta lines show the classification results with the Gamma prior for the best particle on each time (green) and for the best particle on the last time step (magenta).

13 Fig. 7: Friction potential estimates on wet tarmac on each time step. Blue lines show the mean estimates of the individual particles on each time step and the black dotted line shows the true value. a) k = 2400 b) k = 3000 c) k = 3800 Fig. 8: Estimated probability distribution of simulated friction potential of wet tarmac on three different time steps with the test data set. Black solid lines show the estimates of different particles, and the red cross -marker is the real value. On time step the majority of the particles place their probability mass around the friction value of the dry tarmac (.) as in the beginning of the data set there is only measurements of dry tarmac, for which the wet tarmac was misclassified several times. On time step majority of the particles has converged to right value. while some are still located around the value of dry tarmac. On time step all particles have converged to real friction value. 4. Discussion 4.1. Vehicle dynamics estimate for friction potential The article discussed some problematic about vehicle dynamics estimation, but main solutions are already published in literature. The different approaches have their pros and cons and there are summarised in the following:

14 Longitudinal dynamics for friction potential estimation The braking event sometimes produces very high utilization of friction and those conditions are important moments for friction potential estimation. Unfortunately, if the friction potential is not fully utilised, the slip ratio information is needed to evaluate maximum friction coefficient. The slip ratio is more difficult to obtain during braking, because of missing reference velocity (free rolling tyres). Meanwhile the traction situation in 2wd vehicle offers reference velocity measurement and thus more accurate slip ratio estimate for traction wheel. The longitudinal acceleration and engine torque can be used to evaluate longitudinal force. In addition, in fwd vehicle load transfer lightens traction wheels resulting earlier force saturation and thus allowing better evaluation of friction potential. However, in typical calm driving, the traction forces remain quite small, but this depends on driving style. In the future different regenerative systems may offer possibility to brake one axle and simultaneously drive other axle to compensate braking. This would allow vehicle to test friction condition without large energy consumption or disturbing the driver. This kind of test could be triggered when the estimator has no other reliable friction potential estimate and road condition has recently changed Lateral dynamics friction potential estimation The well-known method to evaluate friction potential during cornering is to compare nominal yaw rate to the actual yaw (similar approach than judge ESC intervention). When the nominal and actual vehicle behaviour agrees, the high friction level can be assumed. Then the offset value can be used to evaluate how slippery conditions are in question. This method assumes that there are certain relation between cornering stiffness and maximum friction coefficient, but this is questionable. The improved method would map cornering stiffness and friction values and associate them to certain road conditions. The relation of lateral force to aligning moment is interesting, because two independent measurements of the phenomenon may improve performance of the estimator. However, once again in the literature, the cornering stiffness and aligning moment in wet conditions are not adequately discussed Further research for vehicle dynamics estimators The main problems of the friction potential estimator exist on the accuracy of the identification of parameters such as dynamic rolling radius and vertical load of the wheel. In addition, a further work is to extend lateral dynamics estimator to cover exceptional situations

15 where cornering stiffness cannot be exploited to evaluate μ directly. Also, the estimators could be extended to cover combined slip conditions Data Fusion In this article we have demonstrated that the RBMCDA framework can used to efficiently classify different road types from RoadEye measurements, which can be utilized in estimating the friction potential of different road types from vehicle dynamics.. The aim in this article has not been in optimizing the model in the best possible way, but instead to illustrate the framework, which can be expanded to incorporate more sophisticated a priori information concerning the classification problem at hand. The prior model for data associations proposed in this article is a simple one, and better ones certainly do exist. Still, it illustrates clearly that such kind of prior models are necessary when classifying overlapping road types. The models for the measurement regions were also of very simple form, but in our experiments they showed reasonable fit to real measurements. A very straightforward extension to this would be to assign several Gaussian distributions for individual road types, which would enable to model more complex measurement regions, such as multimodalities. This would be actually very straightforward to implement with the current model framework. Non-linear dynamic and measurement models can also be easily incorporated into the RBMCDA framework by replacing the Kalman filter with some of its non-linear extensions, such as the Extended Kalman filter (EKF) [18] or the Unscented Kalman filter (UKF) [19]. One clear advantage of the proposed approach is that its computational complexity can be controlled with the number of particles such that the more there is available computing resources the better the classification reliability is. With reasonable number of particles 10 the algorithm can be implemented in real time with standard MCU. Further research on the road type classification As was shown in the article [9] the RBMCDA framework extends directly to cases, in which the number of targets can be unknown and time-varying. The extension is based on formulating probabilistic models for target birth and death events, which are estimated with SIR algorithm together with data associations. We have experimented with this extension in the context of road type classification with mixed results. With clearly separable road types (i.e., tarmac, ice and snow) it is possible for the algorithm to learn the right road types, but with overlapping regions (i.e. wet and dry tarmac) the algorithm is in trouble due to absence of additional prior information of the road types. In future work we shall concentrate on

16 building appropriate prior models for learning the right number road types also in cases of overlapping road types On-board demonstration of the data fusion The developed estimator will be demonstrated on winter season The estimator outputs reliable friction potential estimate in all driving conditions. This information can be used to warn a driver and to aid active chassis systems such as emergency braking, adaptive cruise control and collision mitigation. References [1] FRICTI@N final report, available online [2] Gustafsson, F., Slip-based tire-road friction estimation, Automatica, vol. 33, No. 6, pp ,1997 [3] Pavkovic, D., Deur, J., Asgari J., Hrovat D., Experimental analysis of potentials for tire friction estimation in low-slip operating mode, SAE , 2006 [4] Svendenius, J., Tire modeling and friction estimation, PhD Thesis, Department of Automatic Control, Lund University, Sweden, April 2007 [5] Pasterkamp, W.R., The Tyre as Sensor to Estimate Friction, Delft University Press, [6] Casselgren J., Sjödahl, M., Leblanc, J., Angular spectral response from covered asphalt, Applied optics.; Vol. 46, No. 20, p , 2007 [7] Casselgren, J., Road surface classification using near infrared spectroscopy, Licentiate thesis, 43 p, Luleå University of Technology, 2007 [8] Särkkä, S., Vehtari, A. and Lampinen, J. (2004), Rao-Blackwellized Monte Carlo data association for multiple target tracking. In Proceedings of the Seventh International Conference on Information Fusion, volume I, pages [9] Särkkä, S., Vehtari, A. and Lampinen, J. (2007), Rao-Blackwellized Particle Filter for Multiple Target Tracking. Information Fusion, 8(1):2-15. [10] Akashi, H., and Kumamoto, H. (1977), Random sampling approach to state estimation in switching environments. Automatica, 13:429:434. [11] Doucet, A., de Freitas, N. and Gordon, N., editors (2001), Sequential Monte Carlo Methods in Practice, Springer. [12] Kalman, R. E. (1960), A new approach to linear filtering and prediction problems. Transactions of the ASME, Journal of Basic Engineering, 82: [13] Kitagawa, G. (1996), Monte Carlo filter and smoother for non-gaussian nonlinear state space models. Journal of Computational and Graphical Statistics, 5:1-25. [14] Hartikainen, J. and Särkkä, S. (2008), RBMCDA Toolbox for Matlab. Available online at

17 [15] Mehra. R. (1972), Approaches to adaptive filtering. IEEE Transactions on Automatic Control, 17(5): [16] Särkkä, S. and Nummenmaa, A. (2009), Recursive Noise Adaptive Kalman Filtering by Variational Bayesian Approximations, IEEE Transactions on Automatic Control, 54(3): [17] Hartikainen, J. and Särkkä, S. (2009). Extensions to Recursive Noise Adaptive Kalman Filtering by Variational Bayesian Approximations. Working paper. [18] Jazwinski, A. H. (1970), Stochastic Processes and Filtering Theory. Academic Press. [19] Julier, S. J. and Uhlmann, J. K. (2004), Unscented filtering and nonlinear estimation, In Proceedings of the IEEE, 92(3):