Maarten Bieshaar, Günther Reitberger, Stefan Zernetsch, Prof. Dr. Bernhard Sick, Dr. Erich Fuchs, Prof. Dr.-Ing. Konrad Doll 08.02.2017
By 2030 road traffic deaths will be the fifth leading cause of death (WHO report in 2013) 27% of 1.24 million victims worldwide are vulnerable road users (VRUs), e.g., pedestrians and cyclists Due to the ability of VRUs to suddenly start a motion or to change the direction, a dangerous situation may occur within some hundreds of milliseconds VRUs have to be indispensably considered also when reducing accidents and increasing traffic flow by cooperative, automated driving
Use collective intelligence to forecast basic movement primitives and trajectories of VRUs in a distributed way Lay foundations for autonomous driving in mixed traffic scenarios Increase the safety of road users (in particular the safety of VRUs) Improve the traffic flow
Goal Predict Intention of VRU Requirements: Open dynamical system Real-time system Shared communication medium Multi-modal sensor-system Vehicle-, infrastructure-based sensors and smart devices Model and prediction of uncertainties
Cooperative VRU Perception and Intention Detection
Detection and tracking of VRUs in image sequences provided by cameras from vehicles and infrastructure HOG or multiscale features soft computing techniques e.g. SVM, MLP N. Dalal und B. Triggs, Histograms of Oriented Gradients for Human Detection, IEEE Conference on Computer Vision and Pattern Recognition, pp. 886-893, 2005
Multi-view setup to confirm hypothesis M. Goldhammer, K. Doll, U. Brunsmann, A. Gensler und B. Sick, Pedestrian s Trajectory Forecast in Public Traffic with Artificial Neural Networks, in Proceedings of the 2014 22Nd International Conference on Pattern Recognition, Stockholm, Sweden, pp. 4110 4115, 2014
Forward results to intention detection Information: position, geometry and covariances
Movement Primitive Coefficients of Polynomial Approximation Forecast Trajectory using ANN Trajectory Forecast Stefan Zernetsch; Sascha Kohnen; Michael Goldhammer; Konrad Doll Bernhard Sick, Trajectory Prediction of Cyclists Using a Physical Model and an Artificial Neural Network, Intelligent Vehicles Symposium (IV), 2016 IEEE
32px Motion Contour Histograms of Oriented Gradients Detection Extract Edges Binarization Motion History Image 32px Divide Detection in Cells Magnitude Weighted Orientation Histogram Linear SVM
Experimental Results Red rectangle: Pedestrian ROI, Standing Green rectangle: Pedestrian ROI, Walking
Experimental Results Red rectangle: Pedestrian ROI, Standing Green rectangle: Pedestrian ROI, Walking
Artificial Neural Network using Polynomial Approximation Extract Head Trajectories Transform to Ego Coordinate Frame Polynomial Approximation of Velocities Polynomial Coefficients as Input and Output of MLP Predicted Velocities by Evaluation of Polynomials Reconstruction of Predicted Positions
Physical Model Physical model for prediction of starting movement Assumption: Constant Force F res = F a + F i + F air + F r v t = v ss tanh a 0 (t + t 0 ) v ss F res : Resistance F a : Acceleration Resistance F i : Inclination Resistance F r : Rolling Resistance v ss : Steady State Velocity F air : Air Resistance t 0 : Time Delta a 0 : Acceleraton a t = 0
Start Forecast Model Comparison Average Euclidean Error for Prediction of 2.5 s (black: CV Model, red: physical Model, blue: MLP) Error Ellipsis r for Prediction of 2.5 s (0.5 s steps) (black: CV Model, red: physical Model, blue: MLP)
Stop Forecast Model Comparison Average Euclidean Error for Prediction of 2.5 s (black: CV Model, blue: MLP) Error Ellipsis r for Prediction of 2.5 s (0.5 s steps) (black: CV Model, blue: MLP)
Experimental Results
Types of Fusion Feature-Based Result-Based Basic Movement Forecast Basic Movement Forecast Fusion B Classification Features Classification Fusion A Features features from other agents Basic movement forecasts from other agents Polynomial Approximation Sufficient Statistics Ensemble Techniques Model Fusion
Model trajectory as polynomial Movement Primitive Coefficients of Polynomial Approximation Forecast Trajectory using ANN Trajectory Forecast Feature-based fusion on polynomial coefficients Stefan Zernetsch; Sascha Kohnen; Michael Goldhammer; Konrad Doll Bernhard Sick, Trajectory Prediction of Cyclists Using a Physical Model and an Artificial Neural Network, Intelligent Vehicles Symposium (IV), 2016 IEEE
Least-squares polynomial approximation Robust against noise, independent of frequency Consistent abstract representation of sensor data Out-of-order fusion and missing data Efficient up- and downdating of polynomial approximation Orthogonal polynomial basis w = Φ x T Φ x 1 Φ x T y diagonal matrix Coefficients used as input to forecast intentions and trajectories Erich Fuchs; Thiemo Gruber; Jiri Nitschke; Bernhard Sick, Online Segmentation of Time Series Based on Polynomial Least-Squares Approximations, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 12, pp. 2232-2245, Dec. 2010
Weighted polynomial regression Weighting criteria: Timeliness/ Time (Exponential attenuation) Uncertainty (Inverse) measurement models Error model Sensor type and wearing position Results to be published soon Outlier
Stefan Zernetsch; Sascha Kohnen; Michael Goldhammer; Konrad Doll Bernhard Sick, Trajectory Prediction of Cyclists Using a Physical Model and an Artificial Neural Network, Intelligent Vehicles Symposium (IV), 2016 IEEE M. Bieshaar, Cooperative Intention Detection of Vulnerable Road Users, in Organic Computing: Doctoral Dissertation Colloquium, Kassel University Press, to appear Erich Fuchs; Thiemo Gruber; Jiri Nitschke; Bernhard Sick, Online Segmentation of Time Series Based on Polynomial Least-Squares Approximations, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 12, pp. 2232-2245, Dec. 2010 S. Köhler, M. Goldhammer, K. Zindler, K. Doll and K. Dietmeyer, "Stereo-Vision-Based Pedestrian's Intention Detection in a Moving Vehicle," 2015 IEEE 18th International Conference on Intelligent Transportation Systems, Las Palmas, 2015, pp. 2317-2322
N. Dalal und B. Triggs, Histograms of Oriented Gradients for Human Detection, IEEE Conference on Computer Vision and Pattern Recognition, pp. 886-893, 2005 M. Goldhammer, K. Doll, U. Brunsmann, A. Gensler und B. Sick, Pedestrian s Trajectory Forecast in Public Traffic with Artificial Neural Networks, in Proceedings of the 2014 22Nd International Conference on Pattern Recognition, Stockholm, Sweden, pp. 4110 4115, 2014
Taken from http://launchany.com/wp-content/uploads/2016/01/idea.jpg (31 May 2016)