Maarten Bieshaar, Günther Reitberger, Stefan Zernetsch, Prof. Dr. Bernhard Sick, Dr. Erich Fuchs, Prof. Dr.-Ing. Konrad Doll

Transcription

1 Maarten Bieshaar, Günther Reitberger, Stefan Zernetsch, Prof. Dr. Bernhard Sick, Dr. Erich Fuchs, Prof. Dr.-Ing. Konrad Doll

2 By 2030 road traffic deaths will be the fifth leading cause of death (WHO report in 2013) 27% of 1.24 million victims worldwide are vulnerable road users (VRUs), e.g., pedestrians and cyclists Due to the ability of VRUs to suddenly start a motion or to change the direction, a dangerous situation may occur within some hundreds of milliseconds VRUs have to be indispensably considered also when reducing accidents and increasing traffic flow by cooperative, automated driving

3 Use collective intelligence to forecast basic movement primitives and trajectories of VRUs in a distributed way Lay foundations for autonomous driving in mixed traffic scenarios Increase the safety of road users (in particular the safety of VRUs) Improve the traffic flow

4 Goal Predict Intention of VRU Requirements: Open dynamical system Real-time system Shared communication medium Multi-modal sensor-system Vehicle-, infrastructure-based sensors and smart devices Model and prediction of uncertainties

5

6

7 Cooperative VRU Perception and Intention Detection

8 Detection and tracking of VRUs in image sequences provided by cameras from vehicles and infrastructure HOG or multiscale features soft computing techniques e.g. SVM, MLP N. Dalal und B. Triggs, Histograms of Oriented Gradients for Human Detection, IEEE Conference on Computer Vision and Pattern Recognition, pp , 2005

9 Multi-view setup to confirm hypothesis M. Goldhammer, K. Doll, U. Brunsmann, A. Gensler und B. Sick, Pedestrian s Trajectory Forecast in Public Traffic with Artificial Neural Networks, in Proceedings of the Nd International Conference on Pattern Recognition, Stockholm, Sweden, pp , 2014

10 Forward results to intention detection Information: position, geometry and covariances

11 Movement Primitive Coefficients of Polynomial Approximation Forecast Trajectory using ANN Trajectory Forecast Stefan Zernetsch; Sascha Kohnen; Michael Goldhammer; Konrad Doll Bernhard Sick, Trajectory Prediction of Cyclists Using a Physical Model and an Artificial Neural Network, Intelligent Vehicles Symposium (IV), 2016 IEEE

12 32px Motion Contour Histograms of Oriented Gradients Detection Extract Edges Binarization Motion History Image 32px Divide Detection in Cells Magnitude Weighted Orientation Histogram Linear SVM

13 Experimental Results Red rectangle: Pedestrian ROI, Standing Green rectangle: Pedestrian ROI, Walking

14 Experimental Results Red rectangle: Pedestrian ROI, Standing Green rectangle: Pedestrian ROI, Walking

15 Artificial Neural Network using Polynomial Approximation Extract Head Trajectories Transform to Ego Coordinate Frame Polynomial Approximation of Velocities Polynomial Coefficients as Input and Output of MLP Predicted Velocities by Evaluation of Polynomials Reconstruction of Predicted Positions

16 Physical Model Physical model for prediction of starting movement Assumption: Constant Force F res = F a + F i + F air + F r v t = v ss tanh a 0 (t + t 0 ) v ss F res : Resistance F a : Acceleration Resistance F i : Inclination Resistance F r : Rolling Resistance v ss : Steady State Velocity F air : Air Resistance t 0 : Time Delta a 0 : Acceleraton a t = 0

17 Start Forecast Model Comparison Average Euclidean Error for Prediction of 2.5 s (black: CV Model, red: physical Model, blue: MLP) Error Ellipsis r for Prediction of 2.5 s (0.5 s steps) (black: CV Model, red: physical Model, blue: MLP)

18 Stop Forecast Model Comparison Average Euclidean Error for Prediction of 2.5 s (black: CV Model, blue: MLP) Error Ellipsis r for Prediction of 2.5 s (0.5 s steps) (black: CV Model, blue: MLP)

19 Experimental Results

20 Types of Fusion Feature-Based Result-Based Basic Movement Forecast Basic Movement Forecast Fusion B Classification Features Classification Fusion A Features features from other agents Basic movement forecasts from other agents Polynomial Approximation Sufficient Statistics Ensemble Techniques Model Fusion

21 Model trajectory as polynomial Movement Primitive Coefficients of Polynomial Approximation Forecast Trajectory using ANN Trajectory Forecast Feature-based fusion on polynomial coefficients Stefan Zernetsch; Sascha Kohnen; Michael Goldhammer; Konrad Doll Bernhard Sick, Trajectory Prediction of Cyclists Using a Physical Model and an Artificial Neural Network, Intelligent Vehicles Symposium (IV), 2016 IEEE

22 Least-squares polynomial approximation Robust against noise, independent of frequency Consistent abstract representation of sensor data Out-of-order fusion and missing data Efficient up- and downdating of polynomial approximation Orthogonal polynomial basis w = Φ x T Φ x 1 Φ x T y diagonal matrix Coefficients used as input to forecast intentions and trajectories Erich Fuchs; Thiemo Gruber; Jiri Nitschke; Bernhard Sick, Online Segmentation of Time Series Based on Polynomial Least-Squares Approximations, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 12, pp , Dec. 2010

23 Weighted polynomial regression Weighting criteria: Timeliness/ Time (Exponential attenuation) Uncertainty (Inverse) measurement models Error model Sensor type and wearing position Results to be published soon Outlier

24 Stefan Zernetsch; Sascha Kohnen; Michael Goldhammer; Konrad Doll Bernhard Sick, Trajectory Prediction of Cyclists Using a Physical Model and an Artificial Neural Network, Intelligent Vehicles Symposium (IV), 2016 IEEE M. Bieshaar, Cooperative Intention Detection of Vulnerable Road Users, in Organic Computing: Doctoral Dissertation Colloquium, Kassel University Press, to appear Erich Fuchs; Thiemo Gruber; Jiri Nitschke; Bernhard Sick, Online Segmentation of Time Series Based on Polynomial Least-Squares Approximations, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 12, pp , Dec S. Köhler, M. Goldhammer, K. Zindler, K. Doll and K. Dietmeyer, "Stereo-Vision-Based Pedestrian's Intention Detection in a Moving Vehicle," 2015 IEEE 18th International Conference on Intelligent Transportation Systems, Las Palmas, 2015, pp

25 N. Dalal und B. Triggs, Histograms of Oriented Gradients for Human Detection, IEEE Conference on Computer Vision and Pattern Recognition, pp , 2005 M. Goldhammer, K. Doll, U. Brunsmann, A. Gensler und B. Sick, Pedestrian s Trajectory Forecast in Public Traffic with Artificial Neural Networks, in Proceedings of the Nd International Conference on Pattern Recognition, Stockholm, Sweden, pp , 2014

26 Taken from (31 May 2016)