Towards Fully-automated Driving

Transcription

1 Towards Fully-automated Driving Challenges and Potential Solutions Dr. Gijs Dubbelman Mobile Perception Systems EE-SPS/VCA

2 Mobile Perception Systems 6 PhDs, postdoc, project manager, software engineer, assistant prof Research: Visual SLAM, 3D Computer Vision, Deep Learning PAGE 1

3 Automotive Projects PAGE 2

4 Highly Automated Driving (HAD) SAE level 4 and above The driver is no longer the safety fallback Including dynamic urban environments PAGE 3

5 State of the Art PAGE 4

6 State of the Art - Uber PAGE 5

7 Stare of the Art - Autoware PAGE 6

8 How Does It Work PAGE 7

9 State of the Art Digital rail road: nominal vehicle trajectory is predefined in the map. Absolute positioning: RTK- GPS aided by FOG IMU and 3D point-cloud localization Map as a sensor: road and static object geometry are contained in the map Dynamic objects: detect and track from vision, RaDAR, and LiDAR PAGE 8

10 Challenges Scalability: creating HAD maps for absolute positioning is extremely labor intensive. Timeliness: frequent updates are required to keep map fresh. Costly hardware: RTK-GPS, FOGs, and dedicated LiDARS. Limited robustness: adverse weather and lighting conditions and highly-dynamic traffic scenes PAGE 9

11 Challenges Scalable & SAE Level 4+? SAE Level 2 Personally owned Scalable SAE Level 4 Robot Taxi Not scalable PAGE 10

12 Research 1. Research scalable HAD map technologies From absolute to relative positioning Semantic SLAM techniques to automatically create and update maps Crowd source visual observations 2. Research AI to make mobile vehicle less dependent on HAD map Deep learning and real-time inference From pattern recognition to reasoning PAGE 11

13 GPS-based Absolute Positioning Maps needs to be cm accurate in absolute coordinates Vehicle position needs to be cm accurate in absolute coordinates Why do I need my position wrt. the north pole when driving in Eindhoven? PAGE 12

14 Map-based Relative Positioning Route planning (+km scale) does not need cm absolute accuracy Maneuver planning (10 m to 100 m scale) only requires a map that is accurate in the area-of-interest Concept: fully replace absolute GPS positioning by map-based relative positioning and use GPS only as rough initialization and redundancy PAGE 13

15 Map-based Relative Positioning Map has extra localization layer containing landmarks Maps needs to be cm accurate in relative coordinates Vehicle position needs to be cm accurate in relative coordinates Map-based relative localization lowers the accuracy requirements of the map scalability PAGE 14

16 Local Accuracy: How Does It Work Map-based positioning is using the landmarks to locally align the map with reality PAGE 15

17 Local Accuracy: How Does It Work Map-based positioning is using the landmarks to locally align the map with reality PAGE 16

18 Local Accuracy: How Does It Work Map-based positioning is using the landmarks to locally align the map with reality PAGE 17

19 Map-based Relative Positioning Research and develop Highly Automated Driving using map-based relative positioning PAGE 18

20 Stereo Visual Odometry

21 Simultaneous Localization and Mapping PAGE 20

22 Crowdsourcing with Distributed SLAM PAGE 21

23 Research 1. Research scalable HAD map technologies From absolute to relative positioning Semantic SLAM techniques to automatically create and update maps Crowd source visual observations 2. Research AI to make mobile platform less dependent on HAD map Deep learning and real-time inference From pattern recognition to reasoning PAGE 22

24 Ego-lane Boundary Detection PAGE 23

25 Stereo Obstacle Detection PAGE 24

26 Semantic Scene Understanding PAGE 25

27 Make Networks More Efficient Aim: reuse computation for multiple classifiers Our task: Semantic Scene Segmentation Goal: Extend the number of classes, without an extra labeling effort and by maximally reusing feature computation layers PAGE 26

28 Approach: Hierarchical Network PAGE 27

29 Recent results Training on 3-level hierarchy with 3 datasets inference results for 108 classes Cityscapes L2 results Mapillary Vistas L3 results PAGE 28

30 Quantitative Comparison Goal: Train traffic sign sub-classes on GTSDB and test on Cityscapes Compare a flat classifier with our hierarchical classifiers The hierarchical classifier approach does better than a flat classifier approach, even when the flat classifier is trained on the target dataset Nvidia TitanX GPU Tensorflow x 706 ~20 fps PAGE 29

31 Semantic Scene Understanding

32 Future Challenges 90% solved unsolved past now near future far future 100 ms 1 s 10 s Anticipatory systems: Anticipate on possible future events from sensory data from the past PAGE 31

33 Future Research Challenges Extending the prediction horizon for the behavior of road users in highly dynamic urban traffic situations from 1 second to several seconds. Too complex for rule-based approaches Too few training samples, due to combinatorics, for learning-based approaches Requires novel approaches for spatiotemporal and inter-object reasoning PAGE 32

34 AI Paradox Majority of adults: Cannot beat the Chess world champion Can drive a car safely Artificial Intelligence Can beat the Chess world champion Cannot drive a car safely Driving a car safely is more challenging than beating the Chess world champion PAGE 33

35 Conclusion Current approaches for automated driving either do not reach SAE level 4 or are not scalable We require scalable HAD mapping technology Crowdsourcing with Distributed SLAM We require improved AI that can anticipate From pattern recognition to reasoning PAGE 34

36 Future Milestones March to June first highly automated tests (SAE 4) in urban conditions for Autopilot project Open-source pre-release of specific localization, mapping, and sensing components PAGE 35

37 Questions PAGE 36