Road Surface Condition Analysis from Web Camera Images and Weather data. Torgeir Vaa (SVV), Terje Moen (SINTEF), Junyong You (CMR), Jeremy Cook (CMR)

Transcription

1 Road Surface Condition Analysis from Web Camera Images and Weather data Torgeir Vaa (SVV), Terje Moen (SINTEF), Junyong You (CMR), Jeremy Cook (CMR)

2 Motivation Cameras installed along roads for surface condition monitoring Road images available online and updated at varying frequencies (authentication required) Weather measurements available for most road camera stations Surface condition monitoring manually Can we develop automated system for surface monitoring and other purposes?» 494 web cameras» Update frequency 5min 4 hours» Exposed to harsh weather conditions» Static and pan/tilt» Long periods of darkness in winter

3 Project Profile Goal Web service for road surface classification based on web camera images together with weather parameters Methodology Machine learning for pattern classification Manual categorization of road surfaces Feature representation + classification model Web application responding classification results upon request Dry Wet Full Snowy Partial Snowy

4 Problem formulation A supervised machine learning problem for pattern classification Involved key technologies Road surface condition modelling (manual definition of surface condition categories) Supervised learning models (SVM: support vector machine) Feature representation

5 Road surface condition modelling Based on Road maintenance scenario Safe driving purpose Surface conditions in traffic management and accident analysis Subjective observation Four surface categories + Special case (icy) Dry Wet Full Snowy Partial Snowy Icy: frost packed snow freezing rain

6 Feature representation (I): Image features Goal: discriminative features representing different condition categories Luminance and chrominance related features Gradient related features Edge related features Texture related features Road masking Manual masking (automatic update by image registration) Automatic road detection

7 Feature representation (II) Weather features Road surface temperature Air temperature Dew point temperature Relative humidity Precipitation amount Their derivatives within previous 1 hour Feature selection Key feature detection

8 Model training Training image set 9,346 images taken during daytime over 1 year Dry (4,075), wet (1,905), full snowy (1,395), partial snowy (1,971) Model training: feature selection + SVM parameters Threshold T for F-Score Rough search, Fine search Features (F-Score > T) Grid search on SVM parameters Increase T Cross validation Optimum combination (features & SVM)

9 Icy condition modelling: based on weather Frost Packed snow Black ice Freezing rain Icy

10 Icy condition modelling: black ice examples

11 System structure Services / Individuals in Demand Server (with timer) image classification camera ID weather measurement Client classification location info Online Weather Measurements camera ID image Online Images

12 Online classification Predictive model Classification Extract feature vector RH: 90% Precipitation 0.1mm/hour Road temp: -0.6 Air temp: -0.9 Dew point temp: -2.3 Dry: 0.30% FullSnowy: 86.23% Road condition: FullSnowy PartialSnowy: 13.38% Wet: 0.10%

13 Online classification Camera ID: Name: Filefjell v/ Varden, Road: F296, County: 5 Relative humidity 96.9% Classification Precipitation 0.1mm/h Road surface temperature Dry: 0.00% PartialSnowy: 99.30% 0.3 centigrade Air temperature -0.6 centigrade Dew point temp: -1.0 centigrade FullSnowy: 0.69% Road condition: PartialSnowy Wet: 0.00%

14 Online classification Camera ID: Name: Bjøberg II, Road: R52, County: 6 No weather measurement Classification purely and adaptively based on images Dry: 0.00% Classification PartialSnowy: 99.61% Road condition: PartialSnowy FullSnowy: 0.36% Wet: 0.2%

15 Online classification Camera ID: Name: Tuv i Hemsedal, Road: R52, County: 6 No weather measurement Classification purely and adaptively based on images Dry: 95.87% Classification PartialSnowy 0.00% Road condition: Dry FullSnowy: 0.00% Wet 4.13%

16 Online classification Camera ID: Name: Lavangsdalen, Road: E8, County: 19 Relative humidity 63.1% Classification Warning: Frost Precipitation 0.0mm/h Road surface temperature Dry: 0.51% PartialSnowy: 76.72% -9.5 centigrade Air temperature -0.7 centigrade Dew point temp: -6.8 centigrade FullSnowy: 17.28% Road condition: PartialSnowy Wet: 5.48%

17 Result validation and analysis (II) Real road surfaces can have confused conditions, hard to classify to different categories, even by eyes Overall accuracy: ~78% Correct detection Possibly incorrect detection full-snowy partial-snowy partial-snowy wet wet partial-snowy partial-snowy full-snowy

18 Result validation and analysis (IV): Icy conditions Exact comparison with manual detection not performed, due to difficulty in visually checking icy conditions from images System detected results (labels given by the system): frost frost & freezing rain frost packed snow frost freezing rain

19 Result validation and analysis (I) High accuracy: ~ 95% accuracy from cross-validation (i.e., images from the training dataset) ~88% accuracy w.r.t. images with uncontested conditions Dry Full Snowy Wet Partial Snowy

20 Result validation and analysis (III) Comparison against Sintef s manual classification: ~ 80% match rate against manual classification

21 24