Exploiting maritime Big Data, the Blue Hub

Transcription

1 Exploiting maritime Big Data, the Blue Hub Marlene Alvarez Alvarez, Michele Vespe, Harm Greidanus Joint Research Centre the European Commission's in-house science service The Blue Hub:

2 Contents Available (big) data on ship traffic: Ship position reports (AIS, LRIT) Earth observation satellites (e.g., Copernicus) From data to knowledge 2

3 Maritime Situational Awareness (MSA) Which ships are on the sea? What are they doing? Illegal Fishing Piracy Irregular migration Maritime Security Oil Pollution Maritime Traffic 3

4 Ship reporting Data Automatic Identification System (AIS) Long Range Identification and Tracking (LRIT) INMARSAT GPS DC FREQUENCY High (few sec to min) FREQUENCY Fix (to 6 hours) IRREGULAR (spatial/temporal gaps) REGULAR (space and time) ACCESS Free ACCESS Restricted 4

5 Collection of AIS ship reports Collected by coastal receiver networks, continuously Received by satellites, when they pass over Picture: FleetMon.com Picture: exactearth LIMITED COVERAGE GLOBAL COVERAGE CONSTELLATIONS improved frequency revisit 5

6 Earth observation satellites Used for recognition Optical - high resolution - Small area - Daytime, clear skies Used for detection Radar - low resolution - Wide area - Through clouds and night DigitalGlobe Radarsat-1 Fine CSA/MDA 6

7 The Blue Hub exploiting maritime Big Data Vessel Movement Data Data Fusion & Target Tracking Real-Time Maritime Surveillance Anomaly Detection (deviation from normality) Vessel profiling/ intelligence Customs & Agencies Routes prediction Big Data (historical) Data Mining & Knowledge Discovery Routes Extraction Geopolitical issues impact on transport Mapping activities at sea Fishing grounds and fishing efforts Fisheries socioeconomics Fisheries management Ex-post policy evaluation Gridding Shipping Emissions Mapping Offshore and exploration Maritime Spatial Planning Trade Indicators 7

8 Data Fusion & Target Tracking at global scale Vessel Movement Data Data Fusion & Target Tracking Real-Time Maritime Surveillance Ship Name BIG DATA - ~150,000 ships carry transponders - each may send 10,000 messages / day Track built by integrating multiple AIS data providers (ship-type color coded). 8

9 Sentinel-1: Long term ship monitoring 1,557 S-1 images (3 Oct Sep 2015) 41,928 ships detected correlated with AIS AIS Coverage 34,263 ships uncorrelated with AIS Data: Copernicus 9

10 From EU to worldwide tracking and traffic routes Vessel Movement Data Big Data (historical) One-month of EU LRIT CDC data, revealing the main global traffic routes and enabling the implementation of innovative decision support tools Video Link 10

11 Vessel pattern knowledge discovery: from raw data to support to decision Big Data (historical) Data Mining & Knowledge Discovery Routes Extraction Maritime Spatial Planning 2-week raw AIS data Travel Time Histogram Pattern discovery Port area 51.1 Network Single route representation analysis Exit areas Latitude Latitude Entry area Tug Other Type, Haz. Cat. X Passenger Tanker, Tanker, Dredging Fishing Haz. Cat. Y Haz. Cat. Z Tanker, Haz. Cat. X Ship type distribution Tanker Longitude Cargo Cargo, Haz. Cat. OS Cargo, Cargo, Haz. Cat. YHaz. Cat. Z Fernandez Arguedas et al.: Automatic Generation of Geographical Networks for Maritime Traffic Surveillance, Int. Conf. on Information Fusion (2014) Cargo, Haz. Cat. X 11

12 Route prediction Historical LRIT data can be used to predict where a vessel will be up to a few days in advance Big Data (historical) Data Mining & Knowledge Discovery Routes Extraction Routes prediction Zampieri A., Vespe M., Westra M., Alvarez M., Greidanus H., A future for historical LRIT data, International Maritime Organization (IMO) Maritime Safety Committee (MSC) 95th session, London,

13 Anomaly Detection (deviation of normality) - Density map based on 2-y LRIT data - Overlaid real-time positions of all cargo ships - Multi-day past track from one cargo ship Greidanus H, Vespe M., Alvarez M., Detection of Anomalous behaviour in ship reporting data for improved maritime security, Future Security Conference, Berlin,

14 Event-based Knowledge Discovery Pre-Processing Raw Data Trajectory Extraction Georeferenced grid Vessel Id Dynamic Static/Voyage Vessel Objects Vessel based event detection Event type Dynamic Static/Voyage Event Objects Event map generation Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., AIS Event-Based Knowledge Discovery for Maritime Situational Awareness, International Conference on Information Fusion

15 Latitude Vessel-based Event Detection Event Type AOI Enter/Exit Cell Enter/Exit Birth/Death Relevance Access to protected areas or areas regulated by traffic routing schemes Traffic density maps Coverage maps Longitude Start/Stop Proximity Fishing/Steaming AIS on/off switching Detection of stop areas (ports, offshore platforms) Discovery of illegal activities Mapping activities at sea Discovery of illegal activities Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., AIS Event-Based Knowledge Discovery for Maritime Situational Awareness, International Conference on Information Fusion

16 Event Map Generation Event Object MMSI Longitude Latitude SOG COG ROW COL Event timestamp Vessel type Event type Analysis Cube DWH Presentation Layer Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., AIS Event-Based Knowledge Discovery for Maritime Situational Awareness, International Conference on Information Fusion

17 Event Map Generation: Results AOI: [15ᵒ W 17ᵒ E; 43ᵒ N 46ᵒ N] Time interval: October 1st - December 1st, degree Terrestrial AIS data from Italian Coast Guard Distribution of generated events Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., AIS Event-Based Knowledge Discovery for Maritime Situational Awareness, International Conference on Information Fusion

18 Event Map Generation: Results Entry into cells Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., AIS Event-Based Knowledge Discovery for Maritime Situational Awareness, International Conference on Information Fusion

19 Event Map Generation: Results Entry into cells Fishing Vessels Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., AIS Event-Based Knowledge Discovery for Maritime Situational Awareness, International Conference on Information Fusion

20 Event Map Generation: Results Entry into cells Fishing Vessels (Fishing behaviour only) Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., AIS Event-Based Knowledge Discovery for Maritime Situational Awareness, International Conference on Information Fusion

21 Event Map Generation: Results Entry into cells Cargo Vessels Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., AIS Event-Based Knowledge Discovery for Maritime Situational Awareness, International Conference on Information Fusion

22 Event Map Generation: Results Entry into cells Tanker Vessels Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., AIS Event-Based Knowledge Discovery for Maritime Situational Awareness, International Conference on Information Fusion

23 Event Map Generation: Results Entry into cells Passenger Vessels Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., AIS Event-Based Knowledge Discovery for Maritime Situational Awareness, International Conference on Information Fusion

24 Event Map Generation: Results Stop events Alvarez M, Fernandez V., Gammieri V., Mazzarella F., Vespe M., Aulicino G., Vollero A., AIS Event-Based Knowledge Discovery for Maritime Situational Awareness, International Conference on Information Fusion

25 Analysis of geopolitical developments: the declining impact of piracy in the Indian Ocean Big Data (historical) Data Mining & Knowledge Discovery Mapping activities at sea Geopolitical issues impact on transport Left: North- (green) and South-bound (red) traffic crossing the Indian Ocean Right: Traffic extraction during the second semester 2009 using historical LRIT data Ex-post policy evaluation Vespe M., Greidanus H., Alvarez M. : The Declining Impact of Piracy on Maritime Transport in the Indian Ocean: Statistical Analysis of 5-year Vessel Tracking Data, Marine Policy,

26 Analysis of geopolitical developments: the declining impact of piracy in the Indian Ocean Time series of traffic crossing the Indian Ocean in 5 years (each figure shows 6 months), showing the effect of piracy and its progressive decline in deviating maritime traffic Vespe M., Greidanus H., Alvarez M. : The Declining Impact of Piracy on Maritime Transport in the Indian Ocean: Statistical Analysis of 5-year Vessel Tracking Data, Marine Policy,

27 Analysis of geopolitical developments: the declining impact of piracy in the Indian Ocean Time series of traffic crossing the Indian Ocean in 5 years (each figure shows 6 months), showing the effect of piracy and its progressive decline in deviating maritime traffic Piracy Incidents Source: EU Naval Force Somalia Vespe M., Greidanus H., Alvarez M. : The Declining Impact of Piracy on Maritime Transport in the Indian Ocean: Statistical Analysis of 5-year Vessel Tracking Data, Marine Policy,

28 Analysis of geopolitical developments: the declining impact of piracy in the Indian Ocean LRIT derived Speed Over Ground [kn] distributions change over the 5 years: a progressive reduction of speed + the second peak (around 18 knots) disappears mid-2012 Higher speed: - reduces risk of piracy attacks - raises fuel costs Vespe M., Greidanus H., Alvarez M. : The Declining Impact of Piracy on Maritime Transport in the Indian Ocean: Statistical Analysis of 5-year Vessel Tracking Data, Marine Policy,

29 Track and speed profile of vessels Track and speed profile of a trawler showing three clusters of velocities corresponding to in port, fishing (position highlighted in yellow) and steaming behaviours. Vespe M., Gibin M., Alessandrini A., Natale F. Mazzarella F., Osio G., Mapping EU fishing activities using ship tracking data, Journal of Maps,

30 Knowledge Discovery: fishing activities 1) Raw data 2) Fishing Activities 3) Density of Fishing Activities 4) Real time fishing matching Mazzarella F., Vespe M., Damalas D., Osio G.: Discovering Vessel Activities at Sea using AIS Data: Mapping of Fishing Footprints, Proc. 17th Int. Conf. on Information Fusion,

31 EU trawlers Fishing intensity Coverage Vespe M., Gibin M., Alessandrini A., Natale F. Mazzarella F., Osio G., Mapping EU fishing activities using ship tracking data, Journal of Maps,

32 Port activity and trade indicators Trajectories cleaning and splitting 10 October

33 Port activity and trade indicators CB-SMOT 10 October

34 Port activity and trade indicators DB-SCAN 10 October

35 Port activity and trade indicators Concave-hull and buffer 10 October

36 Port activity and trade indicators Semantic annotation 10 October

37 Port activity and trade indicators Weekly Distribution Port of Genoa Port of Gioia Tauro 37

38 Port activity and trade indicators Time Distribution In/Out Port of Genoa Port of Gioia Tauro 38

39 Port activity and trade indicators Port of Genoa: link to Mediterranean Ports 39

40 Port activity and trade indicators Port of Genoa: link to Mediterranean Ports 40

41 Port activity and trade indicators Port of Genoa: link to Mediterranean Ports 41

42 Conclusions Vessel tracking data can be used to understand, map and quantify activities at sea, including how they change over time It is possible to turn large amount of data into elements that can be useful to policy makers The knowledge it is extracted directly from the data, using data mining and knowledge discovery techniques 42

43 Thank you Stay in touch JRC Science Hub: ec.europa.eu/jrc Twitter and YouTube: JRC Audiovisuals Vimeo: LinkedIn: european-commission-joint-research-centre 43