Atmospheric Science and GIS Interoperability issues: some Data Model and Computational Interface aspects

Transcription

1 UNIDATA Boulder, Sep Atmospheric Science and GIS Interoperability issues: some Data and Computational Interface aspects Stefano Nativi University of Florence and IMAA-CNR

2 Outline Service-Oriented Approach for interoperability Atmospheric Science (AS) & GIS Interoperability Scenarios Achievements AS & GIS Interoperability services 1. The THREDDS WCS interface implementation Data Interoperability 2. An Analysis of AS and GIS data models reconciliation 3. The GIS extension for NcML (Proposal) Future challenges Conclusions 2

3 Interoperability approach Service Oriented Approach (SOA): Heterogeneous Applications interoperate by means of services in a Distributed Computing Platform (DCP) According to the paradigm of User / Service Provider Applications are made up of Components component Service component component component Provider Application Distributed Computing Platform User Application stefano.nativi@pin.unifi.it 3

4 Interoperability approach Interoperable Components must share some knowledge: Service Interfaces Computational Interface Defines the syntax of service behaviour Communication Interface Encapsulates transport specific details of service Service Data s Content model Specification of service information content Encoding model Specification of service information encoding rules stefano.nativi@pin.unifi.it 4

5 Interoperability approach component component component Provider Application Shared Data Implementation Distributed Computing Platform Shared Computational Interface Implementation component User Application Loosely coupled Components Shared Communication Interface Implementation SOP: contract SOP: connector 5

6 Interoperability approach To combine pieces like in a puzzle game component Example NcML to OWS Data Mapping OWS Data WMS Intrfc WFS Intrfc WCS Intrfc HTTP GET HTTP POST SOAP/ RPC stefano.nativi@pin.unifi.it 6

7 Interoperability Scenarios Main scenarios: Intra-community Bridge Interoperability Middleware provides a given Information Community (e.g. Atmospheric Science Community) with Facilitating services Mediating services Inter-communities Gateway Interoperability Middleware provides Society with Data/services access to a given Information Community (e.g. AS community) Data model reconciliation services stefano.nativi@pin.unifi.it 7

8 Interop. Bridge Scenario for AS Community To facilitate heterogeneous Atmospheric Science (AS) applications to interoperate in order to achieve complex tasks Application Application Application Atmospheric Science Community Realm Application Application Interoperability middleware Legacy Data Systems 8

9 Interop. Gateway Scenario between AS & GIS Communities To facilitate interoperability among Society s Information Communities. Interoperability between Atmospheric Science Community (AS Community) GIS Community (Geo-Information Community) Interoperability middleware GIS Community Realm Atmospheric Science Community Realm Interoperability middleware Interoperability middleware stefano.nativi@pin.unifi.it 9

10 Interoperability Bridge Available Interoperability Open Specifications Atmospheric Science GIS Data Computational Interface Communication Interface NcML model; ESML model; OPeNDAP model; ADDE model; IDV services; OPeNDAP services; ADDE services; OPeNDAP protocol; ADDE protocol; HTTP; FTP; ISO 191xx model spec.s OpenGIS model spec.s; OWS 1.0 spec.s; HTTP; W3C spec.s 10

11 Interoperability Gateway GIS GIS GIS AS AS AS Interoperability Gateway Communication Interface Computational Interface Data Reconciliation Computational Interface Communication Interface 11

12 Contributions provided (Achievements) Implementation of a whole gateway service between AS & GIS Community Realms 1. Implementation of a WCS interface for the THREDDS middleware Interoperability Data for developing gateways between AS & GIS Community Realms 2. GIS and AS data model reconciliation analysis 3. Specification of a GIS extension for the NcML stefano.nativi@pin.unifi.it 12

13 WCS Interface for THREDDS #1 Co-authors John Caron Ben Domenico Yuan Ho Jeff Webb AS Comm. GIS Comm. THREDDS WCS 13

14 Inter-Communities Interoperability Interoperability Components Open specifications OpenDAP Protocol ADDE Protocol NC API THREDDS model NcML-CS model AS Realm OWS WCS GeoTIFF WCS Computationl Interface HTTP SOAP GIS Realm 14

15 WCS Services WCS use case 1. To access OGC WCS Capabilities document Available geographic Coverage metadata 2. To select an available Geographic Coverage, filtered by Space Time Available Parameter 3. To get the filtered geographic Coverage GeoTIFF format NetCDF format 15

16 WCS Implementation Distributed Computing Platform World Wide Web environment Distributed Computing Platform WCS communications Interfaces HTTP/GET HTTP/POST SOAP/RPC with attachment Communication Interface Implementation 16

17 WCS Computational Interface WCS Computational Interface OpenGIS Web Coverage Service specification ver. 0.7 content-type MIME:text/XML content-type MIME:image/tiff content-type MIME:application/octet-stream Computational Interface Implementation WCS Data OpenGIS OWS model ver. 1.0 OpenGIS WCS data model ver. 0.7 Grid Coverage Rectified Grid Coverage RemoteSensing.Org GeoTIFF ver.1.0 Data Implementation 17

18 Data interoperability Architecture GeoTIFF METAR NCEP Profiler McIDAS Grib NetCDF ADDE OpenDAP Convention XML THREDDS NcML XML OWS-WCS XML Reconciliation 18

19 Data interoperability Architecture METAR NCEP Profiler McIDAS Grib NetCDF ADDE OpenDAP XML Atmospheric Science Realm Convention GIS Realm THREDDS NcML XML GeoTIFF OWS-WCS XML Reconciliation 19

20 THREDDS WCS Implementation Internet GI Systems/ Applications USERS XML HTML WCS protocol requets GeoTIFF data WCS server THREDDS Middleware GeoTIFF Generator WCS Broker Standard Quantities Dataset Query Capabilities Dataset Description XML XML XML XML XML XML XML XML XML XML XML Dataset Inventory Catalogues THREDDS Catalogue Generators, Crawlers and Servers XML XML XML XML XML THREDDS Dataset Query Capability Server THREDDS Dataset Inv Catalogue Server Internet Internet IDV, VGEE, Web Browser, Java API USERS near real-time METAR data DODS: NCEP model data ADDE: NEXRAD Level 3 data LDM stefano.nativi@pin.unifi.it 20

21 THREDDS WCS Implementation THREDDS GeoTIFF data Facilitating Components XML XML XML XML XML XML XML XML XML XML XML WCS server GI Systems/ Applications USERS near real-time METAR data DODS: NCEP model data OPeNDAP ADDE: NEXRAD Level 3 data ADDE LDM stefano.nativi@pin.unifi.it 21

22 THREDDS WCS Implementation Present implementation GeoTIFF NcML coord. sys.s model conventions: CF COARDS AWIPS... NetCDF DODS THREDDS Dataset Inv. Catalogue server Dataset Inv. Catalogue Interface WCS Server THREDDS Client GeoTIFF Generator WCS Interface WCS Protocol Implementation Internet Web interface SOAP service interface HTTP service interface NcML object model Generator HTTP Communication Protocol Implementation Computational Node Computational Component NcML to WCS Transformation SOAP Server Communication Protocol Implementation 22

23 THREDDS WCS Implementation A more Service-oriented implementation GeoTIFF Generator GeoTIFF NcML coord. sys.s model conventions CF COARDS AWIPS... NetCDF DODS THREDDS Dataset Inv. Catalogue server GeoTIFF Generator Dataset Inv. Catalogue Interface THREDDS WCS Broker GeoTIFF Generator Interface THREDDS Client THREDDS Broker Interface WCS Server WCS Interface WCS Protocol Implementation Internet Web interface Computational Node SOAP service interface HTTP service interface Computational Component NcML object model Generator NcML to WCS Transformation HTTP Server Communication Protocol Implementation SOAP Server Communication Protocol Implementation stefano.nativi@pin.unifi.it 23

24 THREDDS WCS Implementation Extended Interoperability Framework SOAP service interface HTTP service interface GeoTIFF Generator GeoTIFF NcML coord. sys.s model conventions CF COARDS AWIPS... NetCDF DODS Dataset Inv. Catalogue server Other Legacy Data System Dataset Inv. Catalogue Services THREDDS WCS Broker GeoTIFF Generator Service THREDDS Broker Service WCS Server WMS Server WCS Services WMS Services Computational Node Internet Internet Computational Component Web interface Other Data Other Data server Data Server Services Other Data Broker Data Broker Service 24

25 THREDDS WCS Demo Server THREDDS Catalogue Monthly Mean Ocean Latent Heat Flux dataset COARDS/CF convention NetCDF dataset Dataset Inv. Catalogue WCS Capability OPeNDAP WCS Coverage (GeoTIFF file) NetCDF file Client Java graphical tool to test WCS client HTTP-GET/POST and SOAP/RPC 25

26 Interoperability side effect Geo Information Browser which shows the content according to ISO SINOTS3G system developed by the Univ. of Florence and IMAA - CNR Add the THREDDS WCS server to the list of Geo Information Browser servers It is possible to get an ISO compliant version of the Monthly Mean Ocean Latent Heat Flux COARDS/CF convention NetCDF dataset Dataset Inv. Catalogue WCS Capability OPeNDAP NetCDF file WCS Coverage (GeoTIFF file) ISO (GeoTIFF file) stefano.nativi@pin.unifi.it 26

27 Interoperability Data Analysis #2 Co-authors Ben Domenico John Caron AS Comm. GIS Comm. 27

28 Interoperability Data Interoperability Data reconciles: AS data model GIS data model Encoding Content Abstract Interoperability Application Schema AS schemas GIS Schemas Interoperability Application Types AS types GIS Types Interoperability Application General AS general Data GIS Data 28

29 Intra-Community Interoperability Data s Available Interoperability Open Specifications Atmospheric Science GIS Encoding Content Abstract NcML; ESML; NetCDF convention models; General Composite model; NetCDF abstract model ISO 19118; OpenGIS GML; SensorML; OpenGIS spec.s ISO TC211 data model spec.s Geo-relational ; General Feature ; stefano.nativi@pin.unifi.it 29

30 AS & GIS Reconciliation Information Community Realms overlap e.g. Geographic aspects of the AS Realm (or Geo-Facets) Geo-Information Community Realm Different Aspects (Concepts & Structures) Atmospheric Science Community Realm Common aspects (Concepts & Structures) 30

31 Common Aspects Most important ones Abstract Coverage Function concept Content & Encoding s Geographic Reference Systems Temporal Reference Systems Grid implicit geometries stefano.nativi@pin.unifi.it 31

32 Different Aspects: Abstract s AS General Measurement data model generates Is a stefano.nativi@pin.unifi.it 32

33 Different aspects: Abstract s GIS General Feature/Coverage data model 1...* stefano.nativi@pin.unifi.it 33

34 Different aspects: Content Metadata describing Satellite swath counts Radar cells Raingauge records AS Measurements Roads Cities Rivers GIS Geo-relational Features Patterns Recognition Geo-Feature extraction Geo-Features Properties Space tasselation 34

35 Different aspects: Encoding Atmospheric Science Encoding of Measurement datasets Metadata related to Measurements Process metadata Sensor metadata Very large hyperspace matrixes of counts GIS Encoding of Geo-relational Features Attributes characterising Geo-Feature Feature Geometry Features Topology 35

36 Concerns using GIS data model for AS datasets Abstract Feature-oriented ontology is not the natural model for conceptualising composite information Coverage is mainly modeled as a unique Feature Content Complex Coverage (e.g. hyperspace grids) are not extensively considered Geographic and Coverage Metadata doesn t cover all AS aspects (e.g. for climatology context or real-time data) Encoding GML types and structures are not sufficient to support all AS-application needs Encoding of large binary datasets in a neutral and standard self-descriptive language is still a problem stefano.nativi@pin.unifi.it 36

37 GIS Extension for NcML #3 Co-authors The NcML Specification Group Current Status: Proposal AS Comm. GIS Comm. 37

38 NcML-G NcML Abstract Geo-Coverage Abstract Extension Abstract Extension Content Encoding (GML-based) NcML-G ML Encoding (Inner-definition) NcML-G IN 38

39 NcML-G: Extension Abstract CompositeMod +Mesurement +Variable +Dimension GeoRelationalMod +Feature +Coverage +Geometry domainof...dataset 1..* contains...variable <<Domain>>...Geometry <<Codomain>...RangeSet GeoCSDimensions 0..* contains characterisedby 0..* GeoCoordinateSyst domainof Feature...Coverage domainof domainof 0..* 0..* DatasetExten DatasetLocatio contains 39

40 NcML-G: Content NcML.CoordinateSystem.CoordinateSystem -name:characterstring...domainextent Is a GeoCoordinateSyst...DomainLocation Is a GeoReferencedCoordinateSyste Is a...coveragegeocoordsys Is a Is a...coveragegeorefcoordsys stefano.nativi@pin.unifi.it 40

41 NcML-G: Encoding s NcML-G IN facilitate Atmospheric Science systems to deal with GIS metadata self-contained implementation of GIS concepts simplified implementation of GIS concepts (ISO 191xx); NcML-G ML facilitate as much as possible GIS systems to import NcML datasets. encode Atmospheric Science geographic aspects, using GML 3.0; NcML-G ML NcML-CS NcML-core XSD/XML GML 3.0 NcML-G IN NcML-CS NcML-core XSD/XML stefano.nativi@pin.unifi.it 41

42 Use Scenarios GIS System imports NetCDF datasets as GML documents NetCDF Conventions NcML-G ML document DOS, ADDE datasets NetCDF file API GIS 42

43 Use Scenarios GIS System imports NetCDF datasets as GeoTIFF file NetCDF Conventions NcML-G document GeoTIFF file DOS, ADDE datasets NetCDF file API API GIS 43

44 Use Scenarios GIS System imports NetCDF datasets as WCS documents + attachments: GeoTIFF attachment NetCDF attachment NetCDF Conventions NetCDF file GeoTIFF file NcML-G document NetCDF file GeoTIFF file WCS document DOS, ADDE datasets NetCDF file API translet GIS-WCS System stefano.nativi@pin.unifi.it 44

45 Future Challenges A Common Service-Oriented Framework for AS and GIS interoperability based on new powerful and open DCP AS Comm. GIS Comm. Distributed Computing Platform stefano.nativi@pin.unifi.it 45

46 Common Service-Oriented Framework Develop, Deploy, Combine interoperability components Loosely-coupled services AS Comm. GIS Comm. Service Oriented Framework 46

47 Common Service-Oriented Framework New DCP environment Web Services environment SOAP/WSDL Standard interface definition mechanisms UDDI Registry services W3C XML security technology Grid Services environment Open Grid Service Architecture (OGSA) Environment for transient Web Services Service semantics Reliability and Security models Lifecycle management Discovery 47

48 Conclusions Interoperability is mainly based on Data model reconciliation Common Interfaces For Inter-community interoperability Data model reconciliation plays a key role Achievements presented WCS implementation for THREDDS An inter-community interoperability gateway AS & GIS Data model reconciliation study Abstract and Content model reconciliation approach NcML GIS extension (NcML-G ML and NcML-G IN proposal) Content and Encoding model reconciliation Future challenges Common Service-Oriented Framework for AS and GIS interoperability, based on new DCP stefano.nativi@pin.unifi.it 48