IMPORTANCE OF OPEN SPATIAL DATA INFRASTRUCTURE FOR DATA SHARING

Transcription

1 IMPORTANCE OF OPEN SPATIAL DATA INFRASTRUCTURE FOR DATA SHARING E.Bank ISLEM GIS 13.cad. N Beysukent Ankara ebank@islem.cm.tr Cmmissin TS-PS-WG IV/4 KEY WORDS: GIS, Infrastructure, Spatial Infrastructures, Interperability, Standarts, Open Systems, Metadata, Inter-Operability ABSTRACT: An exciting trend and visin in Gegraphic Infrmatin System (GIS) is the develpment f Natinal, Cntinental, and Glbal Spatial Data Infrastructures (SDI). Assembling an SDI prtal invlves three key parts: 1. The Metadata Registry and Search Prtal 2. Dynamic mapping and data fusin n the Internet 3. Data mdeling and data dwnlad t acquire data in a useable and well-dcumented frmat fr use. By necessity, these are quickly becming a key part f many GIS implementatins. This presentatin will describe a practical apprach t building specificatins, standards, and reference data mdels fr SDI prgrams.the presentatin will als presents ESRI s GIS sftware platfrm fr pen SDI slutins that can enable lcal, reginal, natinal, and pan-eurpean data sharing. The presentatin will include; Spatial Data Infrastructure Infrmatin Mdeling fr Spatial Data Infrastructure Building Natinal and Cntinental Data Mdels fr SDI Best Practices fr Implementing SDI Using ESRI sftware in Cllabratin with Other GIS s The rle f standards fr SDI and key cntent and metadata specificatins 1. Spatial Data Infrastructure a. What is GIS technlgy? GIS technlgy is making a difference. GIS is mre than sftware r applicatin, it is a wrld f digital data, nline prtals, web services and lcatin services that enable access t infrmatin when and where needed. As an enabling technlgy GIS is making a significant difference tl f gd gverning, better buiness and a healty planet (ESRI. 2004). GIS is an Infrmatin System fr creating, maintaining, managing and using Gegraphic knwledge. GIS is a cmplete platfrm fr wrking (editing, data management, mapping, spatial analysis and visualizatin) with Gegraphic Infrmatin. GIS integrates many types f disciplines/rganizatins like; scial factrs, rads/infrastructure, land use/land cver, imagery, envirnment, base maps, activities like; gereferencing, digital prcessing, map verlay, spatial analysis, visualizatin, and data like; vectr and raster. GIS was a tl fr individual prjects and departments in the past but in recent years GIS has mved very fastly frm being a tl t becme the framewrk fr sharing infrmatin amng rganizatins and acrss sciety. GIS illustrates relatinships, cnnectins and patterns in data. The unique integratin capabilities f GIS bring tgether data frm many surces fr visual display and analysis, creating a mre cmplete picture f a situatin than wuld therwise be pssible. This enables rganizatins t make better decisins based n all relevant factrs. Fr this reasn GIS technlgy has becme an integral part f the infrmatin infrastructure in many rganizatins. GIS has lng been recgnized as an integrating technlgy that can bring highly disparate data int a cmmn map display. Overlaying layers f different data in a cmmn gegraphic space allws yu t integrate and derive new infrmatin t help yu slve prblems and t wrk mre efficiently and effectively. b. What is SDI technlgy? Spatial data is the lcatins and shapes f gegraphic features with descriptins f each feature and differs frm nrmal data with it s vlume and structure. Spatial datasets are interrelated and ften very large. Unlike business queries that may nly return a few recrds, spatial queries fr generating maps are rutinely run against many thusands f features. Spatial data usualy cmes frm acrss an rganizatin and increasingly frm utside surces. Because spatial datasets are fundamentally distributed and dynamic, standarts and interperability have always been a fundamental part f spatial data. There is widespread recgnitin that the data

2 layers and tables in mst GIS implementatins cme frm multiple rganizatins. An rganizatin typically develps sme, but nt all, f its wn spatial data cntent. At least sme f the layers will be frm external surces. Thus GIS data management, by its very nature, is distributed amng many users (ESRI. March 2003). GIS requires a distributed infrmatin system cncept t manage and share spatial data. Because GIS users are hungry fr quality gegraphic infrmatin, there is a fundamental need fr users t share their data. Tday, thusands f rganizatins wrldwide invest billins f dllars annually autmating and integrating map infrmatin fr their fcused GIS prjects. Their need fr up-t-date gegraphic infrmatin drives these effrts. SDI has becme an essential tl when infrmatin 2. Infrmatin Mdeling fr Spatial Data Infrastructure A Spatial Data Infrastructure intercnnects GIS ndes acrss the Wrld Wide Web t prmte infrmatin sharing and access. The SDI cncept binds many GIS ndes int lsely GIS User integratin is critical. It wrks with gegraphic infrmatin frm many surces t supprt a brad range f applicatins. SDI can repurpse infrmatin fr new applicatins that g beynd the data's riginal intent. Fr example, infrmatin cmpiled fr land recrds management can be used fr envirnmental applicatins, utilities, emergency respnse, hmeland security, and many ther applicatins. Spatial data management includes nt nly data cllectin but als data disseminatin. Interperability enables data sharing between rganizatins and acrss applicatins and industries, which results in the generatin and sharing f mre useful infrmatin. cupled infrmatin netwrks. There are three functinal nde in infrmatin mdel f SDI. GIS Users--A wide range f users wh cnnect t the catalgs, search fr useful gegraphic infrmatin, and then cnnect t GIS prtals that prvide the infrmatin access. GIS Users can find the data they need, quickly review and display its cntents, and read r create metadata. GIS Users can als manage spatial data stred in flders n lcal disks r in relatinal databases that are available n their netwrk. Metada Server Figure_1 : Building Blcks f SDI GIS Prtal GIS Prtal GIS Prtal GIS Prtal Metadata Catalgs--A series f well-rganized catalgs that describe and reference gegraphic infrmatin sets at each nde. Metadata is infrmatin abut GIS data describing a cllectin f data. Metadata fr gegraphical data may include the surce f the data, its creatin date and frmat, its prjectin, scale, reslutin, and accuracy, and its reliability with regard t sme standard. GIS Prtal--A GIS prtal that prvides access t FTP dwnlad sites, maps, reprts, Internet GIS services, and s n. GIS Prtal publishes GIS Data & Services & Dcuments. A brad range f GIS users access metadata and services with any client (wireless, html, java,pcket PC, windws). In practice, each nde in an SDI wuld be built t prvide ne r mre f these three capabilities. Fr example, a city might prvide a catalg referencing the available data and services at its Web site alng with a series f data and mapping services. A small GIS shp wuld be able t access and use thse services in their Sftware seat acrss the Internet. In anther case, a state gvernment might prvide the metadata catalg describing its services as well as data and services at ther GIS prtals within the state. The state might als publish GIS Web services with a set f HTML clients t access varius Internet map and data services. At its mst basic level, a Spatial Data Infrastructure is realized thrugh a catalg hlding metadata abut available data and services. As the number f participating rganizatins and users grw, s will the numbers f entries and cmplexity f the metadata catalg. The rle f a metadata server and search tls becmes mre critical. Metadata server makes accessing and managing gegraphic data simple. 3. Building Natinal and Glbal Data Mdels fr SDI GIS prfessinals are learning abut the benefits f a Spatial Data Infrastructure (SDI). The SDI, develped by natinal, state, and lcal gvernments, prmtes the visin f a framewrk fr GIS users t penly share gegraphic infrmatin with ne anther. The SDI is meant t address the needs fr users t intercnnect their existing GIS ndes acrss the Internet (and in many cases ver secure netwrks) in rder t share infrmatin with ne anther penly (i.e., based n standards). Many peple are familiar with the term NSDI, r Natinal Spatial Data Infrastructure, a cncept defined as the technlgies, plicies, and peple necessary t prmte the sharing f gespatial data thrughut all levels f gvernment, the private and nnprfit sectrs, and the academic cmmunity. Recent discussins have revlved arund the GSDI, r Glbal Spatial Data Infrastructure, which describes a framewrk fr data sharing at the glbal level. Many GIS users want t prvide infrmatin prtals t publish a metadata catalg fr their cllectin f GIS data and services. A few clearinghuse sites want t prvide a catalg f infrmatin sets published by many users. Smetimes the data and services will be published fr a user's

3 wn GIS nde, while at ther times the catalg will include data and services at many ther GIS sites. Access t such a clearinghuse nde culd be pen n the Wrld Wide Web, while at ther times it can be restricted and secure fr a select audience. The NSDI, r Natinal Spatial Data Infrastructure, is a cncept develped by the United States federal gvernment and ther state and lcal gvernments. The Glbal Spatial Data Infrastructure (GSDI) is a nnprfit rganizatin cmpsed f individuals, industries, and mre than 50 cuntries whse missin is t supprt ready glbal access t gegraphic infrmatin. The NSDI and GSDI prmte the visin f a framewrk fr GIS users t penly share gegraphic infrmatin and t cllabrate n data develpment (ESRI. March 2003). The visin f a Natinal Spatial Data Infrastructure is based n GIS users penly sharing key sets f GIS infrmatin and the ability f users t search fr and discver infrmatin sets acrss the Wrld Wide Web. Users shuld als be able t access datasets thrugh a variety f mechanisms fr example, via cnnectins t data streaming services, access t map services, and data dwnlads in numerus desired frmats. Catalg prtals require a cmplete infrmatin system fr gegraphy, tls fr data imprt and exprt, tls t validate data, tls t serve catalgs and data, search and discvery tls, tls t explre and use data, and tls t build and manage GIS data. A GIS metadata catalg plays the same rle as a card catalg in a library (ESRI. March 2003). The card catalg in a library has a recrd fr each hlding. All catalg recrds are cmpiled int ne cmprehensive catalg and indexed by authr, subject, and title (and ther keywrds) s that the catalg can be searched. In a GIS, each dataset has a metadata dcument. The cncept behind a GIS catalg prtal is that these metadata dcuments are cllected int a cmprehensive metadata catalg that can be indexed by varius means, such as by gegraphic lcatin. Usually GIS experts intend mre detailed metdata and General Public intend minimal metadata. Minimal Metadata General Public Number f users Mre detailed metadata Full Metadata GIS Experts Figure_2 : Metadata Cmpleteness (Gerc Hgeweg.2004 ). The catalg prtal can then be searched t find candidate datasets fr use. Just as in a library search, each metadata recrd in a GIS catalg prtal can be viewed. Optinally, users can cnnect t and explre the actual dataset t make a decisin abut its ptential use. This is a critical functin necessary fr advanced GIS applicatins. In additin, the GIS Metadata Catalg can be thught f as the rganizing framewrk fr a GIS just as the card catalg reflects the rganizatin f the library as a system, such as the rganizatin f its shelves. Similarly, a GIS netwrk and its catalg prtals can be thught f as the rganizing framewrk fr a multirganizatin GIS (e.g., a state r natinal GIS). Each GIS requires a catalg that dcuments its data hldings and prvides fast access t them. The gal f metadata must be t create cnsistent dcumentatin fr these hldings s that searches wrk acrss GIS systems. Gazetteer Map-based search Search n cmmn fields Metadata search (a) (b) (c) Figure_3 : GIS Catalag (a): Catalg search, (b): Gazetteer, (c): Search result

4 4. Best Practices fr Implementing SDI There are already many examples f catalg prtals many f which are GIS prtals that supprt infrmatin search and discvery. A shrt review f a few very typical Web prtals prvides a helpful cntext t gain a brader understanding f what s available, hw they wrk, and sme f the key characteristics that will be imprtant t build a wrking, practical Spatial Data Infrastructure (ESRI. March 2003). GIS Prtals The URL f the FGDC Clearinghuse, which is a cllectin f GIS sites, knwn as Clearinghuse Ndes. T search many catalgs, yu perfrm a distributed search acrss selected ndes. The distributed search methd has nt wrked well in practice fr numerus reasns (e.g., search sites may nt be nline s n guaranteed results; each site takes a perfrmance hit fr every search; there is ne result set t investigate fr each search nde, making cmparisns difficult; perfrmance is dismal). Recent studies indicate that search users cnnect directly t selected ndes and brwse fr data, which is als quite inefficient. a. The Gespatial Data Clearinghuse is a cllectin f ver 250 spatial data servers, that have digital gegraphic data primarily fr use in Gegraphic Infrmatin Systems (GIS), image prcessing systems, and ther mdelling sftware. These data cllectins can be searched thrugh a single interface based n their descriptins, r "metadata." Click n the name f a Clearinghuse Gateway n the map belw that is clsest t yu and yu will be presented with a selectin f query frms. All entry pints r gateways shwn have exactly the same lists f servers. b. ESRI s central catalg prtal, where users can search fr, discver, and directly cnnect t numerus GIS Web services and Data Dwnlad sites. The Gegraphy Netwrk references datasets and services fr vlunteer GIS participants. Additinal sites have been added ver time t help ESRI learn abut best practices and methds. c. The GIS catalg prtal fr the U.S. Bureau f Land Management and the U.S. Frest Service is built using the same technlgy as the Gegraphy Netwrk. Its gal is t prvide an access prtal t Federal Lands infrmatin including the Public Land Survey System (PLSS) d. Brwsing each Web site with its wn unique layut and design is the mst typical access methd used tday fr finding GIS data. This is the State f Minnesta s site fr GIS data search and acquisitin. Mst GIS use at such sites is t search fr data by brwsing the Web pages at this site e. natinalmappilt/ index.htm The State f Delaware s DataMIL site is an example Web site fr searching and delivering data layers as part f the USGS Natinal Map. Als listed is the State f Texas Gegraphy Netwrk prtal fr searching and finding GIS data Statewide. These are tw in a series f GIS sites that prvide mdern GIS catalg services fr searching.

5 5. Using ESRI sftware in Cllabratin with Other GIS s In 2001, ESRI began t discuss g.net a new architecture fr sharing and using gegraphic infrmatin system (GIS) infrmatin frm distributed surces. g.net is an architectural visin fr hw ESRI users build parts f and participate in a Spatial Data Infrastructure. g.net describes hw ArcGIS, ArcIMS, and ther ESRI sftware technlgy are used. ArcGIS, ArcIMS, ArcSDE, and ther ESRI sftware can be used t implement all f the functinal ndes in a Spatial Data Infrastructure. This architecture is knwn as g.net. ArcGIS tls can be used fr SDI in the fllwing ways (ESRI. March 2003): Client access - Any GIS user wishing t access infrmatin and services remtely can cnnect t and use metadata servers and GIS prtals.arcims includes a range f client applicatins t cnnect t and use standards-based Metadata Servers and GIS prtals. ArcIMS clients can include the Metadata Explrer, varius HTML- and Java -based Webmapping clients, and ther GIS desktps. These include ArcReader, ArcView, ArcEditr, ArcInf, MapObjects fr Java, and ArcPad. Create, manage, and serve metadata- With ArcGIS, users can create and update FGDC- and ISO-cmpliant metadata. Using the standards-based ArcIMS Metadata Server and ArcSDE, users can manage and serve metadata catalgs n a lcal netwrk, a secure netwrk, r the Wrld Wide Web. The ArcIMS Metadata Server can be accessed thrugh standards-based ArcIMS services and thrugh Open GIS prtcls, such as Z Create, manage, and serve GIS infrmatin- GIS users build and manage their gegraphic data and infrmatin using ArcGIS and ArcSDE. They can penly serve data, metadata, nline maps, and ther infrmatin using ArcIMS. ArcIMS services can be accessed with a wide range f clients via GIS and XML-based Web services standards. System Cmpnents GIS Prtal Cmpnents GIS Prtal Applicatin Tl Cmpnents (ArcWeb App Objects) ArcIMS Java Cnnectr ArcIMS System Envirnments HTML, HTTP, XSL, XML, JSP Java Beans, Servlets Spatial Management Cmpnents Data Management Cmpnents ArcIMS App/Spatial Servers ArcIMS ArcSDE RDBMS Data TCP/IP, Sckets SQL Figure_4: ESRI GIS Prtal Architecture (Gerc Hgeweg.2004) 6. The rle f standards fr SDI and key cntent and metadata specificatins Until the mid-'90s, rganizatins purchased gegraphic infrmatin systems that clsely tied applicatins t a native, prprietary spatial data mdel. These early nnrelatinal file structures were highly ptimized fr fast access t data and, being file based, were relatively easy t distribute between sites using the same GIS vendr sftware. Hwever, the ability t share data amng users within an rganizatin was limited by netwrk prtcls such as netwrk file system (NFS). Data sharing between rganizatins with different GIS vendr systems was limited t data cnverters, transfer standards, and later pen file frmats. Sharing spatial data with ther cre business applicatins was rarely achieved. Gradually, GIS mdels evlved int gerelatinal structures where related attribute data culd be stred in a relatinal database that was linked t the file-based spatial features. Hwever, the gerelatinal frmat had limited scalability, and the dual data structure (spatial features stred in prprietary file-based frmat with attributes stred in a relatinal database) meant that the GIS culd nt take full advantage f relatinal database features such as backup and recvery, replicatin, and fail-ver. In additin, supprting large data layers required the use f cmplex tiling structures t maintain perfrmance, and sharing spatial infrmatin with ther cre business applicatins was still nt pssible. In the mid-'90s, new technlgy emerged that enabled spatial data t be stred in relatinal databases (ften referred t as spatially enabling the database), pening a new era f brad scalability and the supprt f large, nntiled, cntinuus data layers. When the new spatially enabled databases were cmbined with client develpment envirnments that culd be embedded within cre business applicatins, the sharing f spatial features with cre business applicatins, such as custmer management systems, became pssible. In additin, these spatially enabled databases allwed rganizatins t take the first steps tward enterprise GIS and the eliminatin f

6 rganizatinal "spatial data islands." Perhaps nt cincidently, the pen GIS mvement was spawned shrtly after the arrival f the first all-relatinal mdels capable f string bth spatial and attribute data in a relatinal database when standards rganizatins, such as the Open GIS Cnsrtium (OGC), the Internatinal Organizatin fr Standardizatin, and the U.S. Federal Gegraphic Data Cmmittee, began prmting the idea f data sharing thrugh spatial data standards. The early wrk f these rganizatins was fcused n sharing simple spatial features in a relatinal database, thereby enabling interperability between the cmmercial GIS vendrs. OGC, an internatinal industry cnsrtium f private cmpanies, gvernment agencies, and universities, published an pen spatial standard called the Simple Features Specificatin. T fully realize the capability and benefits f gegraphic infrmatin and GIS technlgy, spatial data needs t be shared and systems need t be interperable. GIS technlgy prvides the framewrk fr a shared spatial data infrastructure and a distributed architecture. It is very imprtant that a GIS prduct has develped its prducts based n pen standards t ensure a high level f interperability acrss platfrms, databases, develpment languages, and applicatins. The GIS cmmunity in general has been pursuing pen standards and interperability fr many years. Apprachs listed belw play a significant rle in GIS interperability (ESRI. Summer 2003). Data Cnverters (DLG, TIGER, MOSS, GIRAS, IGDS) Standart Interchange Frmats (SDTS, DXF,GML) Open File Frmats (VPF, Shapefiles,DGN) Direct read applicatin prgramming interface (ArcSDE API, CAD reader, ArcSDE CAD Client) Cmmn feature in a DBMS (OGC Simple Feature Specificatin fr SQL) Integratin f standardized GIS Web services (OGC WMS (Web Map Server), OGC WFS (Web Feature Server)) that the GIS cmmunity has been pursuing pen interperability fr many years, and the slutins t achieving this gal have changed with the develpment f new technlgies. In the early days f GIS, the fcus, with rare exceptins, was n individual, islated prjects. Tday the fcus is n the integratin f spatial data and analysis in the missin-critical business prcesses and wrk flws f the enterprise and n increasing the return n investment (ROI) in GIS technlgy and databases by imprving interperability, decisin making, and service delivery. A GIS must prduce useful infrmatin prducts that can be shared amng multiple users, while at the same time prvide a cnsistent infrastructure t ensure data integrity. It is imprtant nt t get caught up in the technlgy and frget this basic principle. Interperability enables the integratin f data between rganizatins and acrss applicatins and industries, resulting in the generatin and sharing f mre useful infrmatin. 8. References ESRI. Summer 2003 Standarts and Interperability ESRI Internatinal User Cnference ESRI. March 2003 Implementing a metadata catalg prtal in a GIS netwrk Gerc Hgeweg.2004 ESRI GIS Prtal Tlkit presentatin 9. Authr Cntact Adress Dr. Emin BANK Surveying Engineer, Ms.Sc. ISLEM Gegraphic Infrmatin Systems&Engineering C.Ltd. 13. Cad N: 14, Beysukent ANKARA, TURKEY Tel: Fax: Mail: ebank@islem.cm.tr 7. Cnclusin The Spatial Data Infrastructure (SDI) cncept describes requirements fr cmputer technlgies, plicies, and peple necessary t prmte the sharing f gegraphic infrmatin thrughut all levels f gvernment, private industry, Nn- Gvernmental Organizatins (NGOs), and the academic cmmunity (ESRI. March 2003). The SDI intercnnects GIS ndes acrss the Internet and, in many cases, ver secure netwrks t share infrmatin with ne anther penly (i.e., based n the best available set f wrking, widely adpted practices and methds). These and ther types f GIS prtals are being built at natinal, state, and lcal levels fr gegraphic infrmatin access and sharing. It is expected that many f these prtals will be central sites that users can readily access and search. In early years, the cnstraints f cmputatinal speed and cst caused GIS vendrs t fcus n practical slutins such as direct file cnversin. Data sharing between rganizatins with different GIS vendr systems was limited t data cnverters, transfer standards, and later pen file frmats. Sharing spatial data with ther cre business applicatins was rarely achieved. Tday, mst GIS prducts directly read and smetimes dynamically transfrm data with minimal time delay. The pint here is