Exploring PostGIS with a Full Analysis Example

Transcription

1 Exploring PostGIS with a Full Analysis Example J.C. MARTINEZ, E. COLL, J. IRIGOYEN Department of Cartographic Engineering, Geodesy and Photogrammetry Polytechnic University of Valencia Camino de Vera, 46022, Valencia SPAIN Abstract: -. This paper explains PostGIS functioning through a small spatial analysis. For this purpose the analysis is firstly performed on a traditional way using a GIS Desktop program (ArcGIS from ESRI), and secondly the same analysis is carried out using SQL language and spatial extensions included on PostGIS, extensions that follow the Simple Features for SQL guidelines, from the Open Geospatial Consortium. Finally, the result from the spatial analysis are assessed and compared. Key-Words: - PostGIS, PostgreSQL, ArcGIS, Spatial Database, SQL Spatial, Open Geospatial Consortium 1 Introduction The software PostGIS [8] (developed by Refractions Research Inc.) add a medium to store geographic objects to the relational database PostgreSQL. In this way, a PostgreSQL server can be used as a backend for a Geographic Information Systems (GIS). As one of the most common task working with a GIS is the spatial analysis, this paper firstly describes the necessary steps and secondly discusses the problems that appear when trying to carry out a spatial analysis with PostGIS, using a spatial database. Thus, for each stage of the analysis, the different queries statements are described and commented. In order to contrast the obtained results, the same analysis is performed also with a commercial program: ArcGIS, from ESRI [1]. The main difference between them is that PostGIS is a program distributed under GPL licence, and ArcGIS is privative software. Furthermore PostGIS is only a spatial database, while ArcGIS is also a complete solution of GIS Desktop. Thus, to visualize, design, etc. PostGIS layers, is necessary to use other software like the ones used in this article: JUMP [5] and QGIS 0.6 [10]. The hardware and the software employed in the analysis described in this paper are: - ArcGIS 9.0 running under Windows XP SP2 operating system - PostGIS 0.9.0, PostgreSQL 7.4.6, GEOS under Suse Linux 9.1 operating system. - Hardware: Notebook with Pentium Centrino processor 1.5 GHz. (Intel). 512Mb RAM 2 Spatial analysis principles The purpose is to found the optimum location for a laboratory with the following requirements: - Located on brushes area (type of soil code equal to 300) - Located on soil suitable for building (type of soil use code higher than 0) - Located close to sewer network (distance lower than 300m.) - Located away from rivers (up to 20m. for small rivers and up to 40m. for big ones) - Area bigger than m Available Cartography Datasets provided for the spatial analysis consist on four layers on shape format [4], and a table on dbf format: - Layer suelos (Fig. 1). Polygonal data layer that consists of 43 features describing soil types. Attribute table schema: (shape: polygon, tsuelo: short int {0, 1, 2, 3},{unsuitable, lower suitability, medium suitability, high suitability}). - Layer usos (Fig. 2). Polygonal data layer that consists of 76 features describing the different kind of soil uses. Attribute table schema: (shape: polygon, tuso: short int {100,200,300, 400,500,600,700},{Urban, Agriculture, Thicket, Wood, Water, Wetland, Badlands}). - Layer river (Fig. 3). Lineal data layer that consists of 106 features describing the existing rivers. Attribute table schema: (shape: line, trio: short int {1, 2},{Minor, Major}).

2 - Layer alcanta (Fig. 4). Lineal data layer that consists of 6 features describing the sewer network. Attribute table schema: (shape: line, id: short int {0}). The attribute id is not used, but it has been created as a layer on a shape format must have at least one field, apart from the geometric one (in ArcGIS 9). Fig. 4. Layer alcanta - Table riodist with the correspondences within rivers and distances in order to complete the proximity analysis. Fig. 1. Layer suelos Trio dist The first step in both spatial analyses, the ArcGIS one as well as the PostGIS one, has been the conversion from the original shape format to the corresponding spatial databases (geodatabase in ArcGIS, or PostGIS tables in PostGIS). The conversion from shape to geodatabase is not compulsory as the analysis operations can be performed in both format. Anyway the conversion adds consistency to the comparison as it increases the criterion standards between both programs. Fig. 2. Layer usos 3 Análisis con ArcGIS In order to perform the spatial analysis, the selected version has been the 9.x, as it adds the new analysis operation engine from ArcInfo that is enclosed in ArcGIS by means of ArcToolbox. The tool modelbuilder [3] has been used in this paper as it allows designing the flowchart in a visual way (Fig. 5). In this way the model, which defines all the spatial operations to perform, can be used to run, modify or repeat on a simple way all the analysis process. Fig. 3. Layer rios

3 The parameters considered in each analysis operation can be seen from the programming code in Jscript, corresponding to the ArcGIS analysis model. The code is the following one: // // ArcGIS generated script //modified by authors // // Create the Geoprocessor object var gp = WScript.CreateObject("esriGeoprocessing.GPDispatch.1"); // Set the necessary product code gp.setproduct("arcinfo"); // Load required toolboxes... gp.addtoolbox("c:/arcgis/arctoolbox/toolboxes/conversion Tools.tbx"); gp.addtoolbox("c:/arcgis/arctoolbox/toolboxes/data Management Tools.tbx"); gp.addtoolbox("c:/arcgis/arctoolbox/toolboxes/analysis Tools.tbx"); // Local variables... var test = "C:\\test"; var bd_mdb = test + "\\bd.mdb"; // Original Data var alcanta_shp = test + "\\alcanta.shp"; var rios_shp = test + "\\rios.shp"; var suelos_shp = test + "\\suelos.shp"; var usos_shp = test + "\\usos.shp"; var riodist_dbf = test + "\\riodist.dbf"; // Geodatabase Featureclasses var alcanta = bd_mdb + "\\alcanta"; var rios = bd_mdb + "\\rios"; var suelos = bd_mdb + "\\suelos"; var usos = bd_mdb + "\\usos"; var riodist = bd_mdb + "\\riodist"; // Intermediate Data var alcantabuf = bd_mdb + "\\alcantabuf"; var riosbuf = bd_mdb + "\\riosbuf"; var vrios_lyr = test + "\\vrios.lyr"; var vrios_lyr2 = test + "\\vrios.lyr"; var vrios = bd_mdb + "\\vrios"; var inter = bd_mdb + "\\inter"; var difbuf = bd_mdb + "\\difbuf"; var final = bd_mdb + "\\final"; var vinter = test + "\\vinter.lyr"; Fig. 5. Spatial analysis (modelbuilder) // Process: Create Personal GDB... gp.createpersonalgdb_management(test, "bd"); // Process: Feature Class To Feature Class... gp.featureclasstofeatureclass_conversion(alcanta_shp, bd_mdb, "alcanta", "", "id id VISIBLE", "DISABLED", "DISABLED", "", "0"); // Process: Buffer... gp.buffer_analysis(alcanta, alcantabuf, "300, Meters", "FULL", "ROUND", "ALL", ""); // Process: Feature Class To Feature Class (2)...

4 gp.featureclasstofeatureclass_conversion(rios_shp, bd_mdb, "rios", "", "trio trio VISIBLE", "DISABLED", "DISABLED", "", "0"); // Process: Make Feature Layer... gp.makefeaturelayer_management(rios, vrios_lyr, "", "", "trio trio VISIBLE"); // Process: Table To Table... gp.tabletotable_conversion(riodist_dbf, bd_mdb, "riodist", "", "trio trio VISIBLE;dist dist VISIBLE", ""); // Process: Add Join... gp.addjoin_management(vrios_lyr, "trio", riodist, "trio", "KEEP_COMMON"); // Process: Copy Features... gp.copyfeatures_management(vrios_lyr2, vrios, "", "0", "0", "0"); // Process: Buffer (2)... gp.buffer_analysis(vrios, riosbuf, "riodist_dist", "FULL", "ROUND", "ALL", ""); // Process: Erase... gp.erase_analysis(alcantabuf, riosbuf, difbuf, ""); // Process: Feature Class To Feature Class (3)... gp.featureclasstofeatureclass_conversion(suelos_shp, bd_mdb, "suelos", "", "tsuelo tsuelo VISIBLE", "DISABLED", "DISABLED", "", "0"); // Process: Feature Class To Feature Class (4)... gp.featureclasstofeatureclass_conversion(usos_shp, bd_mdb, "usos", "", "tuso tuso VISIBLE", "DISABLED", "DISABLED", "", "0"); // Process: Intersect... gp.intersect_analysis(suelos + ";" + usos, inter, "ALL", "", "INPUT"); // Process: Make Feature Layer (2)... gp.makefeaturelayer_management(inter, vinter, "[tsuelo] > 0 AND [tuso] = 300", "", "FID_suelos FID_suelos VISIBLE;tsuelo tsuelo VISIBLE;FID_usos FID_usos VISIBLE;tuso tuso VISIBLE"); // Process: Intersect (2)... gp.intersect_analysis(difbuf + ";" + vinter, final, "ALL", "", "INPUT"); 4 PostGIS Analysis In order to understand the article, the reader must have knowledge of query language and SQL data manipulation (Structured Query Language) [2], and the knowledge of the SFS specification is advisable. PostGIS has been developed by Refractions Research Inc as a research project in open source spatial database technology. PostGIS is released under the GNU General Public License. PostGIS is an extension to the object-relational database PostgreSQL [9] which includes the following characteristics: - Simple Features (SF) as defined by the Open Geospatial Consortium (OGC). - Support for Well-Known Text (WKT) and Well- Known Binary (WKB) representations of GIS objects. - Fast spatial indexing using GiST - Geospatial analysis functions using de GEOS library. - PostgreSQL JDBC extension objects corresponding to the geometries. - OGC access functions defined by the Simple Features Specification (SFS). PostGIS verifies OGC specification [7] called SFS Simple Features for SQL [6], that s the document where the spatial predicates and operators are described. Spatial predicates are functions which compare two spatial objects and return a boolean true/false result indicating the existence (or absence) of a particular spatial relationship. Some examples of spatial predicates are Contains(), Intersects(), Touches(), and Crosses(). The spatial operators take two geometries and return a new derived geometric result. Examples of the operators include Difference(), Union(), Buffer() and Intersection(). The current SFS version is the 1.1, and it is based on SQL92, therefore object-relational concepts (that are integrated on SQL99) are not included. However PostgreSQL would be able to work with them, as it is an object-relational database. 4.1 Spatial analysis As a previous step to the spatial analysis, some tasks are necessary, like creating the database or importing the layers from the shape format. Thus, a script has been created to perform all these tasks: #!/bin/bash createdb test # database produced createlang plpgsql test # language plpgsql psql -f postgis.sql -d test # postgis functionality psql -f spatial_ref_sys.sql -d test #reference systems table #Layer conversion from shp to sql format shp2pgsql rios.shp rios > rios.sql shp2pgsql suelos.shp suelos > suelos.sql shp2pgsql usos.shp usos > usos.sql shp2pgsql alcanta.shp alcanta > alcanta.sql #PostGIS load of layers psql -d test -f rios.sql psql -d test -f alcanta.sql psql -d test -f suelos.sql psql -d test -f usos.sql psql -d test -f test.sql #spatial analysis

5 4.2 Test.sql file Buffer of the layer alcanta 1: create table tmp1 (gid serial); 2: select addgeometrycolumn ('','tmp1','the_geom',-1,'polygon',2); 3: alter table only tmp1 add constraint tmp1_pkey primary key (gid); 4: insert into tmp1(the_geom) select buffer(the_geom,300) from alcanta; Dissolution of the buffer barriers 5: create table alcantabuf (gid serial); 6: select addgeometrycolumn ('','alcantabuf','the_geom',-1,'polygon',2); 7: alter table only alcantabuf add constraint alcantabuf_pkey 8: insert into alcantabuf(the_geom) select geomunion(the_geom) from tmp1; Buffer of the layer rios 9: create table riodist (trio integer primary key,dist float); 10: insert into riodist values (1,40); 11: insert into riodist values (2,20); 12: create table tmp2 (gid serial); 13: select addgeometrycolumn ('','tmp2','the_geom',-1,'polygon',2); 14: alter table only tmp2 add constraint tmp2_pkey 15: insert into tmp2(the_geom) select buffer(r.the_geom,d.dist) from rios as r, riodist as d where r.trio = d.trio; Dissolution of the buffer barriers 16: create table riosbuf (gid serial); 17: select addgeometrycolumn ('','riosbuf','the_geom',-1,'multipolygon',2); 18: alter table only riosbuf add constraint riosbuf_pkey 19: insert into riosbuf(the_geom) select geomunion (the_geom) from tmp2; Intersection of suelos and usos layers. Use of spatial indexes 20: create index suelos_the_geom_idx on suelos using gist 21: select update_geometry_stats('suelos','the_geom'); 22: create index usos_the_geom_idx on usos using gist 23: select update_geometry_stats('usos','the_geom'); 24: create table inter (gid serial,tsuelo integer, tuso integer); 25: select addgeometrycolumn ('','inter','the_geom',-1, 'MULTIPOLYGON',2); 26: alter table only inter add constraint inter_pkey 27: insert into inter (tuso,tsuelo,the_geom) select * from (select u.tuso, s.tsuelo, multi(intersection (u.the_geom, s.the_geom)) as ge from usos as u, suelos as s where u.the_geom && s.the_geom and intersects (u.the_geom,s.the_geom)) as foo; 28: create view vinter as select i.gid as gid,i.the_geom as the_geom from inter as i where i.tuso = 300 and i.tsuelo > 0; Difference between alcantabuf and riosbuf 29: create table difbuf (gid serial); 30: select addgeometrycolumn ('','difbuf','the_geom',-1, 'MULTIPOLYGON',2); 31: alter table only difbuf add constraint difbuf_pkey 32: insert into difbuf (the_geom) select multi(difference (c1.the_geom,c2.the_geom)) from alcantabuf as c1, riosbuf as c2 where intersects (c1.the_geom,c2.the_geom); Intersection of difbuf y vinter layers. Use of spatial indexes. Final layer (Fig. 6) 33: create table final (gid serial); 34: select addgeometrycolumn ('','final','the_geom',-1, 'MULTIPOLYGON',2); 35: alter table only final add constraint final_pkey 36: create index inter_the_geom_idx on inter using gist 37: select update_geometry_stats('inter','the_geom'); 38: create index difbuf_the_geom_idx on difbuf using gist 39: select update_geometry_stats('difbuf','the_geom'); 40: insert into final (the_geom) select multi(intersection (v.the_geom,b.the_geom)) from vinter as v,difbuf as b where v.the_geom && b.the_geom and intersects (v.the_geom,b.the_geom); Fig. 6. Final layer (12 records). Superficies calculation 41: select gid, area (the_geom) as area from final order by area desc;

6 5 Conclusion The two spatial analyses, performed by different software have obtained the same result: 12 areas, 3 of them bigger than m 2. Next table (Table1) shows the calculated surfaces from each program: PostGIS ArcGIS Diferencia 7282, , , , , , , , , , , , , , , , , , , , , ,5 2635, , ,5 1730, , , , , , , , , , , Table 1. Analysis results. (m 2 ) As it is visible in the table, except for the shaded rows, the calculated areas are practically the same, fitting at worst with the third decimal. Differences highlighted on the shaded rows are totally justified by the segments number that each program uses to calculate rounded ends on buffer operations. ArcGIS considers many more segments than PostGIS that uses 8. On the next version of PostGIS, 0.9.1, a third parameter is included on the buffer operator, to specify the segments number. Referring to the time employed, PostGIS spend 5 seconds to perform all the process (pre-view script running on bash), while ArcGIS spend 25 seconds. However this analysis is not representative of the time spend by the algorithms from each program, as the used layers are too small, and consist of few geometric features. Referring to the layers visualization, both Jump and QGIS have work out fine. The positive characteristics of QGIS are the fact that it doesn t load the complete layer on memory as Jump: QGIS directly attacks PostGIS spatial database while Jump performs an import of the layers. Furthermore QGIS renders faster at screen. Jump advantages are the higher number of options (that makes it more similar to a GIS Desktop), and the fact that it s based on the Java Topology Suite (JTS). PostGIS can also be used as a data source with a map server, for example UMN Mapserver [11]. With this combination fast viewers and complete Web GIS can be created. 6 Acknowledgements We want to acknowledge all users that, in an unselfish way contribute to the open source projects, and especially to the PostGIS, PostgreSQL and Jump users list, as well as to Refractions Research Inc. and all their developers: thanks to their fantastic open source solutions. We also want to acknowledge all the institutions that take part in this kind of projects. This work has been partially supported by the research project Information and Management in Local Administration BIA from the Spanish Government (CICYT) and the European Union (ERDF funds). References: [1] ArcGIS 9.x, index.html [2] Celma, M. et al, Bases de datos relacionales, Pearson Prentice Hall, 2003 [3] ESRI, ArcGIS 9. Geoprocessing in ArcGIS, ESRI Digital Book, 2004 [4] ESRI, ESRI Shapefile Technical Description, ESRI White Paper, 1998 [5] JUMP (JUMP Unified Mapping Platform), [6] OGC, Simple Features SQL (SFS), opengis.org/docs/ pdf [7] OGC (Open Geospatial Consortium), opengeospatial.org/ [8] PostGIS, [9] PostgreSQL, [10] QGIS (Quantum GIS), net / [11] UMN Mapserver, index.html