ArcGIS Data Models: Raster Data Models. Jason Willison, Simon Woo, Qian Liu (Team Raster, ESRI Software Products)

Transcription

1 ArcGIS Data Models: Raster Data Models Jason Willison, Simon Woo, Qian Liu (Team Raster, ESRI Software Products)

2 Overview of Session Raster Data Model Context Example Raster Data Models Important Raster GDB Considerations Rasters in ArcGIS 9.0 Questions / Answers / Comments Sample Database and Design Documents For more information, go to

3 What is a Data Model? A practical template for implementing GIS solutions Built on consensus within a discipline Both providers and users Goal is consistency across organizations Great starting points Designed to adhere to emerging standards

4 Why Data Models? Data models provide Quick Start" solutions Best practices Optimized performance Improves data sharing More successful GIS implementations The ArcGIS geodatabase architecture allows Ready-to-use frameworks Centralized location Intelligence/Rules

5 Raster Data Model Context Raster Data Models Essential parts of other data models Just another layer in your data model stacks Base-mapping and/or Analysis needs ESRI s Geodatabase Raster Solution Enterprise or personal geodatabase We will focus on enterprise (ArcSDE) Easy to build specifically to meet your needs

6 Raster Data Model Context

7 Demonstration

8 Where do I start? Determine The purpose of my raster data Your application s scale of use The properties of the raster data All this contributes to your database design Use these guidelines to optimize your solution

9 Types of Geodatabase Rasters Raster Dataset Storage of a single raster dataset Mosaic many inputs to one continuous raster dataset Raster Catalog Container of independent raster datasets Organized as one entity in the database Raster Attributes of a Feature Class Feature class attribute Stored in the geodatabase

10 Raster Datasets Input files Geodatabase Raster Dataset 3712a.tif 3712b.tif UC2004.IMAGE_ c.tif 3712d.tif Single geodatabase raster dataset created from multiple data sources Single row table in the geodatabase Overlapping pixels have options (overwrite, ignore, blend, min, max) Use for DEMs, most base-mapping, and where fast viewing at any scale is important

11 Raster Catalogs Collection of Raster Datasets stored in one table Designed for Overlapping data (time series, stereo-pair photos, scanned maps) Situations where storing each dataset independently is important When you want to organize similar data Use entire catalog, or a subset ArcSDE Raster column UC2004.AERIAL_CATALOG IMAGE NAME DATE 1 WC_ /26/98 2 WC_ /05/01 3 WC_ /04/01 4 WC_ /26/02

12 Raster Attributes Geodatabase raster data is associated with geodatabase features

13 Personal Geodatabases Raster Data in the personal geodatabase Pixels are not stored inside Access Raster Data is stored as referenced files Managed Raster Data is copied as IMGs to a special folder **Note: Raster Datasets are always managed Unmanaged Raster Data is simply referenced by a pathname **Note: Raster Catalogs and attributes can be managed or unmanaged Demonstration

14 Enterprise GDB, Personal GDB, or Raster Files Enterprise GDB Our best multi-user solution Large data volumes (terabytes) Multi-platform / multi-database Personal GDB Good for prototyping and small implementations Limited number of connections Small quantity of data (gigabytes) Windows only

15 Storage / Hardware Two local servers (here at UC2004) 100 megabit network DELL PC Dual 933 MHz processors 1GB of RAM 1TB normal disk storage IBM PC Dual 3 GHz processors 4GB of RAM connected to an INLINE storage device 2 disk arrays, 1.6TB each

16 Raster Data Model Case Studies Ortho-rectified Imagery Global Raster Raster Elevation Scanned Map Time Series Raster Attributes

17 Ortho-rectified Imagery Topography Data Model Cartography Administrative areas Parcels Uses, rights/interests, and ownership Legal description Corners and boundaries Survey control Vectors (roads, rail, buildings) Ortho-rectified Imagery Very accurate imagery

18 Ortho-rectified Imagery Hydrography Data Model Streams Hydrographic points Drainage areas Vector water features Channels Surface Terrain/Elevation Hydro response Ortho-rectified Imagery

19 Ortho-rectified Imagery Data Model (Example Implementation) Texas Natural Resources Information System (TNRIS) Responsible for managing data for all of Texas 1 meter Color-Infrared Digital Ortho-rectified Quarter Quads (DOQQs) Used a Raster Catalog ~ 440 GB of raw data 3000 of provided to ESRI Demonstration

20 TNRIS Ortho-imagery Raster Data Model Purpose: Base-Mapping Update schedule: Irregularly scheduled updates Has portions of the state irregularly updated each year by the counties of Texas (** Not part of the decision process anymore in 9.0 **)

21 TNRIS Ortho-imagery Raster Data Model Scale of Use: Seems a bit too conservative for monitors, OK for printing (** assumes user will not use lossy compression **) (Table courtesy of EMERGE)

22 Orthorectified Imagery Scale of Use Considering Lossy Compression M A P S C A L E 1 : : : : : : : more loss JPEG Compression less loss

23 TNRIS Ortho-imagery Raster Data Model Database Design: Enterprise Geodatabase Size of the data Personal would not have scaled enough Raster Catalog To preserve their base-mapping unit To preserve overlapping corners Lossy compression (JPEG) Base-mapping (not for analysis) Cubic Convolution pyramid resampling Continuous data

24 TNRIS Ortho-imagery Raster Data Model Data Properties: Source Data: 440 GB total raster input data 3000 TIFF files (150 MB each) Enterprise Geodatabase Data: 135 GB in the Geodatabase Raster Catalog ArcGIS Layer: Use scale dependency to view Primarily 10 to 20 rasters at a time Up to 1 : 3780 (1:1 screen to pixel in the monitor) Printing would be different, inline with industry recommendations

25 L.A. County Ortho-imagery Data Model Very similar to TNRIS Base-mapping Lossy compression (JPEG) Differences 0.6 meter resolution Natural color Bilinear Interpolation pyramids Required fast display at any scale Raster Dataset Demonstration

26 Raster Data Model Case Studies Ortho-rectified Imagery Global Raster Raster Elevation Scanned Map Time Series Raster Attributes

27 Global Raster (Example Implementation) EarthSat s SNC of the world Contains all land masses 15m resolution Approximately 4.5 TB of data ~3600 TIFF files (all >1GB) Used 9 independent Raster Datasets Mosaicked a portion of their files into each Demonstration

28 Global Raster Very similar to TNRIS Base-mapping Lossy compression (JPEG) 1 meter resolution (Natural Color) Cubic Convolution pyramids Required fast display at any scale Also required fast loading period 9 Raster Datasets 9 clients loading data into 1 server < 1 week to load into the geodatabase

29 Raster Data Model Case Studies Ortho-rectified Imagery Global Raster Raster Elevation Scanned Map Time Series Raster Attributes

30 Raster Elevation Data Hydrography Data Model Streams Hydrographic points Drainage areas Vector water features Channels Surface Terrain/Elevation Hydro response Orthorectified Imagery

31 Raster Elevation Data Model (Example Implementation) Southern California Association of Governments (SCAG) 5 meter raster elevation dataset L.A. County provided by Emerge/Intermap Demo

32 Raster Elevation Data Model Purpose: Derive additional datasets (analysis) Update Schedule: Very infrequent updates (** Not part of the decision process anymore in 9.0**)

33 Raster Elevation Data Model (Table courtesy of USGS) Scales of Use: Because these are interpolated values, these Scales set by the USGS are valid thresholds. Based on this information, we can infer that 5m data is valid for mapping up to about 1:4000.

34 Raster Elevation Data Model Database Design: Raster Dataset Mosaic together Overlap is the same surface, so it does not need to be preserved Lossless compression (LZ77) Data will be derived (analysis) Cubic Convolution Pyramid resampling Continuous data

35 Raster Elevation Data Model Data Properties: Source Data 2.25 GB, 2 GRIDs 5 meter Intermap DSM (surface) RADAR product Geodatabase Data One large raster of 3 GB Lossless LZ77 ArcGIS Layer Use at any scale (up to 1:4000) Derive other layers as needed (slope, aspect)

36 Raster Data Model Case Studies Ortho-rectified Imagery Global Raster Raster Elevation Scanned Map Time Series Raster Attributes

37 Scanned Maps Topography Data Model Cartography Administrative areas Parcels Uses, rights/interests, and ownership Legal description Corners and boundaries Survey control Vectors roads, rail, buildings Ortho-rectified Imagery Scanned topographic maps

38 Scanned Map (Example Implementation) National Geographic Society TOPO! data Scanned USGS Topographic Maps 7.5 degree extents (1 : series) Southern California Demonstration

39 Scanned Map Raster Data Model Purpose: Base-mapping Update Schedule: Infrequent updates (** Not part of the decision process anymore in 9.0**)

40 Scanned Map Raster Data Model Scales of Use: Map Scale 1:24,000 = 7.5 min (4 per 500dpi) 1:63,360 = 15 min (10.5 per 500dpi) 1:100,000 = 30 x 60 min (16.7 per 500dpi) 1:250,000 = 30 x 60 min (41.7 per 500dpi) Scanned map: useable at the scale it was made Have an inherent scale of use DO NOT get increasingly better at higher scan rates

41 Scanned Map Raster Data Model Database Design: Raster Dataset Stored as RGB Inputs had unmatching colormaps Converted Single-band with colormap to multi-band RGB (** Not necessary anymore in 9.0**) Lossy compression JPEG 50 Not used for analysis Bilinear Interpolation pyramid resampling Produced the best looking output for display

42 Scanned Map Raster Data Model Data Properties: Source Data 1000 s TPQs converted to 1000 s TIFFs 1 : Topographic Maps (scanned by NGS/Topo!) ~ 3 GB of data Geodatabase Data Raster Dataset ~ 10 GB (grows to this size because of RGB storage) ArcGIS Layer Use at 1:24000

43 Raster Data Model Case Studies Ortho-rectified Imagery Global Raster Raster Elevation Scanned Map Time Series Raster Attributes

44 Raster Time Series

45 Raster Time Series (Example Implementation) Hurricane Data Hurricane Mitch (late Oct early Nov, 1998) Shows time slices of hurricane path USGS data AVHRR NDVI data for continent of Africa Every 10 days averaged over the last 21 years Demonstration

46 Time Series Raster Data Model Purpose: Viewing of same extent at different times Update Schedule: Always being updated Just add new data to a raster catalog (** Not part of the decision process anymore in 9.0**)

47 Time Series Raster Data Model Scale of Use: Determined by the data In our cases Regional / Continental / Global extents and applications

48 Time Series Raster Data Model Database Design: Raster Catalog multiple layers, same extent Lossless compression LZ77 Used for visualization, but could be used for analysis Resampling Cubic Convolution

49 Time Series Raster Data Model Data Properties: Source Data 30 input TIFFs for Mitch >700 BILs for Africa Geodatabase Data Raster Catalog Needs to be a Raster Catalog Same extent, with multiple captures in time ArcGIS Layer Use 9.0 built-in time-series renderer 8.3 custom code on developer samples Regional / Continental / Global scales

50 Raster Data Model Case Studies Ortho-rectified Imagery Global Raster Raster Elevation Scanned Map Time Series Raster Attributes

51 Rasters as Attributes of Feature Classes Geodatabase features can store Raster Datasets One or more raster datasets associated with each feature Raster data is stored with the vector data One central location Demonstration

52 Overall Decisions / Considerations Before loading, consider these issues Compression Purpose of Raster data Base map v. analysis Highest quality v. Disk space Will you serve/sell the raster data? Input raster format Heterogeneous v. Homogeneous Cell size, theme, type, colormap, etc Pyramid resampling Discrete v. continuous Preserve overlapping data?

53 Conclusions Raster Data Models are simple on their own part of other data models easy to implement and use Follow these best-practices guidelines Prototype your solution first

54 Evaluations Please fill them out your comments help ESRI become better at meeting your needs each year THANK-YOU!

55 End of Presentation Thank-you for attending! Open to Questions

56