Esri International User Conference San Diego, California Technical Workshops July 25, 2012 Managing Imagery and Raster Data Using Mosaic Datasets Hong Xu, Prashant Mangtani
Presentation Overview Introduction to mosaic dataset Building mosaic datasets Design mosaic datasets for your organization Maintain and update Migrate to mosaic datasets Q&A
Challenges of Managing Imagery Many image sources - Aerial photographs - Satellite imagery from many sensors - DEM and scanned maps - Analytical data High resolution and large volume Need a data model - Store efficiently - Search easily - Process fast - Be able to accessible to multiple clients
What is a Mosaic Dataset A geodatabase model designed for managing collections of images - Catalog, process, search, display, and update Stored as tables in geodatabase - Footprint table references source images - Boundary, raster type - Seamlines, color correction etc Displayed as a composite layer in Map - Boundary - Footprint - Image Mosaic Dataset
Advantages of Mosaic Datasets Dynamic mosaicking - Seamless display at all levels - Overviews to speed up display Processing images on-the-fly - Using raster function - Reduce process time Manage large image collections Fast to create Support images of multiple sensor platforms Streamline update and maintain quality
Use Mosaic Dataset as a Catalog Selection and query - Add selected rasters to Map - Lock the selected rasters View raster and metadata Time aware
Use Mosaic Dataset as an Image Is a raster dataset Closest to the center Provide a seamless mosaic on the fly based on the mosaic rule - Mosaic method to sort the rasters - Closest to center - By attribute - Closest to nadir - North west (default) - Seamline - Mosaic operator to resolve the overlaps - First/Min/Max/Mean/Blend By attribute: cloud cover Export or clip to a raster dataset Use as an input to geoprocessing tool
Share Mosaic Dataset as an Image Service Provide mosaic dataset functionality to web users Access as a catalog - Select/download selected images - Time aware Access as a seamless mosaic REST, Web API, WCS, WMS Require image extension license Session: Publishing image services in ArcGIS (Room 28D) - Thursday 08:30 AM - 09:45 AM - Thursday 03:15 PM - 04:30 PM
Demo: Mosaic dataset basics
Building Mosaic Datasets Cataloging and searching properties Make mosaic dataset look good - seamless mosaic
Basic Workflow Create a mosaic dataset Add rasters - Build footprint - Build boundary Build overviews (optional) Mosaic dataset toolset Automate with model and python
Creating a Mosaic Dataset Create Mosaic Dataset tool - Create footprint table schema OID Shape Raster Name MinPS MaxPS LowPS HighPS Category 1 Polygon <Raster> P01.met 0 90 10 30 Primary Footprint table internal raster catalog with more fields Parameters to set - Define spatial reference - Web Mercator or spatial reference used by your organization - Number of bands and pixel types - Product definition - Used for automatic band matching - Examples: RGB, RGBI, Landset, etc
Adding Rasters Add Rasters to Mosaic Dataset tool - Populate field values OID Shape Raster Name MinPS MaxPS LowPS HighPS Category 1 Polygon <Raster> P01.met 0 90 10 30 Primary 2 Polygon <Raster> P02.met 0 90 10 30 Primary Raster value contains a raster function (process definition) - References images on disk - Process on the fly - Contains metadata, statistics, thumbnails A raster type defines what to add - Understand sensor metadata - Construct the raster function - Add additional fields - Sensor, CloudCover, etc.
Supported Raster Types Define data types can be managed by mosaic dataset Raster dataset - TIFF, Mrsid, JP2000, NITF, etc. Satellite images - Landsat 7 ETM+, Landsat 4-5 TM, Landsat 1-5 MSS - QuickBird, IKONOS, GeoEye-1, WorldView I and II - RapidEye, SPOT, Radarsat 2, FORMOSAT-2, KOMPSAT-2 Aerial images - Match-AT, ISAT, Applanix
Supported Raster Types (Continue) Define data types can be managed by mosaic dataset Lidar and Terrain - LAS, LAS Dataset, and Terrain Military Formats - CIB, CADRG, DTED, HRE Services - Image service, cache map service, WCS, WMS Table - Add existing mosaic datasets by copying all the rows
Tips for Adding Rasters Use DEM orthorectifying sensor data when possible Use Gram-Schmidt pansharpen method Statistics - Do not calculate for processed data - Calculate for sensor data Build pyramids Adding from folder when possible - Required by synchronizing a mosaic dataset Define datum transform if needed (GP environment) Handle invalid spatial reference - Define for missing data - Overwrite wrong spatial reference
Calculate Mosaic Dataset Item Visibility Using Calculate Cell Size Ranges tool OID Shape Raster Name MinPS MaxPS LowPS HighPS Category 1 Polygon <Raster> P01.met 0 90 10 30 Primary 2 Polygon <Raster> P02.met 0 90 10 30 Primary 5 Polygon <Raster> filename1.tif 90 270 90 90 Overviews Mosaic dataset contains rasters of different resolutions MinPS and MaxPS define the visibility of these rasters Calculated based on sources and overlaps - Default cell size range factor is 10 Can be set using field calculator 30m visible range 30-300 5m visible range 0-50 5-30 1m visible range 0-5
Demo: Mosaic dataset cataloging and search
Topics for Building Mosaic Datasets Cataloging and searching Make mosaic dataset look good seamless mosaic
Refine Processing Raster Functions Edit raster function for mosaic items and mosaic dataset Support many functions Edit one item or the mosaic dataset - using raster function editor Edit multiple items - using raster function batch editor The new Edit Raster Function tool - For editing item (s) or mosaic dataset
Handling the Image Background Use Build Footprints and Set NoData tools Build Footprint tool - By radiometry - By geometry Edit using Editor Import Mosaic Dataset Geometry tool Use footprint Define NoData GP tool - Based on a value - Based on a range Use NoData
Build Boundary Define the boundary of the mosaic dataset - Pixels outside the boundary will be clipped Build using Build Boundary tool - Calculated based on footprints Can be modified using Editor Import Mosaic Dataset Geometry tool
Statistics in Mosaic Datasets Use Build Pyramids and Statistics tool Statistics in multiple places - Source raster dataset - Mosaic dataset item (function raster) - Mosaic dataset Source raster Function raster Mosaic dataset Pyramids exist in source dataset Calculate Statistics tool - for mosaic dataset statistics Build Pyramids and Statistics Tool - For statistics of items and source raster dataset
Color Correcting Mosaic Dataset Use Color Balance Mosaic Dataset tool or Color Correction window Based on a calculated color surface - Single color - Color grid - First, second, third polynomial Based on an existing target raster Support excluded area Before After
Building Seamlines Use Build Seamlines tool Used in mosaiking rasters in a mosaic dataset Stored in the seamline table Used by Seamline mosaic method - Blend along the seamlines with a default width Generate seamlines automatically - Radiometry - Geometry - Etc. Edit and import seamlines 1 2
Building Overviews Using Build Overviews tool Cell size=100 OID Shape Raster Name MinPS MaxPS LowPS HighPS Category 1 Polygon <Raster> P01.met 0 90 10 30 Primary 2 Polygon <Raster> P02.met 0 90 10 30 Primary 3 Polygon <Raster> filename1.tif 90 900 90 90 Overviews 4 Polygon <Raster> filename2.tif 90 900 90 90 Overviews Resampled rasters referenced in the footprint table Calculated based on default - TIFF with JPEG compression - Size of 5120x5120 and factor of 3 - Use Define Overview tool to refine the parameters Used for fast display at all levels Add a raster dataset or a service as an overview
Analyzing Mosaic Datasets Using Analyze Mosaic Dataset tool Recommended for publishing as image services Check raster Check footprint geometry Check function chain Check broken path Performance etc.
Updating and Synchronizing Mosaic Dataset Using Synchronize Mosaic Dataset GP tool Data must be added from folder mosaic datasets Update existing items - Any change in source images - Any change in rasters - Option to maintain the existing fields Add new rasters Remove items of broken source
Moving Mosaic Dataset Move all - Copy the FGDB where the mosaic dataset resides - Copy the source and overview images - Use Repair dialog to repair the paths - Or Repair Mosaic Dataset Paths GP tool Extract a portion - Use Distributed Geodatabase toolbar - Create a new folder - Create a target File geodatabase - Run Extract - Copy/move the whole folder
Demo: Make mosaic look good
Design mosaic datasets within your organization
Source and Master Mosaic Datasets Source mosaic dataset - Create from adding source images Master mosaic dataset - Create from source mosaic datasets - Table type by copying rows - Raster Dataset type by adding the whole mosaic dataset Support update mechanism - Synchronize source mosaic if source images updated - Synchronize master mosaic if source mosaic is updated
Reference Mosaic Dataset References an external mosaic dataset or raster catalog Created by specifying - Definition query - Area of interest Boundary = My County Where Sensor = Landsat and Cloud <10% Add NDVI processing It read-only (except boundary) Provides multiple views of the source mosaic dataset Prevents from editing the source mosaic dataset Updates when source mosaic is changed
Server Raster Function (new in 10.1) For publishing image services A function template register with image service - Support client request - Reduced number of services need to be published Created using Function Template Editor - A custom button - Or saved from a raster function Defined when publishing an image service
Mosaic Dataset Design Consideration Create mosaic datasets with data of similar type Elevation Ortho images of same date QuickBird, IKONOS Landsat 5 or 7 Use master mosaic datasets if needed Create referenced mosaic datasets Use server raster function to reduce the number of services Mosaic Dataset to manage data Reference Mosaic Dataset to publish
Demo: Build reference mosaic datasets using raster functions and server raster functions
Migrating to Mosaic Datasets From image service definition (.ISDef) - Create a mosaic dataset - Add using Image Service Definition type From raster process definition - Add using Raster Process Definition type Image service Definitions.\Amberg.ISDef ImageService.ISDef Footprint.dbf Boundary.dbf Seamline.dbf.\RPDefs\rasteris.RPDefs Mosaic dataset Mosaic dataset Footprint feature class Boundary feature class Seamline feature class Function raster datasets
Migrating to Mosaic Datasets (Continue) From a raster catalog - Create a mosaic dataset - Use Table type to add - Raster datasets are re-added as function raster datasets Unmanaged Managed - Find the source data and re-create the mosaic dataset
What is New for Mosaic Dataset in 10.1 Support more raster types - LAS, LASDataset, Terrain, Radarsat2, Kompsat, Formosat, HRE, WMS Many New tools - Analyze Mosaic Dataset - Edit Raster Function - Set Mosaic Dataset Properties - Alter Mosaic Dataset Schema New raster functions - Remap, Band Arithmetic, Attribute Table, Apparent Reflectance, Tasseled Cap Enhancements - Automatic seamline generation - Calculate footprint by geometry - Gram-Schmidt pansharpen method - Support classify and Unique Value renderers - Automatic band matching using production definition
Summary Mosaic dataset is an effective model to manage large image collections Workflow of building mosaic datasets Various tips of building good a looking mosaic dataset Using master mosaic dataset, reference mosaic dataset to manage data in your organization Migrate mosaic datasets from image service definitions
Technical workshops: Additional Information Introduction to Imagery and Raster Data in ArcGIS (Wed:8:30, Fri:9:00) Publishing Image Services in ArcGIS (Thur:8:30 and 3:15) Advanced Image Management using the Mosaic Dataset (Wed: 3:15) Additional resources: Imagery section at resources.arcgis.com Image Blog http://blogs.esri.com/dev/blogs/imagery/default.aspx Fill evaluations online www.esri.com/ucsessionsurveys First Offering ID: 880 Second Offering ID: 990
Questions?