Global Assessment of the New ASTER Global Digital Elevation Model

Transcription

1 Global Assessment of the New ASTER Global Digital Elevation Model James A. Slater, Barry Heady, George Kroenung, William Curtis, Jeffrey Haase, Daryl Hoegemann, Casey Shockley, and Kevin Tracy Abstract In 2009, the US National Aeronautics and Space Administration and Japan s Ministry of Economy, Trade and Industry released a new global digital elevation model (GDEM) derived from reprocessed Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) data from the Terra satellite. An initial, empirical assessment of the accuracy and characteristics of the GDEM was carried out using a globally-distributed sample dataset. Statistical and visual analyses of the GDEM, using known reference DEMs and ground control points (GCPs) for comparison, revealed a systematic bias in the ASTER GDEM elevations, higher average noise levels, and a lower effective ground resolution compared to the reference data, as well as numerous topographic artifacts and anomalies. The GDEM appears to meet its accuracy specifications when compared to the reference DEMs, but compared to the GCPs, some of the areas are questionable. In areas that lack good quality terrain data, the GDEM may prove useful, but must be used with care. Introduction Japan s Ministry of Economy, Trade and Industry (METI) in collaboration with the US National Aeronautics and Space Administration (NASA) sponsored the development of a new global digital topographic model derived from multispectral imaging data collected with the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) onboard the Terra satellite (Jet Propulsion Laboratory, 2009; Yamaguchi et al., 1998). Almost 10 years of data acquired from 1999 to 2008 were reprocessed with new software by Sensor Information Laboratory Corporation (SILC) in Japan. A preliminary version of the new global digital elevation model (GDEM) was released to a number of organizations worldwide for evaluation. The evaluation effort was coordinated by the U.S. Geological Survey (USGS) Earth Resources Observation and Science (EROS) Data Center. Selected results James A. Slater is with the National Geospatial-Intelligence Agency, Basic and Applied Research Office, Sunrise Valley Drive, Reston, VA (cjimblues@gmail.com). Barry Heady is with Integrity Applications Inc., St. Louis, MO, and formerly at the National Geospatial-Intelligence Agency, Geoint Integration Office, Arnold, MO George Kroenung, William Curtis, Jeffrey Haase, Daryl Hoegemann, and Kevin Tracy are with the National Geospatial-Intelligence Agency, Source Directorate, Arnold, MO Casey Shockley is with the National Geospatial-Intelligence Agency, Analysis & Production Directorate, Arnold, MO of that preliminary evaluation are given in a summary report available from the USGS (ASTER Validation Team, 2009). Despite some issues raised by the preliminary evaluation (see below), the new ASTER GDEM was released to the public in June 2009 as a research grade product (Abrams et al., 2010). The purpose of the study reported here was to make an initial, empirical assessment of the quality of the new topographic model for a geographically-representative, global data sample. The intent was to provide a quantitative and qualitative overview and characterization of the ASTER GDEM as delivered and to identify areas needing further consideration or more in-depth analysis. Accuracy and elevation data characteristics were evaluated by statistical and graphical comparisons of the GDEM with known reference terrain elevation data and ground control points. Overall topographic quality was assessed by visual inspection of the GDEM for terrain artifacts and anomalies that represent unrealistic or erroneous depictions of the true terrain. A dataset consisting of 20 Sites in 16 countries was used for the evaluation (Figure 1). Except for one Site in Alaska, none of these Sites is in the United States, since the USGS devoted its resources to an analysis of the new GDEM over the US (ASTER Validation Team, 2009; Abrams et al., 2010). A major interest in this evaluation was to see how the ASTER GDEM compared to Shuttle Radar Topography Mission (SRTM) terrain elevation data (Farr et al., 2007; Slater et al., 2006) in areas below 60 N where their coverage overlaps, and how the GDEM performed in areas above 60 N where global elevation data are sparse and of relatively poor quality. The following sections of this paper discuss the ASTER GDEM data used for the evaluation, the reference data, the evaluation criteria and methodology, the results of the statistical and visual analyses, and a discussion of the results and conclusions drawn from this analysis. ASTER GDEM Data ASTER collects both nadir-pointing and backward-pointing along-track images in the visible and near-infrared band (VNIR Band 3) from 0.76 to 0.86 microns. The resulting stereo pairs are the data source for the new global digital elevation model (GDEM) and cover a geographic area from 83 S to 83 N. All the historical Level 1-A ASTER imagery data were compiled and processed by SILC using new software designed for this project. The accuracy objective for the ASTER GDEM was Photogrammetric Engineering & Remote Sensing Vol. 77, No. 4, April 2011, pp /11/ /$3.00/ American Society for Photogrammetry and Remote Sensing PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING April

2 Figure 1. ASTER GDEM evaluation sites (Gray = SRTM reference data, Black = non-srtm reference data). specified as 20 m vertical error and 30 m horizontal error at a 95% confidence level (ASTER Validation Team, 2009). The data were screened for the presence of clouds, poor stereopair correlation, and outliers based on comparison with a reference DEM or the average of the stacked scenes (repeated observations) for each pixel (Abrams et al., 2010). The GDEM data were provided in 1 1 tiles at a one arcsec (30 m) post spacing, such that each tile contains 3,601 3,601 posts. The ASTER GDEM is referenced to the World Geodetic System 1984 (WGS84) and the elevations are computed with respect to the WGS84 Earth Gravitational Model 1996 (EGM96) geoid (NIMA, 1997). The elevation value at each post is the unweighted average of the accepted scenes in the stack of repeated observations of that pixel. Ocean and sea elevations were set to 0 based on low observed radiance (DN) values and a sea water database derived from a global 30 arcsec elevation data set (GTOPO30) (Gesch et al., 1999). No specific editing was done to identify and delineate lakes and rivers (Bailey, G. B., USGS, personal communication, 2008). The GDEM represents a digital surface model (Fisher and Tate, 2006) for which the observed elevations may refer to bare earth, vegetation, or man-made structures. A Quality Assurance file was provided with each tile, giving the actual number of scenes that were averaged to generate the elevation value at each post. The Quality Assurance file also indicates if a post was corrected using a reference DEM or if it contains an interpolated value. Abrams et al. (2010) provide more details on the creation of the ASTER GDEM. Reference Data All the reference data used in this evaluation were from the public and proprietary holdings of the National Geospatial- Intelligence Agency (NGA). Two types of reference data were used: Digital Terrain Elevation Data (DTED ) and ground control points (GCPs). The global availability and known characteristics of these reference data provided a uniform base for this evaluation and facilitated the comparison of results between geographic areas. The 20 sample geographic areas selected comprise cells with varied terrain relief and elevations ranging from sea level to over 6,700 m (Table 1). These areas were chosen for their global distribution and because the two types of reference data were available. DTED and GCP coordinates are referenced to WGS84 and the elevations are defined as heights above the WGS84 EGM96 geoid. Each of the specific reference data sources is described below. Digital Terrain Elevation Data (DTED ) NGA s DTED holdings cover more than 90 percent of the Earth s land surface. DTED are maintained at two levels of product resolution, DTED 1 (3 arcsec) and DTED 2 (1 arcsec). All of the DTED have the same format and basic data content, although some differences may exist in the characteristics of DTED derived from different sources (e.g., radar versus optical satellite). In general, water bodies have been post-processed such that ocean elevations are set to zero, lakes are flattened to a constant value, and doubleline drains (rivers) are stepped down to fit the terrain, in accordance with the DTED specifications (NIMA, 2000). Accuracy is derived on a cell by cell basis and is driven by the source accuracy. Table 1 gives the absolute vertical and absolute horizontal error ranges for the cells in each sample geographic area. For latitudes of 50 or less, the DTED are gridded at 1 arcsec 1 arcsec post-spacing resulting in a 3,601 3,601 array of points; between 50 and 70 degrees, DTED have 1 2 arcsec spacing (3,600 1,801 array of points); for the latitude intervals from 70 to 75 degrees, 75 to 80 degrees and 80 to 90 degrees, DTED have 1 3 arcsec spacing, 1 4 arcsec spacing, and 1 6 arcsec spacing, respectively. For the comparisons with the ASTER GDEM, the DTED for Sites above 50 N were interpolated to produce elevation grids of 3,601 3,601 posts. Below 60 N, most of the DTED used for the DEM comparisons are from the Shuttle Radar Topography Mission. Above 60 N, where no SRTM data are available, the DTED used in the evaluation were derived from electro-optical stereo imagery or from cartographic sources: specifically, Russia (Sites 1, 2, and 3), Canada and Alaska. In a few areas, two independent sources of DTED were available and used as reference data for the GDEM comparisons (see Table 1). SRTM DTED 2 were derived from data acquired with the C-band Interferometric Synthetic Aperture Radar 336 April 2011 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

3 TABLE 1. REFERENCE DEM DATA CHARACTERISTICS Absolute Absolute Approximate Vertical Error Horizontal Error Elevation Range (m) Range (m) Range (m) (90% L.E.) (c) (90% C.E.) (c) Geographic No. of SW Corner NE Corner Area Cells Coordinates Coordinates Min Max Source Min Max Min Max Afghanistan-1 8 N32 E065 N34 E SRTM DTED IFSAR DEM Afghanistan-2 9 N29 E064 N32 E SRTM DTED Photo. DTED Argentina 25 S41 W071 S36 W SRTM DTED Australia 25 S16 E130 S11 E SRTM DTED Bolivia 4 S20 W068 S18 W SRTM DTED Bosnia 15 N42 E017 N45 E SRTM DTED Photo. DTED Canada 6 N72 W084 N73W Photo. DTED China-1 25 N31 E084 N36 E SRTM DTED China-2 25 N36 E108 N41 E SRTM DTED Iraq 16 N33 E042 N37 E SRTM DTED Photo. DTED Kazakhstan 25 N49 E072 N54 E SRTM DTED Korea 12 N37 E125 N41 E SRTM DTED Photo. DTED Libya 4 N23 E021 N25 E SRTM DTED Nigeria 25 N09 E009 N14 E SRTM DTED Philippines 2 N07 E125 N09 E SRTM DTED Russia-1 2 N75 E064 N77 E Photo. DTED Russia-2 1 N79 E096 N80 E Photo. DTED Russia-3 20 N63 E042 N67 E Carto. DTED Thailand 10 N14 E099 N19 E SRTM DTED U.S.-Alaska 25 N60 W152 N65 W Carto. DTED Source: SRTM DTED 2 or other DTED. Photogrammetrically-derived DTED ; Cartographically-derived DTED ; Interferometric Synthetic Aperture Radar (IFSAR) DEM. (c) L.E. is linear error; C.E. is circular error. (IFSAR) system flown on the Shuttle and represent the radar reflective surface, which is typically similar to optical reflective surface data, except that the radar has been found to penetrate the foliage to some extent, depending on a number of factors such as the type and density of vegetation (Carabajal and Harding, 2006; Hofton et al., 2006; Kellndorfer et al., 2004). This analysis used a new NGA version of these data where the original voids have been filled as much as possible using other existing DEM sources (Grohman et al., 2006) and some interpolation in uniformly flat areas. The data have also undergone additional editing to remove spikes and wells exceeding a 60-meter threshold and anomalous elevation areas caused by phase unwrapping errors. This dataset was chosen as the primary comparison source for the ASTER evaluation where available. Vertical accuracy for SRTM DTED is reported for each cell, but horizontal accuracy is available only on a continental basis. No attempt was made to reduce the elevations to bare earth. The nominal absolute accuracy specifications (NIMA, 2000) for the SRTM DTED 2 were 16 m vertical (90 percent linear error) and 20 m horizontal (90 percent circular error), but as shown in Table 1, the actual errors were significantly better (Rodriguez et al., 2006; Farr et al., 2007). Although, in general, the horizontal accuracy of the 30-meter post-spaced SRTM DTED 2 meets the 20-meter specification, the effective ground resolution is closer to 60 meters (Guth, 2006; Rodriguez et al., 2006; Smith and Sandwell, 2003). Photogrammetrically-derived DTED 2 are generated from electro-optical stereo-pair imagery from proprietary government sources. These data are typically edited to a bare earth surface, except in those areas where the foliage is too dense to see the underlying surface. Water bodies are edited as noted earlier, and spikes and wells are removed using a 10 to 15 meter threshold due to the bare earth nature of the product. Horizontal and vertical accuracies for photogrammetrically-derived DTED are estimated and reported independently for each cell (see Table 1). SPOT5 HRS DTED 2 were derived from SPOT high-resolution stereo-optical imagery by SPOT IMAGE (Bouillon et al., 2006; Spot Image, 2005). These data have been edited very similarly to the photogrammetrically-derived DTED 2 and represent a digital surface model (reflective surface, not bare earth). Horizontal and vertical accuracies for SPOT 5 DTED are estimated and reported independently for each cell. These data were used for GCP comparison purposes only in the Libya sample area. Cartographic source DTED 2 were used in areas above 60 N where other sources were not available. The Alaska DTED 2 data were derived from USGS 1: scale maps and are essentially the same as the USGS National Elevation Dataset (NED), which is distributed at a 2 arcsec (roughly 60 m) grid spacing for Alaska (Gesch et al., 2002). They have the same water finishing characteristics as the other DTED datasets, but are recognized to have horizontal errors larger than those cited in the DTED specification. The Russia Site 3 DTED 2 data were compiled from 1: scale native map source and finished to standard DTED product specifications, including water bodies. As noted in the introduction, the quality of existing elevation data in many areas above 60 N is relatively poor and the DTED used here are no exception. Although this limits the conclusions that can be drawn, it still provides useful insights into the relative quality of the ASTER GDEM. PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING April

4 Commercial IFSAR Data Data collected from a commercial, airborne X-band IFSAR system were used as a reference DEM in one area (Afghanistan Site 1). The DEM was produced at a 5-meter spatial resolution and represents a digital (reflective) surface model. Horizontal positional accuracy is specified as 2.5 m RMSE or better on slopes of 20 degrees or less. Upper bounds on the vertical accuracy are given as 3.0 m RMSE on slopes of 20 degrees or less, 5.0 m RMSE for 20 to 30 degree slopes, and 9.0 m RMSE for slopes over 30 degrees. See for example Mercer (2004). Ground Control Points (GCPs) GCPs are derived photogrammetrically during the triangulation processing of stereo optical satellite imagery at NGA and represent discrete 3-dimensional reference points extracted from the imagery and reduced to bare earth. This is the same imagery used to produce the photogrammetricallyderived DTED, but in this case all available overlapping images are used to generate the control points. Over 1.5 million points are in NGA s proprietary global GCP database. As new images are collected, the GCPs are regenerated using the additional data. There are typically approximately 200 of these points per 1 1 area, although that number can vary based on the terrain type and geographical location. These points generally do not coincide directly with the location of DEM posts. The accuracy of these photogrammetricallyderived control points is better than 10 m vertically and 10 m horizontally at a 90 percent confidence level. Evaluation Criteria and Methodology The objective of this work was to characterize, quantitatively and qualitatively, the new ASTER GDEM on a global, rather than a local or regional, basis. The 20 geographic areas used in the analysis were selected for global distribution, a variety of topographic relief, and (c) the availability of known reference DEMs and GCPs. They were intended to give a representative snapshot of the GDEM and not intended to be a comprehensive global statistical sample. No editing was done on the ASTER GDEM data received from SILC. Since the ASTER GDEM is referenced to WGS84 and its associated EGM96 geoid, no coordinate conversions were applied to any of the data used in the comparisons. Prior to the analysis, water, alternate source data used to fill voids, and interpolated regions were masked out in the GDEM and in the reference DEMs to avoid any undue influence from these areas that might skew the data comparisons and resulting statistics. Both statistical comparisons and visual inspection were employed in the GDEM evaluation. The objective of the statistical analysis was to ascertain how the terrain depicted by the ASTER GDEM compared, on average, to reference data whose characteristics were known. The statistics presented here are the summary values for all the aggregated posts in each entire sample area. Thus, in some areas the statistics represent all the posts in one or two cells, while in others they represent all the posts in 25 cells (see Table 1). ASTER GDEM-to-Reference DEM and ASTER GDEM-to-GCP difference statistics and graphics were compiled for each sample area. For the DEM comparisons, elevation differences were computed for every GDEM post and the corresponding reference DEM post at the same grid point. For the GCP comparisons, the gridded DEM points surrounding the control point were bi-linearly interpolated to estimate the elevation value of each DEM at the control point location. Cumulative frequency distributions were generated for the absolute values of the DEM-GCP elevation differences in each geographic area to check the normality of the elevation distributions in order to estimate the 90 percent linear error (LE) used in the evaluation. In cases where the distributions were approximately normal, the 90 percent error figure was computed by multiplying the calculated standard deviation of the differences by In the other areas where the distributions were non-normal, the elevation difference at the 90 th percentile was extracted directly from the data. See, for example, Hohle and Hohle (2009); Daniel and Tennant (2001). In order to detect any systematic behavior of the ASTER GDEM as a function of the elevation in a given geographic area, the ASTER GDEM-GCP differences and, for comparison, the corresponding Reference DEM-GCP differences were plotted as a function of the reference elevation. These graphics also provide insight into the relative variability of the ASTER GDEM versus the reference DEMs. Noting the statistical independence of the GCPs and the ASTER GDEM, it is possible to estimate the variance of the ASTER GDEM (Hohle and Hohle, 2009) and thus infer whether or not the GDEM meets its vertical accuracy specifications. This was done by applying the relationship: Variance (ASTER-GCP) Variance (ASTER) Variance (GCP). Ideally, the standard deviation of the GCPs should be at least a factor of three better than the ASTER error specification. The proprietary GCP elevations available for this analysis, however, have a maximum 90 percent linear error of 10 meters specified for the whole database, but no individual GCP error estimates. Therefore, the 10 m figure was used in the variance estimates as an upper bound on the GCP error and the results should be viewed as an indicator of the best case error estimates for the ASTER GDEM. Only a cursory examination was made for horizontal registration errors in the GDEM. This was done by computing the RMS elevation difference between the original GDEM and the reference DEM in each geographic area, then systematically shifting one DEM with respect to the other, one post at a time, and re-computing the RMS difference each time. A reduction in the RMS value may indicate a better average fit between the two terrain models, and therefore a possible horizontal offset of the GDEM with respect to the reference DEM. This was done for integer post-spacing increments and provides no information on possible sub-pixel misregistration of the ASTER source imagery. (See, for example, Van Niel et al. (2008)) Since the ultimate accuracy and variability of the ASTER GDEM may be a function of the number of stacked ASTER scene pixels that were averaged to produce the elevation values, it is useful for evaluating the GDEM on a global basis to see how the source data vary from area to area and within specific areas. The Quality Assurance file provides these details. It was used to generate statistical distributions of the number of stacked scenes used in the sample area cells and provides useful insights into the quality of the GDEM. For the visual analysis of the GDEM topography, every sample area was examined using shaded relief plots, elevation-based color map displays, and other tools to look for data artifacts. Visual analysis is essential for discovering artifacts that are frequently hidden or averaged out in the aggregate statistics derived from millions of elevation values (Maune et al., 2001). Extreme random noise, mosaicking artifacts, excessive spikes and wells, and unrealistic terrain depiction discovered through this inspection process, if prevalent in the data, can have a significant impact on the utility of the DEM for certain applications. Evaluation Results ASTER GDEM Comparison with Reference DEMs Table 2 shows the results of comparing the new ASTER GDEM to the reference DEMs for each of the 20 geographic areas 338 April 2011 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

5 TABLE 2. DEM DIFFERENCES FOR ALL GEOGRAPHIC AREAS Mean Elevation Difference (m) ASTER DEM Reference DEM SRTM Other Source Geographic No. of 90% 90% Area Cells Mean L.E. Mean L.E. Afghanistan Afghanistan Argentina Australia Bolivia Bosnia Canada China China Iraq Kazakhstan Korea Libya Nigeria Philippines Russia Russia Russia (c) 15.7 Thailand U.S.A. Alaska (c) 25.1 Commercially produced IFSAR DEM DTED 2. Photogrammetrically-derived stereo optical DTED 2. (c) Cartographic source DTED 2; Alaska based on USGS NED. selected. These areas vary in size from 1 or 2 cells in the case of two Russian Sites and the Philippines to 25 cells for seven of the Sites. It is worth noting again that each of these areas represents a set of contiguous cells and the statistics for each Site represent the whole contiguous area. In 20 of the 23 ASTER GDEM-to-Reference DEM comparisons shown in the table, the mean elevation difference is negative, i.e., the ASTER GDEM elevations are lower than the reference DEM elevations. The three exceptions are Libya (no difference relative to SRTM DTED ), Korea (no difference with respect to the secondary DTED source, although biased negatively compared to SRTM DTED ), and Afghanistan Site 1 (0.6 m higher than the secondary commercial IFSAR source, although slightly negatively biased relative to SRTM DTED ). Since both the SRTM DTED and the ASTER GDEM elevations refer to the reflective surface, and especially given the foliage penetration of the SRTM observations (Carabajal and Harding, 2006), it is not clear why the ASTER elevations fall below the SRTM elevations. For Russia Sites 1 and 2, only photogrammetricallyderived reference DEMs were available, but the ASTER elevations are lower on average than the reference DEM, as was the case for the SRTM DTED comparisons in other areas. The mean DEM difference for Site 1 is comparable to other Sites (ASTER 4.1 m lower than reference source with a 90 percent LE of 12.4 m). The relatively large vertical errors, in particular in Russia Site 2 (29.4 m), may be attributable to very high erroneous elevation values in the GDEM that were not filtered out in the production process. Minimum and maximum elevations in the three ASTER cells evaluated at these Sites differ dramatically from the values in the reference DTED. For example, in Site 1, the maximum DTED elevation is 990 m compared to the maximum ASTER elevation of 2,131 m; and similarly in Site 2, the maximum ASTER elevation is 2,944 m compared to 706 m for the reference source. The DEM evaluations of Russia Site 3 and Alaska were based on cartographic source DTED. The reference data are not very good; Russia Site 3 DTED have vertical and horizontal uncertainties of 17 m and 27 m (Table 1), respectively, so the comparison statistics (-9.7 m mean elevation bias and 15.7 m vertical error for the DEM difference) reflect this inaccuracy in addition to the error in the ASTER GDEM. The results for Alaska, although not very good relatively speaking, are also partly attributable to the 16 to 33 m vertical error estimated for the cartographic reference DTED. It is worth noting that the original pre-release data had large anomalous areas in Russia Sites 2 and 3 that resulted in vertical errors of 160 to 550 m over these areas. Much of this data has been voided out in the public release version of the ASTER GDEM, thus improving the aggregated statistics; however, these Sites still contain extraneous elevation data as indicated by the kilometer-magnitude of the minimum and maximum differences with the reference DEMs cited above. In conjunction with the DEM-to-DEM analysis, each sample area was assessed for any gross geolocation offsets of the ASTER GDEM with respect to the reference DEMs by systematically shifting one DEM relative to the other, one post at a time, and computing the RMS elevation differences. These failed to show any significant horizontal offsets in the GDEM relative to the reference DEMs. The Earth Remote Sensing Data Center in Tokyo, Japan also looked for geolocation errors and found only sub-pixel (under 30 m) average offsets in seven GDEM cells in Japan (Abrams et al., 2010; see also Van Niel et al., 2008). GDEM-to-DEM comparisons over Australia reported by Hirt et al. (2010) are consistent with the above results. Comparing 3 arcsec resolution SRTM elevations to the ASTER GDEM, they found a mean difference of 7.7 m and an RMS value of 11.7 m, with the ASTER GDEM lower than the SRTM terrain data, and no evidence of horizontal offsets. Ground Control Point Analysis The ASTER GDEM and the reference DEMs were independently compared to the GCPs available in each geographic area. As noted earlier, the gridded DEMs were interpolated to estimate the DEM elevation value that coincided with the GCP s horizontal coordinates. A summary of the mean elevation differences of the DEMs with respect to the GCPs and the 90 percent vertical error estimates is given in Table 3. In every area, the mean ASTER elevations are lower than the control points, ranging from close to zero difference (Australia and Libya) to 11 to16 m differences (Canada, Nigeria, Russia). The Canada (34 m) and Russia Sites (34, 27, and 25 m, respectively) also exhibit the largest variability, consistent with the DEM-to-DEM analysis. These results are in contrast to the SRTM-GCP differences in Table 3, which are mostly positive, in effect placing the SRTM elevations above the GCP elevations (as expected). The magnitude of the ASTER variability is two to three times that of the SRTM variability with respect to the control points. The exceptions are Kazakhstan, where the mean SRTM elevations are lower than the GCPs by about 3 m, and Bolivia, where the results for the ASTER elevations are closer to those of the photogrammetrically-derived DTED than to the SRTM DTED. One possible explanation for the GCP results for Bolivia and the corresponding DEM comparison results (Table 2) may be an (unconfirmed) excessive positive bias (in effect, a systematic error) in the Bolivia SRTM elevation data. This would explain the relatively large (10.9 m) GDEM-to-SRTM DEM difference and the 13.3 m relative difference of the ASTER GDEM versus the SRTM DTED with respect to the GCPs. Note also that there are a relatively small number of control points for Bolivia (79) and that 67 of them are in just one of the four Bolivia cells, although this does not appear to be the cause of the observed results. PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING April

6 TABLE 3. COMPARISON OF ASTER AND REFERENCE DEMS WITH GROUND CONTROL POINTS FOR EVALUATION AREAS Mean Elevation Difference (m) DEM-GCP ASTER SRTM OTHER Geographic No. of No. of 90% 90% 90% Area Cells GCPs Mean L.E. Mean L.E. Mean L.E. Afghanistan Afghanistan Argentina Australia Bolivia (e) (e) Bosnia (e) Canada China China Iraq Kazakhstan (e) Korea Libya (e) (e) 0.2 (c) 8.6 Nigeria (e) Philippines (e) (e) Russia Russia Russia (d) 7.5 Thailand U.S.A. Alaska (d) 23.9 IFSAR DEM DTED 2. Photogrammetrically-derived DTED 2. (c) SPOT 5 DTED 2 (10 m vertical accuracy at 90% L.E. for slopes 20%; 18 m for 20 40% slopes). (d) Cartographic source DTED 2. (e) 90% error taken directly from frequency distribution plot of the absolute value of (DEM-GCP) in cases where the distribution did not appear to be approximately normal. In all other cases, 90% figures are computed by multiplying the calculated standard deviation of the differences by The other areas that stand out in terms of mean differences in Table 3 are Nigeria, Canada, and the three Russia Sites. In the case of Nigeria, the relatively large 11.1-m ASTER GDEM-GCP difference is consistent with the large 13.3 m GDEM- SRTM DEM difference shown in Table 2. Here, based on the 2.4 m SRTM-GCP difference for Nigeria in Table 3, the SRTM elevations are not suspect, so it appears that there may be a problem with the GDEM elevations in the Nigeria sample area. As for the Canada and Russia areas, the GCP comparisons with their respective reference DTED indicate that the GCPs in these areas are consistent with the GCPs in the other sample areas. This implies that the large offsets from the GCPs in the Canada and Russia Sites represent a mean bias in the ASTER GDEM. Results obtained by Hirt et al. (2010) also show a systematic negative bias and higher variability for the ASTER elevations compared to ground control and to the SRTM global DEM. In one comparison using 911 GPS/leveling control points, with accuracies of a few centimeters, spread across Australia, the ASTER GDEM-GPS GCP mean difference was 8.2 m (standard deviation of 10.2 m). In order to detect possible elevation dependencies in the ASTER GDEM within each sample area, DEM-GCP differences were plotted as a function of the reference elevation for every area. Figures 2a through 2l show these graphics for the GDEM and the corresponding reference DEM for six representative geographic areas: Alaska, Argentina, Bosnia, China Site 1, Kazakhstan, and Nigeria. The plots reveal the relative variability (noise) in the GDEM versus the reference DEM as well as the distribution of the control points over the elevation range of the data in each area, discernable from the density of the lines representing the individual control point differences. There is generally a mean negative bias in the ASTER elevations and a positive bias in the reference elevations (except for Kazakhstan); however, the magnitudes of the differences in each geographic area do not appear to be elevation-dependent. It is apparent from these plots that the ASTER GDEM is much noisier than the reference DEMs in all cases except for Alaska, where it seems that the NED source data (Gesch et al., 2002) are inherently of poorer quality than the other reference DEM sources. No data have been edited out of these plots, in order to show the actual GDEM data as delivered; however, the vertical axis scales are set to better display the majority of data, forcing large elevation differences to run off the top and bottom of the plots. Vertical Accuracy Estimates for the ASTER GDEM By taking advantage of the independence of the GCPs from the ASTER data, an estimate of the vertical error in the ASTER GDEM was computed using the statistical relationship: Variance (ASTER) Variance (ASTER-GCP) Variance (GCP). Table 4 shows the results for each sample area using a 10 m upper bound for the GCP error and the values from Table 3 for the ASTER-GCP error. Since the same maximum GCP error is assumed for all areas, it accounts for a constant portion of the ASTER-GCP difference error in all areas. In reality, the GCP errors are considerably better than 10 m in some areas, which would effectively increase the estimated ASTER error in those areas. (The estimated ASTER error of 0.0 m in the Bolivia area was rejected as obviously implausible. Note that Bolivia was 340 April 2011 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

7 _ qxd 3/15/11 3:06 PM Page 341 (c) (d) (e) (f) Figure 2. DEM-GCP differences as a function of elevation: ASTER-GCP elevation differences versus SRTM elevation Alaska, NED-GCP elevation differences versus NED elevation Alaska, (c) ASTER-GCP elevation differences versus SRTM elevation Argentina, (d) SRTM-GCP elevation differences versus SRTM elevation Argentina, (e) ASTER-GCP elevation differences versus SRTM elevation Bosnia, (f) Photogrammetric DTED-GCP elevation differences versus DTED elevation Bosnia. the only area where the SRTM-GCP difference error (15.0 m) was greater than the ASTER-GCP difference error (10.0 m) in Table 3, and that it appeared from the comparisons presented earlier that SRTM data may be poor in this area.) Assuming that this is perhaps a best case scenario for an estimate of the GDEM vertical error, the values in Table 4 can be compared to the ASTER GDEM specification of 16.8 m vertical error at 90 percent confidence (equivalent to a PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING 95 percent value of 20 m). This is at least an indication that twelve of the sample areas (including China-2) meet the specification while the other eight may not. Number of ASTER Scenes Used to Compute GDEM Elevations A file was provided with the GDEM that gives the number of ASTER scenes used to compute the elevation value at each post. The number of scenes varies from one for some posts April

8 _ qxd 3/15/11 3:06 PM Page 342 (g) (h) (i) (j) (k) (l) Figure 2. continued (g) ASTER-GCP elevation differences versus SRTM elevation China Site 1 (h) SRTM-GCP elevation differences versus SRTM elevation China Site 1, (i) ASTER-GCP elevation differences versus SRTM elevation Kazakhstan, (j) SRTM-GCP elevation differences versus SRTM elevation Kazakhstan, (k) ASTER-GCP elevation differences versus SRTM elevation Nigeria (l) SRTM-GCP elevation differences versus SRTM elevation Nigeria. to over 30 in other cases. Plate 1a through 1d shows the frequency distribution of the number of passes (scenes) that were averaged per post for typical cells in the areas that were evaluated. There is a significant difference between cell N63E043 in Russia Site 3 in Plate 1d compared to Afghanistan Site 2 cells N29E064 and N31E065 and the two Bolivia cells in Plate 1a. In the Russia cell, 85 percent of the elevation values are based on only 1 or 2 scenes (see Plate 2), while in the Afghanistan 342 April 2011 and Bolivia cells, 99 percent of the elevation values are based on 10 or more scenes. It can also be observed that there is a noticeable difference in coverage between cells in the same geographic area. See for example Afghanistan Site 2 cell N29E066 versus the other two Afghanistan cells in Plate 1a. Also, compare the two adjacent Korea cells in Plate 1c. There are also regional disparities in the number of passes used. Australia and Russia cells shown in Plate 1b and 1d, respectively, have relatively low numbers of passes per post. PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

9 TABLE 4. ESTIMATED ASTER GDEM ELEVATION ERROR BASED ON GCPS ASTER- GCP GCP ASTER Geographic No. of No. of 90% L.E. 90% L.E. 90% L.E. Area Cells GCPs (m) (m) (m) Afghanistan Afghanistan Argentina Australia Bolivia Bosnia Canada China China Iraq Kazakhstan Korea Libya Nigeria Philippines Russia Russia Russia Thailand U.S.A. Alaska % error taken directly from frequency distribution plot of the absolute value of (DEM-GCP) in cases where the distribution did not appear to be approximately normal. In all other cases, 90% figures are computed by multiplying the calculated standard deviation of the differences by Estimated ASTER elevation error for Bolivia is 0.0 in this approximation, which is obviously implausible, and is therefore omitted from this table. A detailed illustration of the pixel-by-pixel and cell-bycell variations in the ASTER GDEM data is shown in Plate 2. This is a composite graphic of the number of stacked scenes used to compute elevations for each post in all 20 cells comprising Russia Site 3. The voided posts (black) are concentrated in several areas and account for 5.7 percent of the total area. Excluding these voids, 76.4 percent of the remaining elevations are based on three or more scenes, and 23.6 percent represent data from only one or two scenes (red and green). This area is of particular interest since it is not covered by SRTM data. Topographic Artifacts and Anomalies in the ASTER GDEM The visual inspection of the ASTER GDEM in the 20 geographic areas detected a number of common and recurring artifacts and anomalies in all of these areas. Figures 3 through 13 illustrate many of these artifacts in the form of shaded relief images of the ASTER GDEM alongside the reference DEM. The general problems can be grouped into the following categories: poor resolution, noisy data, (c) mosaicking artifacts, (d) poorly fit void-fill data, (e) poor coastline definition and water body identification, (f) landform artifacts, and (g) pits and spikes. It was expected that the 30-meter post spacing of the ASTER GDEM would yield visible resolutions similar to those of the SRTM DTED 2 and SPOT5 HRS DEMs that have the same pixel size. This is not the case. The ASTER GDEM has a lower effective resolution than the comparable reference data as can be seen in Figures 3, 8, and 12. In addition, the ASTER GDEM is considerably noisier than the reference DEMs (Figure 4). An important feature of a terrain model is its ability to depict the topography in a seamless manner. Note that the ASTER GDEM is a merger or mosaic of individual elevation posts, each of which is the average value for all of the useable stacked scenes over that post location. It was observed that many areas in the GDEM have small ridge-like structures forming closed loops or open linear features across the terrain (Figure 6). By comparing the locations of these features with a graphical display of the number of scenes used in a given geographic area (see Plate 2), it was discovered that the ridge-like features correspond to the boundaries between areas with different numbers of scenes used to produce the GDEM. They are in effect mosaicking artifacts and typically vary from 1 to 20 m in height; these are also reported in Abrams et al. (2010). Furthermore, the edges of adjacent one-degree cells do not match seamlessly, creating additional discontinuities in the GDEM. There are numerous instances where void areas in the ASTER GDEM were filled with alternate data sources as in Figure 7, leaving obvious discontinuities at the boundary of the ASTER data and the fill data. Unlike DTED products, the data processing plan for the ASTER GDEM did not include any detailed water body delineation and elevation adjustment (Abrams et al., 2010). As a result, inland water bodies are generally undefined or poorly displayed in the topography (Figure 8). Shorelines were also misrepresented and false islands were created in the areas evaluated (Figure 9). Figures 10 and 11 show several landform features that appear in the ASTER GDEM but do not exist in the reference topography. These include trenches, raised linear features that may correspond to roads, and major land structures over an extended area for which there is no evidence in the reference DEM. Hirt et al. (2010) noticed similar artifacts in the GDEM over Australia. It is possible that some individual land features have changed over time but not with the patterns and to the extent seen in many of the geographic sample areas. One other noteworthy problem with the ASTER GDEM is the presence of many pits and spikes in the data that have not been filtered out. Figure 12 shows a number of pits that occurred along ridges in one area and Figure 13 shows an area with a large number of raised surface features containing pits. Neither the pits nor the raised features appear in the reference SRTM DTED. In addition to these examples, 11 of the 25 cells in Alaska and numerous cells in Kazakhstan have very large spikes and wells (hundreds of meters in elevation) that remain in the ASTER GDEM. Discussion and Conclusions The goal of this analysis was to perform an initial assessment of the ASTER GDEM for a global sample of Sites sufficient to identify general characteristics of the terrain data and any specific issues affecting the use of the new DEM. This was accomplished by comparing the GDEM to existing terrain elevation data that were available worldwide and well characterized. Although the reference data ideally should be a factor of 3 to 10 times better than the anticipated accuracy of the GDEM, they are generally not readily available for large areas on a global basis. Since the ASTER GDEM post-spacing and accuracy specification mirrored the specifications for SRTM DTED 2, and the SRTM data represent the most complete and accurate global terrain coverage at this time, it was natural to use these data as the primary DEM reference source to see how the two global DEMs compare. Note that most of the SRTM elevation data (Table 1) have vertical error estimates (90 percent LE) between 4 and 8 meters, which are considerably better than the comparable 16.8 m GDEM specification. Likewise, the reference GCP accuracies used in this analysis are substantially better than the GDEM specification. In addition, the GDEM, the reference DTED (SRTM and non-srtm) and the GCP elevations refer to PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING April

10 _ qxd 3/15/11 3:06 PM Page 344 (c) (d) Plate 1. Distribution of the number of passes (scenes) used per post in the ASTER GDEM for selected cells in Afghanistan and Bolivia, Australia and Nigeria, (c) China Site 1, Kazakhstan and Korea, and (d) Russia Site 3. the same horizontal and vertical datums, thus avoiding coordinate conversion issues. As such, these reference data do provide a valid basis for assessing whether or not the ASTER GDEM is as good, better, or worse than the best comparable global elevation data currently in use. It is expected that other evaluations (e.g., Hirt et al., 2010) will be done with higher accuracy local and regional reference datasets to further characterize the ASTER GDEM. This evaluation focused on statistical comparisons of the GDEM with the reference data and on a detailed visual inspection of the GDEM s depiction of the terrain. A number of general conclusions about the ASTER GDEM can be drawn from this analysis: Elevations in all areas are systematically biased negatively with respect to the reference DEMs and GCPs (Tables 2 and 3, Figures 2a through 2l). The effective ground resolution is noticeably lower than that of the equivalent 30 m reference DEMs (Figures 8 and 12). Based on the aggregated statistics from the GDEM-to-reference DEM comparisons and the coarse geolocation assessment, the GDEM appears to meet its vertical (Table 2) and horizontal specifications. Based on the GCP comparison, assuming a 10 m upper bound on the GCP error, the estimated vertical error in eight of the sample areas (Table 4) may exceed the 16.8 m specification (90 percent LE) in contrast to the DEM-to-DEM results. 344 April 2011 The GDEM is noisier than the reference data by a factor of 2 or more in most areas (Table 3, Figures 2a through 2l). There is a large variability in the number of stacked ASTER scenes used to compute elevation values at each post with as few as one or two in some cases and over 30 in others (Plate 2). Numerous artifacts are present in the data including ridge lines from the mosaicking, pits, spikes, large anomalous areas of elevation, and other unexplained structural features (Figures 3 through 13). There is no inland water body delineation and the coastline definition is poor, making it difficult or impossible to separate land from water in these areas. After taking into account the unexplained negative bias in the ASTER elevations, the ASTER GDEM compares well with the other DEM reference sources in the areas below 60 N; the DEM-to-DEM comparisons were within a range of about 4 to 15 meters (90 percent LE) for these areas. Above 60 N where there are no SRTM source data, specifically in Alaska, Russia, and Canada, the error in the mean elevation differences is greater. In these areas, there are gross anomalies at single posts as well as over larger areas that have sizeable offsets compared to the reference data: as much as 1,000 m in North America and 5,000 m in Russia. These are likely due to unedited cloud cover in the GDEM and may very well be skewing the statistics in the DEM comparisons PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

11 _ qxd 3/15/11 3:06 PM Page 345 Plate 2. The number of scenes (passes) used to compute ASTER GDEM elevation values in Russia Site 3 from the ASTER Quality Assurance file. The red, green and blue areas represent 1, 2 and 3 scenes, respectively. The table at the right shows the number of stacked scenes ( Value ) used to compute the elevation posts, the color code in the graphic, and the actual number of posts corresponding to a given number of scenes. The black areas represent elevations that have been voided out of the GDEM due to anomalies or lack of useable observations. Figure 3. Shaded relief of major topographic features comparing ASTER GDEM, and SRTM DTED for China cell 34 N, 085 E. The ASTER GDEM is noisier and contains numerous pits. (Hirt et al., 2010; Abrams et al., 2010). Some of the DEM differences in the areas above 60 may also be attributable to interpolation error (see for example, Fisher and Tate, 2006) introduced into the reference DTED elevations when the 1 1 arcsec reference grid was created from the original 1 3 arcsec and 1 4 arcsec DTED grids. The variability of the ASTER GDEM elevations with respect to the GCPs is greater than the variability for the PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING GDEM-to-DEM comparisons, ranging from 10 to 25 meters in all areas, except for the Russia Sites (25 to 34 m) and Canada (34 m). The comparable variability in the reference DEMs relative to the GCPs is 5 to 11 m, with only two exceptions in Bolivia (15 m) and Alaska (24 m). In areas where there are other DEM sources besides SRTM to compare to the GCPs (e.g., Libya - SPOT5, Bosnia and Iraq-photogrammetrically-derived DTED ), the error statistics are consistent April

12 _ qxd 3/15/11 3:06 PM Page 346 Figure 4. Shaded relief showing high noise in ASTER GDEM, compared to SRTM DTED for a sample Bolivia cell (at S, W). Figure 5. Shaded relief showing resolution differences and apparent mosaic seam for ASTER GDEM (linear feature in the center) compared to the reference SRTM DTED for Iraq cell 33 N, 42 E. Figure 6. Shaded relief showing ridge-like artifacts in ASTER GDEM compared to SRTM DTED, apparently resulting from the mosaicking process for the sample cell in Korea (at N, E). Figure 7. Shaded relief illustrating poorly fit fill data in the ASTER GDEM compared to NGA DTED as well as small ridge-like mosaicking artifacts in the ASTER GDEM for a cell in Alaska (at N, W). 346 April 2011 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

13 _ qxd 3/15/11 3:06 PM Page 347 Figure 8. Shaded relief of a Thailand water body in the ASTER GDEM and the same area in the SRTM DTED (at N, E). Note also the apparent better resolution of the SRTM data relative to ASTER. Figure 9. Shaded relief showing misrepresentation of coastal shoreline and false islands in ASTER GDEM compared to NGA DTED for a sample cell in Canada (at N, W). Water is black. Figure 10. Shaded relief with an example of land form artifacts in Bosnia cell 44 N, 20 E in the ASTER GDEM that do not appear in the reference SRTM DTED. Figure 11. Shaded relief of trenches in the ASTER GDEM in Thailand that do not appear in the reference SRTM DTED (at N, E). PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING April

14 _ qxd 3/15/11 3:06 PM Page 348 Figure 12. Pits along ridgelines in the ASTER GDEM compared to the reference SRTM DTED for Afghanistan cell 32 N, 066 E. Figure 13. Pits in the ASTER GDEM compared to the reference SRTM DTED for Bosnia cell 44 N, 020 E. with the SRTM DTED results. The higher average noise of the GDEM is exhibited in both the statistics and graphical displays of the data. It was clear from an examination of the ASTER Quality Assurance file that there was great variability in the number of ASTER scenes used to compute the individual elevation values. Cells within the same geographic area can have significantly different coverage as well. In some cases, the higher vertical errors may be attributable to a scarcity of useable data (possibly Russia, for example); however, no generalizations could be made in this evaluation. Two good highly accurate scenes may yield a better elevation value than 10 scenes with widely differing values of less accuracy. The visual inspection turned up a variety of significant data artifacts, anomalous areas, and erroneous landforms. Mosaicking artifacts were common, especially at the boundaries between areas with differing numbers of scenes used in the final GDEM mosaic, manifesting themselves as 1 to 20 m ridges that could be discerned throughout the dataset. Alternate data sources used to fill voids in the ASTER GDEM were not smoothly inserted, leaving distinct physical boundaries and discontinuities between data sources. Large pits and spikes are common in many areas. Structural features in the terrain were observed in the ASTER GDEM, which were totally absent in the reference DEMs. They do not appear to be man-made or natural changes in the topography that occurred between the time of the ASTER observations and the reference DEM collection. Hirt et al. (2010) reported seeing stripe effects that are evidenced as 10 to 20 m steps throughout the entire Australian continent. It is not clear how and why these were created in the ASTER data processing. On the one hand, the ASTER GDEM appears to meet its target specification over the 20 sample geographic areas based on the comparison of the GDEM to the reference DEMs; 348 April 2011 however, relative to the GCPs, the estimated GDEM accuracy appears to fall short of the specification in a significant number of the sample areas. The quality of the ASTER GDEM is less than that of SRTM DTED 2, both in vertical accuracy and effective ground resolution. This evaluation was intended as an overview and first look at the ASTER GDEM. Accordingly, more work will be required to explain the data anomalies and the resolution issue, and it is expected that other studies will take an in-depth look at the effects of land-cover, vegetation, surface roughness, and slope on the quality of the GDEM. Many of the artifacts in the GDEM can be detected and filtered out by users but it would be better if the entire DEM were reprocessed with the goal of removing these artifacts and the systematic errors in some uniform way instead of on an ad hoc basis by individual users. The data in its present form would be very difficult to trust and use without careful review and editing. Within the SRTM coverage area (56 S to 60 N), the ASTER GDEM may be useful in areas outside the US because its 1 arcsec (30 m) resolution, although not fully realized, may be better for some applications than the 3 arcsec (90 m) SRTM elevation data, the only publicly-available SRTM data in those non-us areas. The ASTER data may also be useful for filling voids in the SRTM DTED on a case-by-case basis. Its primary value, though, will be in areas not covered by SRTM (particularly above 60 N), for which it may be the best and in some cases the only source of elevation data at 30 m to 250 m resolution. As such, the ASTER GDEM is a valuable addition to global topographic data. References Abrams, M., B. Bailey, H. Tsu, and M. Hato, The ASTER Global DEM, Photogrammetric Engineering & Remote Sensing, 76(4): PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING