Simulating error propagation in land-cover change analysis: The implications of temporal dependence

Transcription

1 Computers, Environment and Urban Systems xxx (2006) xxx xxx Simulating error propagation in land-cover change analysis: The implications of temporal dependence Amy C. Burnicki a, *, Daniel G. Brown a, Pierre Goovaerts b a School of Natural Resources and Environment, University of Michigan, Dana Building, 440 Church Street, Ann Arbor, MI , USA b BioMedware, 516 North State Street, Ann Arbor, MI 48104, USA Accepted in revised form 5 July 2006 Abstract We examined factors that affect the propagation of error in analyses of land-cover change classified from multi-temporal satellite imagery by simulating multiple versions of land-cover maps at two times, time-1 and time-2. The maps, each with two categories of land-cover, were produced to investigate three specific attributes that affect change-detection accuracy: (1) the pattern of change that produced a time-2 map from a time-1 map, (2) the spatial patterns of the errors that affected both the time-1 and time-2 maps, and (3) the level of temporal dependence (or correlation) between the patterns of error at each time. The simulated maps were analyzed in a change analysis to assess the relative performance of the error-perturbed maps in identifying and quantifying the known landcover changes. Accuracy measures, such as overall percent correctly classified and user s accuracy, were calculated to describe the effects of land-cover errors on the accuracy of the change maps under each experimental setting. The results illustrate that temporal dependence of errors in land-cover maps influences both our ability to detect a variety of land-cover changes and the level of error in change maps. The study also illustrates how spatial simulation can be used to investigate patterns of error propagation where assumptions of spatial and/or temporal independence are violated. Ó 2006 Elsevier Ltd. All rights reserved. * Corresponding author. Address: Department of Geography, University of Wisconsin-Madison, 375 Science Hall, 550 North Park Street, Madison, WI , USA. Tel.: ; fax: address: burnicki@wisc.edu (A.C. Burnicki) /$ - see front matter Ó 2006 Elsevier Ltd. All rights reserved. doi: /j.compenvurbsys

2 2 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx Keywords: Error propagation; Land-cover change; Geostatistical simulation; Uncertainty 1. Introduction Quantifying the extent and rate of land-cover change, and developing accurate projections of future land-cover changes, are important challenges facing the remote sensing and global-change-science communities. Land-cover change has far reaching impacts. Vitousek (1994), in an address to the Ecological Society of America, stated that land-cover change will remain the single most important of the many interacting components of global change affecting ecological systems (p. 1867). However, the errors inherent in quantifying rates and areal extents of land-cover change remain a major uncertainty. The Land- Use and Land-Cover Change Implementation Strategy, produced from a joint initiative of International Geosphere-Biosphere Programme (IGBP) and International Human Dimensions Programme on Global Environmental Change (IHDP: Nunes & Augé, 1999), stated that our current understanding of land-cover change is limited by lack of accurate measurements of its rate, geographic extent, and spatial pattern. Foody (2002) in a review of the current state of accuracy assessment in remote-sensing analyses stated that our limited knowledge concerning the accuracy of land-cover change products directly affects our ability to make accurate predictions of change. Thus, the importance of land-cover change on varying environmental and ecological processes, and the need to better understand the uncertainty involved in determining rates and future occurrences of land-cover change, necessitates the study of patterns of error in land-cover-change products. Many analyses of land-cover change involve comparing land-cover maps derived from satellite images taken at two or multiple points in time and classified to represent landcover types, a process referred to as post-classification change detection. When conducting such an analysis, the accuracy of the resulting map of change is directly dependent on both the accuracies associated with the input classified maps and the interaction of these errors over time. Errors present in post-classification change analyses are primarily attributable to either the misregistration of images or misclassifications resulting from the inaccurate assignment of land-cover classes (Carmel, Dean, & Flather, 2001). This paper is concerned with errors in change resulting from misclassification. Assuming perfectly registered maps, the patterns of error in land-cover maps from two times should be expected to have the strongest influence on the accuracy of the resulting change map. Assuming the errors in the two maps are independent, the overall accuracy of a map of change can be approximated by simply multiplying the individual overall accuracies associated with each classified map (Congalton & Green, 1999; Lunetta & Elvidge, 1998). Studies attempting to assess the uncertainty associated with land-cover change measurements frequently assume temporal independence between errors in individual classified imagery (Carmel et al., 2001; Pontius & Lippitt, 2004). Carmel and Dean (2004) developed an error model to calculate the propagation of uncertainty within classified spatio-temporal datasets that explicitly assumed independence between errors occurring at each time step. While this research succeeded in describing both the magnitude and pattern of error associated with the final map of change, it also illustrated that the presence of a correlation between errors over time significantly affected model results and produced additional uncertainty in model estimates.

3 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx 3 Recent research has confirmed that the patterns of error in a classified land-cover map at one time are correlated with the patterns of error in a land-cover map of the same area at a later time (Carmel, 2004). Liu and Zhou (2004) examined change trajectories created from multi-temporal classified images and illustrated that a significant difference existed between the overall accuracy of change calculated when applying transition rules to the trajectories and that calculated from simply multiplying the individual accuracies of the classified maps. We postulate that this difference can be attributed to the presence of a temporal correlation between classification errors within the time-series. Presumably, the landscape and observational conditions giving rise to classification errors at one time will not have substantially changed in subsequent time steps, resulting in the joint occurrence of classification errors. Recent work by van Oort (2005) lends further support to the importance of incorporating temporal dependencies in land-cover change accuracy assessment analyses. He showed that conditioning misclassification probabilities for time-2 on misclassification probabilities at time-1 improved estimates of change. Therefore, the potential impact temporal dependencies have in creating the pattern of error associated with a map of change must be considered in order to fully comprehend error propagation in analyses of land-cover change. This paper takes an important first step in assessing the accuracy associated with maps of land-cover change by considering the interaction of classification errors within a timeseries. It directly addresses the assumption of temporal independence between patterns of error in classified imagery and investigates the potential impact of these dependencies on the resulting pattern of error in a change map. To assess the impact of temporal dependencies in error, we developed a simulation approach. The usefulness of real-world data for this purpose is limited due to the lack of comprehensive error information accompanying classified maps. Error information, often conveyed in the form of an error matrix, is derived from a sample of the reference data used in map construction and does not exhaustively represent the underlying pattern of error. Furthermore, information regarding the potential existence of a temporal correlation between errors associated with time-series classified maps is seldom reported (van Oort, 2005). Simulation provides the flexibility required to define the structure of the classified maps at time-1 and time-2 and their associated error surfaces and to define the strength of the temporal relationship between error surfaces. Simulation also provides a controlled platform for experimentation, enabling the direct determination of the impact of increasing the temporal correlation between errors when all other parameters are held constant. Finally, simulation enables the generation of multiple realizations for each time-series classified map under specified error conditions. The creation of multiple classified maps at time-1 and time-2 perturbed by error permits a thorough analysis of the impact of increasing temporal dependence on stochastic errors. Our primary interest in developing the simulated model of error in land-cover change was to directly assess the impact of specified patterns of error on the accuracy of the final map of change. A secondary interest that resulted from model development was to determine whether the specified patterns of change interact with the patterns of error to affect the accuracy associated with the final map of change. The simulated surfaces required to implement our approach were produced using the geostatistical simulation algorithm known as simulated annealing. Simulated annealing has seen growing, but limited, application in GIScience and remote sensing research (Aerts & Heuvelink, 2002; Goovaerts,

4 1996; Ware, Jones, & Thomas, 2003). It was chosen for map production in this research due to the flexibility it provides in defining the spatial patterns to be reproduced. The remainder of this paper is organized as follows. Section 2 provides background on the application of simulation in studies investigating the propagation of error in GIS analyses and in land-cover change research. Section 3 describes the development and overall design of our modeling approach to simulate error in classified imagery. It describes how simulated annealing is used to specify the patterns of error associated with each classified map, as well as the degree of temporal dependence between these error patterns and details the methodology used in map generation. Section 4 presents the results of simulation modeling for an initial case study and is followed by a discussion in Section Background The application of simulation to assess uncertainty associated with GIS data has increased over the last decade (Canters, de Genst, & Dufourmont, 2002; Hunter & Goodchild, 1997; Veregin, 1994; Wang, Gertner, Fang, & Anderson, 2005). This is attributable to an increased awareness of the importance of quantifying the accuracy associated with spatial data and a desire to better understand the propagation of errors in GIS analyses. Research has addressed both error propagation modeling derived from statistical theory (Goodchild, Guoqing, & Shiren, 1992; Heuvelink, 1998; Heuvelink, Burrough, & Stein, 1989) and from the application of Monte Carlo simulation (de Genst & Canters, 2000; Fisher, 1991; Kyriakidis & Dungan, 2001). Our motivation for using geostatistical simulation in this research was, in part, related to prior research investigating the propagation of error in GIS overlay operations. The work of both Veregin (1995) and Arbia et al. (1998) provide important keys to understanding what governs error propagation in overlay operations as applied to categorical maps; particularly the AND operation. This operation is applicable to analyses of change because it concerns the identification of map areas that satisfy multiple conditions such as Forest at time-1 AND Agriculture at time-2. Veregin (1995) created a composite error matrix for maps resulting from the AND operations based on the individual error matrices associated with each input layer. This composite matrix was used to estimate the error associated with the overlay operation. Arbia, Griffith, and Haining (1998) defined specific error models for each source map to produce multiple realizations of error-contaminated input maps. These maps were then compared to their error-free counterparts to determine the relative contribution of various sources of error. However, in both studies either the associated error maps or the original input raster maps were assumed to be independent. Accounting, in the overlay operation, for the interactions between errors requires the development of models capable of measuring and quantifying temporal dependencies that influence the error pattern in a final map of change. 3. Methods 3.1. Overall approach ARTICLE IN PRESS 4 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx Fig. 1 outlines the overall framework of our approach to simulating error. The approach can be divided into two main components: (1) generation of a true map of change, and (2) generation of a suite of change maps that are perturbed by specified pat-

5 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx 5 Fig. 1. Diagram of the overall framework for the simulated model of error in post-classification land-cover change illustrating an example model run. terns of error. The true maps of change, resulting from a user-defined pattern of change, are compared to suites of error-perturbed maps of change, resulting from a user-defined pattern of error. This direct comparison enables the determination of the effects of both the patterns of change and patterns of error on the accuracy of the resulting change maps. The true map of change was created by: (a) simulating a continuous surface of classification probabilities for the initial time (time-1), (b) simulating a continuous surface of change probabilities, (c) adding the classification-probability surface for time-1 and the change-probability surface to produce the classification-probability surface for the subsequent time (i.e., time-2), (d) assigning land-cover classes at time-1 and time-2 by classifying the probability surfaces, and (e) producing the true map of land-cover change through map overlay. The error-perturbed maps of change were created by: (a) simulating separate continuous surfaces of error probabilities for both time-1 and time-2, (b) adding the classification-probability surfaces for time-1 and time-2 to their associated series of error-probability surfaces to produce a series of time-1 and time-2 classification-probability surfaces that were perturbed by introduction of error,

6 (c) assigning land-cover classes within the series at both time-1 and time-2 by classifying the error-perturbed probability surfaces using the same classification rule that was used to create the classified maps from the classification-probability surfaces that were not perturbed by error, and (d) producing the series of error-perturbed maps of change through map overlay. In all, three model components were produced by spatial simulation: (1) initial time (i.e., time-1) land-cover maps, (2) subsequent time (i.e., time-2) land-cover maps conveying specific patterns of change, and (3) associated time-1 and time-2 land-cover maps that were perturbed according to a range of spatial and temporal patterns of error. The classification-probability surface represents a map in which each pixel is assigned a probability for belonging to an individual land-cover class. By limiting our analysis to only two possible classes, the value assigned to each pixel, ranging from 0 to 100, can be interpreted as the probability that an individual pixel belongs to land-cover class 1. The change-probability surface represents a map in which each pixel is assigned a probability for experiencing a land-cover transition. Similarly, the error-probability surface represents a map in which each pixel is assigned a probability for experiencing a classification error. The classification-probability surface for time-2, resulting from the addition of the time-1 classification-probability surface and the change-probability surface, represents the probability that an individual pixel belongs to land-cover class 1 after accounting for change. Similarly, the error-perturbed classification-probability surface for time-1 (or time-2), resulting from the addition of the classification and errorprobability surfaces for time-1 (or time-2), represents the probability that an individual pixel belongs to land-cover class 1 after introducing error into the classification. Finally, the resulting land-cover maps for time-1 and time-2, and associated land-cover maps for time-1 and time-2 perturbed by error, are produced by assigning each pixel to either land-cover class 1 or 2, depending on the probability defined as the classification cutoff. In this research, the classification probability cut-off corresponded to the mean, or 50th percentile, of the values in the probability surface Experimental design ARTICLE IN PRESS 6 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx To determine how specified patterns of error affect the accuracy of maps of change, three specific patterns of error were investigated. The first pattern of error allows errors to occur randomly across the surface of the map with a defined level of spatial autocorrelation, and has error patterns at time-1 and time-2 that are independent. The second pattern considers errors that are randomly located with a defined level of spatial autocorrelation at time-1 and time-2, and has a defined level of correlation between time-1 and time-2 error patterns. The third pattern considers errors that have a defined level of correlation to the location of classification boundaries at time-1, a defined level of spatial autocorrelation, and a defined level of correlation between time-1 and time-2 error patterns. Three specific patterns of change were also investigated. The first pattern of change represents the expansion of a single land-cover class through the addition of a trend in the X- direction. The second pattern allows change to occur randomly across the surface of the map with a defined level of spatial autocorrelation. The third pattern considers change that has a defined level of correlation to the location of classification boundaries at

7 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx 7 time-1 and a defined level of spatial autocorrelation. The next section details the simulation methodology used to produce the patterns of error and change considered in this research, as well as the initial land-cover surface Application of simulated annealing Simulated annealing is an optimization algorithm that creates a set of map realizations based on a user-defined objective function. It does not require the formal definition of a random function model, but instead reproduces the user-defined spatial pattern by solving the optimization problem as specified in the objective function (Goovaerts, 1997). Although a single global optimum may exist, many nearly optimal solutions exist and the set of realizations produced represents a set of possible solutions. The objective function is comprised of component statistics defined by the user to quantify the spatial pattern to be reproduced during simulation. Simulated annealing operates in the following manner. An initial random map is created based on a distribution of data values defined by the user. The map is then perturbed by randomly selecting a pixel location, re-sampling a new value from the initial distribution, and calculating the value of the user-defined objective function. Generally, if the map modification leads to a decrease in the difference between the current and target function values, then the perturbation is accepted and a new pixel location is selected for modification. The process continues until the target statistics are reasonably reproduced or further modification of the map does not result in a significant change in the value of the current statistics. An important and defining aspect of simulated annealing is the definition of the criteria used to decide if an observed perturbation is accepted. Simulating annealing allows for the acceptance map modifications that increase the difference between the current and target function values to prevent the solution from becoming stuck in a local optimal solution (Goovaerts, 1997). A cooling schedule is defined to determine the probability of accepting a solution that results in an increase in the difference between the current and target objective function statistics. We used a negative exponential cooling schedule to permit the acceptance of a large proportion of unfavorable perturbations initially and avoid local solutions to the optimization problem. As the simulation algorithm proceeded, the probability of acceptance was decreased relatively slowly to allow the algorithm to converge to a single solution. While this procedure does not guarantee that the optimal solution will be found, the goal of this work was to fully explore the impacts of temporal dependence between errors in classified maps on the accuracy associated with a map of change. It was not necessary to find the global solution to the optimization problem, only to produce a series of solutions that sufficiently reproduced each specified pattern of error to thoroughly examine the impact of these patterns. Three different statistics were used to define the objective functions that produced the initial, change and error-probability surfaces. Objective functions were defined as mathematical combinations of component statistics as follows: OðiÞ ¼ XC c¼1 w c O c ðiþ: ð1þ

8 8 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx where O(i) is the global objective function, i is the ith perturbation, O c (i) is an individual objective function statistic, C equals the number of statistics considered, and w c is the weight assigned to each statistic. The first component statistic was the initial histogram, which defined the data values observed in the simulated surface and was constructed from an initial cumulative frequency distribution. This component measured the difference between the current and target cumulative distribution functions and was defined as: O 1 ðiþ ¼ XK k¼1 ½F ðz k Þ ^F ðiþ ðz k ÞŠ 2 ; ð2þ where F(z k ) is the target cumulative frequency of the distribution at threshold z k, ^F ðiþ ðz k Þ is the cumulative frequency of the distribution at threshold z k calculated for the ith perturbation, and K is the number of thresholds considered. All three probability surfaces simulated in this research were constructed from normal distributions. Normal distributions were chosen due to an intrinsic statistical property associated with the linear combination of independent normal distributions. A distribution resulting from the addition of a series of independent normal distributions (e.g. time-2 = time-1 + change), defined as the linear function: Y ¼ Xn i¼1 a i X i ð3þ where X 1,X 2,...,X n are mutually independent normal variables with means l 1,l 2,...,l n and variances r 2 1 ; r2 2 ;...; r2 n, is also normally-distributed with:! N Xn a i l i ; Xn ; ð4þ i¼1 i¼1 a 2 i r2 i where a 1,a 2,...,a n are real constants (Hogg & Tanis, 1993). Thus, determining the distribution associated with the time-2 and error-perturbed time-1 and time-2 probability surfaces directly resulted from the application of the above theorem. The second component statistic was the semivariogram model, which defined the level of spatial continuity reproduced in the simulated surfaces. It is a measure of the relationship between distance and variance in pixel values, or a measure of the dissimilarity of pixel values separated by a given distance (Goovaerts, 1997). The semivariogram is based on the postulate that pixels located farther apart are less similar, or exhibit greater variance in their values, than pixels located closer together. This component measured the difference between the current and target semivariogram values and was defined as: O 2 ðiþ ¼ XK k¼1 ½cðh k Þ c ðiþ ð^h k ÞŠ 2 ½cðh k ÞŠ 2 ; ð5þ where c(h k ) is the target semivariance at lag k, ^c ðiþ ðh k Þ is the semivariance at lag k calculated for the ith perturbation, and K is the number of lags considered (Goovaerts, 1997). The key parameter in defining the semivariogram model is the range. The range is defined as the distance at which the semivariance equals the variance of the map surface. In other words, it represents the distance beyond which pixels are no longer spatially auto-

9 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx 9 correlated. For all simulated surfaces, the range defined in the semivariogram model represented the distance in which pixels were allowed to exhibit spatial autocorrelation. To incorporate secondary information into the simulation, the third component statistic measured the linear correlation to a secondary variable. This enabled the creation of temporally-dependent error-probability surfaces, where time-2 error patterns were conditioned on their time-1 counterparts, and change and error-probability surfaces that were correlated to time-1 classification boundaries. This statistic measured the difference between the current and target correlation values as defined in: h i 2; O 3 ðiþ ¼ q XY ð0þ ^q ðiþ XY ð0þ ð6þ where q XY (0) is the target linear correlation coefficient between X, previously-generated probability surface, and Y, probability surface currently being simulated, and ^q ðiþ XY ð0þ is the linear correlation coefficient calculated for the ith perturbation (Goovaerts, 1997). Two different objective functions (Eq. (1)) were created from these three statistics. The first objective function considered two statistics: the initial histogram of data values and the semivariogram model, as illustrated in Fig. 2a. The weights assigned to each compo- Fig. 2. Illustration of objective functions considered in simulated annealing. (a) Objective function using two statistics initial histogram and semivariogram model. (b) Objective using three statistics initial histogram, semivariogram model, and linear correlation to secondary map.

10 nent of the objective function were chosen to give emphasis to the reproduction of the semivariogram model. This gave greater importance to the level of spatial continuity observed in each simulated map. The second objective function considered three statistics: the initial histogram of data values, the semivariogram model, and a linear correlation to a previously-generated map realization, as illustrated in Fig. 2b. The weights assigned to the three statistics were chosen to give greater weight equally to both the reproduction of the semivariogram model and linear correlation coefficient. This increased the importance of the level of spatial continuity in the resulting surface and ensured a defined level of association with a secondary map. Simulated annealing was performed using the software package GSLib and program SASim (Deutsch & Journel, 1992). All surfaces generated through simulated annealing had lattice dimensions and were imported into Idrisi (Eastman, 1999) for raster-based processing. This entailed the addition of change and error-probability surfaces to produce time-2 and associated time-1 and time-2 error-perturbed probability surfaces, classification of all probability surfaces into two land-cover class maps, and an overlay operation to produce the final maps of change Generation of model components ARTICLE IN PRESS 10 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx This section details the construction and parameterization of the initial-probability surface, the range of change-probability surfaces, and the range of error-probability surfaces described in Section Initial map The main parameter considered in constructing the initial (time-1) probability surface was the range of the semivariogram model. The objective was to produce an initial map that exhibited a level of spatial autocorrelation that reflected observable real-world patterns. As an example case, a forest/non-forest classification for Washtenaw County, Michigan, located in the southeast corner of the state, was produced using Landsat TM data (30 m resolution). The county was divided into 80 square cells following existing quarter-township boundaries, which were surveyed prior to settlement (Fig. 3a). The grid representing a quarter-township at 30 m resolution was approximately the same size as our simulated surfaces ( ). Indicator semivariogram models were fit to describe the spatial continuity in the forest class in each quarter-township. We produced a statistical distribution of observed range values and used the values corresponding to the first and third quartiles of the distribution to define the target level of spatial autocorrelation in the simulated surfaces. Both the first and third quartiles were chosen in order to examine the possible impact of spatial continuity on model results. Only the results based on the first quartile are presented because the patterns and relative magnitudes of the resulting accuracy measures were the same for both levels of spatial autocorrelation. Simulated annealing was conducted using a normal distribution with a mean of 50 and a standard deviation of 12, chosen to constrain the majority of data values to the range (Values below 0 and above 100 were set to 0 and 100, respectively.) Time-1 probability surfaces were produced that were then classified into a two landcover class map using the 50th percentile of the initial normal distribution as the cut-off.

11 ARTICLE IN PRESS A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx 11 Fig. 3. Land-cover maps displaying quarter-township boundaries for Washtenaw County, Michigan. (a) Forest/ Non-Forest classification. (b) Binary-change map indicating presence/absence of land-cover change.

12 12 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx Maps of change The three patterns of change considered were created using three different methods. First, for the expansion of a single class through the addition of a trend, an initial scaled X trend, ranging from 0 to 1 in the west to east direction, was created and multiplied by two scalar values. The result was two change-probability surfaces representing a trend in the X-direction scaled to represent low and high percentages of change. Low and high amounts of change were calculated based on the first and third quartiles of the distribution of percent change observed in the 80 quarter-townships of Washtenaw County. Second, random, spatially autocorrelated probability surfaces displaying both low and high percentages of change were generated using simulated annealing. Two different normal distributions were used in map generation. Both had means centered at 0 to ensure that each type of change occurred in roughly equal proportions (e.g. forest to non-forest and non-forest to forest). Two standard deviation values were chosen to constrain the magnitude of change to the low and high percent change cases, calculated from the Washtenaw County data, as above. To determine the range of the semivariogram, a binary change/no-change map was produced for Washtenaw County based on two forest/non-forest classifications (Fig. 3b). Indicator semivariogram models were fit to the binary-change class in each quarter-township to produce a distribution of observed range values for change. Based on this distribution, a single range value was selected corresponding to the third quartile of the distribution to reflect a relatively high level of spatial autocorrelation in change. The third quartile was chosen because the majority of range values for binary change were low and a higher level of spatial autocorrelation was preferred. The result was two change-probability surfaces displaying a relatively high degree of spatial continuity in change and exhibiting both relatively low and high magnitudes of change. Finally, maps with change probabilities correlated to the classification boundaries at time-1 were generated using two additional applications of simulated annealing. This required the use of the second objective function, incorporating a correlation coefficient to a secondary map, to describe the intended relationship between the change and time- 1 probability surfaces. First, a secondary variable was calculated that represented the distance to the nearest classification boundary for each pixel in the time-1 surface. Two correlation coefficients were chosen to quantify the relationship between the change probability surface and the distance-to-boundary map reflecting a relatively weak and strong degree of correlation (r = 0.3 and r = 0.7). Negative correlation values were used in order to associate high magnitudes of change probability with short distances from classification boundaries. Simulated annealing was conducted twice using each correlation coefficient and the absolute value of the initial normal distribution corresponding to the low percent of change case, described above. The absolute value of the initial distribution was taken to avoid centering a particular type of change near classification boundaries. To reintroduce the directionality of change, a random binary grid comprised of 1 and 1 was multiplied by each simulated surface. The same semivariogram model used to produce the random, spatially autocorrelated, change surfaces described above was also applied to the generation of correlated change surfaces. The result was two change-probability surfaces displaying a relatively high degree of spatial continuity in change, a relatively low magnitude of change, and having change either weakly or strongly concentrated near the boundaries in the time-1 classification.

13 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx 13 In all, six change-probability surfaces were generated and added to the initial time-1 classification-probability surface. Each of the resulting time-2 probability surfaces were then classified into two land-cover classes using the 50th percentile as the classification cut-off Maps of error The three patterns of error were simulated following similar procedures. First, the spatially autocorrelated but temporally independent error patterns were generated with simulated annealing to produce error-probability surfaces at time-1 and time-2 independently. Lacking comprehensive information about error patterns in Washtenaw County, we set simulation parameters for the error-probability surfaces to reasonable values. A single range value for the semivariogram model was selected so that the error-probability surfaces displayed a smaller degree of spatial autocorrelation, or less spatial continuity, than the change-probability surfaces. The percentage of error allowed to occur between the classified and error-perturbed classified maps was set to 25%. This translates to error-perturbed classified maps having overall accuracies of 75%, which is a reasonable (if low) value given commonly observed levels of accuracy in remote sensing analyses. The initial normal distribution chosen for map generation had a mean centered at 0 to ensure that the different misclassifications were present in roughly equal proportions. A standard deviation value was selected to achieve a misclassification rate of 25%. The result was two sets of independent error-probability surfaces for time-1 and time-2 displaying a relatively low degree of spatial continuity in error and exhibiting a 25% misclassification rate. Second, random, spatially autocorrelated and temporally correlated error-probability surfaces at time-1 and time-2 were created using simulated annealing to generate a time- 1 error-probability surface and a time-2 surface that was correlated with time-1. To generate error-probability surfaces for time-1, simulated annealing was conducted using the previously defined semivariogram range and initial normal distribution. The resulting realizations for time-1 then served as secondary variables in the generation of time-2 errorprobability surfaces. The second objective function, which incorporates a correlation coefficient to a secondary map, was used to produce time-2 surfaces displaying a correlation of either 0.2 or 0.4 to their associated time-1 surfaces. These values reflect the range of temporal correlation values observed in practice (Carmel, 2004). The same semivariogram model and initial normal distribution were used to generate the time-2 error-probability surfaces. The result was a set of error-probability surfaces for time-1 displaying a relatively low degree of spatial continuity and a 25% misclassification rate, and two sets of errorprobability surfaces for time-2 exhibiting either a 0.2 or 0.4 correlation to their associated time-1 error-probability surfaces. Finally, error-probability surfaces that were correlated to classification boundaries and temporally correlated at time-1 and time-2 were created using simulated annealing. The generation of time-1 error-probability surfaces correlated to time-1 classification boundaries followed the same procedure as the generation of correlated change surfaces. The correlation value chosen for error-probability surfaces corresponded to a relatively weak level of correlation (r = 0.3). The same semivariogram model and initial normal distribution were again used in map generation, see above. The resulting realizations for time-1 then served as secondary variables in the generation of error-probability surfaces for time-2. The same simulated annealing procedures used previously to produce temporally correlated error-probability surfaces for time-2 were conducted

14 using both the 0.2 and 0.4 correlation values. The result was a set of error-probability surfaces for time-1 with a relatively low degree of spatial continuity in error, a 25% misclassification rate, and error weakly concentrated near the boundaries in the time- 1 classification. Two sets of error-probability surfaces for time-2 also resulted, exhibiting either a 0.2 or 0.4 temporal correlation to their associated time-1 error-probability surfaces. In all, five sets of error-probability surfaces for time-1 and time-2 were generated and added to the classification-probability surfaces for time-1 and time-2, respectively. This resulted in a series of error-perturbed probability surfaces for time-1 and time-2 that were then classified into two land-cover classes using the 50th percentile as the classification cutoff Analysis ARTICLE IN PRESS 14 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx The end result of model simulation was the production of a single map of true change that was compared to a suite of change maps resulting from the introduction of error. We generated 30 realizations for each error-probability surface at time-1 and time-2 and one realization for the initial map and each change-probability surface. This resulted in the creation of 30 maps of change that were error-perturbed for each combination of parameter values (Fig. 1). The choice of 30 realizations was motivated by the fact that the distribution of statistics calculated when comparing the error-perturbed change maps to their error-free versions would be approximately normal. This enabled a more thorough comparison of the effects of different patterns of error and change on the accuracy of the final maps of change. In all, 30 different combinations of change and error patterns were modeled, corresponding to the six possible change-probability surfaces and the five possible error-probability surfaces. Two statistics were calculated to compare the suite of error-perturbed maps of change to their corresponding maps of true change. The entire simulated surface was used in calculation; i.e. all pixels were included in each case. First, the overall percent correctly classified (PCC) statistic was calculated for each error-perturbed map of change. The overall PCC statistic, a well-known remote sensing measure of overall map accuracy (Congalton, 1991; Lillesand & Kiefer, 2000), was chosen because it provides a good benchmark for evaluating the impact of differing patterns of error on map accuracy. The second statistic calculated was the user s accuracy for correctly predicting the presence of a change in the error-perturbed map of change. The user s accuracy of the change class indicates the probability that a pixel undergoing a land-cover change in the error-perturbed map of change actually experienced a land-cover change in the map of true change. This statistic was considered because the main research objective was to evaluate the performance of the change-detection analysis in correctly identifying land-cover transitions in the presence of error. Separating the model s ability to predict change from its ability to predict persistence provides a better estimate of model performance due to the dominance of persistence in analyses of land-cover change and the ease in predicting stationarity in land-cover over time (Pontius, Shusas, & McEachern, 2004). Thus, user s accuracy of the change class was included to assess the reliability of land-cover transitions predicted by the error-perturbed maps of change. Both statistics were calculated for each of the 30 error-perturbed maps of change and summarized by calculating the mean and standard deviation of the distribution of values.

15 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx Results Overall PCC values increased when the temporal correlation between the time-1 and time-2 error patterns increased (Table 1). Increasing the temporal correlation from 0 to 0.2 to 0.4 produced significant increases in overall PCC values for all patterns of change; i.e. when the standard deviation associated with each PCC value is considered. Differences existed in both the magnitude of overall PCC values and the relative increase in overall PCC values when the temporal dependence between error patterns was increased for the range of change patterns considered. For example, when changes were random and occurred in low percentages, accuracy increased by 2.24% from no temporal correlation to a 0.4 temporal correlation, while the increase was 1.83% when changes were highly correlated to classification boundaries and occurred in low percentages. Larger increases in overall PCC values as a consequence of increasing the temporal correlation from 0 to 0.4 were observed as the amount of change occurring between time-1 and time-2 decreased. For example, the gain in accuracy for the random-change pattern with a low percent of net change was 2.24% when increasing the temporal correlation from 0 to 0.4 as compared to an increase of 1.3% for the random change pattern with a high percent of net change. This pattern held true for change resulting from a trend in the X-direction. Cases where both the change-probability and error-probability surfaces were correlated to the time-1 classification boundaries showed lower overall accuracy values, even in the presence of a temporal correlation. For example, the increase in accuracy for changes highly correlated to classification boundaries and occurring in low percentages was 1.83% when errors were randomly located at time-1 and the temporal correlation was increased from 0 to 0.4, as compared to an increase of only 1.24% when errors were correlated to time-1 classification boundaries. Both change patterns that associated high change probabilities with the classification boundaries showed lower overall accuracies than the other four change patterns when high error probabilities were also associated with classification boundaries (Table 1). In contrast to the trend observed in overall accuracy (Table 1), not all change patterns exhibited an increase in user s accuracy as the temporal correlation between error patterns increased (Table 2). Only two of the six change patterns consistently increased in accuracy as the temporal correlation between time-1 and time-2 error patterns increased from 0 to 0.2 to 0.4 when error patterns were random at time-1. When errors at time-1 were correlated to classification boundaries, only a single change pattern consistently increased in accuracy as the temporal correlation increased. Also, when considering randomly located errors, an increase in temporal correlation had a greater effect on user s accuracy for change patterns exhibiting a higher percentage of change. When changes were random and occurred in low percentages, there was a decrease in user s change accuracy of 0.26% from no temporal correlation to a 0.4 temporal correlation. When changes were random and occurred in high percentages, there was an increase in accuracy of 0.68%. There were, however, some similarities between the patterns of results based on user s accuracy and overall accuracy. First, the user s accuracy results supported the observation that landscapes undergoing various patterns of change were affected differently by temporal dependence, as illustrated by the difference in magnitude of user s accuracy values. Second, in cases where both the change-probability and error-probability surfaces were correlated to the time-1 classification boundaries, the values of user s accuracy of change

16 Table 1 Mean and standard deviation of overall percent correctly classified (PCC) values summarizing 30 runs for each change and error pattern combination Error patterns Change patterns X trend with low % of change X trend with high % of change Random with low % of change Random with high % of change Low correlation to class boundaries High correlation to class boundaries Random at T1 & T2 Random T1 & temporal correlation = 0.2 T2 Random T1 & temporal correlation = 0.4 T2 Low correlation to class boundaries T1 & temporal correlation = 0.2 T Low correlation to class boundaries T1 & temporal correlation = 0.4 T2 16 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx ARTICLE IN PRESS

17 Table 2 Mean and standard deviation of user s change accuracy summarizing 30 runs for each change and error pattern combination Error patterns Change patterns X Trend with low % of change X trend with high % of change Random with low % of change Random with high % of change Low correlation to class boundaries High correlation to class boundaries Random at T1 & T2 Random T1 & temporal correlation = 0.2 T2 Random T1 & temporal correlation = 0.4 T2 Low correlation to class boundaries T1 & temporal correlation = 0.2 T Low correlation to class boundaries T1 & temporal correlation = 0.4 T2 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx 17 ARTICLE IN PRESS

18 were smaller than those observed when errors were random at time-1. For example, the increase in accuracy for changes highly correlated to classification boundaries and occurring in low percentages was 0.18% when errors were randomly located at time-1 and the temporal correlation was increased from 0 to 0.4 as compared to a decrease of 0.51% when errors were correlated to time-1 classification boundaries. Again, both change patterns that associated high change probabilities with the classification boundaries showed a larger magnitude in the decrease of user s accuracy of change when high error probabilities were also associated with classification boundaries. 5. Discussion and conclusions ARTICLE IN PRESS 18 A.C. Burnicki et al. / Comput., Environ. and Urban Systems xxx (2006) xxx xxx While researchers are placing greater emphasis on the ability to provide a more comprehensive and detailed means for assessing the uncertainty of change-detection products, the new advancements must go farther in order to fully comprehend the complex issues that affect change-detection accuracy. Among these complex issues is the recognition that temporal dependencies between error patterns in land-cover maps can significantly affect both the overall magnitude and pattern of error associated with a map of change. A key for future accuracy assessment research in the area of change detection must be the incorporation of time and temporal dependence. Only by understanding how errors propagate through a change-detection algorithm will researchers be able to quantify and present a more thorough and accurate description of uncertainty in land-cover-change products. This work demonstrated the usefulness of a simulation approach in producing and modeling realistic error patterns and their interactions. It illustrated how simulation can be successfully applied to error propagation in land-cover change by defining and controlling the various error inputs and structures. Errors occurring in multi-temporal classified imagery have complex spatial and temporal structures that limit an analytical modeling approach. Simulation provided a means for isolating and testing the assumption of temporal independence. We were able to directly assess the effect of increasing temporal dependence and of alternative error and change patterns on both the accuracy associated with a final map of change and the map s ability to identify known land-cover changes. Several conclusions can be drawn from the results presented above. First, the presence of a correlation between the patterns of error in two land-cover maps improved the overall accuracy of the resulting change map. However, the presence of a correlation between error patterns did not necessarily improve the user s accuracy of the change map in predicting the occurrence of a land-cover transition. Second, the impact of increasing the temporal dependence between error patterns was dependent upon both the amount and pattern of change. The pattern of change played an important role in determining the relative impact that increasing temporal dependence had on both the overall accuracy and user s accuracy of change. The accuracy of change detection in situations with certain change patterns was clearly more sensitive to the presence of a temporal correlation. For example, the overall accuracy of change when the pattern of change was correlated with the classification boundaries showed larger gains, in general, from increased temporal dependence than did the other change patterns. When considering user s accuracy values, the accuracy of change associated with random locations of change was more affected by the increase in temporal dependence. Additionally, modeling results also depended on the percentage of change occurring over time. The presence of a temporal correlation between patterns of error in land-cover had a greater affect on the change accuracy in situations