Bryan F.J. Manly and Andrew Merrill Western EcoSystems Technology Inc. Laramie and Cheyenne, Wyoming. Contents. 1. Introduction...

Transcription

1 Comments on Statistical Aspects of the U.S. Fish and Wildlife Service's Modeling Framework for the Proposed Revision of Critical Habitat for the Northern Spotted Owl. Bryan F.J. Manly and Andrew Merrill Western EcoSystems Technology Inc. Laramie and Cheyenne, Wyoming Contents 1. Introduction Problems with the MaxEnt Habitat Suitability Indices The MaxEnt and Maximum Likelihood Results for Northern Spotted Owls Conclusions References Northern Spotted Owl Modeling Page 1 of 13 6 July 2012

2 1. Introduction The modeling framework described in Appendix C of the Revised Recovery Plan for the Northern Spotted Owl (Strix occidentalis caurina) (U.S. Fish and Wildlife Service, 2011) had three stages. First, a map of spotted owl habitat suitability throughout its U.S. range was constructed based on the MaxEnt model for habitat associations. Second, a spotted owl conservation model was developed based on the results of step 1 using the Moilanen and Kujala (2008) Zonation computer program. Third, a spatially explicit population model that predicts the response of spotted owls to environmental conditions using the HexSim program (Schumaker, 2011) was used. Here we concentrate our comments on the first stage of this analysis because recent research has raised serious questions about the validity of the results obtained from the MaxEnt program in terms of whether the habitat suitability indices that this program produces for different areas are approximately proportional to the probability of spotted owls being present in those areas and, if not, what the habitat suitability indices do represent. Clearly, if the MaxEnt results are suspect then the results from the second and third stages of the modeling process which are based on the MaxEnt results are suspect as well. For this reason we make no comments on the use of the Zonation and HexSim programs. We utilized the vegetation and topographical data as provided by the U.S. Fish and Wildlife Service although we have serious concerns about the validity of these data. These concerns are with two aspects of the data. First, the accuracy of the GNN data seems highly questionable for MaxEnt analyses of the type carried out on northern spotted owls. The GNN database was developed by the LEMMA (Landscape Ecology, Modeling, Mapping and Analysis) team and at their website ( they state that: "In general, GNN vegetation maps are appropriate for landscape to regional-scale analyses but are insufficiently accurate for local or stand-level applications". They illustrate this with the specific example shown here in Figure 1.1 which plots the predicted basal areas of all live trees against the observed basal areas from the 2006 GNN model covering Western Washington for the Northwest Forest Plan project. The observed and predicted basal area are related, but the relationship is very poor so that, 2 for example, if the observed basal area is 60 m the GNN model may predict anything from 2 about 20 to 100 m. This poor level of prediction accuracy may be acceptable for some applications on GNN but is clearly questionable for assessing the habitat suitability of northern plotted owls at the stand level. On the same page on their website they also provide information on the accuracy of GNN vegetation class assignments at the plot level. The example that they provide shows the accuracy varying from 12.5% to 63.1% correct for different classes, which again seems too poor for use for habitat assessment at the stand level. Northern Spotted Owl Modeling Page 2 of 13 6 July 2012

3 The other concern that we have with the data used for the MaxEnt modeling is that subjective decisions were made to combine certain GNN variables before the analysis began. It may be that using the original variables without combining them would have given a more suitable analysis. Essentially, the problem that has been found with the use of the MaxEnt habitat suitability indices is that a high value for a spatial location can occur when the probability of that location being used by the species being considered can either be high or low, while a low habitat suitability index can occur when the probability of use is high. Therefore the true value of a location for conservation purposes is uncertain based on the value of the MaxEnt suitability index. In the next section of these comments we summarize the results found in two recent published papers, one on the use of MaxEnt and the other on the use of MaxEnt and some similar methods. We follow this by a demonstration that the problems with MaxEnt noted in those papers is present when MaxEnt is used with the northern spotted owl data for two of the 11 modeling regions considered by the U.S. Fish and Wildlife Service, so that this is likely the case for the other nine regions as well. Finally, we suggest that the whole modeling procedure used by the U.S. Fish and Wildlife Service needs to be reconsidered and that, in particular, MaxEnt should not be used for the first stage. 2. Problems with the MaxEnt Habitat Suitability Indices The paper by Torres et al. (2012) is not specifically about MaxEnt. This is just one of 11 species distribution models for the occurrence of jaguars in South and Central America that they considered. They used 1409 occurrence records for jaguars in this region with each of the 11 modeling methods and then compared the model results with the observed density of jaguars in 37 different locations also in South and Central America. The purpose of their study was to "test the prediction that environmental suitability derived from species distribution modeling (SDM) could be a surrogate for jaguar local population density estimates". Torres et al. expected that there would be a positive relationship between the model based habitat suitability values and the actual density of jaguars at the 37 locations where this was measured. Surprisingly, they found no significant relationship for eight of the 11 modeling methods. There was a significant relationship at the 5% level for MaxEnt with a simple linear regression but with only 13% of the variation in density accounted for by the habitat suitability index. Thus the MaxEnt results were found to have very little power to predict actual jaguar densities. Torres et al. concluded that MaxEnt and some of the other methods that they considered were able to discriminate to some extent between areas with high and low jaguar densities. However, it was found that low jaguar densities occurred in areas with high and low habitat suitability values whereas high jaguar densities were restricted to areas with high suitability values. Northern Spotted Owl Modeling Page 3 of 13 6 July 2012

4 We believe that these results obtained by Torres et al. may be common to all MaxEnt modeling. This is that if MaxEnt gives a high habitat suitability value at a location then the species may either be absent or present at a high density. In other words, a high habitat suitability index does not necessarily mean that a species is present in an area. Results that we present below suggest that this is also the case with the MaxEnt results for northern spotted owls. The paper by Royle et al. (2012) focuses on the MaxEnt method and is very critical. They argue that "MaxEnt produces a number of poorly defined indices that are not directly related to the actual parameter of interest - the probability of occurrence". They also note that the proponents of MaxEnt and similar methods incorrectly argue that ordinary maximum likelihood estimation is not possible with so called presence-only data (when no direct information about absences is available) and show how the standard maximum likelihood method can be used instead of MaxEnt to estimate true probabilities of occurrence. In addition Royle et al. argue that MaxEnt estimation includes a number of arbitrary aspects such as fixing the constant in a logistic regression and a 'regularization' process that biases parameters. To demonstrate the problems inherent in the use of MaxEnt Royle et al. provided two examples. The first was a simple simulation study that shows that the habitat suitability index from MaxEnt fails to estimate a function that is proportional to the probability of a presence when the true probability function is known. The second example is more interesting because it is based on real data on the Carolina wren (Thryothorus ludovicianus) from the North American Breeding Bird Survey. They used data from survey routes in 2006 and analyzed this three ways. To do this they imposed a 25 m grid over the study area classified grid cells with at least one survey stop as occupied if one or more wrens were detected, or otherwise as unoccupied. For each grid cell they also had data on four land cover variables (the % cover of mixed forest, deciduous forest, coniferous forest and grasslands). Their analyses used with the data were: (a) maximum likelihood using presence only data as described in their paper, i.e. ignoring the information on unoccupied grid cells; (b) MaxEnt estimation with exactly the same data; and (c) standard logistic regression with both presence and absence data. For each analysis they fitted quadratic effects of the four land cover variables. It is clear from these three analyses that standard logistic regression is the only method that makes use of all of the data and hence should give the most reliable results. Therefore, the results from this method are the 'gold standard' for any analysis using presence only data. Figure 2.1 shows the results that Royle et al. obtained in terms of the estimated relative probabilities of grid cells in different locations being used. It can be seen that maximum likelihood has given similar results to logistic regression. Therefore, maximum likelihood with presence only data has worked well with this example. By contrast, the results from MaxEnt are quite different from the gold standard of logistic regression. As noted by Royle Northern Spotted Owl Modeling Page 4 of 13 6 July 2012

5 et al. "MaxEnt's 'logistic output' greatly underestimates the probability of occurrence throughout the core of the species' range and over estimates the occurrence probability in regions where the species was never detected". 3. The MaxEnt and Maximum Likelihood Results for Northern Spotted Owls To investigate any potential bias in the MaxEnt results for northern spotted owls we repeated the U.S. Fish and Wildlife Service's MaxEnt analyses for two of the 11modeling regions that they considered. We used the Gradient Nearest Neighbor (GNN) estimates of existing vegetation, climatic and topographical data layers provided by the USFWS. These data layers represent a smoothing of a 200 ha unit surrounding each 30 by 30 meter pixel within the northern spotted owl range. As noted above in the Introduction, it is important to note that the GNN values for each pixel were not directly measured, but were estimated from available Forest Inventory and Analysis program field plots, satellite imagery and mapped environmental data, and as such have associated prediction errors that will propagate to predicted values for relative habitat suitability. For the purposes of consistency, our modeling efforts were restricted to the data included in the U.S. Fish and Wildlife Service's MaxEnt analyses, although it would be an important exercise to measure the impacts of GNN prediction error and alternative smoothing window sizes (larger and smaller than 200 ha units around each pixel) on any models' prediction of habitat suitability. We also obtained maximum likelihood estimates for probabilities of presence using the same data. Maximum likelihood estimates were obtained using the Maxlike package in R Statistical Sofware (R Development Core Team, 2012) developed by Richard Chandler and Andy Royle of the U. S. Geological Survey (USGS). Data layers were standardized by subtracting the mean and dividing by the standard deviation of each of the layers to make it easier for maximum likelihood methods to converge. We did not examine the question of whether the variables used for the MaxEnt model are most suitable, although that could be considered in the framework of maximum likelihood estimation and can be important in defining the importance of abiotic and biotic variables on the likelihood of spotted owl locations. This information could be useful in determining biological definitions of critical habitat to owl survival and estimation of population impact due to changes in vegetative conditions. Although MaxEnt gives an estimate of the percent contribution a certain variable makes to habitat suitability these impacts are from a 'black box' algorithm and are not directly quantifiable as with maximum likelihood estimation. In our model comparison, we used the same variables as used for the U.S. Fish and Wildlife Service analysis for both our MaxEnt and maximum likelihood analysis to illustrate differences in model output rather than variable selection procedures. We first considered the data for modeling region 7, the Southern Western Cascades. For this region the estimated probabilities of an owl being present were calculated for both the MaxEnt and maximum likelihood models. Figure 3.1 shows the MaxEnt estimates for Northern Spotted Owl Modeling Page 5 of 13 6 July 2012

6 10,000 randomly selected points plotted against their respective maximum likelihood estimates obtained using Maxlike. This shows that there is some relationship between the MaxEnt habitat suitability values and the maximum likelihood estimates of the probability of owls being present, but that this relationship is far from being strong. It appears that if the probability of an owl being present is 0.2 or less then the MaxEnt habitat suitability index can be anything within a range from 0 to over 0.6, while if the MaxEnt habitat suitability index is over 0.2 then the probability of a presence can be anything between about 0.2 to 1.0. Because of the rather vague relationship between the maximum likelihood estimated probabilities of owls being present and the MaxEnt habitat suitability indices these two measures give very different maps for the habitat in the Southern Western Cascades, as shown in Figure 3.2. In particular, the maximum likelihood map shows clear areas of very high probability of a presence but these are not separated from areas with lower probabilities of a presence in the MaxEnt map. Following the MaxEnt and maximum likelihood analyses for Southern Western Cascades, the same analyses were carried out with the U.S. Fish and Wildlife Service data from District 8, Klamath East, to see whether similar differences were found between the results for the two methods of estimation. Figure 3.3 shows the relationship between the MaxEnt habitat suitability indices and maximum likelihood estimates of the probability of an owl being present for the 10,000 randomly selected grid cells in Eastern Klamath. As we expected, the pattern is quite similar to what was found for the Southern Western Cascades, with low to moderately high habitat suitability values occurring when the estimated probability of a presence is quite low while the habitat suitability values vary from being quite low to very high when the probability of a presence is close to one. As a result the maps of habitat suitability for Eastern Klamath in Figure 3.4 again show that the regions with very high probabilities of a presence according to the maximum likelihood model are merged with lower probability areas by the MaxEnt model. 4. Conclusions Based on the results in the Royle et al. (2012) and Torres et al. (2012) papers, the potentially large errors in the GNN data, and the differences that we find between MaxEnt and maximum likelihood estimation applied to the U.S. Fish and Wildlife Service data from the Southern Western Cascades and Klamath East we believe that the results obtained from a MaxEnt analysis may be misleading for habitat modeling in general, and are unreliable with habitat modeling for northern spotted owls in particular. Also, as the U.S, Fish and Wildlife Service analysis used the output from MaxEnt for further analyses using the Zonation and Hexsim programs we believe that the results from these further analyses are also not reliable. We also suggest that there is further work that could be done to verify the superiority of the Royle et al. maximum likelihood estimation method over the methods used for Northern Spotted Owl Modeling Page 6 of 13 6 July 2012

7 MaxEnt. This would involve first assuming that the estimated MaxEnt model for northern spotted owls in the Southern Western Cascades is correct. A number of sets of data could then be simulated using this model and analyzed using MaxEnt and maximum likelihood. Next the estimated maximum likelihood model for the Southern Western Cascades could be assumed to be correct and a number of simulated sets of data produced using this model and analyzed using MaxEnt and maximum likelihood. The best estimation method would clearly then be the one that gives estimated parameters closest to those used to simulate the data for both the MaxEnt and maximum likelihood models. Because of the time needed to carry out these simulations and analyses it has not been possible to do them at this time. We further recommend that additional research be carried out to determine the appropriateness of model input data based on GNN error estimation and smoothing window size. Any modeling effort would be impacted by the validity of this input data and estimated habitat suitability measures should account for the uncertainties in the input data layers. References Moilanen, A. and Kujala, H. (2008). Zonation: Software for Spatial Conservation Prioritization. User Manual v2.0. Metapopulation Research Group, University of Helsinki, Finland. R Development Core Team (2012). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN , URL Royle, J.A., Chandler, R.B., Yackulic, C and Nichols, J.D. (2012). Likelihood analysis of species occurrence probability from presence-only data for modeling species distributions. Methods in Ecology and Evolution 3: Schumaker, N.H. (2008). HexSim (Version ). U.S. Environmental Protection Agency, Environmental Research Laboratory, Corvallis, Oregon. Torres, N.M., P. De Marco Junior, T. Santos, L. Silveira, A.T. de Almeida Jacomo and J.A.F. Diniz-Filho (2012). Can species distribution modeling provide estimates of population densities? A case study with jaguars in the neotropics. Diversity and Distributions 18: U.S. Fish and Wildlife Service (2011). Revised Recovery Plan for the Northern Spotted Owl (Strix occidentalis caurina). U.S. Fish and Wildlife Service, Portland, Oregon. Northern Spotted Owl Modeling Page 7 of 13 6 July 2012

8 Figure 1.1 The example of the accuracy of the GNN predicted basal area of trees from the 2006 GNN model covering Western Washington for the Northwestern Forest Plan project. The GNN predicted basal areas for plots are plotted against the observed basal areas. Northern Spotted Owl Modeling Page 8 of 13 6 July 2012

9 Figure 2.1 A copy of Figure 4 from the Royle et al. (2012) paper showing the Carolina wren distributions estimated from logistic regression (using presence and absence data), maximum likelihood (Maxlike) and Maxent. Northern Spotted Owl Modeling Page 9 of 13 6 July 2012

10 Figure 3.1 The relationship between MaxEnt estimates of habitat suitability and maximum likelihood estimates of the probability of a presence for the Southern Western Cascades. Northern Spotted Owl Modeling Page 10 of 13 6 July 2012

11 Figure 3.2 Habitat maps for the Southern Western Cascades based on the MaxEnt model and the maximum likelihood model. Northern Spotted Owl Modeling Page 11 of 13 6 July 2012

12 Figure 3.3 The relationship between MaxEnt estimates of habitat suitability and maximum likelihood estimates of the probability of a presence for Eastern Klamath. Northern Spotted Owl Modeling Page 12 of 13 6 July 2012

13 Figure 3.4 Habitat maps for the Eastern Klamath based on the MaxEnt model and the maximum likelihood model. Northern Spotted Owl Modeling Page 13 of 13 6 July 2012