Between-observer variation in the application of a standard method of habitat mapping by environmental consultants in the UK

Transcription

1 Journal of Applied Ecology 1999, 36, Between-observer variation in the application of a standard method of habitat mapping by environmental consultants in the UK ANDREW CHERRILL and COLIN MCCLEAN* Ecology Centre, School of Science, Science Complex, University of Sunderland, Sunderland SR1 3SD, UK; and *Environment Department, University of York, Heslington, York YO10 5 DD, UK Summary 1. In the UK, Phase 1 survey is a standard method of habitat mapping that has been used widely for environmental assessment and management planning. In this paper we make the rst rigorous test of the precision with which environmental consultants apply the technique. 2. Six ecologists surveyed independently the same upland site in northern England. In pairwise comparisons between maps, spatial agreement was found to average 25 6% (with a range of 17 3±38 8%) of the area of the study site. The numbers of land cover types that were identi ed ranged from 13 to 21. Four or more surveyors agreed on the classi cation of 19% of the study site, while the area of land upon which all six agreed was only 7 9% of the study site. Spatial errors in the positioning of habitat boundaries occurred, but were a relatively minor source of the di erences between maps. The majority of di erences between maps were due to classi cation errors. Land cover types with similar species compositions were most frequently confused. 3. Spatially referenced eld `target notes' giving additional information on the vegetation mapped in each survey varied in number between 18 and 56. The contents of target notes were inadequate to allow a retrospective assessment of mapping decisions. The total numbers of species listed in target notes varied between surveys from 25 to 145. Sorenson's similarity for species lists derived from pairs of surveys ranged from 18 8% to 63 7%, and was not related to spatial agreement between surveys. 4. Time spent at the eld site was not a correlate of any aspect of the results or cost of the survey. Three surveys conducted by members of a professional institute for ecologists were the most expensive, and also recorded larger numbers of target notes and species than the other surveys. However, their maps were no more similar than other pairs of maps. 5. Analysis of the survey results and comparisons with other methods of vegetation mapping suggest that mapping precision could be increased by (i) placing a greater emphasis on use of aerial photographs and other extant map data prior to (and during) eld work; (ii) making greater provision for mapping of mosaics and increasing the level of oristic information in habitat de nitions; (iii) recording a greater number of more detailed target notes in the eld; and (iv) providing o cebased support to assist in the interpretation of aerial photographs, and the crosschecking of eld surveyors' preliminary classi cations against the contents of target notes and habitat de nitions. The current application of the Phase 1 approach by environmental consultants places too great a reliance on decision-making by the (frequently) unsupported lone surveyor whilst in the eld. Correspondence: Andrew Cherrill ( andrew.cherrill@sunderland.ac.uk).

2 990 Variation between surveyors in habitat mapping Key-words: classi cation errors, Phase 1 survey, quality control, spatial errors, vegetation mapping. Ecology (1999) 36, Introduction Mapping of biological resources is a valuable tool in aiding the identi cation of conservation priorities, and has an important role in monitoring and planning to minimize the deleterious impacts of projects and policies (Harding 1992, 1994; Morris 1995). However, all databases (spatial and otherwise) are likely to contain errors, and it is important that the scale and nature of these errors are known (Allen 1981; Harding 1994; Rich & Woodru 1995; Williams 1996). This paper therefore focuses on the reliability of a method of vegetation mapping that has been used extensively in Britain. Comparisons with alternative methods are drawn and recommendations made that are likely to be widely applicable. Vegetation mapping is intimately linked with vegetation classi cation. Kuchler (1967) identi es two types of interaction. In the rst, eld data on a series of stands are obtained, and these data are used to construct a novel classi cation that is particularly appropriate for the mapping of that locality. In the second, an extant classi cation is available before eld work commences and vegetation is classi ed within the existing framework. This approach ensures that surveys in di erent regions and years use a common framework, enhancing communication and comparative analyses (Kuchler 1967, 1988a). The application of extant classi cations is of particular value in conservation and land use planning, where an appreciation of the overall distribution of natural resources is required (Uhlig & Jordan 1996; Banner et al. 1996; Smith & Carpenter 1996). In Britain, two comprehensive national vegetation classi cations, the National Vegetation Classi cation (Rodwell 1991) and Phase 1 system (Nature Conservancy Council 1990), have been developed and applied widely for these reasons. Phase 1 survey forms the focus of this paper. Classes in the Phase 1 classi cation are de ned on the basis of a range of factors, including land use, soils, physiognomy of vegetation and the presence of species characteristic of particular edaphic conditions (Nature Conservancy Council 1990). Because of the range of criteria used in the Phase 1 classi cation, the classes are arguably best referred to as land cover types, rather than vegetation types. None the less, the aims of Phase 1 survey are to identify sites of potential wildlife conservation value and those requiring more detailed survey. County-wide Phase 1 surveys have proved useful in the setting of conservation priorities, and are also recommended as a basis for the initial assessment of the impact of planning applications (Wyatt 1991; Institute of Environmental Assessment 1995; Morris 1995). Recently, Cherrill & McClean (1995) reported a study in which an area of approximately 25 km 2 was surveyed independently by two organizations. Errors in the classi cations in one or both maps were found to account for at least 41% of the total area. The result emphasized the need for incorporating quality control measures in the design of surveys, but also the need for further work to assess the extent of errors in Phase 1 data. The study can be seen against a background of increasing concern that the quality of scienti c (and particularly ecological) input to environmental assessment and planning is often inadequate in the UK (Coles 1993; Treweek 1996; Thompson, Treweek & Thurling 1997). Similar concerns have also been raised in Australia (Warnken & Buckley 1998), Canada (Beanlands & Duinker 1984) and North America (Reichhardt 1999). The present study compares six Phase 1 vegetation maps of an upland area in England. Five of the six surveys were conducted by professional consultants. The causes of di erences between surveys are identi ed, and comparisons between the Phase 1 methodology and alternative techniques used in Europe and North America (Kuchler 1967; Kuchler & Zonneveld 1988) are used to suggest recommendations, which it is hoped will signi cantly enhance the objectivity of vegetation mapping in the UK. Methods THE PHASE 1 CLASSIFICATION The major divisions in the classi cation are `woodland and scrub', `grassland and marsh', `tall herb and fern', `heathland', `bog', `swamp, marginal and inundation', `open water', `coastland', `rock exposure and waste' and `miscellaneous' (incorporating boundary features and highly arti cial land cover types such as buildings, arable crops and amenity grasslands). Within each division there are subdivisions, which in many cases are further subdivided to give a hierarchical structure to the classi cation. The Phase 1 land cover types are de ned primarily on the basis of the dominant and characteristic plant species, but for certain land cover types this information is supplemented by soil and land use characteristics. `Woodlands', for example, are subdivided into `broadleaved', `coniferous' and `mixed woodlands'. Each of these subdivisions is then split into `semi-natural' and `plantation' land cover types,

3 991 A. Cherrill & C. McClean re ecting land use considerations. Depth of peat is a factor in the de nition of `mire' land cover types, with all `mires' occurring on peat deeper than 0 5 m. Certain grasslands are divided into `acid', `neutral' and `basic' types, although soil conditions are inferred from the presence of indicator species rather than direct measurement of soil ph. Degree of agricultural improvement is a key factor in the classi cation of bogs and grasslands. For example, `blanket bogs' that have been damaged by heavy grazing, burning and drainage, and which consequently have an altered species composition, are classed as `modi ed'. In the classi cation of grasslands, `unimproved' types have not been sown and are dominated by native species in mixtures determined by soil, climate and traditional low-intensity grazing or cutting. `Improved grassland' and `poor semi-improved grassland' are those which have been heavily in uenced by drainage, fertilizers, herbicides and sowing of species of agricultural value. The latter are marginally more species-rich that the former, but are of low potential conservation value. `Acid', `neutral' and `basic semi-improved' grasslands form a transitional group that has been in uenced signi cantly by agricultural practices, but which retains species that are characteristic of `unimproved grassland' and soils of the area. Descriptions of the Phase 1 land cover types are provided by the Nature Conservancy Council (1990). PHASE 1 FIELD METHODOLOGY Surveyors identify areas of relatively homogeneous vegetation and assign each `parcel' to one of the land cover types in the classi cation whilst on site. Boundaries often follow arti cial linear features, but in open unenclosed areas, and where there is variation within elds, surveyors draw their own boundary lines. Aerial photographs are used where available, particularly for locating complex boundaries that are not marked by arti cial boundaries. Land cover types are mapped on to 1 : Ordnance Survey (OS) maps in the eld using standard colour or alphanumeric codes. There is no requirement for soil survey in the methodology, although a knowledge of local soils is advantageous. Surveyors are advised to familiarize themselves with existing information (e.g. soil maps) before commencing work in the eld. In large-scale surveys involving teams of botanists, training in groups and working in pairs is recommended to standardize recording. Information on mapped land cover types and interesting features smaller than the minimum mappable area (0 1 ha at 1 : ) is recorded in `target notes' linked to numbered dots on the map. Target notes may include information on habitat type, dominant plant species, other noteworthy species (e.g. those of conservation value), site management and desirability of further survey. The number of target notes required is not speci ed in the Phase 1 manual, although in general as many as possible should be recorded in the time available (Nature Conservancy Council 1990). On completion of eld work, target notes are typed up and a nal neat version of the eld map is prepared. SELECTION OF SURVEYORS Eight environmental consultancy companies known to undertake Phase 1 survey were contacted and asked to submit an estimate for a Phase 1 survey of the study site (for which a map and brief description were provided). From the replies, ve consultants (responsible for surveys A±E) were selected to encompass the range of estimates (Table 1). In addition to the ve professional consultants, an individual with Phase 1 experience, currently employed by a governmental conservation agency, agreed to carry out a sixth survey (survey F) (Table 1). For brevity, the surveyors and surveys for which they were responsible will be referred to by the same letter (hence surveyor A completed survey A, and so on). Three of the surveys (A, C and D) were conducted by individuals who were members of the Institute of Ecology and Environmental Table 1. The employment status of surveyors, membership of the Institute of Ecology and Environmental Management (IEEM), survey e ort and the cost of each survey Survey e ort Cost of survey Surveyor(s) employed Surveyor(s) IEEM Number of Survey by consultancy member surveyors Days Hours A Yes Yes B Yes No C Yes Yes D Yes Yes E Yes No F No No

4 992 Variation between surveyors in habitat mapping Management (IEEM), which represents the interests of professional ecologists in the UK. Although IEEM has no statutory powers, members share the aim of improving the quality of environmental assessments. Membership is based on peer-review of experience and quali cations. ORGANIZATION OF THE FIELD SURVEYS Surveyors were asked to use the standard Phase 1 approach (Nature Conservancy Council 1990). The report was speci ed to include a colour-coded 1 : scale Phase 1 map, target notes and summaries identifying areas of potential conservation value requiring further survey (reported elsewhere), and any problems encountered in carrying out the eld survey. The study site, of approximately 4 km 2, was at the ADAS Experimental Husbandry Farm, Redesdale, Northumberland (Grid Reference NY8396), and included land rising from the banks of the River Rede to an altitude of 350 m a.s.l. Surveyors had free access to all parts of the farm. A range of land uses was present, including intensi ed (i.e. `improved') and unintensi ed (i.e. `unimproved') grazing, and several broadleaved and coniferous woodlands. A small number of additional fences, not shown on the OS map (Fig. 1), were present, but the study site was characterized by large enclosures (particularly in the south). Field surveys were conducted between 1 July and 1 August Survey dates were arranged so that surveyors would not be present on the site at the same time. On one day this system failed and two surveyors (B and C) were on site at the same time. Surveyors were required to sign a visitors' book, before entering or leaving the study site. This provided information on survey duration (Table 1). Colour aerial photographs of the site taken in 1995, and an o ce to facilitate their use, were made available to the surveyors. PREPARATION OF FIELD MAPS FOR ANALYSIS Surveyors' maps were compared with each other using a Geographical Information System (GIS). All digitizing was performed by a single operator using the ARC/INFO GIS. Boundaries on the OS map used as a basis for mapping by all surveyors were digitized to produce a standard `base map' (Fig. 1). This map was then copied to produce six identical base maps. Individual surveyor's maps were registered to the base map and additional boundaries drawn by the surveyors were digitized. Parcels of land were labelled with a code representing their Phase 1 land cover types. In the case of the Phase 1 land cover type `scattered trees', parcels were coded according to the type of vegetation within which the trees occurred. These non-standard Phase 1 cover types have been allocated codes pre xed A.3. Copies of the original Phase 1 maps were made, and in these new maps the land cover types were aggregated into a smaller number of broad land Fig. 1. The base map of the study site showing information available to all surveyors. The eld marked X is referred to in the text (as Field X).

5 993 A. Cherrill & C. McClean cover categories. In the new map, boundaries between parcels sharing the same land cover category code were dissolved. The list of land cover categories re ected the hierarchical nature of the Phase 1 classi cation of cover types, but also the experience of the authors (Cherrill & McClean 1995). Thus, for example, recently sown grass leys, `improved grassland' and `poor semi-improved grassland' were grouped in the `pasture/silage' category. Nature Conservancy Council (1990) classi es recently sown grass leys as `arable', rather than grassland, but leys were included in the `pasture/ silage' category because they are dominated by sown species that are also abundant in `improved' and `poor semi-improved grassland'. Copies of the maps of the original and aggregate classi cations were made and modi ed by adding line bu ers. Bu ering involved enclosing the land lying within 25 m of each boundary by the addition of arcs running parallel to those already present. The 50-m wide `bu er zones' were eliminated from certain of the analyses described below. This approach re ects the acceptance that eld surveyors could not be expected to mark boundaries on their eld maps with absolute accuracy. Moreover, it is accepted that many boundaries are not distinct, but are better regarded as zones of change (or ecotones) between related vegetation types. Bu ering enabled a comparison of the cover types assigned to the `core areas' of parcels of land, thereby minimizing any subtle spatial di erences between maps in the location of boundaries. The choice of a 25-m bu ering distance was ultimately subjective, but re ected the authors' prior analyses using a range of alternative distances (A. Cherrill & C. McClean, unpublished data), the resolution of Phase 1 mapping at 1 : and the authors' (albeit subjective) interpretation of the width of ecotones present at the study site. All boundaries were bu ered, except those which were present in the original base map. Areas lying within bu er zones were labelled with a new code. The unbu ered maps containing the maps of the original and aggregate classi cations, plus their buffered copies, were stored as separate layers within the GIS, resulting in four separate maps for each survey. AGREEMENT BETWEEN PAIRS OF MAPS The agreement between maps was quanti ed by overlaying them within ARC/INFO as follows: (i) pairwise comparison of the unbu ered maps of the original land cover types; (ii) pairwise comparison of the bu ered maps of the original land cover types; (iii) pairwise comparison of the unbu ered maps of the aggregate land cover categories; and (iv) pairwise comparison of the bu ered maps of the aggregate land cover categories. The results of each overlay were presented as a matrix of correspondence in which the (M i,n j ) th cell gave the area of cover type i in map M classed as cover type j in map N. In each matrix, cover types were placed in the same order along rows and columns. The percentage of land that lay within the diagonal cells of each matrix was used as an index of the extent of the agreement between pairs of maps. Construction of matrices of correspondence was identical for all overlays, except that where bu ering had been used, land lying within a bu er zone in either (or both) maps was excluded. Agreements for pairs of unbu ered maps were expressed as a percentage of the total area included in the overlay analysis. In contrast, the area of agreement for each pair of bu ered maps was expressed as a percentage of the land lying outside of the bu er zones in those maps (rather than as a percentage of the site as a whole). For each pair of surveyors' maps, four estimates of overall agreement were calculated from the comparisons detailed in (i), (ii), (iii) and (iv) above. In addition, four estimates of agreement for a single eld (referred to as Field X, and shown in Fig. 1) were also calculated from these overlays. Field X has an area of 0 27 km 2 and is reverting to semi-natural grassland, having been intensi ed in the past. AGREEMENT BETWEEN ALL SIX MAPS The overall agreement between the six surveys was assessed by combining all six maps. This was done for maps containing unaggregated and aggregate land cover classi cations, with and without bu ers, such that four new maps were created. In each of the new maps, all land parcels had six land cover codes (one from each of the six maps). In analyses involving the bu ered maps, areas contained within a bu er zone in one (or more) of the map layers were excluded from the analysis. The number of occurrences of each code within each parcel was then computed. Each parcel was relabelled with the maximum number of occurrences recorded for any cover type occurring within that parcel. The total areas labelled 1 (agreement between none of the maps), 2, 3, 4, 5 and 6 (agreement between all six maps) were calculated for each of the four new maps. AGREEMENT FOR INDIVIDUAL LAND COVER TYPES Agreement for individual classes in a classi cation could be calculated from a matrix of correspondence in two ways, di ering in whether the marginal row

6 994 Variation between surveyors in habitat mapping or column total is used as a basis for interpretation of the gure in the diagonal cell (Story & Congalton 1986). In the present study, map M was compared to ve others, resulting in ve matrices of correspondence. Five estimates of the agreement of cover type i were therefore calculated by expressing the ve absolute areas of agreement for i (i.e. one from each matrix) as a percentage of the total area of i in the reference map M. Identical calculations were also performed using each of the other ve maps as the reference map. These estimates for cover type i were then summarized as median and range for each reference map (i.e. n = 5 in each case). TARGET NOTES The numbers and contents of target notes in the six maps were compared (i) at the level of the whole study site (by pooling information across target notes within each survey), and (ii) for the single eld referred to as Field X (Fig. 1). Comparisons for the whole study site The contents of target notes were quanti ed by recording the numbers and identities of plant species in each note. Use of measures of species abundance was noted. The identities of plant species recorded in target notes were tabulated in a single list for each survey. Sorenson's similarity index was calculated for pairs of lists derived from di erent surveys (Southwood 1966). Comparisons for a single eld An important consideration in this study was the extent to which the contents of target notes could be used to understand the decisions made by surveyors in assigning parcels of land to a particular land cover type. Target notes for Field X were therefore compared between surveys and cross-referenced with the Phase 1 land cover de nitions. To illustrate this analysis, extracts from the Phase 1 manual and the surveyors' target notes are quoted. In these quotations, and throughout the paper, nomenclature follows Stace (1991). SURVEY EFFORT Surveyors were required to sign a visitors' book on arrival and departure at the study site. The number of hours spent on site and the numbers of days on which visits were made were each used as estimates of survey e ort. These gures were also multiplied by the number of surveyors to give two further estimates of survey e ort. It was not possible, however, to distinguish time spent in the eld from that examining the aerial photographs in the farm o ce. Several surveyors borrowed the photographs overnight and this was noted. PROBLEMS ENCOUNTERED DURING SURVEYS Surveyors were asked to include a section in their reports identifying problems encountered in carrying out the eld survey. Further information was also extracted from the target notes. Results ASPATIAL ANALYSES Comparisons for the whole study site In total, 35 land cover types were recorded by the six surveys. The number of land cover types recorded in individual maps ranged from 13 to 21 (Table 2). Each survey also included small areas of land that were not classi ed. The land cover types with the greatest areas within individual maps were `wet dwarf shrub heath/acid grassland mosaic' (maps A and B), `marshy grassland' (map C), `dry modi ed bog' (map D), `dry dwarf shrub heath/acid grassland mosaic' (map E) and `poor semi-improved grassland' (map F). Of these cover types, only `marshy grassland' was recorded in all six maps (with areas ranging from 0 3% to 23 0% of the study site). The other three `dominant' cover types were each recorded in three maps only. Overall, six land cover types were recorded from all maps. A further four were recorded from ve maps. These gures included the cover types `built' (buildings, roads and unmetalled tracks), `broadleaved semi-natural woodland' and `conifer plantation', which were shown on the original base map (Fig. 1). Combining the 35 land cover types into 12 broader categories increased the consistency of mapping, with eight land cover categories being recorded in at least ve maps. However, some large di erences between maps remained. For example, the areas of `heathland' and `mire' ranged from 0% to 46 4% and 0% to 39 6%, respectively, while the area of `pasture/silage' varied threefold, from 12 2% to 36 7% (Table 2). Comparisons for a single eld The six surveyors identi ed seven land cover types, but four of these were found in a single map only. `Marshy grassland' alone was recorded in more than two surveys (Table 3). The land cover types with the greatest areas within individual maps were `semiimproved acid grassland' (map A), `semi-improved neutral grassland' (maps B and D), `marshy grassland' (maps C and E) and `poor semi-improved

7 995 A. Cherrill & C. McClean Table 2. The areas of land cover types recorded in each surveyor's map as a percentage of the study site's total area. Land cover types are grouped under 13 broad categories (+ = percentage area is less than 0 05). The codes are from Nature Conservancy Council (1990) Map Phase 1 land cover types (and codes) A B C D E F Broadleaved woodland Broadleaved semi-natural woodland (A ) Broadleaved plantation woodland (A ) Felled broadleaved woodland (A.4 1.) Sub-total Coniferous woodland Coniferous semi-natural woodland (A ) Coniferous plantation woodland (A ) Sub-total Mixed woodland Mixed semi-natural woodland (A ) Mixed plantation woodland (A ) Sub-total Scattered trees Trees in improved grassland (A.3.1.) Trees in semi-natural neutral grassland (A.3.2.) Trees in marshy grassland (A.3.3.) Trees in unclassi ed land cover type (A.3.4.) Sub-total Rough grassland Unimproved acid grassland (B.1.1.) Semi-improved acid grassland (B.1.2.) Unimproved neutral grassland (B.2.1.) Semi-improved neutral grassland (B.2.2.) Marshy grassland (B.5.) Sub-total Pasture/silage Arable (grass ley) (J.1.1.) Improved grassland (B.4.) Poor semi-improved grassland (B6.) Sub-total Tall herb and fern Bracken (C.1.1.) Ruderal tall herbs (C.3.1.) Non-ruderal tall herbs (C.3.2.) Sub-total Heathland Dry acid dwarf shrub heath (D.1.1.) Wet acid dwarf shrub heath (D.1.2.) Dry dwarf shrub heath/acid grass mosaic (D.5.) Wet dwarf shrub heath/acid grass mosaic (D.6.) Sub-total Mire Blanket bog (E ) Wet modi ed bog (E.1.7.) Dry modi ed bog (E.1.8.) Acid/neutral ush (E.2.1.) Sub-total Swamp Swamp (F.1.) Rock exposure Acid/neutral natural rock exposure (I ) Quarry (I.2.1.) Mining Spoil (I.2.2.) Sub-total Built Built (buildings, metalled roads, etc.) (J.3.6.) Unclassi ed land Unclassi ed land Total

8 996 Variation between surveyors in habitat mapping Table 3. The areas of land cover types recorded in Field X in each surveyor's map as a percentage of the eld's total area. Land cover types are grouped under the broad categories shown in Table 2. The codes are from Nature Conservancy Council (1990). Map Phase 1 land cover types (and codes) A B C D E F Rough grassland Semi-improved acid grassland (B.1.2.) Semi-improved neutral grassland (B.2.2.) Marshy grassland (B.5.) Sub-total Pasture/silage Improved grassland (B.4.) Poor semi-improved grassland (B.6.) Sub-total Tall herb and fern Bracken (C.1.1.) Mire Acid/neutral ush (E.2.1.) Total grassland' (map F). Aggregating land covers into broader categories resulted in a substantial increase in the consistency of mapping, with ve of the six maps agreeing that over 90% of the eld was `rough grassland' (Table 3). SPATIAL AGREEMENT BETWEEN PAIRS OF MAPS Comparisons for the whole study site Agreement between the Phase 1 land cover type maps was low (Table 4). The maximum agreement between any pair of maps was 38 8%, while the lowest agreement was 17 3%. When land cover types were aggregated into broad land cover categories, the mean agreement between maps was increased from 25 6% to 56 4% for unbu ered maps (paired t-test, t = 10 52, P < ), and from 27 7% to 59 1% for bu ered maps (paired t-test, t = 9 83, P < ). Adding bu ers raised the mean agreement from 25 6% to 27 7% for the original land cover maps (paired t-test, t = 7 44, P < ), and from 56 4% to 59 1% for maps of broad land cover categories (paired t-test, t = 8 56, P < ). With the application of both techniques the greatest agreement achieved between any pair of maps was Table 4. The spatial correspondence between pairs of surveyors' maps assessed using percentage agreement (%). Correspondence was assessed for maps with and without bu ering of boundaries, and for maps with and without aggregation of land cover types into broad categories Maps of unaggregated land cover types Maps of aggregated land cover types Pair of maps Unbu ered maps (%) Bu ered maps (%) Unbu ered maps (%) Bu ered maps (%) A±B A±C A±D A±E A±F B±C B±D B±E B±F C±D C±E C±F D±E D±F E±F Mean SD

9 997 A. Cherrill & C. McClean Table 5. The spatial correspondence between pairs of surveyors' maps for Field X assessed using percentage agreement (%). Correspondence was assessed for maps with and without bu ering of boundaries, and for maps with and without aggregation of land cover types into broad categories Maps of unaggregated land cover types Maps of aggregated land cover types Pairs of maps Unbu ered maps (%) Bu ered maps (%) Unbu ered maps (%) Bu ered maps (%) A±B A±C A±D A±E A±F B±C B±D B±E B±F C±D C±E C±F D±E D±F E±F Mean SD % (Table 4). Aggregation of similar land cover types was much more e ective at increasing agreement than the use of bu ers. Comparisons for a single eld The maximum agreement between any pair of maps for Field X was 92 3%, but in 10 out of 15 pairwise comparisons spatial agreement was zero (Table 5). Aggregation of land cover types resulted in signi cant improvements in agreement for both unbuffered (paired t-test, t = 5 10, P < ) and bu ered maps (paired t-test, t = 4 78, P < ). However, adding bu ers had little e ect on agreement for either the original maps (paired t-test, t = 0 28, P = 0 78) or maps of broad aggregated land cover categories (paired t-test, t = 1 89, P = 0 08) (Table 5). In some cases, agreement was reduced by the addition of bu ers. Visual inspection of the maps for this eld indicated that di erences in the location of boundaries were so large that they could best be attributed to classi cation errors, rather than di ering perceptions of the same ecotones (Fig. 2). SPATIAL AGREEMENT BETWEEN SIX MAPS Overlay of the six maps revealed that the area of land that was classi ed as the same land cover by all surveyors was only 7 9% of the total study area (Table 6, Fig. 3). This gure included the areas of roads, tracks and buildings that were shown on the base map (Fig. 1). Other land cover types contributing to the agreement between all six maps were `improved grassland' and `broadleaved semi-natural woodland' (representing 3 9% and 2 5% of the study area, respectively). These cover types were found in the north of the study site, while in the south there was less agreement between maps on the classi cation of land used for rough grazing (Figs 1 and 3). Aggregating Phase 1 land cover types into broad land cover categories raised the area of overall agreement to 17 8% (again including an area of 1 5% representing the `built' land cover type). Land cover categories contributing to the agreement were `pasture/silage', `rough grassland' and `broadleaved woodland' (contributing areas equivalent to 10 1%, 2 6% and 3 6% of the study area) (Fig. 4). In Table 6 the detailed e ects of aggregating land cover types and introducing bu ers can be seen on the pattern of agreement between the six maps. Thus, for example, the area of land upon which four or more surveyors agreed was 19 0% of the study site, while after aggregation of land cover types this gure rose to 75 2%. Excluding land that lay within 25 m of a boundary in one or more map reduced the areas included in the analyses of the bu ered maps (Table 6). However, as seen in earlier analyses, introduction of bu ers had relatively little e ect on levels of agreement, indicating that only a relatively small proportion of mapping errors was closely associated with boundaries. Even after bu ering of maps of aggregate land cover categories, the area of complete agreement between the six maps was equivalent to only 22 7% of the land lying outside of bu er zones, although four or more surveyors were

10 998 Variation between surveyors in habitat mapping Fig. 2. Phase 1 land cover types recorded in Field X by surveys A, B, C, D, E and F. The locations of target notes are indicated by the solid triangles (numbers refer to Table 11). in agreement on the classi cation of 80 6% of this area (Table 6). SPATIAL AGREEMENT FOR INDIVIDUAL LAND COVER TYPES Estimates of spatial agreement for individual land cover types varied according to which map was used as the reference data set (Table 7). The analyses also showed wide variation in agreement for individual cover types when di erent maps were compared to the same reference map. Thus, for example, when map F was used as the reference data set, agreement for `conifer plantation' varied from 0% to 38 2% with a median of 21 5%. However, when map E was used as the reference data set, a median gure of 85 1% agreement was recorded for the same land cover type (Table 7). With few exceptions there was low correspondence between surveys for individual cover types.

11 999 A. Cherrill & C. McClean Table 6. The overall agreement between six maps assessed as the area of land (as percentage) for which two or more surveys agreed upon the classi cation of land cover Maps of unaggregated land cover types Maps of aggregated land cover types Number of maps agreeing Unbu ered maps (%) Bu ered maps (%) Unbu ered maps (%) Bu ered maps (%) Area included in analysis (km 2 ) Consistently high levels of agreement between maps for individual land cover types were achieved for `improved grassland', `broadleaved semi-natural woodland' and `built' land cover types only (Table 7). Several blocks of woodland were shown on the base map (Fig. 1). However, there was some disagreement as to their composition (`broadleaved', `mixed' or `coniferous') and origin (`semi-natural' or `plantation'). Agreement was, not surprisingly, greatest for the `built' land cover type. Where the spatial agreement for `built' deviated from 100% this re ected the small areas of hard surfaces and outbuildings that were added to the base map by three surveyors. Combining land cover types into broad land cover categories generally resulted in increased estimates of agreement (Table 8). None the less, the e ect of combining cover types into cover categories was variable. The greatest contrast was seen for the land cover categories `heathland' and `mire'. Before aggregation `heathland' and `bog' cover types exhibited low (and typically zero) median agreements (Table 7). However, after aggregation the new category `heathland' showed greatly elevated levels of agreement, while no improvement was seen for the `mire' category (Table 8). TARGET NOTES Comparisons for the whole study site The number of target notes associated with each survey varied between 18 and 56 (Table 9). The content of most target notes was restricted to a brief list of species, with subjective assessments of the abundance of subsets of these. Within each report, the numbers of species recorded varied between target notes; few target notes listed a large number of spe- Fig. 3. The areas of land upon which di erent numbers of survey maps agreed in terms of classi cation to Phase 1 land cover type.

12 1000 Variation between surveyors in habitat mapping Fig. 4. The areas of land upon which di erent numbers of survey maps agreed in terms of classi cation to broad aggregate land cover category. Table 7. Median percentage agreements (with minimum and maximum values in parentheses) for individual Phase 1 land cover types using each map as a reference for comparison with the other ve (n = ve in each case, ± = cover type absent from reference map) Reference map Land cover type A B C D E F Broadleaved semi-natural woodland (76 5±93 8) (60 6±89 3) (64 2±92 3) (89 1±98 4) (83 5±94 3) (62 4±94 9) Broadleaved plantation 15 4 ± (0 0±87 9) (0 0±83 4) (0 0±99 5) (0 0±84 5) (0 0±100 0) Felled broadleaved woodland ± ± ± 0 0 ± ± Coniferous semi-natural woodland ± 0 0 ± ± ± ± Coniferous plantation ± (0 0±100 0) (0 0±100 0) (0 0±100 0) (0 0±85 1) (0 0±38 2) Mixed semi-natural woodland 0 0 ± ± ± ± ± Mixed plantation ± ± (0 0±55 1) (0 0±93 9) (0 0±80 5) (0 0±100 0) Trees in improved grassland ± ± ± 0 0 ± ± Trees in semi-improved neutral grassland ± ± 0 0 ± ± ± Trees in marshy grassland ± ± ± ± Trees in unclassi ed land ± ± ± ± Unimproved acid grassland ± (0 0±37 9) (0 0±31 8) (0 0±62 8) (0 0±38 9) (0 0±43 2) Semi-improved acid grassland (0 0±24 4) (6 1±41 5) (0 0±62 3) (0 1±27 9) (0 1±48 8) (4 1±43 9) Unimproved neutral grassland ± ± 0 0 ± Semi-improved neutral grassland ± (0 0±70 8) (0 0±68 9) (0 0±80 0) (0 0±47 3) (0 0±100 0) Marshy grassland (0 0±10 6) (0 0±100 0) (0 2±30 4) (0 0±62 6) (0 1±64 7) (0 0±37 4)

13 1001 A. Cherrill & C. McClean Table 7. Continued Reference map Land cover type A B C D E F Arable (grass ley) ± 6 5 ± (0 0±100 0) (0 0±32 4) (0 0±100 0) (0 0±33 3) Improved grassland (35 4±98 5) (51 5±99 5) (36 9±97 4) (50 7±88 2) (43 7±86 9) (49 6±98 8) Poor semi-improved grassland ± ± ± 0 0 (0 0±62 7) (0 0±55 1) (0 0±9 5) Bracken (0 5±33 3) (0 0±4 0) (0 0±46 8) (3 6±42 8) (0 6±32 6) (0 0±60 6) Ruderal tall herbs 0 0 ± ± ± (0 0±59 6) (0 0±20 5) (0 0±100 0) Non-ruderal tall herbs ± ± ± ± 0 0 ± Dry acid dwarf shrub heath ± ± (0 0±12 3) (0 0±56 6) (0 0±48 1) (0 0±35 0) Wet acid dwarf shrub heath ± ± 0 0 (0 0±28 5) ± ± 0 0 (0 0±21 3) Dry dwarf shrub heath/acid grassland ± 0 0 ± ± mosaic (0 0±62 4) (0 0±33 0) (0 0±58 6) Wet dwarf shrub heath/acid grassland ± ± ± mosaic (0 0±48 0) (0 0±78 5) (0 0±92 6) Blanket bog ± ± ± ± ± 0 0 Wet modi ed bog ± 0 0 ± ± (0 0±0 7) (0 0±10 3) Dry modi ed bog ± ± ± 0 0 (0 0±51 2) 0 0 (0 0±100 0) 0 0 (0 0±75 3) Acid/neutral ush ± ± ± Swamp ± ± ± 0 0 ± ± Acid/neutral natural rock exposure ± 0 0 ± ± 0 0 ± (0 0±60 8) (0 0±10 1) Quarry ± ± ± ± Mining spoil ± ± ± 0 0 ± ± Built (98 7±99 8) (100 0±100 0) (100 0±100 0) (95 5±96 6) (98 6±98 6) (100 0±100 0) Unclassi ed land (0 0±58 0) (4 3±29 9) (0 0±32 3) (0 0±100 0) (0 0±63 2) (1 9±13 1) cies, and many target notes listed few. There was also signi cant variation between surveys in the number of plant species listed in target notes (Kruskal±Wallis ANOVA, H = 33 5, d.f. = 5, P < 0 001). Survey C recorded a median of 11 species per target note, while the median for survey B was three (Table 9). The total number of plant species identi ed in target notes varied almost sixfold between surveys, from 25 (in survey B) to 145 (in survey C) (Table 6). Overall, 205 species were identi ed across all six surveys. Numbers of sedges recorded varied from two to eight, while the numbers of grasses and herbs recorded varied from one to 26, and three to 59, respectively. Few species of bryophytes were identi- ed in each survey. The similarity of the plant species lists derived from all target notes recorded in each survey ranged from 18 8% to 63 7% (with a mean of 43 4%) (Table 10). The information recorded on the abundance of the species was variable. Only one survey (B) explicitly identi ed a system of scoring species abundance used in the target notes, the stated method being DAFOR (dominant, abundant, frequent, occasional, rare). However, the meaning of the initials was not explained and in the target notes only `dominant' and `abundant' were used (in addition to `sparse', a term not included in the DAFOR approach). In this report, approximately 50% of references to individual species were accompanied by an indication of the species' abundance at the locations identi ed by the target notes.

14 1002 Variation between surveyors in habitat mapping Table 8. Median percentage agreements (with minimum and maximum values in parentheses) for broad land cover categories using each map as a reference for comparison with the other ve (n = ve in each case; ± = cover type absent from reference map) Reference map Land cover category A B C D E F Broadleaved woodland (86 7±91 0) (87 2±90 5) (89 3±92 8) (92 3±93 9) (83 9±88 6) (92 1±96 7) Coniferous woodland ± (0 0±66 8) (0 0±100 0) (0 0±100 0) (0 0±85 2) (0 0±38 2) Mixed woodland ± ± (0 0±40 8) (0 0±93 9) (0 0±80 5) (0 0±100 0) Scattered trees ± ± ± ± Rough grass (20 9±80 1) (23 1±81 8) (34 6±66 3) (32 5±89 5) (28 7±78 0) (25 3±90 2) Pasture/silage (88 5±98 9) (51 6±93 0) (68 9±95 9) (72 6±97 5) (68 7±97 0) (31 6±53 0) Tall herb and fern (0 5±29 6) (0 0±4 0) (0 0±59 1) (3 6±47 1) (0 0±24 3) (0 0±71 1) Heathland 52 2 (0 0±90 3) 49 3 (0 0±86 5) 61 1 (0 0±96 4) ± 53 3 (0 0±100 0) 81 1 (0 0±93 8) Mire ± (0 0±43 0) (0 0±100 0) (0 0±91 4) Swamp ± ± ± 0 0 ± ± Rock exposure ± 0 0 ± ± (0 0±60 8) (0 0±9 8) Built (98 7±99 8) (100 0±100 0) (100 0±100 0) (95 5±96 6) (98 6±98 6) (100 0±100 0) Unclassi ed land (0 0±58 0) (4 3±29 9) (0 0±32 3) (0 0±100 0) (0 0±63 2) (1 9±13 1) Terminology relating to species abundance in surveys A, D, E and F was variable, with `dominant', `frequent' and `occasional' being used inconsistently, along with a wide range of alternatives, e.g. `lots of', `some', `scattered' or `extensive patches of', and `main higher plants are'. In these four reports, between 10% and 50% of references to species within target notes included information (however, vague) on their abundance. Survey C was the only report within which almost all references to species within target notes (i.e. over 95%) gave an indication of the species' abundance. Table 9. The number and content of target notes associated with Phase 1 survey maps A±F Map A B C D E F Number of target notes Median no. of species per note (25th ± 75th percentile range) (2 0±14 0) (1 0±5 0) (6 5±21 0) (1 0±8 0) (1 3±10 8) (3 0±8 0) Number of species identi ed overall Number of species of: Trees Woody shrubs Herbs Grasses Rushes Sedges Other monocotyledons Ferns Horsetails Bryophytes

15 1003 A. Cherrill & C. McClean Table 10. Sorenson's similarity for pairs of species lists derived from target notes associated with each Phase 1 survey map Pair of maps compared Numbers of species recorded Map 1 Map 2 Map 1 Map 2 Both maps Similarity S (%) A B A C A D A E A F B C B D B E B F C D C E C F D E D F E F This report was also consistent in its use of terminology, which adhered to the DAFOR system, although this was not stated in the text. Comparisons for a single eld The number of target notes recorded in Field X varied from zero (surveys B and F) to ve (survey D) (Fig. 2). The manner in which target notes were used to link data to the maps also di ered between surveys. The single target notes recorded in surveys A, C and E each gave a description of the eld as a whole. In survey D, variation within the eld was described by a number of target notes, and information therein was more closely related to their precise locations on the map (Table 11). The target notes for surveys A, C and D gave the impression that the eld was a mosaic of di erent land cover types (Table 11). However, this was not re ected in the accompanying maps that recorded the surveyors' perceptions of the dominant vegetation type (Fig. 2). Survey A, for example, mapped the eld as `semi-improved acid grassland', but noted that patches of `semi-improved neutral grassland' were present. Survey C noted the presence of `semi-improved acid grassland' within `marshy grassland', but mapped only the latter. In Survey D the grassland was said to have a nities to `semiimproved acid grassland', yet the entire eld was mapped as `semi-improved neutral grassland' (Table 11). In each of surveys A, C and D, species listed as occurring in `acid' or `neutral' grasslands included a few given by the Phase 1 manual (Nature Conservancy Council 1990) as indicative of the opposite ph. For example, Juncus squarrosus (when dominant indicative of `acid' grassland) occurred in `neutral' grassland in survey D, while Deschampsia cespitosa (when dominant indicative of `neutral' grassland) occurred in `acid' grassland in survey A (Table 11). The absence of abundance data, however, prevents a retrospective interpretation of the appropriateness of the mapped land covers. `Marshy grassland', which was mapped in three surveys, includes `vegetation with a greater than 25% cover of Juncus acuti orus, J. e usus, J. in exus, Carex species or Filipendula ulmaria' but excludes `grazed Juncus e usus±holcus lanatus/ Deschampsia cespitosa grasslands, which should be classi ed under neutral grasslands, B2' (Nature Conservancy Council 1990). None of the target notes stated the percentage cover of Juncus species, although these rushes were evidently conspicuous to surveyors C, D and E (Table 11). Survey D alone inferred the presence of grazing livestock, while surveys A, C and E also recorded that both H. lanatus and D. cespitosa were present (Table 11). Overall, the target notes suggested that some combination of `semi-improved acid grassland', `semi-improved neutral grassland' and `marshy grassland' may have been appropriate for all or part of Field X. It is clear that most di erences within this eld involved similar vegetation types, as con rmed by the substantial improvement in agreement between maps after aggregation of related land cover types (Table 5). The remaining di erences included small areas of `improved grassland', ` ush' and `bracken', which could arguably have been overlooked. The mapping of `poor semi-improved grassland' by surveyor F was an exception, however. `Poor semiimproved grassland' has a nities with `improved grassland' and `semi-improved neutral grassland', but di ers from other semi-improved grasslands in lacking species indicative of underlying soil ph

16 1004 Variation between surveyors in habitat mapping Table 11. Target notes recorded in Field X in each survey. (The location of each target note is shown in Fig. 2; no target notes were made for this eld in surveys B and F) Survey A Target note 1 ± `Predominantly acid grassland including Agrostis canina, Agrostis capillaris, Agrostis stolonifera, Anthoxanthum odouratum, Deschampsia cespitosa, Festuca spp., Nardus stricta and Poa trivialis. Interspersed with rushes including Juncus articulatus, Juncus conglomeratus, Juncus e usus and Juncus squarrosus with occasional Carex nigra. Mosses included Sphagnum sp. and Polytrichum commune. Herbs included Cirsium palustre and Potentilla erecta. Occasional patches of neutral grassland comprising Cynosurus cristatus, Holcus lanatus and occasional Lolium perenne. Herbs included Cerastium fontanum, Cirsium vulgare, Conopodium majus, Plantago lanceolata and Rumex acetosella. Ditches across eastern part of eld lined by Juncus spp. and shallow fast owing water. Remainder dry.' Survey C Target note 1 ± Marshy grassland; semi-improved acid grassland; running water. `Rush-pasture interspersed with acid grassland covers main part of eld, including abundant Juncus e usus, J. acuti orus, Holcus lanatus, Cynosurus cristatus, Trifolium repens, frequent Ranunculus acris, R. repens, occasional Equisetum palustre, Ranunculus ammula, Deschampsia cespitosa, Juncus articulatus, rare Climacium dendroides, Carex ovalis. Semi-improved grassland on northern slope with abundant Agrostis capillaris, Anthoxanthum odouratum, Cynosurus cristatus, Festuca rubra, frequent Holcus lanatus, Hypochaeris radicata, Juncus squarrosus, Nardus stricta, rare Poa humilis. Stream with slabs of rock in bed, Ranunculus ammula by the water but no associated rush-pasture.' Survey D Target note 1 ± Semi-improved neutral grassland. `Rushy pasture. Remnant acidic grassland species present but sward dominated by neutral species due to enrichment from sheep-grazing. Species include: Festuca ovina, Cynosaurus cristatus, Bellis perennis, Cerastium fontanum, Holcus lanatus, Ranunculus repens, Trifolium repens, Lolium perenne, Cirsium sp., Rumex acetosa, Juncus squarrosus, Carex curta, Anthoxanthum odouratum, Juncus e usus, Deschampsia cespitosa and Poa spp.' Target note 2 ± Marginal vegetation. `Vegetation on steep slope alongside stream, dominated by Juncus acuti orus and other Juncus species. Similar to marshy grassland in the next eld to the east. Debatable whether should be classed as `marginal vegetation' or `marshy grassland' due to slope. 0 Target note 3 ± Semi-improved neutral grassland. `Field with many new drainage channels. Large patches of Juncus e usus with occasional Urtica dioica. Locally enriched mosaic of acid and neutral grassland but more species-poor than streamsides. Occasional patches of Juncus squarrosus and Nardus stricta but mostly Juncus e usus, Cirsium sp., Trifolium repens, Cerastium fontanum, Cynosaurus cristatus, Anthoxanthum odouratum, Poa sp. and Festuca ovina. Upper slopes have a greater proportion of Juncus spp., including Juncus acuti orus, but these areas are not considered to be marshy grassland as they are scattered.' Target note 4 ± Semi-improved neutral grassland. `Juncus e usus along stream-side. Large patch of Cirsium sp.' Target note 5 ± Marginal vegetation. `Juncus e usus along stream-side.' Survey E Target note 1 ± `A semi-improved eld with impeded drainage leading to a proli c growth of rushes, especially Juncus e usus with J. conglomerata, J. squarrosus and Cirsium palustre. Deschampsia cespitosa and Holcus lanatus are also abundant. J. acuti orus present along the stream which runs through this eld. Ditches also contain Polytrichum commune and Sphagnum species.' (Nature Conservancy Council 1990). Mapping of almost 80% of the eld as `poor semi-improved grassland' was di cult to reconcile with the target notes recorded in other surveys. SURVEY EFFORT Five of the surveys were conducted by a single surveyor. Survey D involved two surveyors. Time spent on site varied from 17 to 24 h spread between 2 and 4 days (Table 1). Four measures of survey e ort were the number of days, the number of hours and each of these multiplied by the number of surveyors. None of these measures was signi cantly correlated with cost of survey (P > 0 05 in each case). Neither cost nor any of the four measures of survey e ort were signi cantly correlated with any of the following parameters used to characterize the surveys: number of target notes, number of land cover types, median number of species recorded per target note, or number of species recorded in total (P > 0 05 in each case). The three surveys performed by members of the IEEM (A, C and D) were the three most expensive (Table 1). They also yielded the highest numbers of both target notes and numbers of species. In other respects, their results were similar to those produced by non-members (Table 9). Pairwise comparisons of surveys generated 15 estimates of spatial agreement from map overlay analyses, and 15 estimates of similarity based on comparison of species lists using Sorenson's index (Tables 4 and 10). Those estimates of spatial agreement derived from pairwise comparisons in which both surveyors were members of IEEM (n =3, x = 25 3%, SD = 6 4%) were not di erent from those derived from pairs in which one or neither were members (n = 12, x = 26 9%, SD = 6 8%)