The work from which this copy is made includes this notice: Copyright 2015 by President and Fellows of Harvard College, All rights reserved.

Transcription

1 Citation: Pirokka, M., E.C. Ellis, and P.D. Tredici Personal Remote Sensing: Computer Vision Landscapes. Pages in A. Fard and T. Meshkani, editors. New Geographies #7: Geographies of Information. Harvard Graduate School of Design, Cambridge, MA. The work from which this copy is made includes this notice: Copyright 2015 by President and Fellows of Harvard College, All rights reserved. Further reproduction or electronic distribution is not permitted.

2 Pirokka et al_ng7.indd 178

3 Personal Remote Sensing Computer Vision Landscapes Michalis Pirokka, Erle C. Ellis & Peter Del Tredici Since the invention of aerial photography, remote sensing has played a significant role in efforts to map the ecology of landscapes. 01 Owing to technological innovations, a wide variety of remote sensing techniques are now available to enhance these mapping efforts. Specifically, this article will engage new techniques of mapping ecological and surface structures across landscapes that have been made possible by the combination of lightweight, inexpensive, and publicly available Unmanned Aerial Vehicles (UAV) with computer vision algorithms. 02 These technologies now enable personal remote sensing on demand by landscape and planning professionals to generate high-resolution 2-D and 3-D maps in both visible and near-infrared spectrums for multiple uses, from initial site assessments to sophisticated landscape analyses supporting coastal erosion control, forestry, and habitat management. 03 First from balloons and kites, and later with manned aircraft, satellites, and UAVs, new platforms and techniques for remote sensing and mapping have continued to transform the efforts and the thinking of ecologists, landscape planners, and environmental activists. Now, innovative UAV remote sensing and mapping techniques are entering the design discourse, opening new associations, shifting modes of thinking, and potentially altering the trajectories of the theory, history, and cartography discourses within the design profession. Remote sensing can be a keystone linking the work of designers, urbanists, and ecologists as it employs a visual language and a mode of representation that can immediately be understood across specialization, especially when it accurately replicates real-time dynamics in the physical characteristics of a site. With remote sensing, precise data can be generated to represent both permanent and ephemeral characteristics of a site, offering opportunities for more grounded design development. With remote sensing mapping, terrain becomes more than topography, it becomes a tangible amalgamation of elements, a layering of time, space, and ecology, and the basis for a real-world design process. At multiple levels, 179 Pirokka et al_ng7.indd 179

4 180 the point cloud topographic model produced by remote sensing technology provides a more holistic view of reality than a conventional layered plan. 04 On Methods The introduction of UAVs and 3-D mapping is revolutionizing ecology, allowing a more comprehensive grasp of ecological pattern and process than any prior geospatial methodology. 05 Spatially precise and temporally dynamic, 3-D data enables structural observations to be related to ecological functions, opening up novel spatiotemporal views of ecological phenomena across landscapes. The dramatic evolution in the platforms and flight control of UAVs, their coupling with computer vision, and their migration from military secret to smartphone app have put remote sensing into the hands of those who seek the view from above, from technology enthusiasts to journalists, to environmental professionals, transforming ecological mapping and automating site-specific environmental monitoring. 06 UAVs enable inexpensive, scale-appropriate image acquisition with resolutions defined more by the application than by technological limits. There has never been a lower cost platform for consistent, site-specific 3-D aerial mapping of environmental phenomena to monitor change either in a single visit or over time. The UAV brings four operational advantages as a mapping tool beyond its relatively low cost of operation: deployment on demand, easy repetition as conditions change, the ability to fly under cloud cover (UAVs fly low and slow), and precisionautomated flight plans enabling the acquisition of highly overlapping aerial images needed to support both 3-D reconstructions and perfect orthorectified photo mosaics (buildings and trees are seen only from the top, free of lens and height distortions). Data gathered by UAVs at local scales can be georeferenced and integrated with data obtained from traditional maps as well as spaceborne remote sensing tools, seamlessly integrating views and data from local, regional, and global scales. From the overlapping images collected during a UAV flight, a color-referenced 3-D point cloud, a 3-D landscape geometry, can be generated using structure from motion algorithms. Compared with simple, 2-D layers, these 3-D data products, with their more realistic datarich representation, push scientific studies toward a deeper and more accurate understanding of ecological characteristics. 07 During the past year, a pilot project at the Harvard Graduate School of Design tested the application of UAV technology at Arnold Arboretum of Harvard University in Boston and Harvard Forest in Petersham, Massachusetts. 08 Typical UAV-based surveys resulted in the production of color-referenced point clouds, orthomosaics, tree canopy height projections, and vegetation maps detailed enough to allow for the identification of plant species. The application of this technology in these two projects revealed the potential for this methodology to become an integral part of landscape analysis. By returning to the site during different seasons of the year, when species can be readily distinguished from one another based on leaf-out times, flowering, and autumnal coloration, an even more finely grained understanding of present conditions of a site and its possible future trajectories were obtained. Given this realization, the central question of this project became, How can data obtained by remote sensing be best understood, valued, and deployed by designers and planners? To date, UAV and computer vision technology the cornerstones of personal remote sensing have been applied successfully in population ecology, vegetation dynamics, precision agriculture, archaeology, forestry, and habitat management. These are fields that are best informed through the gathering of precise information in both horizontal and vertical domains, not unlike the information demands of designers working at large scale. The question then becomes, Can these emerging functionalities shape new and more informed forms of design, especially in the fields of landscape architecture and urban design? 09 On Design The design disciplines negotiate precise scales that often can be best addressed through personal remote sensing. The use of remote sensing in design and the friction between the two fields can revolutionize the process of gathering and monitoring spatial data that can be used to inform design development. As a modeling tool of extraordinary precision, it can expose the tectonics of a site. In combination with other techniques of observation and analysis, it can project a new, more up-to-date direction for design. Design aims to give form and lend shape to all elements that influence a site. Typically, however, design lacks a fine scale of time. This can be a crucial factor, as landscape architecture interventions (like ecology itself) are inseparably bound up with time. Can analysis informed by remote sensing be the method that narrows the gap between reality and design? Surely, it takes design a step closer to reality just by sheer virtue of its ability to enable precise understanding of the physical reality of terrain and vegetation across large sites. The potentials of this method within design lay in its ability NG07 Geographies of Information Pirokka et al_ng7.indd 180

5 Above: Point cloud aerial perspective, Common Meadow, Petersham, Massachusetts, October Left: Flight path for an automated flight, Common Meadow, Petersham, Massachusetts, October Pirokka et al_ng7.indd 181

6 182 to identify and correlate up-to-date land use and land cover classifications at both the local and regional scales. Ease of data collection is a major operational advantage that UAV mapping brings to the table for designers. Low operation cost and user-controlled scanning enables repetitive point cloud acquisitions many times across a site on the same day or at least many times per season. By simply associating multiple point clouds with their geolocation, 3-D datasets transform into 4-D datasets, as the variable of time is added to the design equation. This generates the potential for mapping the ephemeral characteristics of a site and, through modeling, of evaluating the specific changes and processes that take place within that site, such as the ebb and flow of tidal waters, seasonal plant changes, animal migration patterns, or the spread of plant pathogens across a landscape, thereby resulting in new associations during design development. With personal remote sensing, we now have the capacity to design for the temporal: this radically shifts the thinking that has characterized design discourse up until this date. Aerial images traditionally have been crucial for understanding and negotiating landscape and urban scales. 10 But can architects and planners produce credible results when feeding their designs directly into a point cloud? Is this to become a new testing method for design interventions? Personal remote sensing can combine the best aspirations of the design discipline with the best ambitions of mapping, imposing scale-appropriate, efficient, and systemized thought and action. It might even support a new conception of sustainable design, as dynamic elements of a site can now be registered and ephemeral ecological associations can now influence the scale and form of a design. By modeling and evaluating the dynamic and the ephemeral, designers can better understand the physicality of the landscape. We are in need of the spatial and temporal precision offered by these new methods: we can no longer afford to ignore the reality of the terrain or to get carried away by intellectual concepts that are independent of the site. 11 Remote sensing can become a significant research topic within design-related disciplines. It is a topic that can be placed in between territory, site design, and ecology, with the perception of geography always present. It is time to bring remote sensing into design and thereby reshape both. The fields of landscape ecology, landscape architecture, urban planning, and urban design stand at a juncture where innovative technologies provide new methods of site analysis that must be dealt with through emerging, new representational techniques. Traditionally designers dealt with ecology and landscape through the static means of the master plan, with a layering of predefined (and most of the time outdated) datasets. Might we now design in near-real-time frameworks, incorporating datasets updated on demand as the design process unfolds? Potentially these technologies will offer new dynamic tools for the practice of design, bringing us one step closer to realizing the informed landscape that fully expresses Above: Images collected through UAV Technology (left), and Tree Canopy Height DEM (right), Bussey Brook Meadow, Arnold Arboretum of Harvard University, October NG07 Geographies of Information Pirokka et al_ng7.indd 182

7 the dynamic articulation of ecological and human processes on a given area of interest. 12 These new mapping techniques differ from earlier practices. Personal remote sensing is an operational methodology that brings the designer into the physical site, back to fieldwork, whether directly or virtually through 3-D reconstructions. 13 And as a methodology, it has the capacity to question and challenge the techniques and toolsets inherited from the past, just by injecting realistic site data and visualization into the praxis. No longer purely speculative, these new mapping techniques help designers go beyond simply portraying preliminary scenarios to conceiving of credible concepts. A mixture of remote sensing and traditional mapping techniques has the potential to produce more interesting results and more workable data than either method by itself. Scale thinking based on mapping can alter the typical priorities that drive design, thereby generating new links between design and cartography and, depending on the scale, new connections between architecture, geography, and ecology. Whereas the images generated from a point cloud can be aesthetically pleasing, their true value resides in the data on actual surface cover that they contain and in their ability to enrich the abstract information found in planning documents and maps that have governed design decisions up until now. On Potentials The conversation currently taking place within the design discourse will no doubt uncover some critical questions related to the usefulness of remote sensing to the field. We anticipate accelerating advances in the monitoring, modeling, and representational techniques related to the ecologies of disturbance, density, complexity, and flux, all at the scale of cities, landscapes, ecosystems, and isolated projects. Any misgivings will be revisited in the course of time, as academics and professionals alike become more familiar with and more dependent upon these new tools. But it is clear that spatial modeling has the potential to offer designers a unique reading of urban environments. Remote sensing should never be conceived as a standalone operation: rather, it should always be used to support the design and planning processes as part of a larger context. The greatest potential for further introduction of these tools in the design discipline lies beyond the technological innovation of UAVs. The reprocessing and reinterpretation of raw digital imaging and other data can become a framework for reexamining the use of aerial images in the world of design, and data availability can become a decisive rather than a limiting factor in any future application of remote sensing in the design discipline. Numerous types of accurate, up-to-date, and highresolution remotely sensed data can be readily accessible and can be integrated in open-source geospatial libraries (for example, the geospatial libraries linked to each figure in this text). The availability and distribution of data can enhance the collective and communal nature of the Above: Pixel comparison. The images reveal that the end user can define the resolution and the precision of the point cloud. Point cloud perspective, Harvard Forest, Petersham, Massachusetts, October Michalis Pirokka, Erle C. Ellis & Peter Del Tredici Pirokka et al_ng7.indd 183

8 Right: Point Cloud Perspective, Brooklyn Bridge Park, Brooklyn, New York, March As this perspective image reveals, the resolution that a point cloud carries qualifies it as a valuable tool for land use and land cover classification. Below: Pier 1, Brooklyn Bridge Park, Brooklyn, New York, March Five years after the opening of the park, Michael Van Valkenburgh Associates can use this image to evaluate the success of their design implementation. Opposite: Property Map, Vegetation and Ecological Functioning Units, Bussey Brook Meadow, Arnold Arboretum of Harvard University October By observing the true orthographic aerial photograph one can start identifying ecological and human processes that take place within the area of interest. 184 NG07 Geographies of Information Pirokka et al_ng7.indd 184

9 Ginkgo - Ginkgo biloba Ginkgo - Ginkgo biloba MIX HARDWOODS MIX HARDWOODS Multiflora rose Rosa multiflora A BT M City of Bosto n A BT M City of Bosto n Multiflora rose Rosa multiflora Orientalorbiculatus bitterswet - Celastrus orbiculatus Oriental bitterswet - Celastrus Sumac - Rhus coriaria L. Cottonwood sect. Aigeiros Sumac - Rhus coriaria L. Cottonwood - Populus sect. Aigeiros- Populus Black Locust - Robinia ps. Oak Quercus L. Goldenrod Solidago L. Treealtissima of Heaven - Ailanthus altissima Tree of Heaven - Ailanthus MESA MESA Hickory - Carya tomentosa Hickory - Carya tomentosa Honeysuckle - Lonicera L. Cottonwood Populus deltoides L. on s st tie Bo ili of Fac y c t Ci ubli P Black cherry Prunus serotina on s st tie Bo ili of Fac ty lic i C ub P sewer line sewer line MBTA - Rhus coriaria L. CottonwoodSumac Milkweed Populus deltoides L. - Asclepias L Gray Birch Betula populifolia Sumac - Rhus coriaria L. Milkweed - Asclepias L Gray Birch Black cherry Betula populifolia Prunus serotina Mugwort Artemisia vulgaris L. Reed Canary Grass - Reed Canary Grass Phalaris arundinacea L.Phalaris arundinacea L. Catalpa Yellow Iris Yellow Iris Catalpa Speciosa Iris pseudacorus L. Cattail Iris pseudacorus L. Typha L. Common reed Common reed WETLAND WETLAND Phragmites australis Phragmites australis Catalpa Catalpa Speciosa Burning Bush Euonymus europaeus L. ARNOLD ARBORETUM ARNOLD ARBORETUM Black Locust Robinia ps. Honeysuckle - Lonicera L. Red Red Maple - Acer rubrum L.Maple - Acer rubrum L. Burning Bush Euonymus europaeus L. Tree of Heaven Ailanthus altissima Cattail Typha L. Cork Tree Phellodendron Amurense Cork Tree Phellodendron Amurense f Oak Quercus L. Goldenrod Solidago L. Hackberry - Celtis Occidentalis Hackberry - Celtis Occidentalis BROOK MEADOWS BUSSEY BROOK BUSSEY MEADOWS so Black Locust Robinia ps. Tea crabapple - Malus hupehensis Tea crabapple - Malus hupehensis Red cedar Red cedar Juniperus virginiana Forsythia - Forsythia suspensa Mugwort Whitty Pear Juniperus virginiana Forsythia - Forsythia suspensa Lilac Artemisia vulgaris L. Whitty Pear Sorbus domestica L. Lilac sp. Sorbus domesticasyringa L. Syringa sp. Mountain Ash Mountain-laurel -Kalmia latifolia L. Mountain Ash Mountain-laurel -Kalmia latifoliaschouten L. Sorbus X Arnoldiana Sorbus X Arnoldiana Schouten Pin oak - Quercus palustris Forsythia Hawthorn - Crataegus Pin oak - Quercus palustris pedicellata Forsythia Forsythia int. spectabilis Hawthorn - Crataegus pedicellata Catalpa Forsythia int. spectabilis Catalpa Catalpa Speciosa Hawthorn - Crataegus nitida Catalpa Speciosa Hawthorn - Crataegus nitida Eastern hemlock - Tsuga canadensis L. Black oak - Quercus velutina Eastern hemlock - Tsuga canadensis L. Black oak - Quercus velutina Hawthorn - Crataegus mollis Northern red oak Hawthorn - Crataegus mollis Northern red oak Quercus rubra maxima Amur maple - Acer ginnala Quercus rubra maxima Amur maple - Acer ginnala Prickly castor oil tree - Kalopanax septemlobus Prickly castor oil tree - Kalopanax septemlobus White oak - Quercus alba White oak - Quercus alba Crabapple Crabapple Black hawthorn - Malus sp. Black hawthorn Manchurian Pear Malus sp. Crataegus douglasii Manchurian Pear Crataegus douglasii Pyrus ussuriensis Honey locust Pyrus ussuriensis Honey locust Gleditsia triacanthos L. Gleditsia triacanthos L. Spring cherry Korean Ash - Prunus subhirtella Spring cherry Korean Ash - Prunus subhirtella sorbus alnifolia sorbus alnifolia Japanese Hop-hornbeam Japanese Hop-hornbeam Ostrya japonica Ostrya japonica Hickory - Carya tomentosa Hickory - Carya tomentosa MBTA w llo Fe ge t & lle n e Co id es rd Pr arva H Black Locust - Robinia ps. Tree of Heaven Ailanthus altissima f Jerusalem Artichoke - Jerusalem Artichoke Helianthus tuberosus Helianthus tuberosus so w llo Fe e & eg nt oll e C id es rd Pr arva H Old Arboretum NurseryOld Arboretum Nursery Oriental bittersweet - Oriental bittersweet Celastrus orbiculatus Celastrus orbiculatus HEMLOCK HILL HEMLOCK HILL 185 FORESTED SLOPE FORESTED SLOPE Michalis Pirokka, Erle C. Ellis & Peter Del Tredici Pirokka et al_ng7.indd 185

10 186 design discipline. This shifts the conversation toward the importance of effective data management and archiving. Due to their large size, UAV data products generate limitations in data sharing. Future efforts must be focused on increasing not only the accuracy but also the availability of datasets for designers. By creating data repositories that can often be managed by libraries, the process of data sharing can be smoothed, enhancing the exchange of research methodologies and ideas. By increasing the availability of practicable and user-friendly datasets, designers can standardize and automate a process of design analysis that will convert remote sensing into a more effective tool for identifying landscape patterns related to form, use, and cover. 14 Remote sensing reveals potentials related to the observation, design, and maintenance of ecological systems and strategies for landscape development. There is an urgent need for in-depth process understanding and for a more profound, precise, and time-based knowledge of land use decisions that drive the urban structure. To fulfill this need, it is likely that novel approaches toward an increasingly interactive remote sensing will become a reality. Monitoring might even happen at the scale of the individual landowner to solve the problem of site accessibility. 15 The designer, as well as other end users will benefit from this. They will be able to update and monitor shared collaborative datasets at any given time, for a close-tooptimal design development process. This interactive form of mapping will be critical for evaluating the success of site restoration and design implementation, cultivating spatial intelligence that helps people better understand and share information about places, and ultimately will transform design information in ways that promote more effective collaboration. Above: Overlaying site topography with point cloud, Bussey Brook Meadow, Arnold Arboretum of Harvard University October NG07 Geographies of Information Pirokka et al_ng7.indd 186

11 An online library including all geospatial data from the scanned sites can be found at: On the use of aerial photography to map landscape ecology, see Olaf Bastian and Uta Steinhardt, eds., Development and Perspectives of Landscape Ecology (Dordrecht: Kluwer Academic Publishers, 2002); and Ronen Kadmon and Ruthie Harari-Kremer, Studying Long-Term Vegetation Dynamics Using Digital Processing of Historical Aerial Photographs, Remote Sensing of Environment 68, no. 2 (May 1999): For UAV definition, see Karen Anderson and Kevin J. Gaston, Lightweight Unmanned Aerial Vehicles Will Revolutionize Spatial Ecology, Frontiers in Ecology and the Environment 11, no. 3 (April 2013): The flight path of a UAV is controlled either autonomously by computers in the vehicle or under the remote control of a pilot on the ground or in another vehicle. Unmanned aircraft are typically launched and recovered by an automatic system or an external operator on the ground. On computer vision algorithms, see N. Snavely, I. Simon, M. Goesele, R. Szeliski, and S. M. Seitz, Scene Reconstruction and Visualization from Community Photo Collections, Proceedings of the IEEE 98, no. 8 (August 2010): See also Jonathan P. Dandois and Erle C. Ellis, High Spatial Resolution Three-Dimensional Mapping of Vegetation Spectral Dynamics Using Computer Vision, Remote Sensing of Environment 136 (September 2013): For remote sensing mapping applications, see Anderson and Gaston, Lightweight Unmanned Aerial Vehicles ; R. Brumana, D. Oreni, M. Alba, L. Barazzetti, B. Cuca, and M. Scaioni, Panoramic UAV Views for Landscape Heritage Analysis Integrated with Historical Maps Atlases, XXIII CIPA Symposium, Prague, Czech Republic (September 2011): 12 16; Dandois and Ellis, High Spatial Resolution ; Woody Turner, Sensing Biodiversity, Science 346, no (October 2014): ; M. J. Westoby, J. Brasington, N. F. Glasser, M. J. Hambrey, and J. M. Reynolds, Structure-from- Motion Photogrammetry: A Low- Cost, Effective Tool for Geoscience Applications, Geomorphology 179 (2012): ; and R. Zahawi, J. P. Dandois, K. D. Holl, D. Nadwodny, L. J. Reid, and E. C. Ellis, Using Lightweight Unmanned Aerial Vehicles to Monitor Tropical Forest Recovery, Biological Conservation 186 (2015): A point cloud is a set of data points in a specific coordinate system. In a 3-D coordinate system, these points are usually defined by X, Y, and Z coordinates and are often intended to represent the external surface of an object. 05. Anderson and Gaston, Lightweight Unmanned Aerial Vehicles ; Dandois and Ellis, High Spatial Resolution Three-Dimensional Mapping ; Woody Turner, Sensing Biodiversity. 06. With regard to the military uses of remote sensing technology, see Laura Kurgan, Close Up at a Distance: Mapping, Technology, and Politics (Brooklyn, NY: Zone Books, 2013), A color-referenced point cloud is one where, in addition to having X, Y, and Z coordinates, has RGB values, which are carried by each point. For Structure from motion and 3-D Visualization, see Snavely et al., Scene Reconstruction and Visualization from Community Photo Collections ; Dandois and Ellis, High Spatial Resolution ; Gregory P. Asner, Roberta E. Martin, Christopher B. Anderson, and David E. Knapp, Quantifying Forest Canopy Traits: Imaging Spectroscopy versus Field Survey, Remote Sensing of Environment 158 (2015): The project began as part of Peter Del Tredici s fall 2014 Research Seminar in Urban Ecology. The work received the Penny White Project Fund award, which allowed students to travel to Baltimore to train at Ecosynth, a lab led by Erle C. Ellis that specializes in UAV remote sensing. During the development of the pilot project, students and their advisors collaborated closely with Janina Mueller, Data Librarian of the Frances Loeb Library. The Ecosynth team at UMBC ( was supported by NSF grant DBI For remote sensing in agriculture, see Jennifer L. R. Jensen and Adam J. Mathews, Visualizing and Quantifying Vineyard Canopy LAI Using an Unmanned Aerial Vehicle (UAV) Collected High Density Structure from Motion Point Cloud, Remote Sensing 5, no. 5 (May 2013): For UAVs in archaeology, see M. Brutto, A. Borruso, and A. D Argenio, UAV Systems for Photogrammetric Data Acquisition of Archaeological Sites, International Journal of Heritage in the Digital Era 1 (January 2012): For forestry and habitat management, see Dandois and Ellis, High Spatial Resolution ; Asner et al., Quantifying Forest Canopy Traits. 10. Bastian and Steinhardt, Development and Perspectives of Landscape Ecology. 11. Christophe Girot, The Elegance of Topology, in Topology: Topical Thoughts on the Contemporary Landscape, ed. Christophe Girot, Anette Freytag, Albert Kirchengast, and Dunja Richter (2013): Wilfried Endlicher, ed., Perspectives in Urban Ecology: Ecosystems and Interactions between Humans and Nature in the Metropolis of Berlin (Berlin and Heidelberg: Springer Verlag, 2011). 13. Erle C. Ellis et al., Measuring Long-Term Ecological Changes in Densely Populated Landscapes Using Current and Historical High Resolution Imagery, Remote Sensing of Environment 100, no. 4 (February 28, 2006): Ibid. 15. Kit Eaton, Civilian Photography, Now Rising to New Level, New York Times, January 2, Image Credits 181, 182 (left), 183: Erle C. Ellis, Peter Del Tredici, Janina Mueller, Hector Tarrido-Picart, and Michalis Pirokka. 182 (right), 186: Peter Del Tredici, Erle C. Ellis, Dana Boswell, Manuel Colon- Amador, Hector Tarrido-Picart, and Michalis Pirokka. 184: Michael Van Valkenburgh Associates, processing and mapping by Janina Mueller and Michalis Pirokka. 185: Peter Del Tredici, Manuel Colon- Amador, Hector Tarrido-Picart, and Michalis Pirokka. 187 Michalis Pirokka, Erle C. Ellis & Peter Del Tredici Pirokka et al_ng7.indd 187

12

13

14