A Shape-based Approach to Spatio-Temporal Data Analysis using Satellite Imagery

Transcription

1 A Shape-based Approach to Spatio-Temporal Data Analysis using Satellite Imagery Thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Computer Science and Engineering by Research by DARPAN BAHETI International Institute of Information Technology, Hyderabad Hyderabad , INDIA November 2018

2 Copyright c Darpan Baheti, 2018 All Rights Reserved

3 International Institute of Information Technology Hyderabad, India CERTIFICATE It is certified that the work contained in this thesis, titled A Shape-based Approach to Spatio-Temporal Data Analysis using Satellite Imagery by Darpan Baheti, has been carried out under my supervision and is not submitted elsewhere for a degree. Date Adviser: Dr. K.S Rajan

4 To my family and friends.

5 Acknowledgments Foremost, I offer my sincerest gratitude to my advisor, Dr. K. S. Rajan, for providing a nurturing environment for exploration and doing research, for his guidance, patience, and enthusiasm. His constant encouragement and support have been a source of motivation throughout the years of my research. I could not have imagined having a better mentor. I would like to thank my loved ones, my parents, brother, and friends, for providing me with unfailing support and continuous encouragement throughout my years of study. I thank Land Processes Distributed Active Archive Centre (LPDAAC), NASA for providing MODIS product dataset of year 2008 to 2012 for the region of interest and National Remote Sensing Centre (NRSC), Indian Space Research Organisation, India for providing AWiFS derived national land use land cover product of West Godavari district of Andhra Pradesh state for crop calendar year v

6 Abstract Many socio-environmental aspects manifest themselves over space and time, interacting at varying scales of these dimensions. Satellite imagery, available repetitively over a region, provide important clues of these observations across these dimensions. But, also pose enormous challenges in terms of data processing, extracting significant patterns (indicating the underlying processes) and be able to further model them as scientific knowledge of the environmental process. Study of cropping practices and changes occurring in it over time for a region is critical to the field of agronomics. The increase of temporal resolution of Earth Observation Satellite platforms has resulted in easier monitoring of the Earth s surfaces. Satellite imagery is an exemplary data source that captures the vegetation responses of the field and such dataset can be used to effectively monitor and track vegetation health over time. The need for large-scale and structured analysis of satellite data for agricultural monitoring is growing. One of the current challenges in understanding agricultural practices is to build a history of land-use outcomes and extracting the local phenological responses that help in building field-based inventory and understanding gross estimate of various cropping systems. Also, monitoring of vegetation health and identification of the major changes in crop practice described here as change-event can provide valuable insights into the possible factors that were responsible for such an event be it biological, physical, hydrological or climatic. In this study, an effort has been made to propose a time-variant analysis method based on the shape characteristics of the vegetation response over time to help identify regions of significant changes. This study attempts to build a spatio-temporal library of vegetation practices and presents an approach, supervised and unsupervised, to process the raw data obtained from the satellite images into representative phenological growth curves. The statistics contributed by the current work is critical to understanding and assessing the changes in cropping practices over time and detecting possible drivers that triggered to cause such change-events. The study covers four agricultural-year periods between 2008 and 2012 over the district of West Godavari, in the south of India. An interesting result from this study was to observe the impact of drought on cropping cycles during and after the drought year for the given region. The work finds that the immediate effects of drought in a given year are limited if the region is well endowed with other resources such as irrigation facility etc. Whereas its impacts on cropping patterns can also be seen in the season following the drought year. And such changes vary across the district depending on where vi

7 vii cropping is spatially located. The approach uncovers that the effect of the 2009 drought year on the agricultural practices vary spatially depending on the access to resources and the time-lag that manifests itself in such processes. In this study, we also find that nearly 80% of the region is well endowed and hence resilient to the climatic vagaries. The major contribution of this work is the analysis of spatio-temporal data to detect changes that may help in capturing episodic and periodic change-events occurring in crop cycles of any given region. With vast data spanning across multiple regions in hand, the proposed shape-based approach that characterize the phenological nature of these time-dependent sequences help to discover regions that are spatially correlated and find crop practices that are prevalent across similarly endowed regions.

8 Contents Chapter Page 1 Introduction Land Cover Dynamics Context and Motivation Why Satellite Data? Thesis Objective and Research Questions Summary of Contribution Outline of the Thesis Literature Survey Satellite Remote Sensing Spectral Image Classification - Supervised and Unsupervised Approaches Multi-Temporal Satellite Datasets MODIS dataset - Vegetation Indices Time-Series Analysis and Classification Dataset and Time-Series Analysis MODIS Satellite Imagery Dataset Characteristics Vegetation Index Data for Accuracy Assessment Study Area Vegetation Classes and Temporal Signature Challenges with Time-series Analysis A Shape-based Supervised Approach for Time-Series Classification Flow Chart Data Preprocessing Data Extraction Cloud Correction Curve Smoothing Ground Truth Information Distance Metric and Clustering Dynamic Time Warping K-medoids Clustering Classification viii

9 CONTENTS ix Optimization - Shape Averaging DTW Barycenter Averaging (DBA) Nearest Neighbour Classification Results Vegetation Cover Map Validation A Feature-based Approach and an Ensembling of the Shape-Dimension Feature Description Cycle Detection and Extraction of Phenological Parameters Decision Tree classification Results Intra-class Variability in Cropping practices Data Driven Unsupervised learning - An approach towards building an automated system for spatio-temporal data analysis Sampling for Identification of Unique Temporal Signatures Clusters of Time-Series Curve Reclassification and Results Change-Event Detection using Spatio-Temporal mapping of Cropping Practices Case Study: Change-Event Dectection in Agriculture Regions of West Godavari district in South India Change Analysis Observation and Inferences Drought Analysis: Source of Irrigation Map - Rainfed v/s Non-rainfed Regions 54 8 Conclusions Bibliography

10 List of Figures Figure Page 1.1 Satellite time-series composition Phenological vegetation cycle for single crop reflected in vegetation index time-series. Different stages in the phenological cycle of a crop are marked Phenological vegetation cycle for double crop reflected in vegetation index time-series Tile layout of MODIS satellite data. Tile h25v07 that covers Andhra Pradesh, Telangana and Karnataka states of India is shown in Figure Illustration of NDVI calculation. The figure illustrates the reflectance property of healthy vegetation v/s unhealthy or sparse vegetation in visible and near Infrared spectrum of light and how reflectance properties can be used to detect the greenness of any vegetation MODIS EVI dataset image EVI image and mask of West Godavari district loaded in QGIS software NRSC reference data Steps to preprocess the raw time-series data System pipeline depicting steps for refining training data and for vegetation cover classification Local minima (considered as noise that needs to be corrected to create smoothened timeseries curve) is present between a left and right local maxima (both local maxima are marked using black dots). The data point at the local minima on the curve is updated (updated to the green dot) using local Min-Max filter described above For a pixel, its raw time-series curve formed using 23 data points for agricultural-year and derived time-series curve obtained after the smoothing step is shown Depicts the resulting alignment of two slightly time-shifted sequences. The optimal warping path is outlined in green in the DTW matrix For each class, core samples of time-series curves extracted using clustering algorithm For each class, core samples of time-series curves extracted using clustering algorithm Representative temporal signature for one agricultural-year. For each class, the vertical spectrum at 23 data points is also shown, that denotes the variability Representative temporal signature for one agricultural-year. For each class, the vertical spectrum at 23 data points is also shown, that denotes the variability Vegetation cover map generated for agricultural-year using proposed shapebased supervised approach and labels of seven classes Rescaled and resampled NRSC reference data for agricultural-year used for validation x

11 LIST OF FIGURES xi 5.1 Feature-based framework for Spatio-temporal analysis Crop-growth cycle and critical points in cycles for deriving phenological parameters are marked Slope-shifting and Extrapolation method to derive Phenological parameters Vegetation cover map generated for agricultural-year using proposed featurebased supervised approach Time-series curve for 115 days crop with peak in Oct-Mid Time-series curve for 140 days crop with peak in Sept-End Time-series curve for 140 days crop with peak in Sept-Mid Time-series curve for 150 days crop with peak in Oct-Start Time-series curve for 150 days crop with peak in Sept-End Time-series curve cluster of No/low Vegetation class Time-series curve cluster of Kharif crop and Rabi crop class Time-series curve cluster of Double cropping class Time-series curve cluster of Plantation and Forest Class Vegetation cover map generated for agricultural-year using the proposed unsupervised approach Double cropping practice spatial intra-variability map. Representative curves for different double-crop classes are shown in Figure Vegetation cover map generated for agricultural-year Histogram chart of change-analysis Each pixel (a parcel of land) was labelled as per the type of change observed. As discussed above, seven type of changes were considered and labelled with different colours. Also, the major regions that showed similar changes (spatio-temporally homogenous regions) for a transition from agricultural-year to are shown. Region A (green label - indicating a positive change from single to double cropping), Region B (brown label - indicating a neutral change that shows a shift in agricultural practice) and Region C (yellow and orange labels - indicating a negative change from double to single cropping practice) are highlighted Time-series curve of a pixel from Region A Time-series curve of a pixel from Region B Time-series curve of a pixel from Region C Time-series curve of a pixel from Region D

12 List of Tables Table Page 3.1 MODIS satellite data info Land cover classes and characteristics of temporal signatures Confusion Matrix of results obtained from the proposed shape-based approach against statistics obtained from NRSC reference data for agricultural-year Confusion Matrix of results obtained from the proposed feature-based approach against statistics obtained from NRSC reference data for agricultural-year Change analysis (Statistics of No change regions) for agricultural-years from to xii

13 Chapter 1 Introduction 1.1 Land Cover Dynamics Land cover is the observed biophysical cover on the Earth s surface. The land cover information provides thematic characterizations of the Earth s terrestrial surface, identifying different components like vegetation, inland water, barren land or human infrastructure. Land cover mapping is a basic assignment for some applications, for example, landscape administration, biodiversity appraisal, and, all the more by and large to help ecological, social, and economic arrangements. As the landscape is continually changing affected by several elements (e.g., urban development, geo-climatic setting, socio-economic factors, forest fires, deforestation, etc), land cover maps turn out to be quickly obsolete, and the requirement for updated and detailed land cover maps is continually expanding to help near real-time spatial decision-making. Understanding the dynamics of land cover and land cover changes can help us to comprehend the drivers that activated such change and could be useful to configuration better strategies and accompany local actors towards more sustainable land administration. 1.2 Context and Motivation Study of spatial distribution of various vegetation covers and changes occurring in it over time for a region provides crucial environmental information. The information plays a vital role in understanding complex ecosystem processes, climate change studies, deforestation studies and many policy applications. Thus, there is a dire need for improved monitoring of vegetation resources. For regions dominated by the agricultural land cover, such information is very valuable and critical to the field of agronomics [1]. The agricultural component of the vegetation landscape is of specific interest because it is intensively managed and continually modified. It also has a direct impact on the ecological processes, hydrological resources, and the economy [2], [3]. For example, a region hit by drought could suffer from fall in agricultural production which would be evident from a change in its corresponding vegetation cover as observed through satellite imagery. Another example could be the introduction of new irrigation infrastructure (e.g., a canal), which can lead to significant changes in vegetation cover over 1

14 the surrounding regions. Post analysis of such impacts could be conducted via studying changes in the vegetation cover, which can enable us to analyze the effectiveness of such infrastructure and what kind of shifts it introduced in cropping practices of that region. It can also provide a base for the improved structuring of future economic development strategies. Such an analysis provides dual benefits in itself. This can lead to better management and distribution of irrigation facilities by laying emphasis on waterdeprived areas and at the same time studying its corresponding socio-economical impacts. Environmental changes either natural or man-made can majorly be categorized into large scale and small scale. Large-scale environmental changes, such as the introduction of new irrigation infrastructures, deforestation, drought, or floods, cause abrupt or sudden changes in vegetation cover. Whereas small-scale changes happen over a prolonged period of time. For example, a region showed change from double cropping to single cropping practice over a time span of years. These shifts are gradual, but detecting such changes could help us analyze underlying environmental factors like soil degradation or long-term change in rainfall pattern etc. Thus, the study of various vegetation covers and changes occurring in it over the spatio-temporal space can provide a holistic perception of where and when those changes occurred, give an insight to the possible drivers that triggered such changes, and their corresponding impacts. 1.3 Why Satellite Data? Satellite-based imagery with its wide and repetitive coverage can provide a good set of samples to assess terrestrial regions. Every pixel of this satellite imagery represents a certain parcel of land on earth. For each such parcel, at a given time, the satellite image gives a snapshot of its current status. This might be in terms of vegetation growth, the rate of urbanization, water spread, or any such dynamic phenomena. Such responses captured over frequent intervals helps generate the time-series of the region, which when analyzed over time and space dimension can help understand various phenomena. For an area, represented by a pixel, the time-series generated using vegetation response - captured in the satellite imagery - is referred to as the temporal signature of that particular region. It captures the variation of greenness attribute over time and indicates its phenological progression. Figure 1.1 shows the temporal variation at a particular pixel as captured and extracted from the twodimensional spatial arrays of satellite imagery. The grouping of images enables us to create time-series for every pixel that demonstrate the time variety of surface characteristics, for example, the reflectance of a particular band of the electromagnetic spectrum or derived indexes captured by satellite sensors. Phenological cycles represent the greening and senescence characteristics in the temporal signature of vegetated land surfaces. In cropping practice terminology, it basically refers to phases from sowing to harvesting. In heterogeneous landscapes, these provide valuable clues on various practices. Each 2

15 Figure 1.1 Satellite time-series composition. vegetation class has a distinct phenological cycle and describes a unique shape in time [4], [5]. Although vegetations tend to vary their annual phenological cycles - may be irregular in the timing of greening and senescence phase, length of growing period, maximum vegetation growth etc - because of different geographic and climatic conditions, these variations are not extreme and shape can remain similar. The shape-characteristic of the temporal signature is exploited in the proposed approach for classification of different vegetation cover. Detecting changes over time for such regions can help in understanding the underlying processes that trigger these changes. Figure 1.2 shows the correlation between the behavior of time-series with different phases of crop/vegetation growth life cycle on the ground. Figure 1.2 [6] shows the typical crop cycle for single cropping practices, whereas, Figure 1.3 [6] shows the crop cycle for double cropping practices. The focus of this research is the examination of such time-series to extract information regarding the underlying processes that aid in building real-world applications. Figure 1.2 Phenological vegetation cycle for single crop reflected in vegetation index time-series. Different stages in the phenological cycle of a crop are marked. 3

16 Figure 1.3 Phenological vegetation cycle for double crop reflected in vegetation index time-series. Our aim is to detect major changes occurring in vegetation covers with emphasis on agricultural landscape using the time-series data while disregarding phenological changes i.e. intra-variations in phenological cycles of a particular agriculture class. These variations are bound to be present due to the high degree of variability between years, across different geography, that is caused due to short-term climatic fluctuations in temperature and rainfall patterns etc [7]. 1.4 Thesis Objective and Research Questions The objective of the thesis is to propose a technique for vegetation land cover classification that exploits the shape-characteristics of the phenological cycles that are visible in the time-series data. And to appreciate the utility of such an approach for analyzing higher-level systems and building models with applications in drought monitoring systems, generating irrigations maps (ground-water irrigation heat map), the effectiveness of new irrigation infrastructure or any other such event. The thesis is centered around understanding the vegetation cover dynamics. For better strategic planning and policy-making decisions, most users and local entities rely on the information that is local to a particular region. Therefore, the information gathered as a result of a continuous assessment of landscape changes need to be available at regional scales. We investigate the general applicability of high temporal resolution satellite imagery datasets available at a moderate spatial resolution for vegetationrelated analysis and devising vegetation cover classification techniques at such regional level. Emphasis has been laid on the identification of different classes prevailing in the agriculturally occupied area. Problems that are addressed in this research are summarized as follows: 1. Can the time-series dataset available from MODIS 250m satellite imagery be used for accurate mapping of vegetation cover at the district level? 4

17 2. How can we exploit the shape-characteristics of time-series data for building a supervised and unsupervised classification systems? 3. Can we detect major changes using the proposed technique? Further, can we capture where, when and, what kind of change occurred? 4. Can any hidden knowledge about the phenomena or the possible drivers that triggered such changes be derived by detecting such events? Is it possible to do pre/post-analysis of such changeevents and extract information? 5. How can we build an automated system with pixel-level classification? 1.5 Summary of Contribution 1. An approach for analyzing spatio-temporal data based on the shape-characteristics has been proposed. Highlights the significance of refining training samples. The pipeline is computationally made faster by calculating representative time-series curves of each class by applying data-mining tools. 2. A novel technique that ensembles the shape-based characteristics (i.e dynamic nature of the vegetation response captured by the satellite) and the feature-based characteristics (derived from the temporal signature formed using vegetation response) of the multi-spectral time-series for vegetation cover classification has been proposed in this research work. Further, each of the agricultural land classes is sub-classified to capture intra-variability within a particular class on basis of phenological parameters. 3. An unsupervised technique to extract unique time-series curves existing in a particular region is proposed. It covers automatic extraction and optimization of reference/training dataset. It outputs a spatial intra-variability map that indirectly captures phenological variations of a particular class and corresponding homogenous spatial clusters where it is prevalent. 4. A qualitative assessment using statistical data and a spatial assessment using sampled data. Change detection and analysis for identification of possible drivers responsible for the change. Correlation of cropping practices with rainfall patterns and investigation of the effect of drought is carried out. 1.6 Outline of the Thesis Chapter 2: surveys several time-series analysis techniques and research for studying various environmental phenomena in the domain of remote sensing. 5

18 Chapter 3: describes the attributes of MODIS and NRSC reference datasets used in this research. It includes the description of the study area as well. This chapter additionally presents distinctive vegetation classes and their typical characteristics that were studied in this work. Also, several challenges that are involved in doing time-series analysis are discussed. Chapter 4: discusses the pre-processing steps and the technique used for time-series data filtering. Talks about the procedure for acquiring an adequate set of training samples that accurately represent the ground truth information. Finally, this section demonstrates how shape-characteristics of time-series curve can be exploited to build a supervised approach for vegetation cover classification. Additionally, data-mining tools such as K-medoids clustering, DTW, and DBA, that are crucial for working of the proposed approach is described. Chapter 5: describes a novel feature-based classification procedure, that consolidates shapepolarity, devised for the characterization of time-series patterns. Demonstrates the method used to derive phenological parameters from EVI time-series that quantifies the vegetation growth process. The utility of phenological parameters towards understanding intra-variability and deriving a season calendar is shown. Chapter 6: proposes a data-driven unsupervised approach for building an automated framework for spatio-temporal data analysis. It presents a spatial segmentation based sampling strategy combined with K-medoids clustering technique that is computationally effective and viably detects unique temporal signatures that exist in a region. A spatial variability map for double cropping practices is also acquired as an output, that reveals spatial clusters that are homogeneous in nature in terms of cropping technique being practiced. Chapter 7: introduces diverse kind of change patterns, either denoting a positive or a negative change. Change analysis of West Godavari district over four agricultural periods was carried out, and major changes were observed on a transition from the agricultural year to Quantitative and qualitative assessments of such changes detected the spatial cluster of regions that were correlated. On further investigation of the impact of drought year on agricultural practices, revealed that such an event can have an immediate effect. However, it can also indirectly manifest over some time. Such an experiment demonstrates the potential application of producing rainfed versus non-rainfed agricultural outline map from the change-event analysis. 6

19 Chapter 2 Literature Survey 2.1 Satellite Remote Sensing Prior to the satellite imagery, the remote sensing started with aerial photographs that were manually interpreted by experts to assemble land cover information [8]. In satellite remote sensing, the major approaches followed for mapping land resources are spectral and temporal image classification. Difference between multi-spectral and multi-temporal (or hyper-spectral and hyper-temporal) data lies in their variable components, spectral domain, and temporal domain respectively Spectral Image Classification - Supervised and Unsupervised Approaches Spectral techniques use the spectral information captured by one or more spectral bands having different wavelengths, and attempts to classify each individual pixel based on its spectral response pattern. Typically, each class of objects on ground describes a different spectral response of the radiance. An object s spectral curve can be regarded as its spectral signature. In an unsupervised approach, generally, the contextual information is also incorporated into the classification. Spectral-spatial unsupervised classification performs natural grouping of pixels based on spectral signatures and assigns each image pixel to a particular class. Unsupervised classification is similar to cluster analysis, most commonly used clustering algorithms for satellite image classification are K-means clustering, ISODATA clustering, and maximum likelihood classification [9]. An unsupervised classification identifies a number of clusters in an output image, and each pixel is assigned to a particular cluster. The clusters generated may not equate well to the classification schema of land cover types of interest. Thus, each cluster needs to be assigned a meaningful label, manually by the user. In heterogeneous landscape, often output contains too many land cover classes that need to be combined to obtain a meaningful characterization of land covers. In other cases, classification method may not identify an adequate number of clusters to describe land cover types of interest. Therefore a cluster must be split into different subclasses. Unsupervised classification is useful when the user cannot accu- 7

20 rately specify training sets for different cover types or there is no pre-existing field data. In spectral and contextual supervised classification, manual delineation of training samples based on expert knowledge remains the preferred choice [10]. The methods use the spectral information of the pixels within each training area to train the classification model. And then based on the spectral properties of a pixel and trained model a class is assigned. Still, several drawbacks remain such as conventional techniques fail at distinguishing a large number of land cover classes with an acceptable misclassification rate as different land covers can share similar spectral signatures. Although methods discussed exploit the whole data, they are not able to identify specific temporal behaviours. In essence, unable to extract and utilize temporal nature. 2.2 Multi-Temporal Satellite Datasets Several studies have firmly established the advantages of multi-temporal image analysis for environmental modelling, land cover mapping etc. The multi-temporal data allows characterizing objects based on their dynamic processes rather than static properties. Multi-temporal data that offers a high temporal resolution, because of high-revisit frequency, are available from satellite platforms such as National Oceanic and Atmospheric Administration (NOAA), Advanced Very High Resolution Radiometer (AVHRR) and Moderate Resolution Imaging Spectroradiometer (MODIS). Studies [11] [12] shows that greenness attribute of vegetation cover can be monitored through its spectral features. Vegetation Indices are mathematical formulations of spectral responses in the red and near-infrared band that can describe changes in green vegetation. Various land-use land-cover maps at national and global scales were prepared using AVHRR NDVI datasets [13] [14]. Main drawbacks of using AVHRR data is poor geometric precision and inadequate radiometric calibration [15] [16]. The use of multi-temporal MODIS vegetation Indices dataset to produce land-use land-cover maps have been demonstrated in several studies [17] [18] [19] MODIS dataset - Vegetation Indices For creating land-cover maps, vegetation classification, and phenological investigation etc., commonly Normalized Difference Vegetation Index (NDVI) fill in as the base. In spite of the fact that NDVI has its points of interest, it is susceptive to atmospheric and soil influences. And in dense canopies, the responsiveness of NDVI towards changes in the amount of vegetation is reduced [20]. In contrast to NDVI, the Enhanced Vegetation Index (EVI) is more precise, has improved sensitivity to vegetation differences in high biomass regions, and can better describe the sharper ends in the growth cycle of vegetation [21] [22]. 8

21 2.3 Time-Series Analysis and Classification Given a satellite data spanning across space and time, several time-series analysis techniques have been used to study various environmental phenomena, characterize vegetation cover, and variations occurring in it over time. Satellite time-series data have been used to map forest disturbance [23], for land cover classification [24], Fourier analysis to monitor seasonal variations in vegetation phenologies and their inter-annual change [25], for detecting trend and seasonal changes [7], wavelet analysis for crop expansion and intensification [26], wavelet analysis in conjunction with time-series models to understand relationship between climate and vegetation dynamics that vary at inter-annual and intra-seasonal scales [27], environmental anomaly indicator system at continental scale using fuzzy approach [28]. For classification of vegetation cover, a number of techniques have been developed using spectral responses and temporal information available from satellite images. Traditional approaches used satellite responses captured at specific periods of time for classification using different separability measurements [29] [3], but the shortcoming of such single-scene or multi-scene satellite imagery is that it cannot describe the dynamic process of vegetation growth and does not provide any insight to such phenomena. In a majority of other studies, features derived from satellite-based time-series data have been used with conventional classification techniques like decision tree etc, for classifying vegetation cover [30]. Features are extracted to capture the information contained in the phenological cycles of vegetation time-series curve [31] [32], but the full temporal detail is not exploited [25] i.e. to say time-series in itself has not been used as an inherent class identifier. Another drawback of such feature-based approaches is that they require adequate training sample that represents a full range of variability, and are highly dependent on the feature-selection techniques used to characterize the phenology. Although the value of time-series data for monitoring vegetation has been firmly established, methods suffered lack of consistency in classification schema used and only a limited number of methods for exploring and understanding phenological seasons from such data series have been developed. Our approach is based on the observation that the different vegetation classes exhibit specific spectral reflectance as functions of time. The objective of this research is to exploit the shape-characteristics of time-series and develop an approach for improved sampling of training data and classification of vegetation cover over a large regional scale. And this process when applied over time and across space can help in providing valuable clues to understand major changes in vegetation covers and regions of change. 9

22 Chapter 3 Dataset and Time-Series Analysis 3.1 MODIS Satellite Imagery MODIS (or Moderate Resolution Imaging Spectroradiometer) is a key instrument aboard the Terra (EOS AM) and Aqua (EOS PM) satellites. Terra s orbit around the Earth is timed so that it passes from north to south across the equator in the morning, while Aqua passes south to north over the equator in the afternoon. Terra MODIS and Aqua MODIS are viewing the entire Earth s surface every 1 to 2 days, acquiring data in 36 spectral bands, or groups of wavelengths. These data have improved our understanding of global dynamics and processes occurring on the land, in the oceans, and in the lower atmosphere. MODIS is playing a vital role in the development of validated, global, interactive Earth system models that may be able to predict global change accurately enough to assist policymakers in making sound decisions including the protection of our environment [GSFC] [33]. MODIS gives a worldwide, repetitive coverage of multi-spectral, multi-temporal imagery and a suite of higher-level science quality products in the help of global environmental change research. The MODIS 250-m Vegetation Index (VI) product (MOD13Q1), which comprises of NDVI (Normalized Difference Vegetation Index) and EVI (Enhanced Vegetation Index) information composited at 16-day interims, holds significant guarantee for regional-scale crop mapping given its resolutions, substantial region scope, and cost-free status. VIs represent dimensionless radiometric measures of green vegetation condition and are associated with biophysical parameters such as biomass and leaf area index (LAI). Therefore, multi-temporal VI data has been broadly utilized for classifying land cover based on their seasonal spectral differences and to characterize major phenological events. 3.2 Dataset Characteristics Figure 3.1 shows the MODIS grids and Table 3.1 shows dataset characteristics. Modis satellite datasets is available at a global scale and is divided up into tiled grids. The products are thus provided and distributed in adjacent non-overlapping tiles that are approximately 10 degrees square (at the equa- 10

23 tor). The MODIS products are available in the sinusoidal tile grids [34]. Tile h25v07 that covers almost the southern part of India (Andhra Pradesh, Telangana and Karnataka states) is the study area in this research. Figure 3.1 Tile layout of MODIS satellite data. Tile h25v07 that covers Andhra Pradesh, Telangana and Karnataka states of India is shown in Figure 3.4. Area Projection Dimensions Resolution Data Format 10 x 10 lat/long Sinusoidal 4800 x 4800 rows/columns 250 meters HDF-EOS Table 3.1 MODIS satellite data info. MODIS vegetation indices provide consistent spatial and temporal comparisons of vegetation canopy greenness, a composite property of leaf area, chlorophyll and canopy structure. Two vegetation index (VI) algorithms are produced globally; the normalized difference vegetation index (NDVI) and the enhanced vegetation index (EVI), which minimizes canopy-soil variations, improve sensitivity over dense biomass regions and improved vegetation monitoring through a reduction in atmosphere influences [35] Vegetation Index When sunlight strikes objects, certain wavelengths of the spectrum are absorbed and other wavelengths are reflected. To determine the density of green on a patch of land, satellite sensors observe the distinct colours (wavelengths) of visible and near-infrared sunlight reflected by the plants. The pigment 11

24 in plant leaves, chlorophyll, strongly absorbs visible light (from 0.4 to 0.7 m) for use in photosynthesis. The cell structure of the leaves, on the other hand, strongly reflects near-infrared light (from 0.7 to 1.1 m). The more leaves a plant has, the more these wavelengths of light are affected, respectively [36]. Different types of vegetation can be identified on the basis of their reflectance properties. Also, plant covers can be distinguished from forest covers based on how reflectance properties vary as a function of time. A vegetation index is a simple numerical indicator that can be used to analyze remote sensing measurements and assess whether the target being observed contains green vegetation or not. It is an empirical measure of vegetation activity at the land surface [37]. Figure 3.2 Illustration of NDVI calculation. The figure illustrates the reflectance property of healthy vegetation v/s unhealthy or sparse vegetation in visible and near Infrared spectrum of light and how reflectance properties can be used to detect the greenness of any vegetation. Figure 3.2 depicts how NDVI is calculated from the visible and near-infrared light reflected by vegetation. Healthy vegetation (left) absorbs most of the visible light that hits it and reflects a large portion of the near-infrared light. Unhealthy or sparse vegetation (right) reflects more visible light and less near-infrared light. A difference between NIR and RED wavelength of reflectance is a unique signature of vegetation. Thus, NDVI (Normalized Difference Vegetation Index) is being defined as NDV I = NIR RED NIR + RED The EVI was developed to optimize the vegetation signal and it improves on the venerable NDVI. It is more sensitive to differences in heavily vegetated areas and better corrects for atmospheric haze as well as the land surface background noise beneath the vegetation. The equation takes the form, EV I = r NIR r red r NIR + C 1 r red C 2 r blue + L G 12

25 where r values are atmospherically corrected (Rayleigh and ozone absorption) surface reflectance, L is the canopy background adjustment term, and C1, C2 are the coefficients of the aerosol resistance term, which uses the blue band to correct for aerosol influences in the red band. As per the MOD13Q1 product description, EVI value varies in the range -0.2 to 1.0. Figure 3.3 shows a sample MODIS EVI image obtained from MOD13Q1 product, covering the southern part of India. Figure 3.3 MODIS EVI dataset image. Figure 3.4 shows a sample EVI image loaded in QGIS software. West Godavari district is highlighted in the image. QGIS was used for the subsetting and masking to obtain the data for the required regions (West Godavari district). Figure 3.4 EVI image and mask of West Godavari district loaded in QGIS software. 13

26 3.3 Data for Accuracy Assessment National land use and land cover dataset derived from the AWiFS sensor and produced on an annual basis by National Remote Sensing Centre (NRSC), Indian Space Research Organisation, Hyderabad, India was used as a reference data for evaluating vegetation cover classification results. The dataset contains 16 classes and is at a spatial resolution of 56-m. Figure 3.5 shows an example of the land cover dataset of West Godavari district for crop calendar year that was used for validation. Before evaluating the classification results, QGIS software was used for the rescaling of the data. Figure 3.5 NRSC reference data. 3.4 Study Area In the present study, we used MODIS 250-m resolution Enhanced Vegetation Index (EVI) product - MOD13Q1. Modis tile h25v07 covers the southern region of India (Andhra Pradesh, Telangana and Karnataka states), where our study area lies. The dataset is composited at 16-day intervals and so it provides 23 observations per year, at a spatial resolution of 250-m. The MODIS data used in this study cover 5-year period from 2008 to 2012, consisting of 23 images per year, giving 115 images in total. Sample MODIS dataset image covering the southern part of India is shown in Figure 3.4. In the present work, the study area covers the West Godavari district of Andhra Pradesh state, located in Southern India (highlighted in Figure 3.4). The district is located in the delta region of the Godavari river and has a large extent of fertile agricultural land. It covers an area of 7,742 km 2. The district is 14

27 situated 80 o 52 and 82 o 28 E, of eastern longitudes and 16 o 32 and 17 o 50 N, of northern latitudes. The vast majority of the cultivable land is irrigated through canals, tanks, and wells. For instance, the central source of irrigation in the district are canals from the Godavari and the Krishna rivers and the network of open head channels from minor rivers like Tammileru, Erra Kaluva, Juleru, Bynere, Ramileruand Gunderu besides a good number of tanks and wells. Of all these water sources the Godavari is the major source of irrigation in this district. The District receives its rainfall generally and predominantly from South West as well as North East monsoon, on which the agricultural activity depends directly or indirectly. 3.5 Vegetation Classes and Temporal Signature In India, agricultural-year is from June to May. The Indian cropping season consists of three primary seasons - (i) Kharif (ii) Rabi and (iii) Zaid. The Kharif cropping season is from June to October/November during the south-west monsoon, crops grown in this period are referred to as Kharif crops (or monsoon crops). The Rabi cropping season is from October/November to March, crops grown in this period are referred to as Rabi crops (or winter crops). The crops grown between March and June are Zaid crops (summer crops). In agriculture, single-cropping is the practice of growing only one crop, whereas multi-cropping refers to the cultivation of two or more crops at the same crop field during a single agricultural-year. Figure 1.2 depicts the single cropping practice whereas Figure 1.3 shows the double cropping practice. In the present study, along with these four agriculture classes - kharif, rabi, zaid, and double cropping (Kharif + rabi etc), two more major vegetation classes were considered, which are plantation and forest. The remaining set of classes such as urban land, fallow land, and waterbody were combined into a single class labeled as no/low vegetation class. Representative temporal signature of each vegetation class shown in Figure 4.6 and Figure 4.7. Table 3.2 summarizes the typical nature of temporal signatures of different land cover classes [38]. 3.6 Challenges with Time-series Analysis 1. Generally, land cover products have limited number of thematic classes and are infrequently updated. Classes considered are inadequate to reflect common agricultural land cover changes. Thus, detailed regional scale cropping patterns based on time-series data need to be mapped on a repetitive basis to monitor change-events. 2. Homogenous cropping practice vs Heterogeneous cropping practice: Tackling the issue of intravariability present within each class and also within the same crop. For instance, different kharif crops (or even same crop, but with different geographical and climatic conditions) will have distinct phenological cycle and different characteristics associated with it, even though all coincides with the monsoon cropping season. So to address the problem, a method needs to developed that 15

28 S. No. Class Characteristics 1 Water bodies It refers to lakes, ponds, inland water, rivers, and streams etc. VI value is very low for such bodies. 2 Urban Refers to human infrastructure such as cities, built-up land, buildings, roads etc. VI values are low and uniform throughout the year. 3 Wastelands and Fallow land 4 Kharif cropping Barren or degraded land that does not or no longer supports vegetation. Includes degraded forest, drought-struck areas, eroded land etc. Deteriorated either due to natural factors or improper management of soil and water resources. Fallow land temporally left uncultivated for a season after being successively cropped. Low EVI value and low amplitudes in the temporal signature and has non-uniform patterns. Single cropping practice where crops (rain-fed) are grown in the monsoon season. Land, where dryland farming is practised as irrigation resources are limited and agricultural activities depend on the rainfall. EVI value reaches the peak in monsoon period (Aug/Sept). Intra variations are present in temporal signatures based on growth cycles (shift of +- 1 month is observed, also shown in Figure 4.6). 5 Rabi Single cropping practice where crops are grown after the rainy season. Regions with better and more sources of irrigation to support agriculture in the winter season. Groundwater irrigation and developed irrigation strategies are used for cultivation. 6 Zaid Summer crops that are grown during a period between winter and monsoon crop season. Areas have good irrigation facilities and fertile soils. 7 Double cropping The region supports two cropping seasons in a year due to better irrigation resources. 8 Plantation Agricultural or non-agricultural areas under tree plantation, adopting certain management techniques. High peak but have low amplitude value, as EVI value is relatively high throughout the year. 9 Forest High EVI value, predominantly remain green throughout the year. Due to shredding of leaves during the autumn season, EVI value drops (Deciduous forest). Table 3.2 Land cover classes and characteristics of temporal signatures. 16

29 ignores intra-phenological variations while being able to differentiate major classes. Further, how to capture and understand intra-variations within each class? 3. A supervised approach based on the trajectory of time-series only works well if the unlabelled time-series curve matches one of the temporal signatures from the library of the extracted training dataset. Thus, database reflecting the ground truth information needs to be collected, that is sufficient and succinct. 4. What kind and set of parameters to extract from time-series data to build a feature-based approach? The challenge is that at various stages of vegetation s intra-annual growth cycle, distinct vegetation types can appear similar and vegetation of the same type significantly dissimilar. 5. How to build an unsupervised approach that utilizes shape-characteristics of time-series data for regional scale vegetation cover classification, as all information or users act at that scale? How to capture intra-variability and tackle issues of heterogeneous cropping practice present in the region? 6. For agriculture land cover analysis and change-event detection, approaches are still not welldefined and there is no systematic framework in place. The challenge is to develop time-series analysis techniques to build a system that helps with the understanding of land cover dynamics and various phenomena associated with it. 17

30 Chapter 4 A Shape-based Supervised Approach for Time-Series Classification 4.1 Flow Chart The flowchart illustrated in Figure 4.1 and Figure 4.2 depicts the methodical workflow adopted in the current study. Figure 4.1 describes the steps to preprocess the raw satellite data. Preprocessing includes steps to filter noises and remove anomalies from the raw time-series data. Figure 4.2 describes the methodology used for supervised classification. Figure 4.1 Steps to preprocess the raw time-series data. 18

31 Figure 4.2 System pipeline depicting steps for refining training data and for vegetation cover classification. 4.2 Data Preprocessing Data Extraction A sequence of contiguous satellite images are stacked to construct raw time-series data using EVI value for each pixel over a period of time. In the present study, time-series are split into segments according to the agricultural-year of India (from June of the first year to May of next year). A timeseries can be thought of as a transition from one agricultural-year to the next. The vegetation pattern for each agricultural-year is processed separately Cloud Correction In case of multi-temporal satellite data, atmospheric influences and sensor malfunctioning often lead to data gaps and data anomalies. Missing values are referred to as data gaps whereas an abrupt fluctuation in the value is defined as data anomaly. Data anomalies can alter the resulting time-series that are not congenial with the gradual process of vegetation growth. Such data may no longer reflect the ground process and can distort the analysis leading to wrong information being elicited. The anomalies need to 19

32 be corrected before any further processing. Clouds and poor atmospheric conditions such as haze usually depress vegetation index values and cause sudden drops, and are to be regarded as noise and removed. MODIS product also provides pixellevel metadata that includes quality assurance (QA) information. One of the QA parameters indicates if the data captured for a particular pixel is not produced due to cloud effects and is unreliable or of unquantifiable quality. The cloud QA parameter is used for the revision of time-series data in the method discussed below to subdue the effect of such erratic data values. For cloud correction, revision of data is carried out by applying time-series filter processing using temporal window operation that extracts the local maximum value. The equation is taken from the first step of local maximum fitting technique devised in [39] and Min Max filtering method for cloud correction explored in [40]. The equation below describes the time series filter : d t = Min[Max(d t w+1, d t w+2,..., d t ), Max(d t, d t+1,..., d t+w 1 )] d t : observed data at time t w : filter window d t : modified data at time t Figure 4.3 Local minima (considered as noise that needs to be corrected to create smoothened timeseries curve) is present between a left and right local maxima (both local maxima are marked using black dots). The data point at the local minima on the curve is updated (updated to the green dot) using local Min-Max filter described above. Window filter size is the number of data points which we use to filter noise at the considering point. If in the window w the change in value is monotonous or if at time t the value is maximum, the filter does not alter it. Only if the value d at the time t locally decreases it is replaced. 20

33 For our case, the window size was determined based on the weather condition. Firstly, for the point in consideration, the number of contiguous cloud days were calculated using the QA parameter (available along with the EVI values at each data point which indicate if the measurement was unreliable due to atmospheric influences). Different window filter size is used for both sides, i.e W left, W right is adjusted to the number of contiguous cloud days at the left and right side of the considered point. We assume data point value to be erroneous and remove noise if the window size is around 2 to 3. Thus cloud QA parameter is used to decide the window sizes. Cloud Filter = Min[Max(W left ), Max(W right )] Curve Smoothing Although above filter processing eradicates extreme signal variations and improves quality of unreliable data points, data analysis remains affected by other negative effects. A number of methods for eliminating data gaps and anomalies to derive the time-series that match with the actual process of change have been formulated, applied, and evaluated. The study in [41] provides an evaluation and comparison of several smoothing algorithms for multi-temporal vegetation index profiles. Popularly used techniques for noise reduction and extracting time-series data of high-quality have been proposed, such as asymmetric function fitting approaches, Gaussian function fitting approach [31], Fourier-based fitting methods [42] and [43], weighted least-squares linear regression method[44] and logistic function fitting etc. In the present study, Adaptive Savitzky-Golay filter technique devised in [45], is used to smoothen the raw time-series data and reduce the noise. The technique is based on two assumptions: (1) The vegetation index data captured by satellite imagery is associated and aptly reflects changes in vegetation component. (2) Atmospheric influences usually cause a drop in VI values. Based on these assumptions, SavitzkyGolay filter was developed to make data approach the upper VI envelope and to best fit the VI variations during the full vegetational season through an iteration process [45]. It is a simplified leastsquares-fit convolution for smoothing and computing derivatives of a set of consecutive values. The convolution can be understood as a weighted moving average filter with weighting given as a polynomial of a certain degree. When applied to a signal, it performs a polynomial least-squares fit within the filter window. This polynomial is designed to preserve higher moments within the data and to reduce the bias introduced by the filter. Savitzy-Golay technique has the advantage of preserving the original shape and features of the signal better than other approaches. It finds the derived curve by fitting successive subsets of adjacent data points over an odd-sized window with a low-degree polynomial by the method of linear least squares. 21

34 A window size of 7 with polynomial order 4 was found appropriate for this dataset. In Figure 4.4, for a sample pixel, its raw time-series and derived time-series curve are shown. Figure 4.4 For a pixel, its raw time-series curve formed using 23 data points for agricultural-year and derived time-series curve obtained after the smoothing step is shown. 4.3 Ground Truth Information Supervised classification approaches are the function of training samples. Selection of correct training samples that accurately denotes the ground information is crucial. In remote sensing, although a number of techniques for classifying land cover objects are explored, typically training samples are manually selected by experts for supervised classification. We propose a procedure for extraction and refinement to obtain good training samples. The next step is to gather a database of time-series patterns that represents the ground truth information. MODIS data along with the NRSC reference data of the previous agricultural-year was used to construct the database [datasets discussed in section 3.1 and section 3.3]. For each vegetation class, representative ground truth data i.e. samples of time-series curves were collected. The database of collected temporal signatures representing the ground truth data will later be required as a training sample in the classification step of our methodology. Temporal patterns corresponding to different vegetation classes in the database need to be error-free as they dictate the accuracy of the classification step. For each vegetation class, the ground truth data was collected from the homogenous regions in order to avoid any inclusion of outliers i.e. time-series curves belonging to any other vegetation class. 22

35 However, due to a coarser resolution of MODIS satellite data, the extracted time-series pattern may not correspond to the ground truth information obtained from NRSC reference data which is at a higher resolution. Due to a coarser resolution of data, features of discrete classes may be present in the majority of the pixels. Therefore random sampling may not correspond to homogenous land cover regions. So instead, pixels that indicate pure land cover class need to be deterministically selected. Moreover, we assume NRSC reference data to be error-free. Therefore, even though homogenous regions were used to collect the ground truth data, outliers may still be present which need to be eliminated. The next step is to identify and extract only the core samples for each vegetation class. We viewed this problem as a time-series clustering problem. For each vegetation class, we perform clustering and select those temporal patterns which are relatively close to the cluster center and eliminate rest of the samples from the temporal signature library by claiming them as potential outliers. In time-series clustering, it is crucial to decide what kind of similarity is important for the clustering application. Accordingly, an appropriate clustering algorithm and an appropriate distance measure should be chosen. In the next section, we discuss the distance metric and the clustering algorithm used in our methodology. 4.4 Distance Metric and Clustering Dynamic Time Warping Dynamic Time Warping (DTW) proposed in [46] [47] is a ubiquitous tool that has been used in various time-series applications. For satellite-based time-series analysis, DTW has been used in research studies [48], [49] and [50]. For time-series based applications, DTW is an accurate similarity measure and it tends to perform better than traditional distance metrics like Euclidean distance, etc [51]. DTW is a dynamic programming algorithm with polynomial time complexity that efficiently finds the optimal alignment. It makes uses of a matrix to store the optimal solutions to subproblems, and the alignment with minimal cost in the matrix is outputted as the warping path [47]. The algorithm is summarized and illustrated in Figure 4.5 [49]. DTW finds the optimal path that minimizes the wrapping cost of one sequence into the other. Dynamic programming can find the path very efficiently. Suppose A is a time-sequence of length n and B is a time-sequence of length m. The cost of the optimal alignment of two sequences A and B respectively aligned up to the i th element of A and j th element of B can be recursively computed by: DT W (A i 1, B j 1 ) DT W (A i, B j ) = d(a i, b j ) + min DT W (A i, B j 1 ) DT W (A i 1, B j ) where A i is a sequence a 1,a 2,...,a i. The overall similarity is then given by DTW(A n, B m ). 23

36 Figure 4.5 Depicts the resulting alignment of two slightly time-shifted sequences. The optimal warping path is outlined in green in the DTW matrix. In the present study, DTW which reflects similarity in shape is used as a distance metric. DTW algorithm finds an optimal alignment between two given time-sequences, by explaining variations in Y-axis (curve) by warping X-axis (time) and provides a distance measures that quantify the similarity between the two sequences. But the crucial observation is that standard DTW algorithm totally disregards the time dimension while finding alignment between two time-sequences. In our study, this can lead to unintuitive alignments, for instance, the temporal pattern of a monsoon crop shown in Figure 4.9 (Kharif) can match to the temporal temporal pattern of a winter crop shown in Figure 4.8 (Rabi), as both are similar in shape but shifted in time. To avoid these temporal inconsistencies, we use a variant of DTW that introduces a temporal constraint, namely the Constrained Dynamic Time Warping (CDTW) method. CDTW introduces a window size on the temporal axis that essentially constrains the possible warping allowed only within that window [47]. Also, CDTW significantly reduces the computation time as compared to DTW. In the present study, time-series are indicative of the annual phenological cycles of different vegetation types. For each vegetation class, cycles tend to be irregular - in terms of timing of greening and senescence phase, length of growing period etc, but irregularities are not that extreme and typically temporal shift of one to two months is observed. Thus, in the present study, owing to this fact we use +/-2 as the window size which basically allows matching within the temporal shift of two months overcoming the problem of DTW K-medoids Clustering For time-series clustering, the k-medoids clustering technique is adopted, being one of the standard approaches [52] [53]. For time-series data, traditional k-means clustering technique tends to fail - in 24

37 capturing the shape-characteristics of time-series curve - as it is modelled to work in the euclidean space [54]. Rather than computing average of cluster members, k-medoids finds a new cluster center by selecting an existing data point within each cluster that best represents its cluster center. k-medoids aims at minimizing the intra-cluster sum of squares, by using the proximity of objects to the medoids of the clusters formed by the algorithm. For each vegetation class, k-medoids clustering technique along with CDTW as a distance metric was used for the refinement of ground truth data to extract core samples. Partitioning Around Medoids (PAM) algorithm that is based on a greedy method is the standard implementation of k-medoids clustering [55]. Steps for the algorithm can be summarized as: 1. Initialize: Out of the n data members, k points are randomly selected as the medoids. 2. Assignment step: Each member in the data is associated to the closed medoid (DTW distance metric). 3. While the cost decreases: Update step: For each medoid m and each non-medoid data point o (1) Swap m and o, recalculate the cost of the configuration (sum of distances of points to their medoid) (2) Revert the swap operation if the total cost increases. In this study, we additionally added iteration parameter to the K-medoids calculation, where clustering was performed on the whole dataset for 10 times. The iteration that yields clusters with the least intra-cluster sum of squares was chosen. The results obtained as a result of K-medoids clustering are shown in Figure 4.6 and Figure 4.7 for various classes. 4.5 Classification In this study, time-series classification is considered as a curve-matching problem. For a particular pixel, its unlabelled time-series curve is compared against the representative temporal signatures of different classes, present in the library derived from the ground truth data. For classification of time-series data, Neighbour algorithm (given a query, it finds the proximal data point within the training set) with DTW as a distance metric is a viable solution and in several cases, it outperforms other classification techniques [51]. However, this approach has one drawback, it is computationally too expensive for any real-time application with large database. In NN classification, the training set is used to classify query sequences and its size dictates the time-complexity of the algorithm. Thus, the cost depends upon the number of data points present in the training set against which the distances are computed. To make the process cost-effective the size of training set can be decreased by implementing clustering and matching the query to the centroids of the clusters. This approach is referred as Nearest centroid classifier in which the training set size is reduced down to one average example per class [55]. 25

38 Figure 4.6 For each class, core samples of time-series curves extracted using clustering algorithm. 26

39 Figure 4.7 For each class, core samples of time-series curves extracted using clustering algorithm. 27

40 Figure 4.8 Representative temporal signature for one agricultural-year. For each class, the vertical spectrum at 23 data points is also shown, that denotes the variability. 28

41 Figure 4.9 Representative temporal signature for one agricultural-year. For each class, the vertical spectrum at 23 data points is also shown, that denotes the variability. 29

42 4.5.1 Optimization - Shape Averaging To deal with the problem of expensive computational complexity, we reduce the size of training data. In our study, for time-series analysis purpose, this can be done by constructing a representative curve for each vegetation class using core samples (extracted earlier); and then later use the representative curves for the classification (Figure 4.8 and Figure 4.9). Here, we take advantage of two critical facts that each vegetation class has a distinct temporal pattern and core samples belonging to a particular vegetation class typically have similar shape-characteristics. In the present study, DTW Barycenter Averaging (DBA) technique demonstrated in [56] is deployed to construct the representative time-series curves. DBA is an averaging method for time-series data based upon DTW distance metric. Furthermore, application of such techniques and its relevance for time-series classification has been shown in [57]. Temporal signature of different vegetation classes using representative curves (obtained using DBA) are shown in Figure 4.8 and Figure 4.9. For each class, the vertical spectrum at 23 data points is also shown in figures, that denotes the variation in time-series curves that were present in the core samples DTW Barycenter Averaging (DBA) DBA is an iterative algorithm that follows an expectation maximization approach. At each iteration, the average time-series is refined and updated. The algorithm has two steps and it briefly works as follows: 1. The n series to be averaged are labeled S 1,...,S n. 2. Intilization: Begin with an initial average series T avg. 3. Alignment: In the dataset D, for every sequence S to be averaged the best alignment of S with T avg under DTW is calculated and the path is stored. 4. Update: Construct a new average T avg by giving each point an updated value: the average of every point from S connected to it in the DTW path. The detailed step-by-step instructions along with the pseudocode for the algorithm are described in the research paper [56]. The final result is subject to the initial average sequence, thus a good initialization is extremely important. The purpose of the DBA is to preserve the shape-characteristics such as extreme magnitudes in y-axis and timing of those extremes on the x-axis of input sequences. In the initialization step, the medoid member of the dataset is selected as T avg. The intra-cluster sum of squares from all data points in the set is used as a distance metric to determine the medoid member. The author proves that the algorithm seeded with the medoid converges. 30

43 4.5.2 Nearest Neighbour Classification Finally, for the classification of input data, NN-CDTW i.e. Nearest neighbour with CDTW as a distance metric was used. As each class is represented by a unique representative curve, each pixels time-series curve is classified based on its CDTW distance threshold to these. NN is computationally less demanding than others, given the data vector size. 4.6 Results Vegetation Cover Map The time-series curve of each pixel is labeled using the proposed approach. Figure 4.10 shows the vegetation cover map generated with the MODIS time-series data for the agricultural-year over the West Godavari district. The derived vegetation cover map shows the extent and spatial distribution of different vegetation covers - seven vegetation classes were considered Validation Results generated were compared with the land cover reference dataset obtained from NRSC. The NRSC data with 16 classes at a resolution of 56m was merged and resampled to obtain the 7 classes at 250m, to match the MODIS data (Figure 4.11). Classes with similar temporal characteristics in NRSC data were grouped together as they exhibit similar vegetation temporal patterns. No/low vegetation included Built-up, Wasteland, Rann, Current Fallow, Grassland, Littoral Swamp, Snow Cover and Water Bodies, while Forest covered Evergreen, Deciduous, and Scrub Forest. The rest were unaltered. In the down-scaling, statistical mode was considered as the class in non-homogenous regions. Thus, after rescaling of NRSC reference data, the pixel-based comparison was carried out with the derived vegetation cover map to evaluate the accuracy using confusion matrix. The confusion matrix shows cross-validation results for 6 classes, as zaid class was not present in West Godavari district. The results derived from the confusion matrix (Table 4.1) yield an overall accuracy of 81.43%. Based on spatial visualization it can be empirically observed that major classification error occurs at marginal regions that are heterogeneous or transitional in nature. The errors can be attributed to mixed pixels, i.e., where pixels in MODIS data contain a mixture in surface reflectance due to its coarse resolution. Another reason for misclassification could be due to rescaling and resampling of NRSC reference data. It can be observed (Figure 4.10 and Figure 4.11) that our proposed approach performs better in classifying regions that are relatively homogenous in nature. The error between water body (no/low vegetation) and plantation can be attributed to littoral swamps present in wetland regions. Whereas error between plantation and forest could be due to a high correlation between their phenological cycles. And 31

44 the temporal signature of other no/low vegetation classes tends to be irregular, as EVI is not designed for such land covers, which can also lead to errors. Predicted (No. of pixels (relative %)) (Agri. year ) No/low Kharif Rabi Double Plantation Forest vegetation No/low vegetation Kharif Rabi Double Plantation Forest Table 4.1 Confusion Matrix of results obtained from the proposed shape-based approach against statistics obtained from NRSC reference data for agricultural-year

45 Figure 4.10 Vegetation cover map generated for agricultural-year using proposed shape-based supervised approach and labels of seven classes. Figure 4.11 Rescaled and resampled NRSC reference data for agricultural-year used for validation. 33

46 Chapter 5 A Feature-based Approach and an Ensembling of the Shape-Dimension Figure 5.1 Feature-based framework for Spatio-temporal analysis. We propose an approach for analyzing spatio-temporal data based on shape-based polarity parameter that allows creating a new set of features which improves the overall classification accuracy and precision in agricultural classes. In vegetation cover classification, few challenging tasks are to distinguish agricultural component from other non-agricultural or vegetation components; and to be able to differentiate between different single croppings and double croppings. As mentioned earlier, the temporal signature of different vegetation class has different cyclic behaviour. For a particular class, even though its temporal signature may contain different states of variations, these variations are not to be considered as dissimilar classes. The cyclic nature captured from the temporal profiles can be parameterized to build classification techniques. However, techniques must be able to distinguish cycles in order to classify different vegetation covers and change patterns. 34

47 The core of this work is to present the new feature set obtained from time-series data, that produces an adequate set of properties to describe different vegetation patterns. The set of features extracted from time-series database are then fed to data-mining algorithms, that automatically generates vegetation cover classification models. The steps followed for the proposed method are depicted in Figure 5.1. The major step of the methodology includes detection of cycle and extraction of phenological features. The next step is the learning phase where the classification system is trained using the core samples of time-series database of different vegetation covers that were extracted earlier as ground truth data. The data mining algorithm creates a classification model based on the sample set, and then it is applied to the entire data set to create a vegetation cover map. 5.1 Feature Description Vegetation-growth cycle can be defined as the recurring cycle of activities related to the canopy emergence and senescence. In general, vegetations describe a systematic pattern of growth. The cropgrowth pattern can be visualized as an ideal bell-shaped curve, where the vegetation-greenness of crop first gradually increase up to the senescence period and then after the harvest, the vegetation-greenness starts to decline. The time-series curve shown in Figure 5.2 [58] represents typical crop growth cycle and indicates the crop health over time. To monitor any phenomenon, we require a set of parameters that are considered as good descriptors of the state and processes exhibited by such phenomenon. We can assess the deviation over time using the representative parameters. In vegetation analysis, the entire phenological cycles representing different phases often repeat yearly. Analysis of time-series data was carried out on an annual basis. The period starts in June of one year and finishes in May of the second year as per the agricultural year of India. In this work, we characterize vegetation patterns using several parameters that are listed below. Further, we segmented annual time-series into two parts as per the major cropping seasons of India, i.e kharif season and rabi season respectively. We calculate statistical features for both of the seasons separately; phenological parameters are calculated by detecting cycles. And shape-based polarity feature is integrated using distance vector. Thus the feature vector encapsulates the statistical nature of time-series in two seasons, phenological values and also the shape-characteristics of temporal signature. 1. Phenological Parameters Phenological parameters extracted are (a) Time for the start of the season (b) Seasonal amplitude (c) Time for the end of the season (d) Length of the growing period (LGP) (e) Time for the mid of the season. 2. Statistical Features We also included basic statistical features such as the mean, maximum/peak value, minimum/valley value, time of the valley, cycle s sum. 35

48 3. Shape-based Polarity Feature It s a distance vector of a given time-series curve with respect to different classes. We construct the distance vector by calculating DTW distance between the current time-series curve and representative time-series curve of each vegetation classes, that were constructed earlier using shape-averaging data mining tool DBA. In essence, entries of distance vector basically indicate the polarity in shape-dimension of given temporal signature towards different classes. Figure 5.2 Crop-growth cycle and critical points in cycles for deriving phenological parameters are marked. 5.2 Cycle Detection and Extraction of Phenological Parameters In order to process the time-series and correctly detect actual cycles, some basic quality criteria must be fulfilled [31]. Thus, the steps performed during data preprocessing of time-series such as curve smoothing and cloud correction are crucial. Cloud correction helps in eradicating random spikes and downfalls in time-series and curve smoothing overall improves the quality of time-series data. As seen in the methodology discussed below these steps are critical for finding actual peaks and valleys. The smoothed time series composed of 23 data points, obtained after executing Savitsky-Golay filtering mentioned in Chapter 4, is processed for finding parameters. In the current work, basic methodology is followed to detect cycle and to extract phenological parameters from it. Initially, all the critical points (maxima and minima in time-series) are found using the derivative technique. It is calculated from successive differences of adjacent data points, change of sign 36

49 of slopes indicates critical points (positive to negative indicates peak and likewise negative to positive indicates valley). After that, empirical thresholds are imposed on the values, i.e minima and maxima greater or lower than certain values are discarded. Further, some set of constraints are levied on their values to filter out critical points and in order to find potential peaks that actually corresponds to a cycle. Constraints included such as local maxima that are at least separated by minimum distance are only considered. Basically, it forces only the highest local maxima to be selected within a certain contiguous time window (as per the value of minimum distance set in time axis) and similarly for local minima. We use a gap of 3 data points as a minimum distance value for finding peaks and gap of 6 data points as the minimum value for finding valleys. Lastly, we only keep maximum peak between any two valleys. As a result, we obtain peaks that correctly corresponds to the actual phenological cycles. We can identify the position of the peak which denotes the parameter E - Time for the mid of the season. Also to note that the valleys found using above methodology may not represent the time for the start of the season or the time for the end of the season. Rest of the parameters can be extracted using the peaks detected. Figure 5.3 Slope-shifting and Extrapolation method to derive Phenological parameters. The slope-shifting method is then deployed to calculate the position of left-minimum and rightminimum for a given peak. Every triplet of left minimum, peak and right minimum denotes a cycle and all the five phenological parameters can be computed. In the slope-shifting method, slope refers to the angle made by a line joining a data point (value at some time t on time-series curve) and the peak point (calculated in the previous step) with the time-axis. Instead of using peak point for slope calculation, a certain buffer is allowed at both ends for computing left and right minimum. To find the position of left-minimum, we keep on shifting data points towards left starting from the peak position till the 37

50 slope drops below the certain empirical threshold and likewise position of the right minimum is also calculated. Further, we make certain corrections to these values as per our assumption that cycles are ideally bell-shaped. Minimum of values at left-minimum and right-minimum position is considered as the base level. Then the extrapolating step is carried out that corrects the position of another minimum by calculating the intersection point of extrapolated slope line and the base level. The position of minimums is updated to the position of the intersection point. So both of the minima are on the same base level representing as close as possible to a bell-shaped curve. All the phenological parameters can be computed from the extracted points. Figure 5.3 shows the ideal bell-shaped time-series curve along with the technique used to detect phenological parameters. 5.3 Decision Tree classification A feature vector constructed using several parameters captures different aspects of time-series curve - statistical metrics, physical phenomena, and polarity feature indicating the shape nature of the curve. The proposed set of features are employed in an experiment for detecting and distinguishing vegetation cover patterns. Core samples of time-series database collected as ground truth are used to train a random forest classification model which is based on decision tree data mining algorithm. The trained model is then deployed to label unknown time-series curves for the vegetation cover classification. For each pixel-time trajectory, besides assigning a label to it, the classification model also outputs the probability of it towards each class. 5.4 Results Vegetation cover map generated with the MODIS time-series data using the proposed approach is shown in Figure 5.4 which depicts the extent of different vegetation covers for the agricultural-year over the West Godavari district. In the present study, for quantitative validation, classes of reference map were combined as per the classification schema. NRSC reference map was used to compare the extent and quantitatively calculate statistics for all the classes as shown in Table 5.1. The results derived from the confusion matrix yield an overall accuracy of 81.80%. In comparison with the previous results, it is observed that the results obtained have better precision in differentiating agricultural component from the non-agricultural component. The accuracy in agricultural classes is relatively higher, though marginally, as the phenological features extracted are devised to better describe agricultural patterns than the other classes. Also, low accuracy and error are observed in distinguishing components that have higher vegetation values. One of the reasons for this inaccuracy can be attributed to the phenological parameter extraction method as it is not devised by keeping such vegetation covers in mind. Overall the results provide more clarity to the classes of interest. 38

51 Figure 5.4 Vegetation cover map generated for agricultural-year using proposed feature-based supervised approach. 39

52 Predicted (No. of pixels (relative %)) (Agri. year ) No/low Kharif Rabi Double Plantation Forest vegetation No/low vegetation Kharif Rabi Double Plantation Forest Table 5.1 Confusion Matrix of results obtained from the proposed feature-based approach against statistics obtained from NRSC reference data for agricultural-year Intra-class Variability in Cropping practices Kharif class composite of a wide range of different crop types that are grown in southwest monsoon season. Different crop varieties have different temporal signatures and have certain phenological variations. Besides that even for same crop type several factors such as different farming practices, soil management, local geographical and climatic conditions, labour availability can lead to cause shifts in sowing and harvesting dates etc. The distribution of crop varieties and mapping of cropping practices is required to efficiently examine agricultural production. The main idea of present research is to show that satellite imagery time-series and phenological based method can aid for better agricultural monitoring at crop scale. 40

53 The length of growing period for kharif crops is generally around 100 to 150 days. Kharif crops with either different sowing and harvest dates, amplitude, or different peak positions are shown below in Figure 5.5 to Figure 5.9. Phenological parameters capture the intra-variability in cropping seasons and the objective is to demonstrate that it can be used for spatial mapping and building season calendars for various crop varieties present in the region. The shortcoming of the parameter-based method is that it induces more variability as they are subject to smoothing and parameter extraction techniques. Figure 5.5 Time-series curve for 115 days crop with peak in Oct-Mid. Figure 5.6 Time-series curve for 140 days crop with peak in Sept-End. 41

54 Figure 5.7 Time-series curve for 140 days crop with peak in Sept-Mid. Figure 5.8 Time-series curve for 150 days crop with peak in Oct-Start. Figure 5.9 Time-series curve for 150 days crop with peak in Sept-End 42

55 Chapter 6 Data Driven Unsupervised learning - An approach towards building an automated system for spatio-temporal data analysis In this work, we propose an approach for building an automated system for vegetation monitoring at a district scale. So far, current supervised approaches give us a macro-level understanding of vegetation covers present in a region based on available training temporal patterns. Thus, it is difficult to track deviations that occur in agricultural practices i.e shifts in crop cycles over a period of time. An unsupervised approach has been explored for vegetation cover classification at a regional scale based on the shapecharacteristics of time-series curve. The objective is to build a data-driven automated system at the district scale. The motivation behind an unsupervised analysis is to better appreciate the unique shape of different vegetation cover classes (or even the intra-class variations present in time-series curves of the same class) that exists in the given region. Supervised approaches require the availability of ground truth information to obtain a suitable training set that corresponds well to vegetation cover classes of interest. Furthermore, supervised classification contains analyst bias. Also, the reference maps provided by resources such as NRSC are not 100% accurate, infrequently updated and provides little insight regarding changes occurring in cropping practices and long-term shifts in phenological cycles. The purpose here is to do an unsupervised analysis to build a training set from the data itself and then use extracted training set in the supervised stage for vegetation cover classification. However, supervised classification is generally expected to attain greater accuracies as they are trained on user knowledge. Also, the drawback of unsupervised approach is that the accuracy of the classification will likely decrease as the complexity of the scene increases. Unsupervised algorithms may detect the classes that are most different, i.e outliers (not depicting any class in particular) that may not accurately differentiate subtly different classes that you could train within a supervised algorithm. Also, usage of training data obtained as a result of an unsupervised analysis would likely yield more error because the training set for a class may contain more mixed samples than the supervised approach. Nonetheless, an unsupervised approach based on shape characteristics of time-series curve has been explored that can help to build a data-driven automated system and provide insight into different vegetation covers and intra-class variability present in the region. 43

56 6.1 Sampling for Identification of Unique Temporal Signatures In the proposed approach, firstly an unsupervised analysis is performed to extract unique temporal signatures belonging to different vegetation classes that exist in the region. The next step is the supervised stage, that uses the unique representative curves detected to reclassify the whole region. In an unsupervised stage, the methodology is divided into two phases - (1) grid sampling and (2) multi-level clustering. Now in order to find unique time-series curve prevalent in the region, the stratified random sampling method is carried out. In the current study, stratified random sampling refers to the division of the entire region into sub-regions, random samples are then selected from each sub-region. It ensures that each sub-region captures different classes present within its locality and when all sub-regions are combined, it adequately represents the whole region. The main advantage of such sampling is that it captures key characteristics of diverse classes present in the data. However, it is important to note here that a particular class may be present in multiple sub-regions, i.e sub-regions can have an overlapping class. But the purpose is to capture all the unique temporal signatures from each sub-regions. Multilevel clustering performed in later stage potentially mitigates the issue of overlapping by recombining such classes. In sampling phase, spatial segmentation of the region is carried out. Besides the reasons discussed above, it also helps from the computational standpoint. As it focuses on each sub-regions separately, the volume of the data to process is reduced. Also indirectly it takes advantage of the redundancy of the data (as large homogeneous regions may get divided). The whole region is divided into 5 x 5 grid, each grid representing a sub-region. Typically, the size of the district is about 500 * 500 pixels, so each sub-region is of size 100 * 100 pixels, i.e roughly 10K pixels. Now for each sub-region, we randomly select around 1K pixels as samples. Stratified grid sampling is performed so that all the local vegetation type is captured, that may not be present elsewhere in the region. In the next phase, two-level clustering is carried out. For each sub-region, k-medoids clustering is performed on its samples with the value of the number of cluster centers set to around 20. As a result, we obtain 20 clusters that represent different classes present within that sub-region. Some classes may be similar, which depends on the number of cluster centers set beforehand and the actual number of classes present within that sub-region. Although, the value is set sufficiently large so that no local class gets ignored. For each cluster formed, we keep 20 core samples and discard others considering them as outliers. Now for each sub-region, we have around 20 (number of clusters) * 20 (number of core samples from each cluster) core samples. So for the whole region, in total, we sampled around 25 (number of sub-regions) * 400 (number of samples per sub-regions) time-series curves, i.e around 10K samples. From each cluster, only core samples are taken into consideration, with an assumption that the samples of majority class which may be present in multiple sub-regions do not dominate second level clustering (because the same amount of samples for minority class is selected from each sub-region). The 10K time-series samples selected serves as the representative samples that capture temporal signatures of all the different classes present in the entire region. After that, the second level of K-medoids clustering is performed on the extracted samples 44

57 obtained from each grid. As a result, we obtain 20 clusters for the entire region. Each cluster formed represents a unique class and time-series samples belonging to each class is then used as representative curves for the supervised stage. The spatial segmentation based grid sampling approach allows enforcing a local cluster criterion. Clusters are formed that represents vegetation types present in the sub-region locality. So indirectly clusters capture homogenous regions in the data space, i.e spatially correlated regions, in which particular vegetation types are dense. The multi-level clustering algorithm proposed here categorizes data points that are either core members of the clusters or are at the border - located relatively far from the cluster centers. In this study, to avoid outlier bias, only those data points that belong to the cluster core are processed further at both levels. The cluster centers are found using K-medoids clustering algorithm and these cluster centers are outputted as identified vegetation type. 6.2 Clusters of Time-Series Curve Some of the noticeable and distinguishable temporal signatures obtained as a result of spatio-temporal multi-level clustering are shown in Figure 6.1. Time-series curve cluster formed is representative of diverse vegetation present in the region, and specifically indicates a different kind of cropping techniques being practiced in the agricultural components of the region. It can also be observed that the clustering technique based on the shape characteristics of time-series curve proposed here captures intra-variability present within same cropping class that is present in the region. The double cropping is predominantly practiced in the West Godavari district due to better resource availability. It is expected that the spatial segmentation based sampling and multi-level clustering should find intra-variability present in the double cropping practices. It is also evident from the clusters formed that is shown in Figure 6.1 to Figure 6.4. Besides that few clusters which contained mixed samples were also formed which were difficult to connect to a specific vegetation. One of the reasons for conflicts in resolving clusters can be attributed to the temporal frequency of the time-series data, as it only captures one value for 16 days. The set of challenges posed by temporal resolution of the data need to be tackled. 45

58 Figure 6.1 Time-series curve cluster of No/low Vegetation class. Figure 6.2 Time-series curve cluster of Kharif crop and Rabi crop class. 46

59 Figure 6.3 Time-series curve cluster of Double cropping class. 47

60 Figure 6.4 Time-series curve cluster of Plantation and Forest Class. 6.3 Reclassification and Results The extracted representative curves from an unsupervised phase are labeled using expert domain knowledge. Labeled representative curves are then used along with the supervised method discussed in chapter-4 to perform vegetation cover classification over the region. The vegetation cover map derived is shown in Figure 6.5. The results were validated with NRSC reference data and showed an overall accuracy of 75.8%. An interesting output of the unsupervised approach is that it detects five different crop patterns that all correspond to double cropping practices and indicates intra-variability. But the algorithm identifies them as independently different crops - either having different amplitude or growing period - rather than clubbing them into a single class. Each of the clusters is labelled separately as they indicate a different type of cropping system based on the vegetation profiles that we get as an output. Also, each cropping pattern is spatially clustered. It was observed that the regions having similar cropping practice are located in a spatial vicinity that might indicate certain local cropping practices of sowing and harvesting. 48

61 Figure 6.5 Vegetation cover map generated for agricultural-year using the proposed unsupervised approach. The double cropping practice spatial variability map derived for West Godavari district is shown in Figure 6.6. Each color represents a unique cropping practice. It can be observed from the variability map, that we are also able to detect regions that are spatially correlated and homogenous in terms of double cropping type present. The variation in spatio-temporal attributes of vegetation phenology are in sowing and harvesting practices. At regional scale such complex variations can be attributed to dissimilarity in the community, climate, soils type and quality, fertilizers and seed varieties used, land management, monsoon cycles and its progression etc. To estimate the spatio-temporal distribution of crop varieties and understand cropping dynamics of the regions, the agriculturist can analyze intravariability data obtained as a result of the above categorization along with other spatial data that, directly or indirectly, govern corresponding cropping activities. 49

62 Figure 6.6 Double cropping practice spatial intra-variability map. Representative curves for different double-crop classes are shown in Figure