MACHINE LEARNING FOR GEOLOGICAL MAPPING: ALGORITHMS AND APPLICATIONS

Transcription

1 MACHINE LEARNING FOR GEOLOGICAL MAPPING: ALGORITHMS AND APPLICATIONS MATTHEW J. CRACKNELL BSc (Hons) ARC Centre of Excellence in Ore Deposits (CODES) School of Physical Sciences (Earth Sciences) Submitted in fulfilment of the requirements for the degree of Doctor of Philosophy University of Tasmania May, 2014

2

3 i Did you ever fly a kite in bed? Did you ever walk with ten cats on your head? Did you ever milk this kind of cow? Well, we can do it. We know how. If you never did you should. These things are fun and fun is good. Dr. Seuss

4 ii

5 iii DECLARATION OF ORIGINALITY This thesis contains no material which has been accepted for a degree or diploma by the University or any other institution, except by way of background information and duly acknowledged in the thesis, and to the best of my knowledge and belief no material previously published or written by another person except where due acknowledgement is made in the text of the thesis, nor does the thesis contain any material that infringes copyright. AUTHORITY OF ACCESS This non-published content of the thesis (see below) may be made available for loan and limited copying and communication in accordance with the Copyright Act STATEMENT REGARDING PUBLISHED WORK CONTAINED IN THESIS Chapter 4 of this thesis is published under a Creative Commons Attribution (CC BY) licence. You are free to copy, communicate and adapt the work, so long as you attribute the authors. To view a copy of this licence, visit The publishers of the papers comprising Chapters 5 to 6 hold the copyright for that content, and access to the material should be sought from the respective journals. Matthew J. Cracknell May 2014

6 iv Machine learning for geological mapping

7 v STATEMENT OF CO-AUTHORSHIP The following people and institutions contributed to the publication of work undertaken as part of this thesis: Matthew James Cracknell, ARC Centre of Excellence in Ore Deposits (CODES), School of Earth Sciences, University of Tasmania = Candidate Anya Marie Reading, ARC Centre of Excellence in Ore Deposits (CODES), School of Earth Sciences, University of Tasmania = Author 1 Andrew William McNeill, Mineral Resources Tasmania, Department of Infrastructure Energy & Resources (DIER) = Author 2 Author details and their roles: Paper 1, Geological mapping using remote sensing data: A comparison of five machine learning algorithms, their response to variations in the spatial distribution of training data and the use of explicit spatial information : Located in Chapter 4 Candidate was the primary author and with Author 1 contributing to its development, refinement and presentation.

8 vi Machine learning for geological mapping Paper 2, The upside of uncertainty: Identification of lithology contact zones from airborne geophysics and satellite data using Random Forests and Support Vector Machines : Located in Chapter 5 Candidate was the primary author and with Author 1 contributing to development, refinement and presentation. Paper 3, Mapping geology and volcanic-hosted massive sulfide alteration in the Hellyer Mt Charter region, Tasmania, using Random Forests and Self-Organising Maps : Located in Chapter 6 Candidate was the primary author and with Author 1 contributing to its refinement and presentation and Author 2 contributing to its formalisation and development. We the undersigned agree with the above stated proportion of work undertaken for each of the above published (or submitted) peer-reviewed manuscripts contributing to this thesis: Signed: Anya M. Reading Supervisor School Of Earth Sciences University of Tasmania Jocelyn McPhie Head of School School Of Earth Sciences University of Tasmania Date:

9 vii ABSTRACT Machine learning algorithms are designed to identify efficiently and to predict accurately patterns within multivariate data. They provide analysts computational tools to aid predictive modelling and the interpretation of interactions between data and the phenomena under investigation. The analysis of large volumes of disparate multivariate geospatial data using machine learning algorithms therefore offers great promise to industry and research in the geosciences. Geoscience data are frequently characterised by a restriction in the number and distribution of direct observations, irreducible noise in these data and a high degree of intraclass variability and interclass similarity. The choice of machine learning algorithm, or algorithms and the details of how algorithms are applied must therefore be appropriate to the context of geoscience data. With this knowledge, I aim to employ machine learning as a means of understanding the spatial distribution of complex geological phenomena. I conduct a rigorous and comprehensive comparison of machine learning algorithms, representing the five general machine learning strategies, for supervised lithology classification applications. I also develop and test a novel method for obtaining robust estimates of the uncertainty associated with machine learning algorithm categorical predictions. The insights gained from these experiments leads to the further development and comparison of new methods for the incorporation of spatial-contextual information into machine learning supervised classifiers. In using machine learning algorithms for geoscience applications, I have developed bestpractice methodologies that address the challenges facing geoscientists for geospatial supervised classification. Guidelines are established that detail the preparation and integration of disparate spatial data, the optimisation of trained classifiers for a given application and the robust statistical and spatial evaluation of outputs. I demonstrate, through a case study in a region that is prospective for economic mineralisation, the combination of supervised and unsupervised machine learning algorithms for the critical appraisal of pre-existing geological maps and formulation of meaningful interpretations of geological phenomena.

10 viii Machine learning for geological mapping The experiments conducted as part of my research confirm the efficacy of machine learning algorithms to generate accurate geological maps representing a variety of terranes. I identify and explore key aspects of the spatial and statistical distributions of geoscience data that affect machine learning algorithm performance. My research clearly identifies Random Forests as a good first-choice algorithm for the prediction of classes representing lithologies using commonly available multivariate geological and geophysical data. Furthermore, Random Forests prediction uncertainty is shown to be closely related to ambiguous and/or erroneous classifications and, thus provides a practical means of indicating variable levels of confidence. Spatial-contextual information is best incorporated into machine learning supervised classifiers via the pre-processing of input variables and/or the post-regularisation of classifications. My findings indicate that a trade-off between optimal predictive models and interpretable explanatory models exists, whereby, intuitively interpretable models are not necessarily the most accurate. The practical application of machine learning algorithms requires the implementation of three key stages: (1) data pre-processing; (2) algorithm training; and (3) prediction evaluation. This methodology provides the foundation for generating accurate and geologically meaningful predictions with minimal user intervention and assists in the formulation of robust interpretations of complex geological phenomena. For example, classifications obtained by Random Forests are useful for critically appraising interpreted geological maps. Clusters produced by Self-Organising Maps indicate the presence of discrete, spatially contiguous and geologically significant sub-classes within individual lithological units, which represent regions of contrasting primary composition and alteration styles. My results may be widely applied to a broad range of practical geoscience challenges such as ore deposit targeting, geo-hazard risk assessment, engineering and construction projects, hydrological and environmental modelling and ecological studies. The applications of machine learning algorithms detailed in this thesis align well with state-of-the-art Big Data online infrastructure and virtual laboratories currently emerging in Australia.

11 ix CONTENTS DECLARATION OF ORIGINALITY... III AUTHORITY OF ACCESS... III STATEMENT REGARDING PUBLISHED WORK CONTAINED IN THESIS... III STATEMENT OF CO-AUTHORSHIP...V ABSTRACT...VII CONTENTS...IX LIST OF TABLES... XV LIST OF FIGURES... XVII LIST OF ABBREVIATIONS...XXI ACKNOWLEDGEMENTS... XXIII CHAPTER 1 INTRODUCTION Machine learning Geological maps Research scope and hypothesis Major research questions to be addressed Thesis structure...7 CHAPTER 2 MACHINE LEARNING THEORY AND IMPLEMENTATION Machine learning Supervised versus unsupervised learning Supervised classification Classification strategies Statistical learning algorithms Instance-based learners Logic-based learners Support Vector Machines Perceptrons Supervised classifier implementation Data pre-processing Classifier training...27

12 x Machine learning for geological mapping Prediction evaluation Unsupervised clustering Clustering strategies Partitioning algorithms Hierarchical algorithms Self-Organising Maps Unsupervised clustering implementation Conclusions CHAPTER 3 A REVIEW OF MACHINE LEARNING FOR GEOSCIENCE CLASSIFICATION APPLICATIONS Machine learning non-geoscience applications Machine learning geoscience applications Classification of 0D data Classification of 1D data One temporal dimension One spatial dimension Classification of 2D data Land cover/vegetation mapping Geological mapping Supervised classification Unsupervised clustering Combined supervised and unsupervised methods Practical machine learning implementation Data Data pre-processing Prediction evaluation Integrated workflow Conclusions CHAPTER 4 GEOLOGICAL MAPPING USING REMOTE SENSING DATA: A COMPARISON OF FIVE MACHINE LEARNING ALGORITHMS, THEIR RESPONSE TO VARIATIONS IN THE SPATIAL DISTRIBUTION OF TRAINING DATA AND THE USE OF EXPLICIT SPATIAL INFORMATION Abstract Introduction Machine learning for supervised classification Machine learning algorithm theory Naïve Bayes k-nearest Neighbours... 73

13 Contents xi Random Forests Support Vector Machines Artificial Neural Networks Geology and tectonic setting Data Methods Pre-processing Classification model training Prediction evaluation Results Discussion Machine learning algorithms compared Influence of training data spatial distribution Using spatially constrained data Conclusions Acknowledgements Description of supplementary information...91 CHAPTER 5 THE UPSIDE OF UNCERTAINTY: IDENTIFICATION OF LITHOLOGY CONTACT ZONES FROM AIRBORNE GEOPHYSICS AND SATELLITE DATA USING RANDOM FORESTS AND SUPPORT VECTOR MACHINES Abstract Introduction The lithology prediction problem Random Forests Support Vector Machines Data Tectonic setting and history Data sources Data pre-processing Methods Training and evaluating algorithms Variance Results Discussion Conclusions Acknowledgements...119

14 xii Machine learning for geological mapping CHAPTER 6 MAPPING GEOLOGY AND VOLCANIC-HOSTED MASSIVE SULFIDE ALTERATION IN THE HELLYER MT CHARTER REGION, TASMANIA, USING RANDOM FORESTS AND SELF-ORGANISING MAPS Abstract Introduction Geological setting Random Forests Self-Organising Maps Data and Methods Source data Data sampling Training Random Forests and variable selection Implementing Self-Organising Maps Results Geological classification using Random Forests Discrimination of geological sub-classes using Self-Organising Maps Discussion Conclusions Acknowledgements CHAPTER 7 SPATIAL-CONTEXTUAL MACHINE LEARNING SUPERVISED CLASSIFIERS: LITHOSTRATIGRAPHY CLASSIFICATION EXAMPLE Abstract Introduction Pre-processing methods Focal operators Image segmentation Training data selection Post-processing methods Combination methods Study aims Data Lithostratigraphy classification target Geophysical data input variables Pre-processing Methods Data sampling Global pixel-based classifiers...162

15 Contents xiii Spatial-contextual classifiers Pre-processing Algorithm training Post-processing Prediction evaluation Results Discussion Spatial-contextual classifiers compared Issues of spatial scale Geological interpretations Conclusions CHAPTER 8 SYNTHESIS AND DISCUSSION Algorithms Supervised classification Implementation Decision structures Accuracy comparison Spatial-contextual classifiers Prediction uncertainty Unsupervised clustering Applications Data pre-processing Data preparation Variable extraction Variable selection Classifier training Training and test data Classifier induction Classification post-processing Evaluation and interpretation Statistical evaluation Interrogating decision structures Complementary interpretation Extended research implications Integrated workflow using R Wider geoscience applications Big Data CHAPTER 9 CONCLUSIONS

16 xiv Machine learning for geological mapping REFERENCES APPENDIX A MACHINE LEARNING ALGORITHM SENSITIVITY TO IMBALANCED CLASS DISTRIBUTIONS A.1. Introduction A.2. Methods A.3. Results A.4. Discussion and Conclusions APPENDIX B VARIANCE AND ENTROPY FOR MULTICLASS CLASSIFICATION UNCERTAINTY APPENDIX C SUPPLEMENTARY INFORMATION C.1. Data C.2. MLA software and parameters APPENDIX D R PACKAGES APPENDIX E DATA SOURCES AND PRE-PROCESSING APPENDIX F R CODE AND SCRIPTS README.txt...275