DISCRETE INVERSE AND STATE ESTIMATION PROBLEMS With Geophysical The problems of making inferences about the natural world from noisy observations and imperfect theories occur in almost all scientific disciplines. This book addresses these problems using examples taken from geophysical fluid dynamics. It focuses on discrete formulations, both static and time-varying, known variously as inverse, state estimation or data assimilation problems. Starting with fundamental algebraic and statistical ideas, the book guides the reader through a range of inference tools including the singular value decomposition, Gauss Markov and minimum variance estimates, Kalman filters and related smoothers, and adjoint (Lagrange multiplier) methods. The final chapters discuss a variety of practical applications to geophysical flow problems. Discrete Inverse and State Estimation Problems: With Geophysical is an ideal introduction to the topic for graduate students and researchers in oceanography, meteorology, climate dynamics, geophysical fluid dynamics, and any field in which models are used to interpret observations. It is accessible to a wide scientific audience, as the only prerequisite is an understanding of linear algebra. is Cecil and Ida Green Professor of Physical Oceanography at the Department of Earth, Atmospheric and Planetary Sciences, Massachusetts Institute of Technology. After gaining his Ph.D. in geophysics in 1966 at MIT, he has risen through the department, becoming its head for the period between 1977 81. He subsequently served as Secretary of the Navy Research Professor and has held senior visiting positions at many prestigious universities and institutes across the world. His previous books include Ocean Acoustic Tomography (Cambridge University Press, 1995) with W. Munk and P. Worcester, and The Ocean Circulation Inverse Problem (Cambridge University Press, 1996).
DISCRETE INVERSE AND STATE ESTIMATION PROBLEMS With Geophysical CARL WUNSCH Department of Earth, Atmospheric and Planetary Sciences Massachusetts Institute of Technology
cambridge university press Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo, Delhi, Mexico City Cambridge University Press The Edinburgh Building, Cambridge cb2 8ru, UK Published in the United States of America by Cambridge University Press, New York Information on this title: /9781107406063 C. Wunsch 2006 This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published 2006 First paperback edition 2011 A catalogue record for this publication is available from the British Library isbn 978-0-521-85424-5 Hardback isbn 978-1-107-40606-3 Paperback Additional resources for this publication at /9781107406063 Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate.
To Walter Munk for decades of friendship and exciting collaboration.
Colour plate section appears between pages 182 and 183, and is also available for download in colour from www. cambridge.org/9781107406063
Contents Preface page ix Acknowledgements xi Part I Fundamental machinery 1 1 Introduction 3 1.1 Differential equations 4 1.2 Partial differential equations 7 1.3 More examples 10 1.4 Importance of the forward model 17 2 Basic machinery 19 2.1 Background 19 2.2 Matrix and vector algebra 19 2.3 Simple statistics: regression 29 2.4 Least-squares 43 2.5 The singular vector expansion 69 2.6 Combined least-squares and adjoints 118 2.7 Minimum variance estimation and simultaneous equations 125 2.8 Improving recursively 136 2.9 Summary 143 Appendix 1. Maximum likelihood 145 Appendix 2. Differential operators and Green functions 146 Appendix 3. Recursive least-squares and Gauss Markov solutions 148 3 Extensions of methods 152 3.1 The general eigenvector/eigenvalue problem 152 3.2 Sampling 155 3.3 Inequality constraints: non-negative least-squares 164 3.4 Linear programming 166 3.5 Empirical orthogonal functions 169 3.6 Kriging and other variants of Gauss Markov estimation 170 vii
viii Contents 3.7 Non-linear problems 171 4 The time-dependent inverse problem: state estimation 178 4.1 Background 178 4.2 Basic ideas and notation 180 4.3 Estimation 192 4.4 Control and estimation problems 214 4.5 Duality and simplification: the steady-state filter and adjoint 229 4.6 Controllability and observability 232 4.7 Non-linear models 234 4.8 Forward models 248 4.9 A summary 250 Appendix. Automatic differentiation and adjoints 250 5 Time-dependent methods 2 256 5.1 Monte Carlo/ensemble methods 256 5.2 Numerical engineering: the search for practicality 260 5.3 Uncertainty in Lagrange multiplier method 269 5.4 Non-normal systems 270 5.5 Adaptive problems 273 Appendix. Doubling 274 Part II Applications 277 6 Applications to steady problems 279 6.1 Steady-state tracer distributions 280 6.2 The steady ocean circulation inverse problem 282 6.3 Property fluxes 309 6.4 Application to real oceanographic problems 311 6.5 Linear programming solutions 326 6.6 The β-spiral and variant methods 328 6.7 Alleged failure of inverse methods 331 6.8 Applications of empirical orthogonal functions (EOFs) (singular vectors) 333 6.9 Non-linear problems 335 7 Applications to time-dependent fluid problems 340 7.1 Time-dependent tracers 341 7.2 Global ocean states by Lagrange multiplier methods 342 7.3 Global ocean states by sequential methods 351 7.4 Miscellaneous approximations and applications 354 7.5 Meteorological applications 356 References 357 Index 367 Colour plates between pp. 182 and 183.
Preface This book is to a large extent the second edition of The Ocean Circulation Inverse Problem, but it differs from the original version in a number of ways. While teaching the basic material at MIT and elsewhere over the past ten years, it became clear that it was of interest to many students outside of physical oceanography the audience for whom the book had been written. The oceanographic material, instead of being a motivating factor, was in practice an obstacle to understanding for students with no oceanic background. In the revision, therefore, I have tried to make the examples more generic and understandable, I hope, to anyone with even rudimentary experience with simple fluid flows. Also many of the oceanographic applications of the methods, which were still novel and controversial at the time of writing, have become familiar and almost commonplace. The oceanography, now confined to the two last chapters, is thus focussed less on explaining why and how the calculations were done, and more on summarizing what has been accomplished. Furthermore, the time-dependent problem (here called state estimation to distinguish it from meteorological practice) has evolved rapidly in the oceanographic community from a hypothetical methodology to one that is clearly practical and in ever-growing use. The focus is, however, on the basic concepts and not on the practical numerical engineering required to use the ideas on the very large problems encountered with real fluids. Anyone attempting to model the global ocean or atmosphere or equivalent large scale system must confront issues of data storage, code parallelization, truncation errors, grid refinement, and the like. Almost none of these important problems are taken up here. Before constructive approaches to the practical problems can be found, one must understand the fundamental ideas. An analogy is the need to understand the implications of Maxwell s equations for electromagnetic phenomena before one undertakes to build a high fidelity receiver. The effective engineering of an electronic instrument can only be helped by good understanding ix
x Preface of how one works in principle, albeit the details of making one work in practice can be quite different. In the interests of keeping the book as short as possible, I have, however, omitted some of the more interesting theoretical material of the original version, but which readers can find in the wider literature on control theory. It is assumed that the reader has a familiarity at the introductory level with matrices and vectors, although everything is ultimately defined in Chapter 2. Finally, I have tried to correct the dismaying number of typographical and other errors in the previous book, but have surely introduced others. Reports of errors of any type will be gratefully received. I thank the students and colleagues who over the years have suggested corrections, modifications, and clarifications. My time and energies have been supported financially by the National Aeronautics and Space Administration, and the National Science Foundation through grants and contracts, as well as by the Massachusetts Institute of Technology through the Cecil and Ida Green Professorship.
Acknowledgements The following figures are reproduced by permission of the American Geophysical Union: 4.8, 6.16 6.20, 6.23 6.26, 6.31, 7.3 7.5 and 7.7 7.10. Figures 2.16, 6.21, 6.27, 6.32, 7.1 and 7.11 are reproduced by permission of the American Meteorological Society. Figure 6.22 is reproduced by permission of Kluwer. Errata A correction list compiled by the author can be found here: http://ocean.mit.edu/~cwunsch xi