ENTROPY-BASED PARAMETER ESTIMATION IN HYDROLOGY

Transcription

1 ENTROPY-BASED PARAMETER ESTIMATION IN HYDROLOGY

2 Water Science and Technology Library VOLUME 30 Editor-in-Chief V. P. Singh, Louisiana State University, Baton Rouge, U.S.A Editorial Advisory Board M. Anderson, Bristol, U.K. L. Bengtsson, Lund, Sweden S. Chandra, New Delhi, India M. Fiorentino, Potenza, Italy W. H. Hager, Zurich, Switzerland N. Harmancioglu,Izmir, Turkey U. C. Kothyari, Roorkee, India A. R. Rao, West Lafayette, Indiana, U.S.A. B. M. Sahni, Aurangabad, India D. Stephenson, Johannesburg, South Africa The titles published in this series are listed at the end of this volume.

3 ENTROPY-BASED PARAMETER ESTIMATION IN HYDROLOGY by VIJAY P. SINGH Water Resources Program, Department of Civil and Environmental Engineering, Louisiana State University, Baton Rouge U.S.A. SPRINGER-SCIENCE+BUSINESS MEDIA, B.V.

4 A C.I.P. Catalogue record for this book is available from the Library of Congress. ISBN DOI / ISBN (ebook) Printed on acid-free paper All Rights Reserved 1998 Springer Science+Business Media Dordrecht Originally published by Kluwer Academic Publishers in 1998 No part of the material protected by this copyright notice may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording or by any information storage and retrieval system, without written permission from the copyright owner

5 To: Anita Vinay, and Arti

6 TABLE OF CONTENTS Page PREFACE... xii ACKNOWlEDGMENTS... xv CHAPTER 1 ENTROPY AND PRINCIPLE OF MAXIMUM ENTROPY Entropy Theory Principle of Maximum Entropy Entropy-Based Parameter Estimation Parameter-Space Expansion Method Entropy as a Criterion for Goodness of Fit Dependence of Entropy on Sample Size... 9 REFERENCES METHODS OF PARAMETER ESTIMO\TION Method of moments for Continuous Systems Method of Moments for Discrete Systems Method of Probability Weighted Moments Method of Mixed Moments Method of L-Moments Method of Maximum Likelihood Estimation Method of Least Squares Comparison with Entropy Method Problems of Parameter Estimation REFERENCES UNIFORM DISTRIBUTION Specification of Constraint Construction of Zeroth Lagrange Multiplier Estimation of Parameter Distribution Entropy REFERENCES EXPONENTIAL DISTRIBUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimation

7 viii REFERENCES NORMAL DISTRIBUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimation REFERENCES TWO-PARAMETER LOGNORMAL DISTRIBUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimation REFERENCES THREE-PARAMETER LOGNORMAL DISTRIBUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimation Comparative Evaluation of Estimation Methods REFERENCES EXTREME VALUE TYPE I DISTRIBUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimation Evaluation of Parameter Estimation Methods Using Field Data Evaluation of Parameter Estimators Using Monte Carlo Experiments. 126 REFERENCES LOG-EXTREME VALUE TYPE I DISTRIBUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimation... " Comparative Evaluation of Parameter Estimation Methods REFERENCES EXTREME VALUE TYPE III DISTRmUTION Ordinary Entropy Method

8 10.2 Parameter-Space Expansion Method Other Methods of Parameter Estimation Comparative Evaluation of Estimation Methods REFERENCES GENERALIZED EXTREME VALUE DISTRIBUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimation REFERENCES WEIBULL DISTRIBUTION Ordinary Entropy Method... " Parameter-Space Expansion Method,..., Other Methods of Parameter Estimation Comparative Evaluation of Parameter Estimation Methods REFERENCES GAMMA DISTRIBUTION Ordinary Entropy Method Parameter-Space Expansion Method, Other Methods of Parameter Estimation Comparative Evaluation of Estimation Methods REFERENCES PEARSON TYPE III DISTRIBUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimatilon Comparative Evaluation of Estimation Methods REFERENCES LOG-PEARSON TYPE III DISTRIBUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimation Comparative Evaluation of Estimation Methods REFERENCES ix

9 x 16 BETA DISTRIBUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimation... "., 281 REFERENCES TWO-PARAMETER LOG-LOGISTIC DISTRmUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimation Comparative Evaluation of Parameter Estimation Methods REFERENCES THREE-PARAMETERLOG-LOGISTICDISTRmUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimation Comparative Evaluation of Parameter Estimation Methods REFERENCES TWO-PARAMETER PARETO DISTRmUTION OrdinaryEntropyMethod Parameter-Space Expansion Method Other Methods of Parameter Estimation Comparative Evaluation of Estimation Methods Using Monte Carlo Experiments REFERENCES TWO-PARAMETER GENERALIZED PARETO DISTRIBUTION Ordinary Entropy Method Parameter-Space Expansion Method Other Methods of Parameter Estimation Comparative Evaluation of Parameter Estimation Methods REFERENCES THREE-PARAMETER GENERALIZED PARETO DISTRmUTION Ordinary Entropy Method Parameter-Space Expansion Method

10 21.3 Other Methods of Parameter Estimation Comparative Evaluation of Parameter Estimation Methods REFERENCES TWO COMPONENT EXTREME VALUE DISTRIBUTION Ordinary Entropy Method Other Methods of Parameter Estimation Comparative Evaluation of Parameter Estimators REFERENCES..., SUBJECT INDEX... " xi

11 PREFACE Since the pioneering work of Shannon in the late 1940's on the development of the theory of entropy and the landmark contributions of Jaynes a decade later leading to the development of the principle of maximum entropy (POME), the concept of entropy has been increasingly applied in a wide spectrum of areas, including chemistry, electronics and communications engineering, data acquisition and storage and retreival, data monitoring network design, ecology, economics, environmental engineering, earth sciences, fluid mechanics, genetics, geology, geomorphology, geophysics, geotechnical engineering, hydraulics, hydrology, image processing, management sciences, operations research, pattern recognition and identification, photogrammetry, psychology, physics and quantum mechanics, reliability analysis, reservoir engineering, statistical mechanics, thermodynamics, topology, transportation engineering, turbulence modeling, and so on. New areas finding application of entropy have since continued to unfold. The entropy concept is indeed versatile and its applicability widespread. In the area of hydrology and water resources, a range of applications of entropy have been reported during the past three decades or so. This book focuses on parameter estimation using entropy for a number of distributions frequently used in hydrology. In the entropy-based parameter estimation the distribution parameters are expressed in terms of the given information, called constraints. Thus, the method lends itself to a physical interpretation of the parameters. Because the information to be specified usually constitutes sufficient statistics for the distribution under consideration, the entropy method provides a quantitative way to express the information contained in the distribution. Furthermore, it also provides a way to derive a distribution from specified constraints. In other words, the hypotheses underlying the distribution have an easy interpretation. These advantages notwithstanding, the entropy-based parameter estimation has received comparatively little attention from the hydrologic community. It is hoped that this book will stimulate interest in this fascinating area of entropy and its application in hydrology, environmental engineering, and water resources. The subject matter of the book spans 22 chapters. The entropy concept and the principle of maximum entropy are introduced in Chapter 1. Also introduced therein are the entropy-based method of parameter estimation and parameter-space expansion method. The chapter is concluded with a brief account of the use of entropy as a criterion for goodness of fit and the dependence of entropy on the sample size. A short discussion of five other popular methods of parameter estimation, including the methods of moments, probability weighted moments, L moments, maximum likelihood estimation, and least squares, is presented in Chapter 2. Also presented there is a brief account of errors and statistical measures of performance evaluation, including bias, consistency, efficiency, sufficiency, resilience, standard error, root mean square error, robustness, and relative mean error. The next two chapters present the base distributions-uniform and exponential. The uniform distribution is discussed in chapter 3. Beginning with specification of the constraint, the chapter goes on to discuss construction of the zeroth Lagrange multiplier, estimation of the parameter, and the entropy of the distribution. Chapter 4 presents parameter estimation for the exponential distribution. The discussion on the entropy-based parameter estimation method is divided into the ordinary entropy method and the parameter-space expansion method. The first method includes specification of constraints, construction of the zeroth Lagrange multiplier, relation between Lagrange multipliers and constraints, relation between Lagrange multipliers and parameter, relation between parameter and constraint and distribution entropy. The second method discusses specification of constraints, derivation of the entropy function and the relation between distribution parameter and constraint. The chapter is concluded with a discussion of the xii

12 methods of moments, maximum likelihood estimation, probability weighted moments and L moments. The three succeeding chapters are devoted to normal and lognormal distributions. The organization of presentation, comprising both the ordinary entropy method and the parameter space expansion method, is similar in each chapter. The ordinary entropy method includes specification of constraints, construction of the zeroth Lagrange multiplier, relation between Lagrange multipliers and constraints, relation between Lagrange multipliers and parameters, relation between parameters and constraints, and distribution entropy. The parameter space expansion method includes specification of constraints, derivation of the entropy function, and relation between distribution parameters and constraints. Chapter 5, devoted to parameter estimation for the normal distribution, is concluded with a discussion of the methods of moments, maximum likelihood estimation and probability weighted moments. Chapter 6 treats the two-parameter lognormal distribution and is concluded with a short account of the methods of moments, maximum likelihood estimation and probability weighted moments. Chapter 7 is devoted to parameter estimation for the three-parameter lognormal distribution. In addition to discussing the methods of moments (regular and modified), maximum likelihood estimation (regular and modified) and probability weighted moments, the chapter is concluded with a comparative evaluation of these parameter estimation methods, including the entropy method, using Monte Carlo experimentation and field data. The next four chapters are devoted to the extreme-value distributions. The organization of discussion of the entropy-based parameter estimation is the same as in the preceding chapters. Chapter 8 discusses parameter estimation for extrf:me-value type I or Gumbel distribution. The other methods of parameter estimation included in the chapter are the methods of moments, maximum likelihood estimation,least squares, incomplete means, probability weighted moments, and mixed moments. The chapter is concluded with a comparative evaluation of the parameter estimation methods using both field data and Monte Carlo simulation experiments. Chapter 9 discusses parameter estimation for log-extreme-v~lue type I or log-gumbel distribution. It goes on to discuss the methods of moments and maximum likelihood estimation, and is concluded with a comparative evaluation of the parameter estimation methods using annual flood data. Chapter 10 discusses parameter estimation for extreme-value type III distribution. It also includes a discussion of the methods of moments and maximum likelihood estimation and a comparative evaluation of the parameter estimation methods. Chapter 11 discusses parameter estimation for the generalized extreme-value distribution, and is concluded with a discussion of the methods of moments, probability weighted moments, L-moments and maximum likelihood estimation. The next five chapters discuss parameter estimation for the Weibull distribution and Pearsonian distributions. The organization of presentation of the entropy-based parameter estimation is the same as in the preceding chapters. Chapter 12 discusses parameter estimation for the Weibull distribution. It goes on to discuss the methods of moments, maximum likelihood estimation, probability weighted moments and least squares, and is concluded with a comparative evaluation of the methods using rainfall data and Monte Carlo experiments. Chapter 13 discusses parameter estimation for gamma distribution. It also presents a discussion of the methods of moments, cumulants, maximum likelihood estimation,least squares, and probability weighted moments. By applying these parameter estimation methods to unit hydrograph estimation and flood frequency analysis, the chapter is ended with a comparative assessment of the methods. Chapter 14 discusses parameter estimation for Pearson type III distribution. It also includes a discussion of the methods of moments, maximum likelihood estimation and probability weighted moments; as well as a comparative assessment of the estimation methods using annual maximum discharge data. Chapter 15 discusses parameter estimation for log Pearson type III distribution. It also includes a treatment of the methods of moments (direct as xiii

13 XIV well as indirect), maximum likelihood estimation, and mixed moments. Finally, it presents a comparative evaluation of the estimation methods using field data as well as Monte Carlo experiments. Chapter 16 discusses parameter estimation for beta distribution (or Pearson type I) distribution and is concluded with a discussion of the methods of moments and maximum likelihood estimation. The next two chapters present two and three parameter log-logistic distribution. The organization of presentation of the entropy-based parameter estimation is similar to that of the preceding chapters. Chapter 17 discusses parameter estimation for two-parameter log-logistic distribution. It also treats the methods of moments, probability weighted moments and maximum likelihood estimation; and is concluded with a comparative evaluation of the estimation methods using Monte Carlo simulation experiments. Chapter 18 discusses parameter estimation for threeparameter log-logistic distribution. It also presents the methods of moments, maximum likelihood estimation and probability weighted moments; as well as a comparative assessment of the parameter estimation methods. The next three chapters present Pareto distributions. The organization of presentation of the entropy-based parameter estimation is the same as in the preceding chapters. Chapter 19 discusses parameter estimation for the 2-parameter Pareto distribution. It also discusses the methods of moments, maximum likelihood estimation and probability weighted moments. The chapter is ended with a comparative assessment of parameter estimation methods using Monte Carlo simulation. Chapter 20 discusses parameter estimation for the 2-parameter generalized Pareto distribution. It also discusses the methods of moments, maximum likelihood estimation and probability weighted moments, and presents a comparative assessment of the parameter estimation methods. Chapter 21 discusses parameter estimation for the 3-parameter generalized distribution. It also includes a discussion of the methods of moments, maximum likelihood estimation and probability weighted moments; as well as a comparative evaluation of the parameter estimation methods. Chapter 22 discusses parameter estimation for the two-component extreme-value distribution. The first method is the ordinary entropy method that includes a discussion of specification of constraints, construction of zeroth Lagrange multiplier, relation between parameters and constraints, and estimation of parameters, including point estimation and regional estimation. The chapter also includes a discussion of the methods of maximum likelihood estimation and probability weighted moments, and is concluded with a comparative evaluation of the parameter estimation methods. V.P. Singh Baton Rouge, Louisiana

14 ACKNOWLEHGMENTS Much of the material reported in the book was developed while I was working on research projects funded by various organizations, including the United States Geological Survey, Louisiana Water Resources Research Institute, and the U.S. Department of the Army, Corps of Engineers. I worked with several of my graduat(: students and colleagues on entropy-related projects. Of particular mention are: Dr. A. K. Rajagopal, Naval Research Laboratory, with whom I worked for about 4 years and our collaborative work led to the development ofthe parameterspace expansion method; Professor M. Fiorentino and Dr. P. Claps of the Department of Environmental Engineering and Physics of the University of Basilicata at Potenza, Italy, with whom I worked on the two-component extreme value distribution and geomorphologic laws; Dr. Donald Barbe of the Department of Civil and Environmental Engineering, University of New Orleans, Louisiana, and Professor J.F. Cruise of the Department of Civil and Environmental Engineering, University of Alabama, Huntsville, Alabama, with whom I worked on hydraulic applications of entropy. Messrs. K. Arora, S.K. Jain, K. Singh, and H. Guo; Dr. P.P. Krstanovic, and Dr. F.x. Yu were some of my students who helped me continue to work on entropy. I am grateful to all of these individuals. ][ndeed this book draws significantly from the fruits of their labor. My family (wife, Anita; son, Vinay; and daughter, Arti) allowed me to work during evenings, weekends, and holidays. They supported me and were always there when I needed them. Without their love and support this book would not have been completed. As a small token of my gratitude, I dedicate the book to them. xv