Maximum-Entropy and Bayesian Spectral Analysis and Estimation Problems

Transcription

1 Maximum-Entropy and Bayesian Spectral Analysis and Estimation Problems

2 Fundamental Theories of Physics A New International Book Series on the Fundamental Theories of Physics: Their Clarification, Development and Application Editor: ALWYN VAN DER MERWE University of Denver, U.S.A. Editorial Advisory Board: ASIM BARUT, University of Colorado, U.S.A. HERMANN BONDI, Natural Environment Research Council, U.K. BRIAN D. JOSEPHSON, University of Cambridge, U.K. CLIVE KILMISTER, University of London, U. K. GONTER LUDWIG, Philipps-Universitiit, Marburg, F.R.G. NATHAN ROSEN, Israel Institute of Technology, Israel MENDEL SACHS, State University of New York at Buffalo, U.S.A. ABDUS SALAM, International Centre for Theoretical Physics, Trieste, Italy HANS-JORGEN TREDER, Zentralinstitut fur Astrophysik der Akademie der Wissenschaften, G.D.R.

3 Maximum-Entropy and Bayesian Spectral Analysis and Estimation Problems Proceedings of the Third Workshop on Maximum Entropy and Bayesian Methods in Applied Statistics, Wyoming, U.S.A., August 1-4, 1983 edited by c. Ray Smith u.s. Army Missile Command, Redstone Arsenal, Alabama, U.S.A. and Gary J. Erickson Department a/electrical Engineering, Seattle University, Seattle, Washington. U.S.A. D. Reidel Publishing Company A MEMBER OF THE KLUWER ACADEMIC PUBLISHERS GROUP Dordrecht / Boston / Lancaster / Tokyo

4 Libnry of Congress Cataloging in Public:ation Data Maximum Entropy Workshop (3rd: 1983: Laramie, Wyo.) Maximum-entropy and Bayesian spectral analysis and estimation problems. (Fundamental theories of physics) Includes index. 1. Entropy (Information theory)-congresses. 2. Bayesian statistical decision theory-congresses. I. Smith, C. Ray, II. Erickson, Gary J. III. Title. IV, Series. Q370.M ' ISBN-13: e-isbn-13: DOl: / Published by D. Reidel Publishing Company, P.O. Box 17, 3300 AA Dordrecht, Holland. Sold and distributed in the U.S.A. and Canada by Kluwer Academic Publishers, 101 Philip Drive, Assinippi Park, Norwell, MA 02061, U.S.A. In all other countries, sold and distributed by Kluwer Academic Publishers Group, P.O. Box 322,3300 AH Dordrecht, Holland. All Rights Reserved 1987 by D. Reidel Publishing Company, Dordrecht, Holland Softcover reprint of the hardcover 1st edition 1987 No part of the material protected by this copyright notice may be reproduced or utilized in any form or by any means, electronic or mechanical including photocopying, recording or by any information storage and retrieval system, without written permission from the copyright owner

5 To the memories of our fathers, Robert Austin Smith and Phillip Christian Erickson

6 CONTENTS Preface ix BAYESIAN SPECTRUM AND CHIRP ANALYSIS E. T. Jaynes 1 ON ENTROPY RATE Athanasios Papoulis 39 STATE SPACES AND INITIAL ESTIMATES IN MINIMUM RELATIVE-ENTROPY INVERSION WITH APPLICATION TO SPECTRUM ANALYSIS AND IMAGE ENHANCEMENT John E. Shore 51 RELATIVE-ENTROPY MINIMIZATION WITH UNCERTAIN CONSTRAINTS: THEORY AND APPLICATION TO SPECTRUM ANALYSIS Rodney W. Johnson 57 A PROOF OF BURG'S THEOREM B. S. Choi and Thomas M. Cover 75 A BAYESIAN APPROACH TO ROBUST LOCAL FACET ESTIMATION Robert M. Haralick 85 THE MAXIMUM ENTROPY METHOD: THE PROBLEM OF MISSING DATA William I. Newman 99 ON THE ACCURACY OF SPECTRUM ANALYSIS OF RED NOISE PROCESSES USING MAXIMUM ENTROPY AND PERIODOGRAM METHODS: SIMULATION STUDIES AND APPLICATION TO GEOPHYSICAL DATA Paul F. Fougere. 127 RECENT DEVELOPMENTS AT CAMBRIDGE Stephen F. Gull and John Skilling PRIOR KNOWLEDGE MUST BE USED John Skilling and Stephen F. Gull HOW THE BRAIN WORKS: THE NEXT GREAT SCIENTIFIC REVOLUTION David Hestenes 173

7 viii CONTENTS MAXIMUM ENTROPY IN STRUCTURAL MOLECULAR BIOLOGY: THE FIBER DIFFRACTION PHASE PROBLEM Richard K. Bryan 207 A METHOD OF COMPUTING MAXIMUM ENTROPY PROBABILITY VALUES FOR EXPERT SYSTEMS Peter Cheeseman 229 SPECIAL-PURPOSE ALGORITHMS FOR LINEARLY CONSTRAINED ENTROPY MAXIM IZATION Yair Censor, Tommy Elfving, and Gabor T. Herman 241 BAYESIAN APPROACH TO LIMITED-ANGLE RECONSTRUCTION IN COMPUTED TOMOGRAPHY Kenneth M. Hanson and George W. Wecksung 255 APPLICATION OF THE MAXIMUM ENTROPY PRINCIPLE TO RETRIEVAL FROM LARGE DATA BASES Paul B. Kantor 273 TWO RECENT APPLICATIONS OF MAXIMUM ENTROPY Lee H. Schick 283 A VARIATIONAL METHOD FOR CLASSICAL FLUIDS Ramarao Inguva, C. Ray Smith, T. M. Huber, and Gary Erickson. 295 UPDATING INDUCTIVE INFERENCE N. C. Dalkey 305 PARALLEL ALGORITHMS FOR MAXIMUM ENTROPY CALCULATION Stuart Cernan 317 Subject Index 319

8 PREFACE This volume has its origin in the third Workshop on Maximum-Entropy and Bayesian Methods in Applied Statistics, held at the University of Wyoming, August 1 to 4, It was anticipated that the proceedings of this workshop could not be prepared in a timely fashion, so most of the papers were not collected until a year or so ago. Because most of the papers are in the nature of advancing theory or solving specific problems, as opposed to status reports, it is believed that the contents of this volume will be of lasting interest to the Bayesian community. The workshop was organized to bring together researchers from different fields to examine critically maximum-entropy and Bayesian methods in science, engineering, medicine, economics, and other disciplines. Some of the papers were chosen specifically to kindle interest in new areas that may offer new tools or insight to the reader or to stimulate work on pressing problems that appear to be ideally suited to the maximum-entropy or Bayesian method. Certain facets of publishing a book are inherently unrewarding and frustrating. Or so it seems until the task is completed, and one has the pleasure of acknowledging publicly those who have helped along the way. Adequate thanks to Martha Stockton are impossible. The camera-ready copy prepared by Martha has benefited substantially by her editorial, proofreading, and drafting assistance. Dr. David Larner and Professor Alwyn van der Merwe, both affiliated with Reidel, provided encouragement and friendship at critical times. We are happy that Reidel has agreed to publish future proceedings of these workshops. Others who have made our work easier or more rewarding include Evelyn Haskell, Marce Mitchum, and our friends of the SDC Passive Sensors Division. Dr. Rabinder Madan of the Office of Naval Research has provided continual encouragement and assisted us in obtaining much-needed funding. August 1987 C. Ray Smith Gary J. Erickson ix