Springer Series in Information Sciences 26. Editor: Thomas S. Huang. Springer-Verlag Berlin Heidelberg GmbH

Transcription

1 Springer Series in Information Sciences 26 Editor: Thomas S. Huang Springer-Verlag Berlin Heidelberg GmbH

2 Springer Series in Information Sciences Editors: Thomas S. Huang Teuvo Kohonen Manfred R. Schroeder Managing Editor: H. K. V. Lotsch 30 Self-Organizing Maps By T. Kohonen 2nd Edition 31 Music and Schema Theory Cognitive Foundations of Systematic Musicology By M. Leman 32 The Maximum Entropy Method By N. Wu 33 Steps Towards 3D Active Vision By T. Vieville 34 Calibration and Orientation of Cameras in Computer Vision Editors: A. Grün and T. S. Huang 35 Speech Processing: Fundamentals and Applications By B. S. Atal and M. R. Schroeder Volumes 1-29 are listed at the end of the book.

3 C. K.Chui G. Chen Discrete Hoc Optimization With Applications in Signal Processing and Control Systems Second Edition With 38 Figures Springer

4 Professor Charles K. Chui Department of Mathematics, and Department of Electrical Engineering, Texas A&M University, College Station, TX , USA Dr. Guanrong Chen Department of Electrical Engineering, University of Houston, Houston, TX , USA Series Editors: Professor Thomas S. Huang Department of Electrical Engineering and Coordinated Science Laboratory, University of Illinois, Urbana, IL 61801, USA Professor Teuvo Kohonen Helsinki University of Technology, Neural Networks Research Centre, Rakentajanaukio 2 C, FIN Espoo, Finland Professor Dr. Manfred R. Schroeder Drittes Physikalisches Institut, Universität Göttingen, Bürgerstrasse 42-44, D Göttingen, Germany Managing Editor: Dr.-Ing. Helmut K. V. Lotsch Springer-Verlag, Tiergartenstrasse 17, D Heidelberg, Germany Library of Congress Cataloging-in-Publication Data. Chui, Charles K.: Discrete H~ optimization: with applications in signal processing and control systems 1 Charles K. Chui; Guanrong Chen ed. - Berlin; Heidelberg; New York; Barcelona; Budapest; Hong Kong; London; Milan; Paris; Santa Clara; Singapore; Tokyo: Springer, (Springer se ries in information sciences; 26). I. Auf!. u.d.t.: Chui, Charles K.: Signal processing and systems theory ISBN Pp. ISSN OnO-678X ISBN ISBN (ebook) DOI / The Ist Edition appeard under the titie: Signal Processing and Systems Theory This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under the German Copyright Law. Springer-Verlag Berlin Heidelberg 1992, 1997 Originally published by Springer-Verlag Berlin Heidelberg New York in 1997 The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Data conversion by K. Matthes, Heidelberg Cover design: design & production GmbH, Heidelberg SPIN: Printed on acid-free paper

5 Dedicated to the memory of our friend Professor Xie-Chang Shen ( )

6 Preface This is the second edition of our 1992 monograph, published under the title of Signal Processing and Systems Theory: Selected Topics. The need for the change in the current title was motivated by recent advancement of HOC-optimization in both continuous and discrete settings as well as the current trend in the development of similar and more effective theories and techniques for hybrid systems with continuous plants and digital feedback controllers. The new title also gives a better description of the contents of the book, concerning the fundamental theories and methodologies for discrete HOC-optimization studies and applications to such problems as optimal filter design and system reduction. In the preparation of this edition, we took advantage of the opportunity to correct several typos and update the list of references. We are again grateful to Dr. H. Lotsch of Springer-Verlag, for his encouragement and enthusiastic promotion of this book. College Station Houston February 1997 Charles K. Chui Guanrong Chen

7 Preface to the First Edition It is well known that mathematical concepts and techniques always play an important role in various areas of signal processing and systems theory. In fact, the work of Norbert Wiener on time series and the pioneering research of A. N. Kolmogorov form the basis of the field of communication theory. In particular, the famous sampling theorem, usually attributed to Wiener's student Claude Shannon, is based on the Paley-Wiener theorem for entire functions. In systems engineering, the important work of Wiener, Kalman, and many other pioneers is now widely applied to real-time filtering, prediction, and smoothing problems, while optimal control theory is built on the classical variational calculus, Pontryagin's maximum principle, and Bellman's dynamic programming. There is at least one common issue in the study of signal processing and systems engineering. A filter, stable or not, has a state-space description in the form of a linear time-invariant system. Hence, there are common problems of approximation, identification, stability, and rank reduction of the transfer functions. Recently, the fundamental work of Adamjan, Arov, and Krein (or AAK) has been recognized as an important tool for at least some of these problems. This work directly relates the approximation of a transfer function by stable rational functions in the supremum norm on the unit circle to that of the corresponding Hankel operator by finite-rank bounded operators in the Hilbert space operator norm, so that various mathematical concepts and methods from approximation theory, function theory, and the theory of linear operators are now applicable to this study. In addition, since uniformly bounded rational approximants on the unit circle are crucial for sensitivity considerations, approximation in the Hardy space HOC also plays an important role in this exciting area of research. This monograph is devoted to the study of several selected topics in the mathematical theories and methods that apply to both signal processing and systems theory. In order to give a unified presentation, we have chosen to concentrate our attention on discrete-time methods, not only for digital filters, but also for linear systems. Hence, our discussions of the theories and techniques in Hardy spaces and from the AAK approach are restricted to the unit disk. In particular, the reader will find that the detailed treatment of multi-input/multi-output systems in Chap. 6 distinguishes itself from the bulk of the published literature in that the balanced realization approach

8 X Preface to the First Edition of discrete-time linear systems is followed to study the matrix-valued AAK theory and HOC-optimization. The selection of topics in this monograph is guided by our objective to present a unified frequency-domain approach to discrete-time signal processing and systems theory. However, since there has been considerable progress in these areas during recent years, this book should be viewed as only an introductory treatise on these topics. The interested reader is referred to more advanced and specialized texts and original research papers for further study. We give a fairly rigorous and yet elementary introduction to signals and digital filters in the first chapter, and discrete-time linear systems theory in the second chapter. Hardy space techniques, including minimum-norm and Nevanlinna-Pick interpolations will be discussed in Chap. 3. Chap. 4 will be devoted to optimal Hankel-norm approximation. A thorough treatment of the theory of AAK will be discussed in Chap. 5. Multi-input/multi-output discrete-time linear systems and multivariate theory in digital signal processing will be studied in the final chapter via balanced realization. The first author would like to acknowledge the continuous support from the National Science Foundation and the U.S. Army Research Office in his research in this and other related areas. To Stephanie Sellers, Secretary of the Center for Approximation Theory at Texas A&M University, he is grateful for unfailing cheerful assistance including making corrections of the 'lex files. To his wife, Margaret, he would like to express his appreciation for her understanding, support, and assistance. The second author would like to express his gratitude to his wife Qiyun Xian for her patience and understanding. During the preparation of the manuscript, we received assistance from several individuals. Our special thanks are due to the Series Editor, Thomas Huang, for his encouragement and the time he took to read over the manuscript, Xin Li for his assistance in improving the presentation of Chap. 5, and H. Berens, I. Gohberg, and J. Partington for their valuable comments and pointing out several typos. In addition, we would like to thank J. A. Ball, B. A. Francis, and A. Tannenbaum for their interest in this project. Finally, the friendly cooperation and kind assistance from Dr. H. Lotsch and his editorial staff at Springer-Verlag are greatly appreciated. College Station Houston August 1991 Charles K. Chui Guanrong Chen

9 Contents 1. Digital Signals and Digital Filters Analog and Digital Signals Band-Limited Analog Signals Digital Signals and the Sampling Theorem Time and Frequency Domains Fourier Transforms and Convolutions on Three Basic Groups Frequency Spectra of Digital Signals z-transforms Properties of the z-transform Causal Digital Signals Initial Value Problems Singular and Analytic Discrete Fourier Transforms Digital Filters Basic Properties of Digital Filters Transfer Functions and IIR Digital Filters Optimal Digital Filter Design Criteria An Interpolation Method Ideal Filter Characteristics Optimal IIR Filter Design Criteria 35 Problems Linear Systems State-Space Descriptions An Example of Flying Objects Properties of Linear Time-Invariant Systems Properties of State-Space Descriptions Transfer Matrices and Minimal Realization Transfer Matrices of Linear Time-Invariant Systems Minimal Realization of Linear Systems SISO Linear Systems Kronecker's Theorem Minimal Realization of SISO Linear Systems System Reduction Sensitivity and Feedback Systems Plant Sensitivity Feedback Systems and Output Sensitivity 71

10 XII Problems Contents Sensitivity Minimization Approximation in Hardy Spaces Hardy Space Preliminaries Definition of Hardy Space Norms Inner and Outer Functions The Hausdorff-Young Inequalities 3.2 Least-Squares Approximation Beurling's Approximation Theorem An All-Pole Filter Design Method A Pole-Zero Filter Design Method A Stabilization Procedure 3.3 Minimum-Norm Interpolation Statement of the Problem Extremal Kernels and Generalized Extremal Functions An Application to Minimum-Norm Interpolation Suggestions for Computation of Solutions 3.4 Nevanlinna-Pick Interpolation An Interpolation Theorem Nevanlinna-Pick's Theorem and Pick's Algorithm Verification of Pick's Algorithm Problems. 4. Optimal Hankel-Norm Approximation and Hoo Minimization 4.1 The Nehari Theorem and Related Results Nehari's Theorem The AAK Theorem and Optimal Hankel-Norm Approximations 4.2 s-numbers and Schmidt Pairs Adjoint and Normal Operators Singular Values of Hankel Matrices Schmidt Series Representation of Compact Operators Approximation of Compact Hankel Operators 4.3 System Reduction Statement of the AAK's Theorem Proof of the AAK Theorem for Finite-Rank Hankel Matrices Reformulation of AAK's Result 4.4 Hoo-Minimization Statement of the Problem An Example of Hoo-Minimization Existence, Uniqueness, and Construction of Optimal Solutions Problems

11 Contents XIII 5. General Theory of Optimal Hankel-Norm Approximation Existence and Preliminary Results Solvability of the Best Approximation Problem Characterization of the Bounded Operators that Commute with the Shift Operator Beurling's Theorem Operator Norms of Hankel Matrices in Terms of Inner and Outer Factors Properties of the Norm of Hankel Matrices Uniqueness of Schmidt Pairs Uniqueness of Ratios of Schmidt Pairs Hankel Operators Generated by Schmidt Pairs The Greatest Common Divisor: The Inner Function ~~(z) Basic Properties of the Inner Function ~~(z) Relations Between Dimensions and Degrees Relations Between ~~(z) and s-numbers AAK's Main Theorem on Best Hankel-Norm Approximation Proof of AAK's Main Theorem: Case Proof of AAK's Main Theorem: Case Proof of AAK's Main Theorem: Case Problems HOO-Optimization and System Reduction for MIMO Systems Balanced Realization of MIMO Linear Systems Lyapunov's Equations Balanced Realizations Matrix-Valued All-Pass Transfer Functions Optimal Hankel-Norm Approximation for MIMO Systems Preliminary Results Matrix-Valued Extensions of the Nehari and AAK Theorems Derivation of Results 230 Problems 241 References 247 Further Reading 251 List of Symbols 255 Subject Index 257