Springer Series in Information Sciences 26 Editor: Thomas S. Huang Springer-Verlag Berlin Heidelberg GmbH
Springer Series in Information Sciences Editors: Thomas S. Huang Teuvo Kohonen Manfred R. Schroeder Managing Editor: H. K. V. Lotsch 30 Self-Organizing Maps By T. Kohonen 2nd Edition 31 Music and Schema Theory Cognitive Foundations of Systematic Musicology By M. Leman 32 The Maximum Entropy Method By N. Wu 33 Steps Towards 3D Active Vision By T. Vieville 34 Calibration and Orientation of Cameras in Computer Vision Editors: A. Grün and T. S. Huang 35 Speech Processing: Fundamentals and Applications By B. S. Atal and M. R. Schroeder Volumes 1-29 are listed at the end of the book.
C. K.Chui G. Chen Discrete Hoc Optimization With Applications in Signal Processing and Control Systems Second Edition With 38 Figures Springer
Professor Charles K. Chui Department of Mathematics, and Department of Electrical Engineering, Texas A&M University, College Station, TX 77843-3368, USA Dr. Guanrong Chen Department of Electrical Engineering, University of Houston, Houston, TX 77204-4793, USA Series Editors: Professor Thomas S. Huang Department of Electrical Engineering and Coordinated Science Laboratory, University of Illinois, Urbana, IL 61801, USA Professor Teuvo Kohonen Helsinki University of Technology, Neural Networks Research Centre, Rakentajanaukio 2 C, FIN-02150 Espoo, Finland Professor Dr. Manfred R. Schroeder Drittes Physikalisches Institut, Universität Göttingen, Bürgerstrasse 42-44, D-37073 Göttingen, Germany Managing Editor: Dr.-Ing. Helmut K. V. Lotsch Springer-Verlag, Tiergartenstrasse 17, D-69121 Heidelberg, Germany Library of Congress Cataloging-in-Publication Data. Chui, Charles K.: Discrete H~ optimization: with applications in signal processing and control systems 1 Charles K. Chui; Guanrong Chen. - 2. ed. - Berlin; Heidelberg; New York; Barcelona; Budapest; Hong Kong; London; Milan; Paris; Santa Clara; Singapore; Tokyo: Springer, 1997. (Springer se ries in information sciences; 26). I. Auf!. u.d.t.: Chui, Charles K.: Signal processing and systems theory ISBN 3-540-61959-3 Pp. ISSN OnO-678X ISBN 978-3-540-61959-8 ISBN 978-3-642-59145-7 (ebook) DOI 10.1007/978-3-642-59145-7 The Ist Edition appeard under the titie: Signal Processing and Systems Theory This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under the German Copyright Law. Springer-Verlag Berlin Heidelberg 1992, 1997 Originally published by Springer-Verlag Berlin Heidelberg New York in 1997 The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Data conversion by K. Matthes, Heidelberg Cover design: design & production GmbH, Heidelberg SPIN: 10559093 5413144-5 4 3 2 10- Printed on acid-free paper
Dedicated to the memory of our friend Professor Xie-Chang Shen (1934-1991)
Preface This is the second edition of our 1992 monograph, published under the title of Signal Processing and Systems Theory: Selected Topics. The need for the change in the current title was motivated by recent advancement of HOC-optimization in both continuous and discrete settings as well as the current trend in the development of similar and more effective theories and techniques for hybrid systems with continuous plants and digital feedback controllers. The new title also gives a better description of the contents of the book, concerning the fundamental theories and methodologies for discrete HOC-optimization studies and applications to such problems as optimal filter design and system reduction. In the preparation of this edition, we took advantage of the opportunity to correct several typos and update the list of references. We are again grateful to Dr. H. Lotsch of Springer-Verlag, for his encouragement and enthusiastic promotion of this book. College Station Houston February 1997 Charles K. Chui Guanrong Chen
Preface to the First Edition It is well known that mathematical concepts and techniques always play an important role in various areas of signal processing and systems theory. In fact, the work of Norbert Wiener on time series and the pioneering research of A. N. Kolmogorov form the basis of the field of communication theory. In particular, the famous sampling theorem, usually attributed to Wiener's student Claude Shannon, is based on the Paley-Wiener theorem for entire functions. In systems engineering, the important work of Wiener, Kalman, and many other pioneers is now widely applied to real-time filtering, prediction, and smoothing problems, while optimal control theory is built on the classical variational calculus, Pontryagin's maximum principle, and Bellman's dynamic programming. There is at least one common issue in the study of signal processing and systems engineering. A filter, stable or not, has a state-space description in the form of a linear time-invariant system. Hence, there are common problems of approximation, identification, stability, and rank reduction of the transfer functions. Recently, the fundamental work of Adamjan, Arov, and Krein (or AAK) has been recognized as an important tool for at least some of these problems. This work directly relates the approximation of a transfer function by stable rational functions in the supremum norm on the unit circle to that of the corresponding Hankel operator by finite-rank bounded operators in the Hilbert space operator norm, so that various mathematical concepts and methods from approximation theory, function theory, and the theory of linear operators are now applicable to this study. In addition, since uniformly bounded rational approximants on the unit circle are crucial for sensitivity considerations, approximation in the Hardy space HOC also plays an important role in this exciting area of research. This monograph is devoted to the study of several selected topics in the mathematical theories and methods that apply to both signal processing and systems theory. In order to give a unified presentation, we have chosen to concentrate our attention on discrete-time methods, not only for digital filters, but also for linear systems. Hence, our discussions of the theories and techniques in Hardy spaces and from the AAK approach are restricted to the unit disk. In particular, the reader will find that the detailed treatment of multi-input/multi-output systems in Chap. 6 distinguishes itself from the bulk of the published literature in that the balanced realization approach
X Preface to the First Edition of discrete-time linear systems is followed to study the matrix-valued AAK theory and HOC-optimization. The selection of topics in this monograph is guided by our objective to present a unified frequency-domain approach to discrete-time signal processing and systems theory. However, since there has been considerable progress in these areas during recent years, this book should be viewed as only an introductory treatise on these topics. The interested reader is referred to more advanced and specialized texts and original research papers for further study. We give a fairly rigorous and yet elementary introduction to signals and digital filters in the first chapter, and discrete-time linear systems theory in the second chapter. Hardy space techniques, including minimum-norm and Nevanlinna-Pick interpolations will be discussed in Chap. 3. Chap. 4 will be devoted to optimal Hankel-norm approximation. A thorough treatment of the theory of AAK will be discussed in Chap. 5. Multi-input/multi-output discrete-time linear systems and multivariate theory in digital signal processing will be studied in the final chapter via balanced realization. The first author would like to acknowledge the continuous support from the National Science Foundation and the U.S. Army Research Office in his research in this and other related areas. To Stephanie Sellers, Secretary of the Center for Approximation Theory at Texas A&M University, he is grateful for unfailing cheerful assistance including making corrections of the 'lex files. To his wife, Margaret, he would like to express his appreciation for her understanding, support, and assistance. The second author would like to express his gratitude to his wife Qiyun Xian for her patience and understanding. During the preparation of the manuscript, we received assistance from several individuals. Our special thanks are due to the Series Editor, Thomas Huang, for his encouragement and the time he took to read over the manuscript, Xin Li for his assistance in improving the presentation of Chap. 5, and H. Berens, I. Gohberg, and J. Partington for their valuable comments and pointing out several typos. In addition, we would like to thank J. A. Ball, B. A. Francis, and A. Tannenbaum for their interest in this project. Finally, the friendly cooperation and kind assistance from Dr. H. Lotsch and his editorial staff at Springer-Verlag are greatly appreciated. College Station Houston August 1991 Charles K. Chui Guanrong Chen
Contents 1. Digital Signals and Digital Filters 1 1.1 Analog and Digital Signals... 1 1.1.1 Band-Limited Analog Signals 1 1.1.2 Digital Signals and the Sampling Theorem 3 1.2 Time and Frequency Domains........ 7 1.2.1 Fourier Transforms and Convolutions on Three Basic Groups........ 7 1.2.2 Frequency Spectra of Digital Signals 12 1.3 z-transforms 13 1.3.1 Properties of the z-transform 13 1.3.2 Causal Digital Signals 16 1.3.3 Initial Value Problems.... 16 1.3.4 Singular and Analytic Discrete Fourier Transforms 19 1.4 Digital Filters 21 1.4.1 Basic Properties of Digital Filters..... 22 1.4.2 Transfer Functions and IIR Digital Filters 26 1.5 Optimal Digital Filter Design Criteria 30 1.5.1 An Interpolation Method... 30 1.5.2 Ideal Filter Characteristics 32 1.5.3 Optimal IIR Filter Design Criteria 35 Problems 37 2. Linear Systems 47 2.1 State-Space Descriptions 47 2.1.1 An Example of Flying Objects 47 2.1.2 Properties of Linear Time-Invariant Systems 49 2.1.3 Properties of State-Space Descriptions.... 50 2.2 Transfer Matrices and Minimal Realization..... 54 2.2.1 Transfer Matrices of Linear Time-Invariant Systems 54 2.2.2 Minimal Realization of Linear Systems 58 2.3 SISO Linear Systems................. 61 2.3.1 Kronecker's Theorem............. 62 2.3.2 Minimal Realization of SISO Linear Systems 66 2.3.3 System Reduction 68 2.4 Sensitivity and Feedback Systems........ 70 2.4.1 Plant Sensitivity 70 2.4.2 Feedback Systems and Output Sensitivity 71
XII 2.4.3 Problems Contents Sensitivity Minimization 74 77 3. Approximation in Hardy Spaces... 3.1 Hardy Space Preliminaries........ 3.1.1 Definition of Hardy Space Norms 3.1.2 Inner and Outer Functions... 3.1.3 The Hausdorff-Young Inequalities 3.2 Least-Squares Approximation. 3.2.1 Beurling's Approximation Theorem 3.2.2 An All-Pole Filter Design Method 3.2.3 A Pole-Zero Filter Design Method 3.2.4 A Stabilization Procedure 3.3 Minimum-Norm Interpolation. 3.3.1 Statement of the Problem. 3.3.2 Extremal Kernels and Generalized Extremal Functions 3.3.3 An Application to Minimum-Norm Interpolation 3.3.4 Suggestions for Computation of Solutions 3.4 Nevanlinna-Pick Interpolation. 3.4.1 An Interpolation Theorem............ 3.4.2 Nevanlinna-Pick's Theorem and Pick's Algorithm 3.4.3 Verification of Pick's Algorithm Problems. 4. Optimal Hankel-Norm Approximation and Hoo Minimization 4.1 The Nehari Theorem and Related Results 4.1.1 Nehari's Theorem. 4.1.2 The AAK Theorem and Optimal Hankel-Norm Approximations 4.2 s-numbers and Schmidt Pairs....... 4.2.1 Adjoint and Normal Operators. 4.2.2 Singular Values of Hankel Matrices. 4.2.3 Schmidt Series Representation of Compact Operators 4.2.4 Approximation of Compact Hankel Operators 4.3 System Reduction. 4.3.1 Statement of the AAK's Theorem 4.3.2 Proof of the AAK Theorem for Finite-Rank Hankel Matrices 4.3.3 Reformulation of AAK's Result 4.4 Hoo-Minimization. 4.4.1 Statement of the Problem. 4.4.2 An Example of Hoo-Minimization 4.4.3 Existence, Uniqueness, and Construction of Optimal Solutions Problems 84 84 85 86 88 89 89 91 93 95 100 100 101 102 104 105 105 109 111 113 118 119 119 123 127 127 129 130 134 137 137 139 147 149 149 151 154 161
Contents XIII 5. General Theory of Optimal Hankel-Norm Approximation 169 5.1 Existence and Preliminary Results 169 5.1.1 Solvability of the Best Approximation Problem 170 5.1.2 Characterization of the Bounded Operators that Commute with the Shift Operator.... 172 5.1.3 Beurling's Theorem 174 5.1.4 Operator Norms of Hankel Matrices in Terms of Inner and Outer Factors 177 5.1.5 Properties of the Norm of Hankel Matrices 180 5.2 Uniqueness of Schmidt Pairs............ 186 5.2.1 Uniqueness of Ratios of Schmidt Pairs 186 5.2.2 Hankel Operators Generated by Schmidt Pairs 188 5.3 The Greatest Common Divisor: The Inner Function ~~(z) 190 5.3.1 Basic Properties of the Inner Function ~~(z) 190 5.3.2 Relations Between Dimensions and Degrees 192 5.3.3 Relations Between ~~(z) and s-numbers 194 5.4 AAK's Main Theorem on Best Hankel-Norm Approximation 196 5.4.1 Proof of AAK's Main Theorem: Case 1 198 5.4.2 Proof of AAK's Main Theorem: Case 2 199 5.4.3 Proof of AAK's Main Theorem: Case 3 200 Problems 201 6. HOO-Optimization and System Reduction for MIMO Systems 208 6.1 Balanced Realization of MIMO Linear Systems 208 6.1.1 Lyapunov's Equations 209 6.1.2 Balanced Realizations.......... 211 6.2 Matrix-Valued All-Pass Transfer Functions.. 213 6.3 Optimal Hankel-Norm Approximation for MIMO Systems 220 6.3.1 Preliminary Results........... 220 6.3.2 Matrix-Valued Extensions of the Nehari and AAK Theorems 225 6.3.3 Derivation of Results 230 Problems 241 References 247 Further Reading 251 List of Symbols 255 Subject Index 257