Christakis Charalambous, B.Sc.(Eng.} A Thesis. Submitted to the Faculty of Graduate Studies. in Partial Fulfilment of the Requirements.

Size: px

Start display at page:

Download "Christakis Charalambous, B.Sc.(Eng.} A Thesis. Submitted to the Faculty of Graduate Studies. in Partial Fulfilment of the Requirements."

Oswin Hines
5 years ago
Views:

1 NONLINEAR LEAST pth APPROXDfATION AND NONL:':NEAR PROGRAMMING WITH APPLICATIONS IN THE DESIGN OF NETWORKS AND SYSTEMS by Christakis Charalambous, B.Sc.(Eng.} A Thesis Submitted to the Faculty of Graduate Studies in Partial Fulfilment of the Requirements for the Degree Doctor of Philosophy Mc~fastet' University February 1973 ~ Christakis Charalambous 1973

2 NONLINEAR LEAST pte APPROXI~~TION WITH APPLICATIONS

3 DOCtOR OF PHILOSOPHY (1973) McMaster University Hamilton, Ontario title AutHOR SUPERVISOR : Nonlinear Least pth Approximation and Nonlinear Programming with Applications in the Design of Networks and Systems Christakis Charalambous B.Sc.(Eng.)(University of Surrey) J.W. Bandler B.Sc.(Eng.), Ph.D. (University of London) D.I.C. (Imperial College) NUMBER OF PAGES: ix, 156 SCOPE AND CON'IEN'IS: the purpose of this thesis is to present a unified treatment of nonlinear least pth and nonlinear minimax approximation problems, and a new method for nonlinear programming. Least pth approximation with values of p in the range 1,000 to l~~oo,ooo in conjunction with the Fletcher-Powell and Fletcher algorithms have been successfully applied to a variety of network and system optimization problems. A comparison is made between two new ~lgorithrns for nonlinear minimax approximation and some of the existing ones. Also, the new approach to nonlinear programming is compared with the well-known SUMT. A critical review of unconstrained optimization is also included. ii

4 ACKNOtf:LEDGE~mNTS The author wishes to express appreciation to Dr. J.W. Bandler for his guidance and encouragement in the preparation of this thesis. The author thanks the other members of the Supervisory Committee, Dr. C.M. Crowe, Dr. T.M.K. Davison and Dr. E. Della Torre for their interest and encouragement. The author wishes to thank Dr. R.E. Seviora whose early suggestion~ resulted in a turning point in this work. Thanks go to B.L. Bardakjian, J. Chen, V.K. Jha, N.D. ~~rkettos, P. Liu, J.R. ropovic and T.V. Srinivasan who implemented some of the ideas presented in this thesis. Discussions with W. Kinsner are acknowledge~. The author is grateful for the generous support of the National Research Council of Canada through grants A7239 and C154, and through the award of an NRC Scholarship. The author's special thanks go to his wife Mary without whose patience this work could not have been undertaken. Thanks go. to Mrs. K. Paulin for her expert typing of the manuscript. iii

5 TABLE OF CONTENTS CHAPTER 1 - INTRODUCTION CHAPTER 2 - UNCONSTRAINED OPTIMIZATION Fundamental Concepts and Definitions Multidimensional Gradient Strategies Steepest Descent Newton Method Huang's Generalized Algorithm (1970) Special Case Special Case Fletcher Algorithm (1970a) CHAPTER 3 - GENERALIZED LEAST pth APPROXDIATION Introduction Leaet pth Approximation for Single Specified Function The Error Function Continuous Approximation Discrete Approximation Generalized Least pth Objectives The Error Functions Case 1 - Specification Violated Case 2 - Specification Satisfied Discussion Conclusions CHAPTER 4 - CONDITIONS FOR OPTIMALITY Introduction Objective Functions for Both Cases Case 1 - Specification Violated Case 2 - Specification Satisfied Assumptions Two Theorems Theorem Proof for Case Proof for Case Theorem Proof for Case Proof for Case Optimality Conditions for Complex Error iv

6 CHAPTER 4 - Continued Page Examples Second-Order Model of a Fourth- Order System Quarter-Wave Transmission-Line Transformer Conclusions 46 CHAPTER 5 - PRACTICAL LEAST pta OPTIMIZATION Introduction Definitions The Objective Function Case 1 - Specification Violated Case 2 - Specificntion Satisfied Examples Design Examples Two- and Three-Section Transmission- Line Transformer Five-Section Transmission-Line Filter Lumped-Distributed-Active Filter Discussion Conclusions 81 CHAPTER 6 - NONLINEAR MINIMAX OPTI}lIZATION Introduction Background Theory Derivation of Algorithm Computational Procedure Assumptions Lemma Lemma Lemma Theorem Theorem Theorem Derivation of Algorithm Computational Procedure Lemma LerJma Theorem Theorem Theorem Examples Problem Problem v

7 CHAPTER 6 - Continued Page Design Examples Second-Order Model of a Fourth- Order System Three-Section Transmission-Line Transformer Nonlinear ~animax Optfmization with Constraints Conclusions 119 CHAPTER 7 - NONLINEAR PROGRAMMING USING MINIMAX TECHNIQUES Introduction The Nonlinear Programming Problem An Equivalent Minimax Problem Theorem Possible Implementation l2~ Comments Examples The Post Office Parcel Problem (Rosenbrock 1960) The Beale Problem (Beale 1967) The Rosen-Su:uki Problem (Rosen and Suzuki 1965) Quadratic Function with Equality Constraints Conclusions 136 CHAPTER 8 - CONCLUSIONS 139 REFERENCES 142 APPENDIX 150 AUTHOR INDEX 154 vi

8 --, I LIST OF FIGURES Figure Fig. 3.1 Fig. 3.2 Fig. 3.3 Example of a design problem for which it is generally impossible for the response to exceed the specification. Case 1 is applicable. Example of a design problem in which the response exceeds the specification. Case 2 is applicable. Sketches to illustrate the behaviour of components of possible generalized least pth objectives. f(x) is convex, continuous with continuous derivatives. p>l in (b) and (c). p~l in (d) Fig. 4.l(a) Fig. 4.1 (b) Fig. 4.2 Optimum error for least pth approximation with p equal to 2 for the problem in Section 4.6.l~ Optimum error for least pth approximation with p equal to 10,000 for the problem in Section Pi(P) or li(p), as appropriate, calculated at specified values of time and for certain values of p for the problem in Section Fig. 5.1 Contours of U for p = Fig. 5.2 Contours of U for p = 2 55 Fig. 5.3 Contours of U for p = Fig. 5.4 Contours of U for p = Fig. 5.5 Optimization from Zl=1. 0, Z2=3.0, (a) Fletcher, 61 (b) Fletcher and Powell. Fig. 5.6 Optimization from Zl=1. 0, Z2=6.0, (a) Fletcher, 62 (b) Fletcher and Powell. F:f.g. 5.7 Optimization from Zl=3.5, Z2=6.0, (a) Fletcher, 63 (b) Fletcher and Powell. vii

9 Figure Page Fig. 5.8 Optimization from Zl=3.5, Z2=3.0, (a) Fletcher, 64 (b) Fletcher and Powell. Fi~. 5.9 Optimization from Zl=1.5, Z2=3.0, Z3~6.0, 65 tl/t~=0.8, t2/~=1.2, t3/lq=0.a, (a) Fletcher, (b) letcher an Powell. Fig. 5!10 5-section transmission-line lowpass filter. 66 Fig Optimized response of the circuit of Fig subject to the constraints imposed for Problem 1, Section Fig Fig Fig Fig Fig Fig Optimized response of the circuit of Fig subject to the constraints imposed for Problem 2, Section Passband details of the optimized response shown 71 in Fig Third-order lumped-distributed-active lowpass 72 filter. Optimized, gain of the circuit of Fig subject 77 to the constraints imposed for Problem 1, Section Optimized gain of the circuit of Fig subject 78 to the constraints imposed for Problem 2, Section Optimized gain of the circuit of Fig subject 79 to the constraints imposed for Problem 3, Section Fig. 6.1 Fig. 6.2 Fig. 6.3 Fig. 6.4 Fig. 6.5 Contours of U (~) for Problem ", Contours of U 2 (t,~) for Problem 6.1 with ~ = o. 99 Contours of U2<t'~) for Problem 6.1 with 100 ~= e. Contours of U2(t,~) for Problem 6.1 with 101 ~=2.036l+E. Contours of Uoo<t) for Problem viii

10 Figure Page Fig. 7.1 Contours for the Post Office Parcel Problem 127 when a=200. Fig. 7.2 Contours for the Post Office Parcel Problem 128 when a=245. Fig. 7.3 Contours for the Post Office Parcel Problem 129 when a=300. Fig. 7.4 Contours for the Post Office Parcel Problem 130 when a"'106 Fig. 7.5 Contours for the quadratic function when a-l. 137 Fig. A.1(a) Original network. 151 Fig. A.1(b) Adjoint network. lsi ix

11 CHAPTER 1 INTRODUCTION This thesis is basically centered around the theoretical and practical implementation of least pth approximation for cases where we have upper and lower response specifications, such as are encountered in filter design, and on new methods for nonlinear programming. Since, throughout the thesi~we will often have unconstrained objective functions to be minimized, Chaptar 2 is devoted to the most important gradient algorithms for unconstrained optimization, such as those by Huang (1970), Fletcher and Powell (1963), and Fletcher (1970a). Usually least pth approximation is applied to the approximation of single specified functions by a network or system response. Chapter 3 extends the usefulness of least pth approximation to a wider variety of network and system design problems and a wider range of specifications than appear to have been considered previously from the least pth point of view. See Eandler and Charalambous (197la, 1972c). Using the above extension of least pth approximation it w~s possible to derive the necessary and sufficient conditions for optimality in generalized nonlinear least pth approximation problems for p + ~ (Bandler and Charalambous 1971b, 1972b, 1973a). In the limit the conditions for a minimax approximation are derived as is to be expected. See, for example, Band1er (1971). This is discussed in Chapter 4. 1

12 2 It is very well known that 1ea~t pth approximation with a sufficiently large value of p cant in princip1e t be used to achieve a near minimax solution. Chapter 5 shows how to eliminate the i11 conditioning which arises when the value of p is extremely large. The important feature of this approach is the use that can be made of efficient gradient minimization techniques such as the Fletcher and Powell method (1963) and the more recent Fletcher method (1970a) in conjunction with least pth objective functions employing extremely large values of Pt typically 1 t OOO to 1 t OOO t OOO. At the time when this approach was developed the largest value of p successfully used and reported in the literature wast to the author's know1edge t 10. Application of this method to microwave design problems and to 1umpeddistributed-active filters is included (Band1er and Chara1ambous 1971c t 1972dt 1972e). See also Band1er t Chara1ambous and Tam (1972). Chapter 6 presents two new algorithms for nonlinear minimax optimization. The nonlinear optimization problem is solved by transforming it into a sequence of least pth optimization problems with a b-i.y!ue value of p (Chara1ambous and Band1er J.973a). From the experimental results which are available it seems that these methods are faster than any of the existing methods (Chara1ambous and Band1er 1973b). Chapter 7 presents a new approach to nonlinear programming (Band1er and Chara1ambous 1972at 1973b). The original nonlinear programming problem is formulated as an unconstrained minimax problem. Under reasonable restri~tions it is shown that a point satisfying the Kuhn-Tucker necessary conditions for optimality (1950) of the original nonlinear programming problem also satisfies the necessary conditions for

13 3 optimality of the minimax problem. Several numerical examples compare the new approach with the well-known SUMT method of Fiacco and McCormick (1964a, 1964b). The adjoint netw~rk approach for evaluating the gradients of the objective function with respect to network parameters was used for network design problems (Director and Rohrer 1969, Bandler and Seviora 1970). Throughout the thesis one function evaluation includes evaluation of all first derivatives. The digital computer used for all the numerical results was a CDC Original contributions are: (i) (ii) Generalized least pth objectives. The conditions for optimal minimax approximation derived from the generalized least pth objectives. (iii) The scaling procedure allowing least pth approximation with extremely large values of p. (iv) (v) Two new algorithms for minimax optimization. A new approach to nonlinear programming.

14 CHAPTER 2 UNCONSTRAINED OPT"tMIZATION 2.1 Fundamental Concepts and Definitions The uncol.atrained optimization problem is to calculate the minimum value of the scalar valued function U where (2.1) and (2.2) U is called the objective nunction and the column vector t contains the k real independent variables. The term "unconstrained" implies that the value of each variable can be any real number. Maximizing a function is the same as minimizing the negative of the function, so only the minimization problem will be considered. A point t is called a global minimum of U(t) if (2.3) for all t. If the strict inequality holds for t ~ to be unique. i the minimum is said..,.., If (2.3) holds only in the neighbourhood of t, then t is called a local minimum of U. 4

15 l I 5 given by The first three terms of the multidime~sionaltaylor series are where (2.4) 6 (2.5) represents the incremental change in the parameters, a a~l v 6.. '" a aljl2 a aljlk (2.6) is the first partial derivative operator with respect to the parameter vector t, and a 2 u aljl2 1 a 2 u aljllaljl2 a 2 u aljllaljlk G 6 "" '" a 2 u aljl2 aljl l a 2 u a 2 u ;2 aljl2 aljl k (2.7) 2.-1

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

, MODELLING AND CONTROL OF SUSTAINED OSCILLATIONS IN THE CONTINUOUS EMULSION POLYMERIZATION OF VINYL ACETATE. by Mark James Pollock

, MODELLING AND CONTROL OF SUSTAINED OSCILLATIONS IN THE CONTINUOUS EMULSION POLYMERIZATION OF VINYL ACETATE by Mark James Pollock A Thesi s Submitted to the Schoo~ of Graduate Studies in Partial Fulfilment