On the Connection between Balanced Proper Orthogonal Decomposition, Balanced Truncation, and Metric Complexity Theory for Infinite Dimensional Systems

Size: px

Start display at page:

Download "On the Connection between Balanced Proper Orthogonal Decomposition, Balanced Truncation, and Metric Complexity Theory for Infinite Dimensional Systems"

Duane Barber
5 years ago
Views:

1 1 American Control Conference Marriott Waterfront, Baltimore, MD, USA June 3-July, 1 FrA6.6 On the Connection between Balanced Proper Orthogonal Decomposition, Balanced Truncation, and Metric Complexity Theory for Infinite Dimensional Systems Seddik M. Djouadi Abstract In this paper, the connection between two important model reduction techniques, namely balanced proper orthogonal decomposition POD) and balanced truncation is investigated for inite dimensional systems. In particular, balanced POD is shown to be optimal in the sense of distance minimization in a space of integral operators under the Hilbert- Schmidt norm. Whereas balanced truncation is shown to be a particular case of balanced POD for inite dimensional systems for which the impulse response satisfies certain finite energy constraints. POD and balanced truncation are related to certain notions of metric complexity theory. In particular both are shown to minimize different n-widths of partial differential equation solutions including the Kolmogorov, Gelfand, linear and Bernstein n-widths. The n-widths quantify inherent and representation errors due to lack of data and loss of ormation. I. INTRODUCTION In this paper, we study the possible connections between two important model reduction techniques, namely, balanced truncation introduced in [14], and balanced proper orthogonal decomposition POD) introduced in [] for finite dimensional systems. Both model reduction techniques have been widely applied in diverse areas, see for example, [1][9][6][][3] [4][5]. Balanced truncation has been extended to inite dimensional systems in [7][18]. Balanced POD has been extended to the inite dimensional setting in [8]. Balanced POD is a tractable method for computing approximate balanced truncations, that has computational cost similar to POD [], where connections between balanced POD and balanced truncation are discussed. In particular, it is pointed that balanced truncation may be viewed as POD of a particular dataset, using the observability Gramian in an inner product []. In this paper, we take a different direction and show that the balanced POD and balanced truncations coincide when using a particular balanced realization for inite dimensional systems. This is carried out by first showing that balanced POD is optimal, as far as best approximation of a certain class of impulse responses of stable linear inite dimensional systems by stable impulse responses of finite dimensional systems in some L -norm. The latter is shown to be equivalent to optimal approximation of Hilbert-Schmidt Hankel operators by finite rank Hankel S.M. Djouadi is with the Electrical Engineering & Computer Science Department, University of Tennessee, Knoxville, TN djouadi@eecs.utk.edu. This work was supported by the Air Force Office of Scientific Research under contract AF-FA , and National Science Foundation under contract NSF-CMMI operators in the Hilbert-Schmidt norm. We relate balanced POD and balanced truncation to certain notions in metric complexity theory. In particular both are shown to minimize different n-widths of inite dimensional systems, including the Kolmogorov, Gelfand, linear and Bernstein n-widths. The n-widths quantify inherent and representation errors due to lack of data and loss of ormation. For example, the Kolmogorov n-width quantifies the representation error due to inaccurate representation of the range space of the system Hankel operator. It represents the loss of ormation in the ormation processing stage. The Kolomogorov n-width characterizes the representation complexity of the model reduction problem. The Gelfand n-width characterizes the experimental complexity of the ormation collecting stage using simulation or identification. It is related to the inherent error due to lack of data and inaccurate measurements. The inverse of the Gelfand n-width gives the least number of measurements needed to reduce the modeling uncertainty to a predetermined value. The rest of the paper is organized as follows. Section II contains the problem formulation. In section III a key balanced truncation is presented and is based on [18]. In section IV truncation of the balanced realization is shown to yield balanced POD. Section V discusses the relationship between balanced POD, balanced truncation and metric complexity theory. Section VI provides concluding remarks. II. PROBLEM FORMULATION In this paper the class of stable linear inite dimensional systems G with m inputs, p outputs described by the following, G :ẋt) Axt)+But) yt) Cxt) 1) where A is the generator of a strongly continuous semigroup T ) see for e.g. [17] for definition), B a linear bounded operator from R m into H, C a linear bounded operator from H into R p, H a Hilbert space representing the state space. The Hilbert space H will be taken to be the universal) Hilbert space l for concreteness. In what follows we assume that the initial state x.we /1/$6. 1 AACC 4911

2 have then see for e.g. [11]) xt) and consequently yt) t t T t τ)buτ)dτ, t ) CTt τ)buτ)dτ, t 3) The corresponding impulse response ht) is then defined by ht) CTt)B, t. We assume that the impulse response satisfies h ) L 1 R p m ) L R p m ) 4) The output is given by the convolution yt) t ht s)us)ds 5) We would like to approximate the behavior of the inite dimensional system 5) by finite dimensional models of order, say n. To do so we consider the impulse response of the system 5), and formulate the problem as a distance minimization as follows μ ht) h n t) dt h n where h n is an impulse response of order n<, and the norm is defined as 6) ht) trh h)t) 7) where tr ) denotes the trace, and h denotes the adjoint of h. The corresponding Hankel operator to ht), denoted Γ: L R m ) L R p ), is defined as Γut) ht τ)uτ)dτ 8) The change of variables τ s transforms 44) into Γut) ht + s)us)ds 9) Note that 9) is an integral operator with kernel ht + s). The assumption 4) guarantees that the operator Γ is a Hilbert-Schmidt operator defined on L R m ). Moreover, the Hilbert-Schmidt norm of Γ, denoted HS, can be written as [16] Γ HS ht) dt 1) Let its spectral factorization be given by Γ σ i χ i ζ i,χ i L R m ),ζ i L R p ) 11) where σ i are the Hankel singular values of the system G ordered in decreasing order, i.e., σ 1 σ σ n 1 σ n, denotes the tensor product see e.g. [16] for the definition), and {χ i } n 1 and {ζ i } n 1 are orthonormal sets in L R m ) and L R p ), respectively. The Hilbert Schmidt norm of Γ is also given by Γ HS i σ i trh h)s)ds Γφ i 1) where {φ i } is any orthornomal basis in L R m ). Next, consider the optimal distance minimization μ n : Γ Γ n HS 13) n< where Γ n is an operator acting from L R m ) into L R p ) of rank at most n. The optimization 13) is the shortest distance from the operator Γ to the subspace of finite rank Hilbert-Schmidt operators acting from L R m ) into L R p ). The latter will be denoted by S n. Note that this distance problem is posed in an inite-dimensional space. The latter is the space of Hilbert-Schmidt operators with the HS-norm which ia Hilbert operator space. If the minimizing operator is itself a Hankel operator then the corresponding impulse response will solve the optimization 6) A Theorem in [16] asserts that the subspace of finite rank Hilbert-Schmidt operators defined from L R m ) into L R p ), S n, can be represented in terms of tensors as S n {S α j υ j τ) ϕ j t), α j, j1 υ j τ) L R m ),ϕ j t) L R p )} 14) where {υ j } and {ϕ j } can be chosen to be orthonormal sets in L R m ) and L R p ), respectively. The space of Hilbert-Schmidt operators is in fact a Hilbert space with the inner product [16], denoted, ), if A and B are two Hilbert-Schmidt operators defined on L R m ), A, B) :trb A) 15) where tr denotes the trace, which in this case is given by the sum of the eigenvalues of the operator B A which is necessarily finite [16]. Note that the inner product 16) induces the Hilbert-Schmidt norm A HS tra A). As it stands by the principle of orthogonality the shortest distance minimization is solved by the orthogonal projection of Γ onto the subspace S n. Let us call this orthogonal projection P S. Using the polar representation of compact operators [16], ΓUΓ Γ, where U is a partial isometry and Γ Γ the square root of Γ, which is also a Hilbert-Schmidt operator, and admits a spectral factorization of the form [16] Γ Γ h σ i χ i χ i 16) i 491

3 where σ i >, σ i as i, are the eigenvalues of Γ h Γ h, and ν i, form the corresponding orthonormal sequence of eigenvectors, i.e., Γ Γ χ i σ i χ i, i 1,,. Note that Uχ i : ζ i. Both {χ i } and {ζ i } are orthonormal sequences in L R m ) and L R p ), respectively. The sum 16) has either a finite or countably inite number of terms. The above representation is unique. Our objective now is to compute the orthogonal projection P S. To do so, note that the subspace S n can be re-written as S n Span{υ j τ) ϕ j t), j 1,,,n, υ j L R m ),ϕ j L R p )} 17) First, we show that the minimizing tensors υ j τ) ϕ j t), j 1,,,n, correspond to the Hankel operator tensors χ i ζ i, i 1,,,n. To see this let {φ j } be a basis in L R m ), then we have Γ HS Γφ i 18) Let α j be scalars, with υ j φ j, j 1,,n. The minimization 13) can be written as μ n Γ α j υ j ϕ j 19) α j,υ j,ϕ j j1 It follows from 18) that the RHS of 19) squared satisfies Γ α j υ j ϕ j Γ α j υ j ϕ j )φ i j1 Γφ i in+1 i j1 α j <υ j,φ i >ϕ j j1 Γφ i + Γφ i α i ϕ i Γ HS Γφ i + Γφ j α j ϕ j ) Therefore, we have Γ α j,υ j,ϕ j j1 j1 α j υ j ϕ j { Γ HS α j,υ j,ϕ j Γυ i + Γυ j α j ϕ j } j1 1) The minimization can be accomplished by maximizing n Γυ i with respect to υ i, υ i 1, i 1,,n, and minimizing n j1 Γυ j α j ϕ j, under υ j ϕ j 1. The maximization is solved by taking υ j χ j, and the minimization by putting α j σ j, ϕ j ζ j, j 1,,n, since Γχ j σ i <χ j,χ i >ζ i σ j ζ j ) j1 j1 j1 Hence, the optimal subspace S n can be written as S n Span{χ j τ) ζ j t), j 1,,,n} 3) The optimal rank n operator, denoted Γ o n, in 13) is therefore given by Γ o n σ i χ i ζ i 4) From 4) we see that Γ o n is a Hilbert-Schmidt operator Γ o n : L R m ) L R p ) Γ o nf)t) σ i χ i τ)fτ) dτ ζ i t) 5) The corresponding Hilbert-Schmidt kernel is kτ,t): σ i χ i τ) ζ i t) 6) The optimal distance minimization 13) is then μ n Γ Γ o n HS σi 7) in+1 The optimal Γ o n is a not necessarily a Hankel operator. In fact, Γ o n is a Hankel operator if there exists an integral kernel N such that [11] kτ,t)nτ + t), τ, t 8) In the next section, we show that a particular balanced realization realizes the optimal operator Γ o n as a Hankel operator. That is, the corresponding kernel kτ,t) of Γ o n may be taken to satisfy 8). Moreover, truncating this balanced realization will be shown to correspond to balanced POD. III. A PARTICULAR BALANCED REALIZATION In this section, we introduce a particularly important balanced realization proposed in [18], and which play a crucial role in the sequel. The importance of this realization stems from the fact that it uses the Hankel singular vectors and values in the state space representation. Let Ã, B and C be linear bounded operators: Ã : H H, B : R m H, C : H R p, where Ã is the initesimal generator of a strongly continuous semigroup, T t), onh such that ht) C T t) B, a.e. in t. That is C,Ã, B) is a realization of ht). We will assume that T t) is exponentially stable. Define the controllability and observability operators denoted Ψ c and Ψ o, respectively by [11] Ψ c : L R m ) H Ψ c u : T t) Buτ)dτ Ψ o : H L R p ) Ψ o x : C T t)x, t 9) The operators Ψ c and Ψ o realize the Hankel operator Γ as ΓΨ o Ψ c 3) 4913

4 The realization Ã, T t), B) is balanced if the following controllability and observability gramians Ψ c Ψ c Ψ oψ o T t) B B T t) dt 31) T t) C C T t)dt 3) are both equal to the same positive diagonal operator, i.e., Ψ c Ψ c Ψ oψ o Σ : diag{σ 1,σ, } 33) Following [18] we use the following realization on l, for i, j 1,,, σj T ij t) ζi τ)ζ j t + τ)dτ 34) σ i B σ 1 χ 1 ),, σ i χ i ), )T 35) C σ 1 ζ 1 ),, σ i ζ i ), ) 36) Note that T t) Tij t) ) is a strongly continuous contraction semigroup on l. Moreover, the corresponding initesimal generator Ã ) Ã ij satisfies [18] Ã ij σj σ i ζi τ) dζ jτ) dτ 37) dτ This realization produces bounded controllability and observability operators which satisfy [18] Ψ c Ψ c Ψ oψ o Σ : diag{σ 1,σ, } 38) That is the realization C,Ã, B) is a balanced realization. In the next section, we show that truncating this balanced realization results in balanced POD. IV. TRUNCATION OF THE BALANCED REALIZATION YIELDS BALANCED POD Following [18] and as in the finite dimensional case, we define the n-th order truncations of C,Ã, B) as Ãr : ) σ j Ãr ij : ζi τ) dζ ) jτ) dτ σ i dτ i, j 1,,,n B r : σ 1 χ 1 ),, σ n χ n ) ) T C r : σ 1 ζ 1 ),, σ n ζ n ) ) 39) The truncated system Ãr, B r, C r ) has state space R n, is stable and balanced [18], i.e., the corresponding controllability and observability gramians satisfy Ψ rc Ψ rc Ψ roψ ro Σ : diag{σ 1,σ,,σ n } 4) Therefore, the reduced model 39) corresponds exactly to balanced truncation. Now its impulse response, denoted h r t), is given by h r t) C r eãrt Br, a.e. t. In terms of the singular vectors χ i and ζ i, i 1,,,n, the semigroup eãr t can be written as eãr t σj ζi τ)ζ j t + τ)dτ, i, j 1,,n 41) σ i The Hankel operator, denoted Γ r, associated to the truncated balanced system, Ãr, B r, C r ),isgivenby Γ r : L R m ) L R p ) Γr u ) t) h r t + τ)uτ)dτ 4) The corresponding spectral decomposition is clearly Γ r σ i χ i ζ i 43) and therefore, Γ r Γ o n and Γ r solves the optimal distance minimization 13). In addition, since Γ r is Hankel its impulse response solves the optimization 6), i.e., μ ht) h n t) dt h n ht) h r t) dt 44) and both optimizations 6) and 13) are equal, i.e., μ μ n. The above derivation shows that balanced POD and balanced truncation are both the result of the optimal approximation 13), and in fact coincide when using the realization 36) proposed in [18]. In the next section, balanced POD and balanced truncation are related to key notions in metric complexity theory. V. BALANCED POD, BALANCED TRUNCATION AND METRIC COMPLEXITY THEORY In this section, we discuss role of balanced POD and truncation in optimizing different n-widths defined in [13]. The main object here is the Hankel operator Γ associated to the system. Recall that Γ is defined as Γ : L R m ) L R p ) Γu)t) ht + τ)uτ)dτ 45) Under the assumption on the impulse response ht) stated above the operator Γ is Hilbert-Schmidt. Since in model reduction we are interested in approximation by finite dimensional models, and in particular, n-parameter affine models such as in balanced POD, we introduce the representation error of Γ as the best optimal) set distance between the range space of Γ, ΓL R m ) ) and the finite dimensional subspaces, X n,ofl R p ), i.e., ΓL dist R m ) ) ),X n : sup ) f g 46) g X n f ΓL R m ) where the imum is taken over the n-dimensional subspaces X n of L R p ). By definition of the representation error 46), the minimum 4914

5 representation of the range space ΓL R m ) ) by an n- dimensional subspace X n is ΓL d n R m ) ) ),L R p ) ΓL dist R m ) ) ),X n 47) X n L R p ) This is exactly the Kolmogorov n-width of ΓL R m ) ). If the imum in 47) is attained for some subspace Xn, o then Xn o is said to be an optimal subspace for ΓL d n R m ) ) ), L R p ). The optimal subspace gives the optimal n-dimensional affine model for ΓL R m ) ). The Kolmogorov n-width quantifies the representation error due to inaccurate representation of the set ΓL R m ) ) : It represents the loss of ormation in the ormation processing stage. The Kolomogorov n-width characterizes the representation complexity of the model reduction problem. The inverse function of d n ) was called metric dimension function by Zames [19], and viewed as an appropriate measure of the metric complexity of uncertain systems. It is the dimension of the smallest subspace whose elements can approximate arbitrary points of ΓL R m ) ) to a specified tolerance. The n-width in the sense of Gel fand, is defined as [13] d n Γ L R m ) ) ) ; L R p ) : sup Γf 48) L n f 1, f L n where the imum is taken over all subspaces L n of Γ L R m ) ) of codimension at most n. A subspace is said to be of co-dimension n if there exist n independent bounded linear functionals {f i } n on L R p ) for which If L n {g : g L R p ),f i g),,,...,n} 49) d n Γ L R m ) ) ) ; L R p ) sup{ f : f Γ L R m ) ) L n } 5) where L n is a subspace of codimension at most n, then L n is an optimal subspace for d n Γ L R m ) ) ) ; L R p ) 51) The Gel fand n-width characterizes the experimental complexity of the ormation collecting stage using simulation or identification. It is related to the inherent error due to lack of data and inaccurate measurements. The inverse of the Gel fand n-width gives the least number of measurements needed to reduce the modeling uncertainty to a predetermined value. The linear n-width is defined by δ n Γ L R m ) ) ; L R p ) ) : sup Γφ P n φ 5) P n φ 1, φ L R m ) where P n is any continuous linear operator from L R m ) into L R p ) of rank at most n. The n-width in the sense of Bernstein of Γ L R m ) ) is defined by [13] Γ L R m ) ) ) ; L R p ) sup sup{λ : λsx n+1 ) L R p )} X n+1 sup X n+1 x Γ L R m ) ) X n+1) x 53) where X n+1 is any n+1)-dimensional subspace of L R p ), and SX n+1 ) is the unit ball in X n+1, i.e., SX n+1 ){y X n+1 : y 1} 54) The basic results of this section are summarized in the following theorem which tells us that balanced POD and balanced truncation optimize the different n-widths. Theorem 1: Let the operator Γ and {σ i }, {χ i }, {ζ i } be defined as above. Then d n Γ L R m ) ) ) ; L R p ) d n Γ L R m ) ) ) ; L R p ) 55) δ n Γ L R m ) ) ) ; L R p ) Γ L R m ) ) ) ; L R p ) σ n+1, n, 1,, Furthermore, the singular vectors {χ i } and {ζ i } are optimal for the n-widths in the following sense i) The subspace spanned by the vectors {ζ i }, X n Span{ζ 1,,ζ n }, is optimal for d n Γ L R p ) ) ) ; L R p ) 56) ii) The subspace L n { χ L R m ), is optimal for d Γ n L R m) ) ; L R p ). iii) The linear operator Q n φ χτ)χ i τ)dτ, 57) } i 1,,,n 58) φτ)χ i τ)dτχ i 59) is optimal for δ n Γ L R m ) ) ) ; L R p ). iv) The subspace X n+1 Span{ζ 1,,ζ n+1 } 6) is optimal for Γ L R m ) ) ) ; L R p ). 4915

6 Sketch of the Proof. From Chapter II [13], we have d n Γ L R m ) ) ) ; L R p ) δ n Γ L R m ) ) ) ; L R p ) d n Γ L R m ) ) ) ; L R p ) Γ L R m ) ) ) ; L R p ) 61) Note that from 53) we have Γ L R m ) ) ) ; L R p ) sup x 6) ) X n+1 X n+1 x Γ L R m ) Let {ψ 1,,ψ n+1 } be a basis for X n+1, and since for x Γ L R m ) ) X n+1, x Γy, y L R m ) with Γy X n+1 Span{ψ 1,,ψ n+1 }, and therefore for some ϕ X n+1,wehave x Γy < ΓΓ ϕ, ϕ > 63) <ϕ,ϕ> where <, >represents the inner product in L R p ), and it follows that and thus x Γ L R m ) ) X n+1) x x < ΓΓ ϕ, ϕ > ϕ X n+1,ϕ <ϕ,ϕ> Γ L R m ) ) ) ; L R p ) sup ψ 1,,ψ n+1 ϕ Span{ψ 1,,ψ n+1},ϕ 64) < ΓΓ ϕ, ϕ > σ n+1 65) <ϕ,ϕ> where the supremum is taken w.r.t. all sets of n+1) linearly dependent vectors {ψ i } in L R p ). Moreover, the supremum is attained for ψ i ζ i, i 1,,n+1, and ϕ ζ n+1. The linear n-width satisfies [13] δ n Γ L R m ) ) ) ; L R p ) σ n+1 66) REFERENCES [1] K. Willcox and J. Peraire, Balanced Model Reduction via the Proper Orthogonal Decomposition, AIAA JOURNAL Vol. 4, No. 11, November, pp [] C. W. Rowley, Model reduction for fluids using balanced proper orthogonal decomposition, International Journal of Bifurcation and Chaos 153) : , March 5. [3] Ilak, M. and Rowley, C. W., Reduced-Order Modeling of Channel Flow Using Traveling POD and Balanced POD,. 3rd AIAA Fow Control Conference, San Francisico, June 6 [4] M. Ilak and C. W. Rowley Modeling of transitional channel flow using balanced proper orthogonal decomposition, Physics of Fluids 3413), March 8. [5] M. Ilak and C. W. Rowley Reduced-order modeling of channel flow using traveling POD and balanced POD AIAA Paper , 3rd AIAA Flow Control Conference, June 6. [6] Seddik M. Djouadi, R. Chris Camphouse, and James H. Myatt, Empirical Reduced-Order Modeling for Boundary Feedback Flow Control, Journal of Control Science and Engineering Volume 8 8), Article ID , 11 pages.doi:1.1155/8/ [7] N.J. Young, Balanced Realizations in Infinite Dimensions, Opeartor Theory Advances and Applications, vol. 19, Birkhauser Verlag Basel, [8] J.R. Singler and B. King, Balanced Proper Orthogonal Decomposition for Model Reduction of Infinite Dimensional Linear Systems, Proc. of the International Conference on Computational and Mathematical Methods in Science and Engineering, CMMSE 7, Chicago, June 7. [9] R.C. Camphouse, S.M. Djouadi and J.H. Myatt, Feedback Control for Aerodynamics, Proc. of the IEEE Conference on Decision and Control, 6. [1] K. Zhou, J.C. Doyle and K. Glover, Robust and Optimal Control, Prentice-Hall, [11] N.K. Nikolski, Operators, Functions and Systems: An Easy Reading. Volume : Model Operators and Systems, Mathematical Surveys and Momographs Vol. 93, American Mathematical Society,. [1] F. Riesz and B.Sz.-Nagy, Functional Analysis, Dover, 199. [13] A. Pinkus, n-widths in Approximation Theory, Springer-Verlag, [14] Moore, B., Principal Component Analysis in Linear Systems: Controllability, Observability, and Model Reduction, IEEE Transactions on Automatic Control, Vol. AC-6, No. 1, 1981, pp [15] L.M. Silverman and M. Bettayeb, Optimal Approximation of Linear Systems, Proceedings of the American Control Conference, [16] Schatten R. Norm Ideals of Completely Continuous Operators, Springer-Verlag, Berlin, Gottingen, Heidelberg, 196. [17] R. F. Curtain and H.J. Zwart, An introduction to inite-dimensional linear systems theory, Springer [18] K. Glover, R.F. Curtain and R. Partington, Realisation and Approximation of Linear Infinite Dimensional Systems with Erroe Bounds, SIAM J. Control and Optiz., vol. 6, No. 4, 1988), [19] G. Zames, On the metric complexity of causal linear systems: ε- entropy and ε-dimension for continuous time. IEEE Trans. Automat. Control ), no., 3. Thus all the n-widths equal σ n+1, and i), iii) and iv) hold. VI. CONCLUSION In this paper, using tools from operator and system theory we showed that balanced POD and balanced truncations coincide when using a particular balanced realization for inite dimensional systems. In this regard, Hilbert-Schmidt Hankel operator associated to the system when it exists plays a crucial role. Connections to different n-widths which are important notions in metric complexity theory are explored, and show that balanced POD and truncations optimize the n-widths. 4916

Let T (N) be the algebra of all bounded linear operators of a Hilbert space L which leave invariant every subspace N in N, i.e., A T (N), AN N.

Let T (N) be the algebra of all bounded linear operators of a Hilbert space L which leave invariant every subspace N in N, i.e., A T (N), AN N. 2009 American Control Conference Hyatt Regency Riverfront, St. Louis, MO, USA June 10-12, 2009 FrA04.4 Commutant Lifting for Linear Time-Varying Systems Seddik M. Djouadi Abstract In this paper, we study