Convergence of eigenvalues in state-discretization of linear stochastic systems

Transcription

1 University of Wollongong Research Online Faculty of Engineering and Information Sciences - Papers: Part A Faculty of Engineering and Information Sciences Convergence of eigenvalues in state-discretization of linear stochastic systems Jose A. de Dona University of Newcastle Graham C. Goodwin University of Newcastle Richard H. Middleton University of Newcastle Iain Raeburn University of Wollongong, raeburn@uow.edu.au Publication Details de Dona, J. A., Goodwin, G. C., Middleton, R. H. & Raeburn, I. (). Convergence of eigenvalues in state-discretization of linear stochastic systems. SIAM Journal on Matri Analysis and Applications, (4), -. Research Online is the open access institutional repository for the University of Wollongong. For further information contact the UOW Library: research-pubs@uow.edu.au

2 Convergence of eigenvalues in state-discretization of linear stochastic systems Abstract The transition operator that describes the time evolution of the state probability distribution for continuousstate linear systems is given by an integral operator. A state-discretization approach is proposed, which consists of a finite rank approimation of this integral operator. As a result of the state-discretization procedure, a Markov chain is obtained, in which case the transition operator is represented by a transition matri. Spectral properties of the integral operator for the continuous-state case are presented. The relationships between the integral operator and the finite rank approimation are eplored. In particular, the limiting properties of the eigenvalues of the transition matrices of the resulting Markov chains are studied in connection to the eigenvalues of the original continuous-state integral operator. Keywords linear, stochastic, discretization, state, convergence, systems, eigenvalues Disciplines Engineering Science and Technology Studies Publication Details de Dona, J. A., Goodwin, G. C., Middleton, R. H. & Raeburn, I. (). Convergence of eigenvalues in statediscretization of linear stochastic systems. SIAM Journal on Matri Analysis and Applications, (4), -. This journal article is available at Research Online:

3 SIAM J. MATRIX ANAL. APPL. Vol., No. 4, pp. c Society for Industrial and Applied Mathematics CONVERGENCE OF EIGENVALUES IN STATE-DISCRETIZATION OF LINEAR STOCHASTIC SYSTEMS JOSÉ A. DE DONÁ, GRAHAM C. GOODWIN, RICHARD H. MIDDLETON, AND IAIN RAEBURN Abstract. The transition operator that describes the time evolution of the state probability distribution for continuous-state linear systems is given by an integral operator. A state-discretization approach is proposed, which consists of a finite rank approimation of this integral operator. As a result of the state-discretization procedure, a Markov chain is obtained, in which case the transition operator is representedby a transition matri. Spectral properties of the integral operator for the continuous-state case are presented. The relationships between the integral operator and the finite rank approimation are eplored. In particular, the limiting properties of the eigenvalues of the transition matrices of the resulting Markov chains are studied in connection to the eigenvalues of the original continuous-state integral operator. Key words. state-discretization, Markov chains, integral operators, finite rank approimation AMS subject classifications. 6J, 47N3 PII. S Introduction. The understanding and use of links between continuous and discrete features of systems has proven to be a fruitful area. In some cases, knowledge of properties from discrete processes gives insight into the continuous counterpart. On the other hand, the mathematical methods associated with the continuum (e.g., differential equations, integrals) very often turn out to be easier for analytical treatment than those associated with discrete coordinate aes (e.g., difference equations, summations) []. The algorithms available for solving discrete-state problems, for eample, parameter estimation in Markov chains, have a very high computational compleity when the number of states is large. Typically the number of computations required is proportional to the square of the number of states, for eample, the Viterbi algorithm or the probability grid filter (see, e.g., [3]). On the other hand, the tools available for solving linear continuous-state problems (e.g., the celebrated Kalman filter) are well known for their simplicity. With these advantages of continuous-state algorithms over discrete-state ones in mind, we are interested in looking at continuous-state approimations to discrete-state problems. This line of research is aimed at identifying the appropriate geometric structures in discrete-state models which are amenable to linear continuous-state approimations. As part of this program of characterization of the class of discrete-state systems suitable for continuous-state approimation, we are studying the inverse process, Receivedby the editors November, 997; acceptedfor publication (in revisedform) by A. C. M. Ran February 7, 999; publishedelectronically March,. This work was supportedby the Centre for IntegratedDynamics andcontrol, The University of Newcastle, Australia. Part of this work has been published, in a preliminary form, by the same authors in Consistent state-discretization of linear systems, in Proceedings of the 36th IEEE Conference on Decision and Control, San Diego, CA, 997, pp Department of Electrical andcomputer Engineering, The University of Newcastle, Newcastle, NSW 38, Australia (eejose@ee.newcastle.edu.au, eegcg@cc.newcastle.edu.au, rick@ee.newcastle. edu.au). Department of Mathematics, The University of Newcastle, Newcastle, NSW 38, Australia (iain@maths.newcastle.edu.au).

4 STATE-DISCRETIZATION OF LINEAR STOCHASTIC SYSTEMS 3 namely, the properties of discrete-state systems that arise in the state-discretization of continuous-state linear systems. In particular, this paper eamines the asymptotic eigenstructure of the discrete-state systems (Markov chains) generated by the state-discretization of linear continuous-state systems. Related work can be found in [4, 6, ] in a general setting. However, since we are restricting our study to linear continuous-state systems, we are able to provide more specific information about the structure of the discrete-state approimation. This paper is organized as follows. In section we present some properties of state space formulations of linear systems that are relevant for their state-discretization. An operator which provides the time evolution of continuous-state probability density functions, together with its eigenvalues and eigenfunctions, is discussed in that section. In section 3 a state-discretization approach for linear systems is presented, and properties of the resulting discrete-state Markov chains are discussed. In particular, it is proved that the eigenvalues of the transition matrices obtained converge to a subset of the eigenvalues of the original continuous-state operator. Section 4 is dedicated to an eample in order to illustrate the material presented.. Discrete-time linear state space models. Let a linear discrete-time system be described by the state space equation () k+ = A k + Bv k, where k, k+ R n are the states at discrete-times k and k+, respectively, v k R m is an independent and identically distributed (iid) sequence of random vectors with finite variance, and the initial state is assumed to have an arbitrary distribution with finite variance and independent of v k for all k. A and B are real matrices of appropriate dimension, and we assume A such that all its eigenvalues belong to the interior of the unit disk. For simplicity of presentation we will further assume that the matri A is invertible (see, e.g., ()). This condition is naturally satisfied in the contet of time-discretization of a continuous time system (see, e.g., []). We will denote the joint probability density function (PDF) of a vector of n random variables y, evaluated at the n-tuple (,,..., n ), by f y () f y (,,..., n ), where denotes the vector =(,..., n ) T R n. We assume here that all the PDFs of interest have compact support and belong to the Hilbert space L ([a,b ] [a n,b n ]) (e.g., the uniform distribution). This assumption is necessary in the proofs below, and from a computational point of view, in most interesting applications densities do not significantly differ from zero outside a suitable compact set. The PDF of k+ can then be epressed in terms of those of k and v k (for eample, by using the auiliary variables method; see, e.g., [8]) as follows: () f k+ () = det(a) (Tf k )(), + + f k ( A ( Bξ) ) f v (ξ)dξ...dξ m where R n, ξ R m and () defines a conve operator on the conve space of PDFs. Since we are interested in spectral properties, we work with the etension of this operator to the linear space L ([a,b ] [a n,b n ]). Note that this etension is the linear operator T on L ([a,b ] [a n,b n ]) defined by (). In () we omitted k in f vk since v k is iid.

5 4 DE DONÁ, GOODWIN, MIDDLETON, AND RAEBURN In what follows we use the n-dimensional characteristic function (which eists for all distributions; see, e.g., [9]) and denote (3) (4) Φ k (iw) =F{f k ()} = Eep{ iw T k }, Φ v (iw) =F{f v ()} = Eep{ iw T v}, where F{ } stands for the Fourier transform and i. In epression (3) w R n, and in (4) w R m. In the Fourier transform domain () becomes (5) Φ k+ (iw) =Φ k (A T iw)φ v (B T iw) ( T Φ k )(iw), where w =(w,...,w n ) T R n and T is the linear operator on Fourier transforms (which corresponds to the linear operator T ). Since v k R m is iid and the eigenvalues of matri A are less than unity in modulus, there eists a stationary PDF, denoted f, such that satisfies f = Tf in () (see, for eample, Theorems.3 and.7 in [7] for a proof of the more general case where the coefficients in the autoregressive model are random). We let Φ denote the Fourier transform of the stationary distribution, satisfying (6) Φ (iw) =( T Φ )(iw) =Φ (A T iw)φ v (B T iw). For simplicity, the following theorem is formulated in the Fourier domain, although an equivalent result can be obtained in the state domain (see Remark.). Theorem.. The comple numbers λ T and the functions V T given by (7) (8) λ T =(λ ) l (λ ) l...(λ r ) lr, V T (iw) =(v T iw) l (v T iw) l...(vr T iw) lr Φ (iw) are eigenvalues and eigenfunctions of the operator T, where λ,...,λ r are the eigenvalues corresponding to each block of the Jordan canonical form of matri A, r n is the number of blocks, v,...,v r are the corresponding eigenvectors, and l,...,lr are arbitrary nonnegative integers. Proof. Substituting epression (8) into (5), we get ( TV T )(iw) V T (A T iw)φ v (B T iw) =(v T A T iw) l (v T A T iw) l...(vr T A T iw) lr Φ (A T iw)φ v (B T iw). Now, by using (6) and the fact that λ i and v i are eigenvalues and eigenvectors of A, wehave ( TV T )(iw) =(λ ) l (λ ) l...(λ r ) lr (v T iw) l (v T iw) l...(v T r iw) lr Φ (iw), which, from (7) and (8), can be written as ( TV T )(iw) =λ T V T (iw). Remark.. The eigenvalues of T given by (7) are also eigenvalues of T (denoted λ T ). The eigenfunctions of T can be evaluated by noting that (iw j ) l Φ (iw) isthe Fourier transform of l f ()/ l j (see, e.g., [4]). Hence, from (8) it can be seen that the eigenfunctions of T corresponding to the eigenvalues λ T are linear combinations of high order partial derivatives of f () with respect to the components of = (,..., n ) T.

6 STATE-DISCRETIZATION OF LINEAR STOCHASTIC SYSTEMS 5 3. State-discretization of linear state space models. The linear operator T defined by () provides the time evolution of the continuous-state probability density functions. The state-discretization of the system () is performed by approimating this operator and the PDFs by discrete-state counterparts. We will restrict our study to a bounded region of the state space: an n-dimensional hypercube X =[a,b ] [a n,b n ] of volume V X. As eplained before, we assume that all the PDFs of interest have compact support inside this region and belong to L ([a,b ] [a n,b n ]). To discretize the region X of the state space we use the idea of refinements [6, ], i.e., we divide X into N n subsets of size V X /N n denoted e, e,...,e N n, and define a Markov chain on the resulting discrete state space. The probability vectors of the Markov chain are given by (9a) where (9b) p k (P { k e },...,P{ k e N n}) T, P { k e j } = f k ()d...d n, e j and the conditional PDF of k+ is computed according to () as () f k+ ( k e j )= det(a) f k ( A ( Bξ) k e j ) fv (ξ)dξ...dξ m. In (), f k ( k e j ) is the conditional PDF of the random variable k evaluated at ( ), given the condition that k belongs to the set e j and is defined as (see, e.g., [8]) f k (y) f k ()d...d n if y e j, e () f k (y k e j )= j otherwise. The transition matri between time k and k +,Q k = {q (k) ij }, i, j N n, is then defined as () q (k) ij P { k+ e i k e j } = e i f k+ ( k e j )d...d n. Notice that we have made eplicit the dependence of Q k on k. If this is not the case, we say that the Markov chain has stationary or homogeneous transition probabilities, and we denote by Q the transition matri. Notice also that the matri Q k is defined by () and (), and these depend on the PDF f k (see ()). In order to simplify the computation of the matri Q k, we approimate the PDFs f k used in () by functions that assume piecewise-constant values in each subset e i (see the discussion preceding (4)). This approach has the advantage that the transition matrices are stationary (see the independence from f k in the elements q ij of Q in (4)).

7 6 DE DONÁ, GOODWIN, MIDDLETON, AND RAEBURN We denote by λ T the eigenvalues of the operator T and by λ Q (N) the eigenvalues of the transition matri Q obtained with a partition of N n cells. Notice that the spectrum of Q, σ(q), has N n points. We are interested in the accumulation (or limit) points of these sets as N. The following theorem provides a connection between these accumulation points and the eigenvalues of the operator T. Theorem 3.. The set of nonzero limit points, as N, of eigenvalues λ Q (N) is a subset of the set of eigenvalues λ T of the operator T. Proof. To maintain the notation as simple as possible, we will prove the theorem for the case of scalar systems (n = ). The proof for the multidimensional case is similar, but with far more complicated notation. We consider a scalar linear system described by (3) k+ = a k + bv k, where a < and b. As eplained before, it is assumed that all distributions have compact support. In particular we suppose that supp f v [ B,B] is fied. In the scalar case, the operator T defined by () can be written as (Tf k )() = + k(, y)f k (y)dy, where the kernel is given by k(, y) =f v ( ay b )/ b and satisfies + k(, y)d =. Here we choose A b B/( a ) to ensure that the support of Tf k lies in [, A] whenever supp f k [, A] (and, in a practical situation, we also choose A large enough to ensure that the support of the initial state distribution satisfies supp f [, A]). Then T becomes the integral operator on L ([, A]): (4) (Tf)() = A k(, y)f(y)dy with k L ([, A] [, A]). To perform the state-discretization of this system we divide the interval X = [, A] inton = N subsets of length A N, denoted e,...,e N, and we define the operator P N on L ([, A]): (5) N P N I N,j, j= where the operators I N,j are defined as follows: (6) (I N,j f)() ={ N A e j f(z)dz if e j, otherwise for f L ([, A]). Notice that the operator P N is a projection onto the N-dimensional subspace of L ([, A]) consisting of functions with N piecewise constant values, and that P N f = E[f(U) G N ], where G N is the sigma-algebra generated by the e i, U is the identity mapping, and the probability space (Ω, F,P) is such that Ω = R and P is the uniform probability measure on [, A].

8 STATE-DISCRETIZATION OF LINEAR STOCHASTIC SYSTEMS 7 Direct computation shows that the image of the operator P N TP N is constant over each cell e i, taking the value [( (P N TP N f)() = N N )( )] N (7) k(, y)dyd f(z)dz A e j A e j j= e i for e i. We need to prove now that the operator P N TP N converges, as N,toT in the natural norm of operators on L ([, A]). For this purpose we use k i,j to denote the scaled integral of k(, y) over the cell e i e j : ( ) N (8) k i,j = k(, y)dyd, A e i e j and we define a piecewise-constant function on [, A] [, A] by (9) N N n k N(, y) = k i,j X ei ()X ej (y), i= j= where X ei ( ) is the indicator function X ei () =if e i and otherwise. From (7), (8), and (9) it can be seen that the operator P N TP N is given by an integral operator with kernel n k N (, ), i.e., () (P N TP N f)() = A n k N(, y)f(y)dy. Then, by using Hölder s inequality, it can be shown from (4) and () that () T P N TP N ( A A k(, y) n k N(, y) dyd ). If the function k is continuous, it is uniformly continuous in the compact interval [, A] [, A]. It follows that the functions n k N converge uniformly to k. Thus, in the case of continuous k, it follows from the inequality A that the right-hand side of () converges to zero as the number of divisions N tends to infinity. It is a standard result from measure theory that continuous functions on [, A] [, A] constitute a dense subspace of L ([, A] [, A]) (see, e.g., []). Hence, for an arbitrary k L ([, A] [, A]), we can approimate k in the -norm by a continuous function g (i.e., k g ). For any N, the corresponding discrete approimations n k N and ng N, given by (8) and (9), satisfy n k N n g N = A A = i,j X ei ()X ej (y) k i,j g i,j dyd i,j ( ) A ( ) N A N A A (k(, y) g(, y))x ei ()X ej (y)dyd i,j ( ) N (k g)x ei A X ej.

9 8 DE DONÁ, GOODWIN, MIDDLETON, AND RAEBURN Now, writing the identity (k g)x ei X ej [(k g)x ei X ej ][X ei X ej ] and applying Hölder s inequality gives n k N n g N ( ) N (k g)x A X ei ej X ei X ej i,j () = i,j (k g)x ei X ej = k g (or we could observe that k n k N is the orthogonal projection on the span of {X ei ()X ej (y)}, and hence is norm-decreasing). Since we have already seen that n g N g for the continuous function g, and since from () we have (3) n k N k n k N n g N + n g N g + g k k g + n g N g, we deduce that n k N k. Therefore, we conclude from () that the operator P N TP N converges to T in norm. The transition matri Q of the resulting Markov chain is defined by () and (). As eplained before, we simplify the computation by taking a piecewise-constant approimation of f k (i.e., of the form f k = P N f k ). Then, the elements of the matri Q are given by q ij = N A e i e j k(, y)dyd. Therefore, the ith element of the product of the matri Q and a vector p = {p j }; j N, is given by [( (Qp) i = N N ) ] (4) k(, y)dyd p j. A e j j= e i From (7) and (4) it can be seen that the matri Q (acting on vectors) and the operator P N TP N (acting on piecewise-constant functions) are conjugate. To see this, consider a function f = i v ix ei and a vector v = {v i }; i N. Then, by noticing that N A e j f(z)dz = v j, it can be seen from (7) and (4) that P N TP N f = i (Qv) ix ei. It follows immediately that the eigenvalues of the operator P N TP N and those of the matri Q are the same, and that the eigenfunctions of P N TP N are given by the piecewise-constant held versions f vq = i v Q i X ei of the eigenvectors v Q = {v Qi } N i= of Q. It remains to prove that the set of nonzero limit points of the eigenvalues of the operator P N TP N is a subset of the set of eigenvalues of the operator T. For this purpose, let s suppose that λ is a limit point of a sequence of eigenvalues λ N of P N TP N ; then we have to show that λ is an eigenvalue of T. As we proved before, the operator P N TP N converges to T in the natural norm of operators on L ([, A]). Hence, (λ N I P N TP N ) converges to (λi T ) in norm. Since the set of invertible operators is open [, Theorem.], if (λ N I P N TP N ) is not invertible (i.e., λ N belongs to the spectrum of P N TP N ) for all N then (λi T ) is not invertible (i.e., λ belongs to the spectrum of T ). Since the operator T is compact [3], and for a compact linear operator every spectral value, with the possible eclusion of zero, is an eigenvalue [5, Theorem 8.4 4], the theorem is proved. 4. Eample. In order to illustrate the convergence of eigenvalues and eigenvectors we consider the following scalar state space model: (5) k+ =.5 k + v k,

10 STATE-DISCRETIZATION OF LINEAR STOCHASTIC SYSTEMS 9.3 First eigenfunction.5 3 Second eigenfunction Third eigenfunction Fourth eigenfunction Fig.. Eigenfunctions of the operator T corresponding to the four eigenvalues of larger magnitude (,.5,.5, and.5, respectively). where v k Uniform[, ] is an iid sequence, and Uniform[, ]. Notice that, by (7), (8), and Remark., the eigenvalues of the operator T defined by () for the scalar case are the nonnegative integer powers of λ =.5, and the eigenfunctions of T are the stationary PDF f and its derivatives. In Figure the stationary PDF f and its three first derivatives are shown. The eigenfunctions are ordered according to the magnitude of the corresponding eigenvalue (i.e., the first eigenfunction corresponds to the larger eigenvalue, etc.). It is easy to see, for the scalar system (3) and supp f v [ B,B], that f has support equal to [ b B/( a ), b B/( a ) ]. For this eample, supp f =[ 4, 4]. Since all distributions in this eample have compact support equal or contained in the interval [, ] of the real line, the region in the state space to be discretized is chosen as the interval X =[, ]. This interval is divided into N subsets of size /N and a Markov chain is defined on the discrete state space {e,e,...,e N }. The transition probability matri Q is computed using () and (). In Figure the eight larger eigenvalues of the matrices Q corresponding to different values of N (denoted λ Q (N)) are shown on a logarithmic scale. Notice that they converge to the eigenvalues corresponding to the continuous state operator T : (6) log(λ Q (N)) log(.5) log(.5l ) log(.5) l ; for l =,,...,7 as N. Notice also that the number of points N needed for convergence of the successive eigenvalues is eponentially increasing. The reason for this can be found in Figure. The successive eigenfunctions have peaks, the number of which increases proportionally to the powers of two. Therefore, in order to have a faithful representation of them with the eigenvectors of the Q matrices, the number of turning points of the eigenvectors increases by powers of two. In general, for a particular N,

11 DE DONÁ, GOODWIN, MIDDLETON, AND RAEBURN N Fig.. Eight larger eigenvalues of matrices Q on logarithmic scale (log(λ Q (N))/log(.5)) as a function of the number of subintervals N. First eigenvector Second eigenvector 3.. N N Third eigenvector Fourth eigenvector 5 7 N N Fig. 3. Eigenfunctions of P N TP N corresponding to the four eigenvalues of larger magnitude for N =,,, and 3 subintervals. unless an eigenfunction of P N TP N approimates one of T, we cannot epect the corresponding eigenvalues to be close approimations. In Figure 3 the eigenfunctions corresponding to the four eigenvalues of larger modulus of the operators P N TP N for N =,,, and 3 are shown to illustrate their convergence to the eigenfunctions of T depicted in Figure. The eigenfunctions are ordered, as in Figure, according to the magnitude of the corresponding eigenvalue.

12 STATE-DISCRETIZATION OF LINEAR STOCHASTIC SYSTEMS 5. Conclusions. Initial results in the characterization of the class of discretestate systems amenable to a linear continuous-state approimation have been presented. In particular, we have eplored in this paper the properties of discrete-state systems that arise in the inverse process, namely, in the state-discretization of linear continuous-state systems. The method to perform the state-discretization presented consists of a finite rank discrete approimation of the transition integral operator corresponding to the linear continuous-state system. As a result of this statediscretization procedure a Markov chain is obtained. In this work we have shown that the eigenvalues of the resulting Markov chain transition matrices converge to those of the transition operator of the underlying continuous-state linear model. This convergence has been illustrated by a scalar eample. Acknowledgments. The authors wish to thank Professor Ale Novikov and Dr. Nino Kordzakhia from the Department of Statistics at the University of Newcastle for valuable discussions. REFERENCES [] K. Åström, Introduction to Stochastic Control Theory, Academic Press, New York, 97. [] D. R. Co and H. D. Miller, The Theory of Stochastic Processes, Chapman andhall, London, 965. [3] I. Gohberg and S. Goldberg, Basic Operator Theory, Birkhäuser, Boston, 98. [4] P. E. Kloeden and E. Platen, Numerical Solution of Stochastic Differential Equations, Springer-Verlag, Berlin, New York, Heidelberg, 99. [5] E. Kreyszig, Introductory Functional Analysis with Applications, John Wiley & Sons, New York, London, Sydney, 978. [6] H. J. Kushner and P. G. Dupuis, Numerical Methods for Stochastic Control Problems in Continuous Time, Springer-Verlag, Berlin, New York, Heidelberg, 99. [7] D. F. Nicholls and B. G. Quinn, Random Coefficient Autoregressive Models: An Introduction, Springer-Verlag, Berlin, New York, Heidelberg, 98. [8] A. Papoulis, Probability, Random Variables and Stochastic Processes, McGraw Hill, New York, Toronto, London, 965. [9] M. B. Priestley, Spectral Analysis and Time Series, Academic Press, New York, 98. [] J. S. Rosenthal, Markov chain convergence: From finite to infinite, Stochastic Process. Appl., 6 (996), pp [] W. Rudin, Real and Comple Analysis, McGraw Hill, London, 97. [] W. Rudin, Functional Analysis, McGraw Hill, New York, Toronto, London, 973. [3] P. Terwiesch and M. Agarwal, A discretized nonlinear state estimator for batch processes, Comput. Chem. Engrg., 9 (995), pp [4] K. B. Wolf, Integral Transforms in Science and Engineering, Plenum Press, New York, 979.

13 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.