Covariance Calculation for Floating Point. State Space Realizations

Transcription

1 Covariance Calculation for Floating Point 1 State Space Realizations Sangho Ko, Robert R. Bitmead Abstract This paper provides a new method for analyzing floating point roundoff error for digital filters by using Finite Signal-to-Noise models whose noise sources have variances proportional to the variance or power of the corrupted signals. With this model, a new expression for output error covariance of floating point arithmetic is derived in case of double or extended precision accumulation. The output error covariance shows that the optimal state space realization for floating point is the same as that of that fixed point case except two in cases : when the filter has poles extremely close to the unit circle or when final quantization to precisions shorter than single precision is employed. An explicit formula is found for determining the minimum number of mantissa bits for stable realization. Index Terms Covariance, optimal state space realizations, roundoff noises, multiplicative white noises. I. INTRODUCTION From the 1970 s, many researchers have conducted studies to minimize the errors in digital signal processing computations caused by finite wordlength effects. The finite wordlength effect may be divided S. Ko and R. R. Bitmead is with the Department of Mechanical and Aerospace Engineering,University of California at San Diego, La Jolla, CA USA. (so, rbitmead)@ucsd.edu.

2 into two categories of coefficient error and roundoff error [1]. Here, only the effect of roundoff errors will be considered. 2 The roundoff errors due to fixed point arithmetic are modelled by additive white noise sequences independent of the signal and with fixed variance. Mullis and Roberts [1] and Hwang [2] independently developed results on the properties of the output errors of the digital filters and determined the optimal fixed point state space realization. Since the roundoff errors in floating point arithmetic are correlated with the signal that is quantized into a finite precision number, the errors cannot be modelled by the standard white noise and the expression and the analysis of the errors are more complex compared to that of fixed point arithmetic. With this inherent complexity, the optimal state space realization in floating point arithmetic is nown only for special cases. In the case of double precision accumulation, it may be shown [3] that the optimal state space realization is similar in nature to that of fixed point arithmetic, and for the case of extended precision accumulation (a few additional mantissa bits, but not double length) Bomar et al. [4] found that the floating point roundoff noise gain is identical in form to the fixed point gain. The previous wor to optimize fixed point realization is directly applicable to the floating point realizations. In both papers, the state error covariance equation in finite precision is expressed by a function of the state covariance in infinite precision, so their equations are not recursive and thereby the stability issues of the covariance equation could not be addressed. We apply tools of multiplicative noise processes to to extend and refine these earlier results. Selton [5] introduced a new noise model for linear systems, so called Finite Signal-to-Noise Model, or simply the FSN model. Since this model assumes that the variance of the additive noise corrupting a signal is proportional to the variance of the signal, it is well suited to analyzing floating point roundoff error. One important property of the systems with FSN model is that they can be destabilized in mean square due to noises [5], whereas the traditional additive white noise model cannot be destabilized by noise alone. This new model describes many physical systems more realistically than the traditional

3 additive white noise model. Recently de Oliveira and Selton [6] provided two LMI (Linear Matrix Inequalities) which are necessary and sufficient conditions for mean square state feedbac stabilization of linear systems with FSN model. The FSN model reduces to the same mathematical problems as for multiplicative noises. Existence conditions of state feedbac stabilizability for linear systems with state and control dependent noise in continuous time were derived [7] and a parametrization method was suggested for calculating exact stability bounds for systems with multiplicative noise [8]. 3 The problem of floating point arithmetic is analyzed in a different way using FSN models in the case of extended or double precision accumulation. The expression of the output error covariance is derived by solving a recursive state covariance equation and so stability problems are considered and an expression for the upper bound of roundoff noise for stable filter realization is derived, which can be used for determining the minimum number of final mantissa bits for stability. The new expression for output error covariance compensates for the approximation which was made in the previous studies and it is reconfirmed that the optimal state space realization of floating point arithmetic is the same as that of fixed point except in some special cases. In this paper, matrices will be denoted by upper case boldface (e.g., A) orcalligraphic uppercase (e.g., A), column matrices (vectors) will be denoted by lower case boldface (e.g.,x), and scalars will be denoted by lower case (e.g., y) orupper case (e.g., Y ). For a matrix A, A T denotes its transpose. For asymmetric matrices P > 0 or P 0 denotes the fact that P is positive definite or positive semi-definite, respectively. II. FLOATING POINT ARITHMETIC A. Floating Point Arithmetic Noise The floating point binary representation of a number is given by x = x m 2 xe (1) where x m is a fractional part called the mantissa, and x e is called the exponent [9]. Typically, the mantissa is normalized such that 0.5 < x m < 1. Since the number is represented by a multiplication

4 of the mantissa and the exponent, the resolution between two successive floating point numbers depends on the magnitude of these numbers, with the quantization error being proportional to this magnitude of both of these numbers. It is suggested that the noise due to floating point computation is largely due to the mantissa truncation and is therefore proportional to the signal amplitude. Hence, the quantization error in floating point arithmetic cannot be modelled by traditional additive white noise. 4 Floating point multiplication and addition roundoff errors can be described by [4] FL(x 1 x 2 )=x 1 x 2 (1 + ɛ), (2) FL(x 1 + x 2 )=(x 1 + x 2 )(1 + δ) where FL( ) denotes floating point quantization and ɛ and δ are white noises with zero mean value and the variances of, approximately σɛ 2 σ2 δ (0.18)2 2B (3) where B is the number of mantissa bits. Many floating point digital signal processors in use today are classified as single-precision devices, but internally perform register-to-register calculations with additional mantissa bits [4]. For example, the Texas Instruments TMS320C30/C40 family of processors, the Analog Devices ADSP21020 family, and the AT&T DSP32 family all use a 32-bit mantissa for register-to-register operations, while the Motorola DSP96002 uses a 31-bit mantissa. Only the final result of a sum of products calculation is quantized bac to the 24-bit-mantissa single-precision format. The mean and the variance of this final quantization are respectively zero and ση 2, (4) (0.167)2 2B where B represents final mantissa bit(=24). Therefore we can consider only the final roundoff error, since B>B and ση 2 σ2 ɛ,σ2 δ.

5 5 B. Effects of Floating Point Errors on Digital Filters Digital filters are represented by the state equations x +1 = Ax + Bu y = Cx + Du, (5) where u is the scalar input, y is the scalar output, and x is the n-length state vector. A, B, C, D are n n, n 1, 1 n, and 1 1 real constant matrices, respectively. Due to the floating point quantization, the actual filter is implemented as ˆx +1 = Èx + B û ŷ = C ˆx + D û, (6) where ˆx, ŷ, and û is the actual state, the actual output, and the actual input, respectively. And, the other matrices are defined as follows: à =[α ij ()], B =[β i ()], C =[ c j ()], D = D. (7) Here a ij (1 + ɛ ij ) n p=1 (1 + δ ip)(1 + η i ), j =1, 2 α ij () = a ij (1 + ɛ ij ) n p=j 1 (1 + δ ip)(1 + η i ), j =3,...,n β i () =b i (1 + ɛ i,n+1 )(1 + δ i,n )(1 + η i ), c j (1 + ɛ n+1,j ) n p=1 (1 + δ n+1,p)(1 + η n+1 ), j =1, 2 c j () = c j (1 + ɛ n+1,j ) n p=j 1 (1 + δ n+1,p)(1 + η n+1 ), j =3,...,n (8) D() =D(1 + ɛ n+1,n+1 )(1 + δ n+1,n )(1 + η n+1 ), where ɛ ij = ɛ ij (), δ ij = δ ij () are multiplication and addition errors and η i = η i () are final quantization errors, all of which are modelled as zero mean independent white noises.

6 6 If the products of very small error terms are neglected in (8), then (6) is reduced to (9). ˆx +1 = (A + A )ˆx +(B + B )û ŷ = (C + C )ˆx +(D + D )û, (9) where a 11 m 11 ()... a 1n m 1n () b 1 m 1,n+1 () A =....., B =. a n1 m n1 ()... a nn m nn () b n m n,n+1 () [ ] C = c 1 m n+1,1 ()... c n m n+1,n (), D = Dm n+1,n+1 (). (10) And, ɛ ij + n p=1 δ ip + η i, j =1, 2 m ij () = ɛ ij + n p=j 1 δ ip + η i, j =3,...,n m i,n+1 () =ɛ i,n+1 + δ i,n + η i ɛ n+1,j + n p=1 δ n+1,p + η n+1, j =1, 2 m n+1,j () = ɛ n+1,j + n p=j 1 δ n+1,p + η n+1, j =3,...,n (11) m n+1,n+1 () =ɛ n+1,n+1 + δ n+1,n + η n+1. As discussed in Section A, since the error of final quantization into 24-bit mantissa is much bigger than the other errors caused by intermediate register-to-register arithmetic, we can consider only the final

7 7 roundoff errors η i (). Then, the matrices in (10) can be reduced to a 11 η 1... a 1n η 1 a a 1n { A =..... = diag [η 1 η 2... η n ]. }.... = η i E i A a n1 η n... a nn η n a n1... a nn η 1 b 1 b { } B =. = diag [η 1 η 2... η n ] 1. = η i E i B η n b n b n ] C = [c 1 η n+1... c n η n+1 = η n+1 C (12) D = η n+1 D, where E i is the n n square elementary matrix that has 1 in the i i element and zero elsewhere. Therefore, we can express the actual state and output equation by ( ) ( ) ˆx +1 = I + η i E i Aˆx + I + η i E i Bû ŷ =(1+η n+1 ) Cˆx +(1+η n+1 ) Dû. (13) C. Calculation of Floating Point Output Error Covariance To find the expression of output error covariance we define the state error e and the output error y as in (15) and (17). e +1 x +1 ˆx +1 (14) = Ae η i E i Aˆx η i E i Bu (15) y y ŷ (16) = Ce η n+1 Cˆx η n+1 Du (17) Here it is assumed that û = u. This is often the case in practice when the input itself has been generated by a finite wordlength device, and hence is nown exactly.

8 8 When the input is a white noise with variance of σu 2, from (13) and (15) state and state error covariance equation will be given by X +1 = A X A T + σ 2 η Ē +1 = AĒA T + σ 2 η E i A X A T E i + σuσ 2 η 2 E i A X A T E i + σuσ 2 η 2 where X E{ˆx ˆx T } and Ē E{ê ê T }. E i BB T E i + σubb 2 T (18) E i BB T E i, (19) From (18) and (19), finiteness of both covariances depends only on X. That is, if X lim X exists, then Ē lim Ē also exists. The second and third terms of the right hand side of (18) are caused by the floating point roundoff error. The state covariance recursion can be destabilized by the second term when the variance of the floating point error is relatively greater than the stability margin of the matrix A. Hence, the floating point errors can cause stability problems for systems lie very narrow band filters where poles are very near to the unit circle. Since ση 2 1, the third term on the righthand side of X equation (18) can be neglected compared to the fourth term [ We note that because (18) is linear, the effect of this term is to increase the covariance X proportionally with the magnitude of this neglected term relative to the final term. It cannot affect stability of the recursion.]. Hence, (18) is reduced to X +1 = A X A T + ση 2 E i A X A T E i + σubb 2 T. (20) Equation (20) can be reexpressed by using Kronecer products [10] as [ ] x +1 = (A A)+ση 2 (E i E i )(A A) x + q = A x + q, where A [ (A A)+ση 2 n (E i E i )(A A) ], x Vec ( ) ( X, and q Vec σ 2 u BB T ). Then, mean square stability of the state covariance X is equivalent to ρ(a) < 1, where ρ( ) represents the spectral radius of a matrix. The following lemma gives equivalent conditions for mean square stability of (20). (21)

9 9 Lemma 1 Consider the system represented by (21) and suppose (A, B) is a controllable pair with A having all eigenvalues inside the unit circle. Denote A 1 I (A A) and A 2 σ 2 η n (E i E i )(A A). Then, ρ(a) < 1 if and only if ρ(a 2 A 1 1 ) < 1. Proof: Suppose ρ(a) < 1. Then, X = lim X is finite, nonnegative definite and satisfies the following algebraic equation: X = A XA T + σ 2 η E i A XA T E i + σu 2 BBT. (22) Further, if lim X is finite, then ρ(a) < 1. Wenext establish positivity and monotonicity property of X as a function of σ 2 η. Consider two values σ 2 η = σ2 1, σ2 η = σ2 2 with σ2 1 σ2 2 for which X is finite and converges to limits denoted by X σ1 and X σ2, respectively. Then, the difference X σ X σ1 X σ2 satisfies X σ = A X σ A T + σ1 2 E i A X σ A T E i + ( σ1 2 ) n σ2 2 E i A X σ2 A T E i. (23) The nonnegativity of the driving term of (23) and ρ(a) < 1 ensures that Xσ 0. Whence, This is the desired monotonicity result. σ 2 1 σ2 2 X σ1 X σ2. (24) When σ η =0, Xσ satisfies X σ = A X σ A T + σ 2 u BBT with A stable and (A, B) controllable. Thus X σ > 0 [11]. By monotonicity, we have X σ1 X σ2 > 0. (25) The monotonicity of X with respect to σ 2 η implies that either lim X is finite for all σ 2 η (which is demonstrably untrue [6]) or there exists a σsup 2 such that ση 2 <σsup 2 implies that lim X is finite, positive definite and ση 2 σsup 2 implies that { } X is unbounded. Thus, σ 2 η <σsup 2 implies ρ(a) < 1 and ση 2 σ2 sup implies ρ(a) 1. Bycontinuity of the eigenvalues of A with σ2 η,wehaveatσ2 η = σ2 sup, ρ(a) =1. Consider the following vectorized version of (22): [ ] x = (A A)+ση 2 (E i E i )(A A) x + q = A x + q, (26)

10 10 where x Vec ( X). We have that, if a solution to (26) exists, then is is given by x =(I A) 1 q.wenow further, from the monotonicity argument, that this x exists for any σ 2 η <σ2 sup but isdiscontinuous at σ2 η = σ2 sup.(we actually now that X, the solution to (22), has a different number of positive eigenvalues either side of σsup 2 and from (25), we also now that for ση 2 <σsup 2, X is bounded below.) This implies that the matrix (I A) is singular at this point and so λ(a) =1. Then, from the relation (I A)= ( I A 2 A 1 ) 1 A1, the matrix ( I A 2 A 1 ) 1 is singular with the invertible A1 and thus λ(a 2 A 1 1 )=1, which means ρ(a 2 A 1 1 ) = 1, since the matrix A 2A 1 1 is proportional to σ 2 η. Hence, ρ(a) < 1 if and only if ρ(a 2 A 1 1 ) < 1. Theorem 1 The following statements are equivalent: (i) The floating point state covariance represented by (20) is mean square stable. (ii) There exists P > 0 satisfying the LMI (iii) Proof: APA T P + σ 2 η E i APA T E i < 0. (27) [ { }] 1 ση 2 < ρ (E i E i )(A A)(I A A) 1 It is shown [12] that (i) is equivalent to (ii). By Lemma 1, ρ(a) < 1 is equivalent to ρ(a 2 A 1 1 ) < 1, from which (iii) can be obtained and hence (i) is equivalent (iii). (28) Together with (4), the upper bound for final roundoff error variance given by (28) can be used to determine the minimum number of final mantissa bits that is required for stable realization of filters which have poles very near to the unit circle. From (21) and if ρ(a 2 A 1 1 ) < 1, x = A 1 1 =0 ( A2 A 1 ) 1 q. (29)

11 11 Multiplying on both sides by A 1 = I (A A) yields ( A 1 x = A2 A 1 ) 1 q. (30) =0 Therefore, where r ση 2 x =(A A) x + q + j=1 r, (31) =1 ( Ei A j) ( E i A j) q. (32) Then, unvectorizing x yields { } (A X = A XA T + Q + R = A i T Q + R ) i =1 i=0 =1 ( = σu 2 K + ) (A A i T R ) (33) i, i=0 =1 where r Vec(R ) and R can be expressed as a recursive equation. From (32), r = σ 2 η j=1 j=1 ( Ei A j) ( E i A j) = ση 2 ( Ei A j) ( E i A j) Vec(R 1 ) = σ 2 η j=1 Vec [ E i A j R 1 A Tj ] E i. q (34) Therefore, with R 1 = σ 2 u σ2 η E i R = σ 2 η j=1 E i A j R 1 A Tj E i (35) A j BB T A T j E i = σu 2 σ2 η diag [ AKA T ] ii. (36) j=1

12 12 The steady state covariance of the output error is given by where Ē satisfies Y lim E { y 2 () } = CĒCT + σ 2 ηc XC T + σ 2 uσ 2 ηd 2, (37) Ē lim Ē = AĒAT + ση 2 E i A XA T E i + σuσ 2 η 2 E i BB T E i. (38) In (33), X consists of two terms and lets us see the effect of each term on Ē and Y separately. X(1), Ē (1), and Y (1) represent the effect of the first term and X (1), Ē (2), and Y (2) represent the effect of the second term. First, when only the first term of X is considered, that is X (1) = σu 2 K,wewill have from (38) Ē (1) = AĒ(1) A T + σuσ 2 η 2 ( E i AKA T + BB T ) E i. = AĒ(1) A T + σu 2 σ2 η E i KE i [ ] (A = σu 2 σ2 η A T E i KE ) i. =0 (39) Then, from (37) Y (1) σ 2 uσ 2 η { [ ] (A = C A T E i KE ) } i C T + CKC T + D 2 =0 { ( = tr A T ) [ ]} C T CA E i KE i + CKC T + D 2 =0 { [ ]} = tr W E i KE i + CKC T + D 2. (40) Therefore, { } Y (1) = σu 2 σ2 η CKC T + D 2 + K ii W ii (41) which is the same as the expression for output error covariance of Bomar et al. and Rao. Therefore, it can be said that if the second term of state covariance recursion in (20) is neglected, then the output error covariance will be the same as the previous studies.

13 13 Now, to see the effect of the second term of X, denote the effect of particular R on Y (2), X(2) and Ē(2) by Y (2), X(R ) (2) and Ē(R ) (2), respectively. Then, we have Y (2) = = =1 Y (2) CĒ(R ) (2) C T + σηc 2 X(R ) (2) C T. =1 (42) From (33), (35) and (38), it can be shown that { σηc 2 X (R ) (2) C T = σηtr 4 CA l E s A t ( R 1 A T ) t ( Es A T ) } l C T l=1 s=1 t=1 { = σηtr 4 ( A T ) } t diag[w]ii A t R 1 t=1 (43) and = ση 4 tr { W 1R 1 }, { CĒ (R ) (2) C T = ση 2 tr C A l E s A t ( R A T ) t ( Es A T ) } l C T l=0 s=1 t=1 { = ση 2 tr diag[w] ii A t ( R A T ) } t t=1 { = ση 4 tr ( A T ) t diag[w]ii A t E s A l ( R 1 A T ) } l Es t=1 s=1 l=1 = σ 4 ηtr { W 2 R 1 }, (44) where W ( A T ) l diag[w 1 ] ii (A) l with W 0 = W. (45) l=1

14 14 Hence, Y (2) = ση 4 tr { (W 1 + W 2 ) R 1 } { } = σηtr 6 A Tt diag[w 1 + W 2 ] ii A t R 2 t=1 = σ 6 η tr { (W 2 + W 3 ) R 2 }. (46) Therefore, we have Y σ 2 u σ2 η = Y (1) + =1 σ 2 u σ2 η Y (2) = CKC T + D 2 + tr { Wdiag [K] ii } + = σ 2 η tr { (W 1 + W ) R 1 } = σ 2 u σ2(+1) η tr { (W 1 + W ) diag [ AKA T ] ii}. =1 = CKC T + D 2 + tr { Wdiag[K] ii } + σ 2 η tr { Wdiag CKC T + D 2 + tr { Wdiag[K] ii } + where σ 2 η 1 and K AKAT are used. =1 { [ ] } ση 2 tr (W 1 + W ) diag AKA T ii [ ] } AKA T ii { [ ] } ση 2 tr W diag AKA T ii + ( 1+ση 2 ), =1 { [ ] } ση 2 tr W diag AKA T ii (47) Lemma 2 =1 ση 2 tr { W diag [ AKA T ] } ii (48) is a geometric series and the ratio of each two consecutive terms, r, isgiven by { [ tr diag l=1 Al diag [ AKA T ] ( ii A T ) ] } l W 1 ii r = tr { diag [ AKA T ] ii W }, (49) 1 where Proof: See Appendix. W 1 = ( A T ) l diag [W]ii A l. (50) l=1

15 15 Direct application of Lemma 2 yields Theorem 2. Theorem 2 For a given infinite precision digital filter represented by (5), the steady state covariance of output error, Y,isgiven by Y = Y (1) + Y (2) (51) Y (1) = σuσ 2 η 2 { CKC T + D 2 + tr (Wdiag [K] ii ) } (52) Y (2) = σuσ 2 η 2 ση 2 1 rση 2 tr { W 1 diag [ AKA T ] } ii, (53) for a white noise input signal of variance σu 2 when the filter (5) is implemented in digital signal processor utilizing accumulation for internal register-to-register, where W ii is the i ith element of the observability gramian W of matrix pair (A, C), K is the controllability gramian of matrix pair (A, B), ση 2 is the variance of the final quantization error given by (4) and r is given by (49). According to the two previous studies in [3] and [4], the output error covariance is given by { } Y previous = σu 2 σ2 η CKC T + D 2 + K ii W ii (54) which is the same as Y (1) in Theorem 2. For very stable A, Y (1), that is Y previous well approximates to Y, since for ση 2 σ 1, usually 2 η 1 rσ 2 η tr { W 1 diag [ AKA T ] ii} tr (Wdiag [K]ii ) and therefore Y (2) can be neglected in Y expression (51). But, for matrices A that have poles very near to the unit circle, it cannot be neglected. Since the term Y (2) comes from the second term in the right hand side of the state covariance equation given by (20), if that term is neglected then it yields the same expression for output covariance as the one given by the two previous studies, which is illustrated in the following example of frequency sampling filter which has poles on a circle with radius slightly less than one.

16 Example 1 Consider the first resonator of type 2 frequency sampling filter ([13], [14]) relative to linear phase FIR filters, with real valued impulse response coefficients, and with length N = 60. The 16 transfer function is given by H(z) = 2 H 0 sin (π/n) 1 2az 1 π/n + a 2 z 2 (55) with H 0 =1and a =1 2 L. This filter has two zeros at the origin and two poles near to the unit circle. The bigger L, the closer to the unit circle. To see the effect of the final quantization, consider two cases : one for L =10and the other L =42,anextreme case. Table I compares the output error covariance of direct form of state space realization and fixed point optimal realization for each L. In Table I, Bits -column represents the minimal required mantissa bits for stable realization calculated TABLE I COMPARISON OF OUTPUT ERROR COVARIANCE. Realization L=10 L=42 Form Y (1) Y (2) Bits Y (1) Y (2) Bits Direct Fixed point optimal from (4) and (28). Asshown, when L =10, Y (2) can be neglected compared to Y (1),but in case of the extreme value L =42, Y (2) is as big as Y (1).Table I shows also that the fixed point optimal realization still results in much smaller output error covariance than the direct form, even for the case of L =42. As shown in Example 1, when the poles are not extremely close to the unit circle, the output covariance due to the second term of state covariance (20) can be neglected and in this case the optimal realization is the same as the fixed point case. But, if a filter has poles extremely close to the unit circle, the additional term Y (2) can be as big as Y (1) and cannot be neglected. In addition to this case, the additional

17 17 term Y (2) can be also big when ση 2 is big, that is, when the final quantization to precision shorter than single precision (e.g., final quantization to short precision) is employed. So, in these cases, the optimal realization structure will be different from that of fixed point case, since the optimal realization structure should satisfy simultaneously the following conditions : Minimize tr (Wdiag [K] ii ), and Minimize tr { W 1 diag [ AKA T ] ii}, and finally Minimize r which is given by (49). Solving the above minimization problem to find the optimal structure will be very complex, but we can observe that the fixed point optimal realization which minimizes tr (W) with the constraint K ii =1for all i =1,...,n also reduces the output covariance, although it does not minimize it. This is confirmed also in Example 1. III. CONCLUDING REMARKS The floating point roundoff errors coming from implementing digital filters appear in the form of multiplicative noises, so the traditional method of using additive white noise cannot be applied to analyze the effect of the errors. In this paper, the floating point error effects on digital filters were analyzed by using the newly introduced FSN models which have noise sources whose variances are linearly proportional to the variances of the corrupted signal. With this new model, a new expression for the output error covariance of digital filters was derived when implemented in floating point digital signal processor using accumulation and it was reconfirmed that the optimal state space digital filter realization of floating point arithmetic will be the same as that of fixed point arithmetic.

18 18 REFERENCES [1] C. T. Mullis and R. A. Roberts, Synthesis of minimum roundoff noise fixed point digital filters, IEEE Trans. Circuits Syst., vol. CAS-23, pp , [2] S. Y. Hwang, Minimum uncorrelated unit noise in state-space digital filtering, IEEE Trans. Acoust.,Speech, Signal Processing, vol. ASSP-25, pp , [3] B. D. Rao, Floating point arithmetic and digital filters, IEEE Trans. Signal Processing, vol. 40, pp , [4] B. W. Bomar, L. M. Smith, and R. D.Joseph, Roundoff noise analysis of state space digital filters implemented on floating point digital signal processors, IEEE Trans. Circuits Syst., vol. 44, pp , [5] G. Shi and R. E. Selton, State feedbac covariance control for linear finite signal-to-noise ratio models, in Proc. CDC, New Orleans, LA, USA, 1995, pp [6] M. C. de Oliveira and R. E. Selton, State feedbac control of linear systems in the presence of devices with finite signal-to-noise ratio, Int. J. Contr., vol. 74, pp , [7] J. L. Willems and J. C. Willems, Feedbac stabilizability for stochastic systems with state and control dependent noise, Automatica, vol. 12, pp , [8] T. Sasagawa and J. L. Willems, Parametrization method for calculating exact stability bounds of stochastic linear systems with multiplicative noise, Automatica, vol. 32, pp , [9] M. Gevers and G. Li, Parametrizations in control, estimation and filtering problems. London: Springer-Verag, [10] J. W. Brewer, Kronecer products and matrix calculus in system theory, IEEE Trans. Circuits Syst., vol. CAS-25, pp , [11] R. E. Selton, T. Iwasai, and K. M. Grigoriadis, A unified algebraic approach to linear control design. London: Taylor and Francis, [12] S. Boyd, L. E. Ghaoui, E. Feron, and V. Balarishnan, Linear matrix inequalities in system and control theory. Philadelphia: SIAM, [13] L. R. Rabiner and R. W. Schafer, Recursive and nonrecursive realizations of digital filters designed by frequency sampling techniques, IEEE Trans. Audio and Electroacoustics, vol. AU-19, pp , [14] L. R. Rabiner and B. Gold, Theory and application of digital signal processing. New Jersey: Prentice-Hall, Inc., [15] R. A. Horn and C. R. Johnson, Matrix analysis. New Yor: Cambridge University Press, 1985.

19 19 APPENDIX Proof of Lemma 2 Consider the ratio of ( +1)-th term to th term of the series (48), which is denoted by r. r = tr { diag [ AKA T ] { [ ii W } tr diag +1 tr { diag [ AKA T ] l=1 Al diag [ AKA T ] ( ii A T ) ] } l W ii W } ii = tr { diag [ AKA T ] ii W } (56) The numerator and the denominator of r can be expressed by Vec-operator as follows: ( Numerator of r = Vec E s A l diag [ AKA T ] ( ii A T ) ) T l Es Vec(W ) = { s=1 s=1 l=1 l=1 ( E s A l E s A l) Vec ( diag [ AKA T ] ii) } T Vec(W ) = Vec(K) T (A A) T ( (E s E s ) A T A T ) l s=1 l=1 (E s E s ) Vec(W ) s=1 (57) Similarly, denominator of r = Vec(K) T (A A) T (E s E s ) Vec(W ). (58) Let M e n (E i E i ), M a l=1 (A A)l, A [2] (A A), v Vec(W ), and Vec(K). From (45), s=1 v = A [2]T M e v 1. (59) Then, and r r 1 = T A [2]T Me M T a M e v T A [2]T M e v numerator of = T A [2]T M e M T a M em T a M ev 1 T A [2]T M e M T a M ev 1 r T A [2]T M e v 1 T A [2]T M e M T a M ev 1 T A [2]T M e v 1 T A [2]T M e M T a M ev 1 = T A [2]T M e v 1 v 1 T r M em T a M e M T a M e A [2] 1 ) = tr (M a M e A [2] T A [2]T M e v 1 v 1 T M em a M e. (60) (61)

20 20 Since M e M T a M e = M e M a M e, M e v 1 v T 1 M em a M e = M e v 1 v T 1 M em e M a M e = M e M a M e M e v 1 v T 1 M e (62) = M e M T a M e M e v 1 v T 1 M e. Therefore, numerator of r ( ) = tr M a M e A [2] T A [2]T M e M T a r M em e v 1 v 1 T M e. (63) 1 Similarly, denominator of r = T A [2]T M e M T a r M ev 1 v 1 T M em a M e A [2] 1 ( ) = tr M a M e A [2] T A [2]T M e M T a M e v 1 v 1 T M e ( ) = tr M a M e A [2] T A [2]T M e M T a M e M e v 1 v 1 T M e. (64) From (63) and (64), r /r 1 =1. Hence, r is constant for all. Taing =1yields (49).