A DELAY-DEPENDENT APPROACH TO DESIGN STATE ESTIMATOR FOR DISCRETE STOCHASTIC RECURRENT NEURAL NETWORK WITH INTERVAL TIME-VARYING DELAYS

ICIC Express Letters ICIC International c 2009 ISSN 1881-80X Volume, Number (A), September 2009 pp. 5 70 A DELAY-DEPENDENT APPROACH TO DESIGN STATE ESTIMATOR FOR DISCRETE STOCHASTIC RECURRENT NEURAL NETWORK WITH INTERVAL TIME-VARYING DELAYS Chin-Wen Liao, Chien-Yu Lu, Kai-Yuan Zheng and Chien-Chung Ting Department of Industrial Education and Technology National Changhua University of Education No.1, Jin-De Road, Changhua 500, Taiwan lcy@cc.ncue.edu.tw Received March 2009; accepted May 2009 Abstract. This paper deals with the problem of state estimation for discrete stochastic recurrent neural network with interval time-delays. The activation functions are assumed to be globally Lipschitz continuous. Attention is focused on the design of a state estimator which ensures the global stability of the estimation error dynamics. A delay-dependent condition with dependence on the upper and lower bounds of the delays is given in terms of a linear matrix inequality (LMI) to solve the neuron state estimation problem. When this LMI is feasible, the expression of a desired state estimator is also presented. In addition, slack matrices are introduced to reduce the conservatism of the condition. A numerical example is provided to demonstrate the applicability of the proposed approach. Keywords: Recurrent neural network, Stochastic systems, Linear matrix inequality, State estimators, Interval time-delays 1. Introduction. In the past few decades, recurrent neural networks (RNNs) have been intensively studied. Many researchers found successful various applications in many fields such as pattern recognition, image processing, optimization problems, and associative memory. Many of these applications heavily depend on the dynamic behaviors. In practical condition, delayed systems are often encountered and time delay is frequently a source of instability and oscillations in the system. Therefore, dynamics in a neural network often have time delays due to lots of reasons, such as the finite signal propagation time in biological networks and the finite switching speed of amplifiers in electronic neural networks. The situation of time delay could make RNNs have bad performance, and even make system instable. So, many researchers focus on stability analysis for delayed RNNs. A lot of literatures of this issue have been reported in literature [1-11]. State estimation is a subject of great practical and theoretical important which has received much attention in recent years [12-19]. Since, the neuron states are not always fully available in the neural networks outputs in many practical applications. In this kind of cases, it is necessary to estimate the neuron states through measurements. Through available output measurements, many problems are about to estimate the neuron states in which the dynamic of the estimation error is asymptotically globally or exponentially stable. Recently, the state estimation problem for recurrent neural networks with time delays was studied in [12-1], where an effective linear matrix inequality (LMI) [20] approach was developed to solve the problem. The state estimation problem for recurrent neural networks with mixed time delays has been dealt with in [15-17], where sufficient conditions for the existence of estimator have been presented in terms of LMIs. A class of Markovian recurrent neural networks with mixed time delays was presented, where the neural networks have a finite number of modes, and the modes may jump from one to another according to a Markov chain in [18]. [19] presented the design problem of state 5

C.-W. LIAO, C.-Y. LU, K.-Y. ZHENG AND C.-C. TING estimator for a class of neural networks of neutral-type with interval time-varying delays, where a sufficient condition for existence of state estimator for the networks is given in terms of LMI. However, it should be pointed out the aforementioned results are continuous delayed RNNs. Recently, the problem of state estimation for discrete-time recurrent neural networks with interval time-varying delay was considered in [21], where a sufficient condition with dependence on lower and upper bounds of delay was proposed and an LMI approach was developed. So far, no state estimation results on discrete stochastic recurrent neural network with interval time varying delays are available in the literature, and remain essentially open. The objective of this paper is to address this unsolved problem. In this paper, the aim is to deal with the state estimation problem for discrete stochastic recurrent neural network with interval time-varying delays. The interval time-varying delay includes both lower and upper bounds of delays. A delay-dependent condition for the existence of estimators is proposed and the criterion is formulated in accordance with an LMI, which introduces into slack matrices and reduces the conservatism of the criterion. A general full order estimator is sought to guarantee that the resulting error system is globally asymptotically stable. Desired estimators can be obtained by the solution to certain LMIs, which can be solved numerically and efficiently by resorting to standard numerical algorithms [20]. Finally, an illustrative example is provided to demonstrate the effectiveness of the proposed method. 2. Problem Statement and Preliminaries. Consider the following discrete stochastic recurrent neural network with interval time-delays described by x(k + 1) = Ax(k) + W 1 f(x(k) + W 2 f(x(k τ(k)) + [C x(k) + C h x(k h(k))]ω(k), (1) where x(k) = (x 1 (k), x 2 (k),, x n (k)) T is the state vector, A = diag(a 1, a 2,, a n ) is real constant with entries a i < 1, i = 1, 2,, n, W1 n n and W2 n n are the interconnection matrices representing the weighting coefficients of the neurons, C and C h are known real constant matrices. f(x(t)) = [f 1 (x 1 (t)),, f n (x n (t))] T R n is the neuron activation function with f(0) = 0, τ(k) and h(k) are the time-varying delay of the system satisfying τ m τ(k) τ M, k N, (2) h m h(k) h M, k N, () where 0 < τ m < τ M and 0 < h m < h M are known integers. ω(k) is a scalar Wiener process (Brownian Motion) defined on a complete probability space (Ω, F, P ) which is assumed to satisfy E{ω(k)} = 0, E{ω 2 (k)} = δ, k = 0, 1, 2,, () where δ > 0 is a known scalar. In order to establish main results, it is necessary to build the following assumption which the activation function (1) is assumed to bounded and satisfy the following assumption. Assumption 2.1. The neuron activation functions in (1), g i ( ), satisfy the following Lipschitz condition. 0 g i(x) g i (y) α i, (i = 1, 2,, n), (5) x y where G i R n m are known constant matrices such that x, y R and x y. Assumption 2.2. The neuron activation functions in (1) are bounded. In order to observe the neuron states. The recurrent neural network measurements are assumed to satisfy y(k) = Dx(k) + [Ex(k) + E h x(k h(k))]ω(k), () where y(k) R m is the measurement output and D, E and E h are constant matrix with appropriate dimension.

ICIC EXPRESS LETTERS, VOL., NO., 2009 7 For (1) and (5) of the system, we consider the following full-order estimator ˆx(k + 1) = Aˆx(k) + W 1 f(ˆx(k) + W 2 f(ˆx(k τ(k)) + [C ˆx(k) + C hˆx(k h(k))]ω(k) +L[y(k) Dˆx(k) (Eˆx(k) + E hˆx(k h(k)))ω(k)], (7) where ˆx(k) is estimator of the neuron states and L R m n is the estimator gain matrix to be determined. In this article, we found a suitable L R m n such that ˆx(k) approaches x(k) asymptotically. Let e(k) = x(k) ˆx(k) (8) be the state estimator error. Then with (1), (5) and (), the state error e(k) satisfies the following equation: e(k + 1) = (A LD)e(k) + W 1 f(e(k)) + W 2 f(e(k τ(k))) +[(C LE)e(k) + (C h LE h )e(k h(k))]ω(k), (9) where f(e(k)) = f(x(k)) f(ˆx(k)), f(e(k τ(k))) = f(x(k τ(k))) f(ˆx(k τ(k))), e(k h(k)) = x(k h(k)) ˆx(k h(k)). It is obvious to find out from Assumption 2.1 that the solution of (1) exists for all k 0 and is unique.. Main Results. In this section, an LMI based condition will be established. The globally delay-dependent state estimation condition given in (9). We used the LMI approach to solve the estimator gain matrix if the system (9) is globally asymptotically stable. Now, we derive the conditions under which the neural network dynamics of (1) is globally stable. For mathematical formulation, we define Z 1 = ρ 1 P, Z 2 = ρ 2 P, Z = ρ P, Z = ρ P, Y = P L, (ρ 1, ρ 2, ρ, ρ, are given scalars). The following theorem reveals to solving the state estimation problem formulated involving several scalar parameters. Theorem.1. Under Assumption 2.1 and Assumption 2.2, given scalars 0 τ m < τ M, 0 h m < h M, the network output (), the error-state dynamics (9) and system (1) with interval time varying delays τ(k) and h(k) satisfying (2) and () is globally asymptotically stable. If there exist matrices P > 0 i > 0 (i = 1, 2, ), Z i > 0 (i = 1, 2,, ), and diagonal matrix R i > 0, R 2 > 0 and S i, H i, T i, Γ i, Φ i, Θ i (i = 1, 2,, ) of appropriate dimensions such that the following LMI holds Ω τ M S τ Mm H τ Mm T h M Φ h Mm Γ h Mm Θ Ā C τ M S T τ M Z 1 0 0 0 0 0 0 0 τ Mm H T 0 (τ Mm )(Z 1 + Z 2 ) 0 0 0 0 0 0 τ Mm T T 0 0 τ Mm Z 2 0 0 0 0 0 h M Φ T 0 0 0 h M Z 0 0 0 0 h Mm Γ T 0 0 0 0 h Mm (Z + Z ) 0 0 0 h Mm Θ T 0 0 0 0 0 h Mm Z 0 0 Ā T 0 0 0 0 0 0 P 0 C T 0 0 0 0 0 0 0 δ 1 P τ M Ā T 1 τ M CT 1 τ Mm Ā T 2 τ Mm CT 2 h M Ā T h M CT h Mm Ā T h Mm CT

8 C.-W. LIAO, C.-Y. LU, K.-Y. ZHENG AND C.-C. TING τ M Ā 1 τ M C1 τ Mm Ā 2 τ Mm C2 h M Ā h M C h Mm Ā h Mm C τ M Z 1 0 0 0 0 0 0 0 0 δ 1 τ M Z 1 0 0 0 0 0 0 0 0 τ Mm Z 2 0 0 0 0 0 0 0 0 δ 1 τ Mm Z 2 0 0 0 0 0 0 0 0 h M Z 0 0 0 0 0 0 0 0 δ 1 h M Z 0 0 0 0 0 0 0 0 h Mm Z 0 0 0 0 0 0 0 0 δ 1 h Mm Z < 0 where Ω = Ω(i, j), i = 1,, 9, j = 1,, 9, Ω 11 = (τ M τ m + 1)Q 1 + Q 2 + Q + Q + Q 5 + Q P + S 1 + S1 T + Φ 1 + Φ T 1, Ω 12 = S 1 + S2 T + H 1 + T 1, Ω 1 = S T + H 1, Ω 1 = S T T 1, Ω 15 = Γ 1 + Φ T 2 Φ 1 + Θ 1, Ω 1 = Γ 1 + Φ T, Ω 17 = Θ 1 + Φ T, Ω 18 = Σ T R1 T, Ω 19 = 0, Ω 22 = Q 1 S 2 S2 T +H 2 H2 T +T 2 T2 T, Ω 2 = H 2 +H T S T + T T, Ω 2 = H T + S T T T + T 2, Ω 25 = 0, Ω 2 = 0, Ω 27 = 0, Ω 28 = 0, Ω 29 = R2 T, Ω = H H T Q, Ω = H T T, Ω 5 = 0, Ω = 0, Ω 7 = 0, Ω 8 = 0, Ω 9 = 0, Ω = Q T T T, Ω 5 = 0, Ω = 0, Ω 7 = 0, Ω 8 = 0, Ω 9 = 0, Ω 55 = Q 2 Φ 2 Φ T 2 +Γ 2 +Γ T 2 +Θ 2 +Θ T 2, Ω 5 = Γ 2 +Γ Φ T +Θ T, Ω 57 = Φ T + Γ T + Θ T Θ 2, Ω 58 = 0, Ω 59 = 0, Ω = Q Γ Γ T, Ω 7 = Γ T Θ, Ω 8 = 0, Ω 9 = 0, Ω 77 = Q Θ Θ T, Ω 78 = 0, Ω 79 = 0, Ω 88 = R 1 R1 T, Ω 89 = 0, Ω 99 = (R 2 Σ 1 ) (R 2 Σ 1 ) T, Ā = [(A T P D T Y T ) 0 0 W1 T P W2 T P ] T, Ā 1 = [(ρ 1 (A I) T P ρ 1 D T Y T ) 0 0 ρ 1 W1 T P ρ 1 W2 T P ] T, Ā 2 = [(ρ 2 (A I) T P ρ 2 D T Y T ) 0 0 ρ 2 W1 T P 2 W2 T P ] T, Ā = [(ρ (A I) T P ρ D T Y T ) (10) 0 0 ρ W1 T P ρ W2 T P ] T, Ā = 0 0 ρ W T 1 P ρ W T 2 P ] T, C = [(C T P E T Y T [(ρ (A I) T P ρ D T Y T ) Ch T P ET h Y T ) 0 0] T, C 1 = [ρ 1 (C T P E T Y T ) C 2 = [ρ 2 (C T P E T Y T ) 0 0 ρ 2 (Ch T P ET h Y T ) 0 0 ρ (Ch T P ET h Y T ) 0 0] T, C = [ρ (C T P E T Y T ) 0 0 ρ 1 (Ch T P ET h Y T ) 0 0 0 0] T, 0 0] T, C = [ρ (C T P E T Y T ) 0 0 ρ (Ch T P ET h Y T ) 0 0] T, S = [S1 T S2 T S T S T 0 0 l 0 0 0] T, H = [ H1 T H2 T H T H T 0 0 0 0 0 ] T, T = [ T1 T T2 T T T T T 0 0 0 0 0 ] T, Γ = [ Γ T 1 0 0 0 Γ T 2 Γ T Γ T 0 0 ] T, Φ = [ Φ T 1 0 0 0 Φ T 2 Φ T Φ T 0 0 ] T, Θ = [ Θ T 1 0 0 0 Θ T 2 Θ T Θ T 0 0 ] T, in which τ Mn = τ M τ m, h Mn = h M h m. Then the system described by (1) is globally stochastically asymptotically robustly stable in the mean square. L = P 1 Y. In this case, a desired the estimator gain matrix L is given as

ICIC EXPRESS LETTERS, VOL., NO., 2009 9. Numerical Example. Consider the discrete time recurrent neural network (1) with parameters [ as follows ] [ ] [ ] [ ] 0. 0 0.1 0.5 0.25 0.2 1 0 A =, W 0 0.2 1 =, W 0. 0.1 2 =, C =, C 0.01 0.0 0 1 h = [ ] [ ] 0.1 0.2 0.52 0, Σ =, ρ 1 0 0 0.07 1 = ρ 2 = ρ = ρ = 0.1. Take the activation function as g(x) = 1/2( x + 1 + x 1 ). The stochastic process {ω(k)} satisfies () with δ = 0.. In this example, we assume the activation functions satisfy Assumption 2.1 with α 1 = 0.52, [ α 2 = ] 0.07. [ For the network ] output, [ the parameter ] D, E and E h 1 0 0.1 0 0.2 0.1 is given as. D =, E =, E 0 1 0 0.1 h =. Using the Matlab 0. 0.1 LMI Control Toolbox to solve the LMI (10) for all interval time-varying delays satisfying τ(k) = + sin(kπ/2)(i.e. the lower bound τ m = 2 and the upper bound τ M = ) and h(k) = 2 + sin(kπ/2) (i.e. the lower bound h m = 1 and the upper bound h M = ), the feasible [ solution is sought ] as [ ] [ ] 0.0009 0.000 292.92 2.020 1.719 8.9825 P = 0.000 0.001 1 = 2.020 51.80 2 =, 8.9825 25.72 [ ] [ ] [ ] 227.572 5.8995 227.5788 5.8977 227.521 5.9559 Q = 5.8995 2.575 = 5.8977 2.577 5 =, 5.9559 2.917 [ ] [ ] [ ] 227.58 5.9202 192.28 0 15.0292 0 Q =, R 5.9202 2.515 1 =, R 0 195.080 2 =, 0 158.79 [ ] 0.0022 0.002 Y =. Therefore, by Theorem.1 the state estimation problem is 0.002 0.00 solvable, and a desired estimator gain is given by L = P 1 Y as L = [ 0.72 0.752.5.907 5. Conclusions. In this study, we investigate the problem of state estimation for discrete stochastic recurrent neural network with interval time-delays. A sufficient condition for solvability of this problem, which takes into account the interval time-delays, has been derived. The exponential state estimator is designed to estimate the neuron states and the dynamics of estimation error is globally exponentially stable. Finally, a numerical example has been presented to demonstrate the effectiveness of the proposed approach. ]. REFERENCES [1] C. Y. Lu, T. J. Su, Y. H. Su and S. C. Huang, A delay-dependent approach to stability for static recurrent neural networks with mixed time-varying delays, Int. J. Innovative Computing, Information & Control, vol.52, pp.11-172, 2008. [2] P. Balasubramaniam and R. Rakkiyappan, Global asymptotic stability of stochastic recurrent neural networks with multiple discrete delays and unbounded distributed delays, Applied Mathematics and Computation, vol.20, pp.80-8, 2008. [] Q. Zhang, X. Wei and J. Xu, A generalized LMI-based approach to the global asymptotic stability of discrete-time delayed recurrent neural networks, Int. J. Innovative Computing, Information & Control, vol., pp.19-199, 2008. [] Y. Chen and W. Su, New robust stability of cellular neural networks with time-varying discrete and distributed delays, Int. J. Innovative Computing, Information & Control, vol., pp.159-155, 2007. [5] Y. Guo, New results on input-to-state convergence for recurrent neural networks with variable inputs, Nonlinear Analysis: Real World Applications, vol.9, pp.1558-15, 2008. [] Y. Lv, W. Lv and J. Sun, Convergence dynamics of stochastic reaction diffusion recurrent neural networks with continuously distributed delays, Nonlinear Analysis: Real World Applications, vol.9, pp.1590-10, 2008. [7] J. Yu, K. Zhang, S. Fei and T. Li, Simplified exponential stability analysis for recurrent neural networks with discrete and distributed time-varying delays, Applied Mathematics and Computation, vol.205, pp.5-7, 2008.

70 C.-W. LIAO, C.-Y. LU, K.-Y. ZHENG AND C.-C. TING [8] Q. Song, Exponential stability of recurrent neural networks with both time-varying delays and general activation functions via LMI approach, Neurocomputing, vol.71, pp.282-280, 2008. [9] R. Rakkiyappan and P. Balasubramaniam, Delay-dependent asymptotic stability for stochastic delayed recurrent neural networks with time varying delays, Applied Mathematics and Computation, vol.198, pp.52-5, 2008. [10] Q. Song, Novel criteria for global exponential periodicity and stability of recurrent neural networks with time-varying delays, Chaos, Solitons & Fractals, vol., pp.720-728, 2008. [11] L. Wang, Z. Zhang and Y. Wang, Stochastic exponential stability of the delayed reaction diffusion recurrent neural networks with Markovian jumping parameters, Physics Letters A, vol.72, pp.201-209, 2008. [12] Z. Wang, Ho, D. W. C. and X. Liu, State estimation for delayed neural networks, IEEE Trans. Neural Networks, vol.1, pp.279-28, 2005. [1] H. Huang, G. Feng and J. Cao, An LMI approach to delay-dependent state estimation for delayed neural networks, Neurocomputing, vol.71, pp.2857-287, 2008. [1] Y. He. Wang, M. Wu and C. Lin, Delay-dependent state estimation for delayed neural networks, IEEE Trans. Neural Networks, vol.17, pp.1077-1081, 200. [15] T. Li and S. Fei, Exponential state estimation for recurrent neural networks with distributed delays, Neurocomputing, vol.71, pp.28-8, 2007. [1] T. Li, S. Fei and Q. Zhu, Design of exponential state estimator for neural networks with distributed delays, Nonlinear Analysis: Real World Applications, vol.10, pp.1229-122, 2009. [17] Y. Liu, Z. Wang and X. Liu, Design of exponential state estimators for neural networks with mixed time delays, Physics Letters A, vol., pp.01-12, 2007. [18] Z. Wang, Y. Liu and X. Liu, State estimation for jumping recurrent neural networks with discrete and distributed delays, Neural Networks, vol.22, pp.1-8, 2009. [19] J. H. Park, O. M. Kwon and S. M. Lee, State estimation for neural networks of neutral-type with interval time-varying delays, Applied Mathematics and Computation, vol.20, pp.217-22, 2008. [20] S. Boyd, L. G. Ei, E. Feron and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory, Philadelphia, PA: SIAM, 199. [21] C. Y. Lu, A delay-range-dependent approach to design state estimator for discrete-time recurrent neural networks with interval time-varying delay, IEEE Trans Circuits and Systems II: Express Briefs, vol.55, pp.11-117. 2008.