Bifurcation Analysis of Chaotic Systems using a Model Built on Artificial Neural Networks

Transcription

1 Bifurcation Analysis of Chaotic Systems using a Model Built on Artificial Neural Networs Archana R, A Unnirishnan, and R Gopiaumari Abstract-- he analysis of chaotic systems is done with the help of bifurcation diagrams and Lyapunov exponents he qualitative changes in dynamics of the system are evaluated with the help of bifurcation diagrams hey provide models of transitions and instabilities as some control parameter is varied Another ey component for the chaotic dynamics analysis is the Lyapunov exponent For attractors of maps or flows, the Lyapunov exponents also sharply discriminate between the different dynamics A positive Lyapunov exponent may be taen as the defining signature of chaos Neural Networ (RNN) trained with improved nonlinear Kalman filter is proposed for modeling the system in this paper As a case study the famous Lorenz and Rossler chaotic systems are analyzed with the proposed method he bifurcation diagrams are plotted and Lyapunov exponents are calculated KeyWords-- Bifurcation,Lyapunov exponents, Artificial Neural Networ, Extended Kalman Filter, chaotic systems C IINRODUCION HAOIC dynamic systems are an interesting field to many researchers and the available methods to analyse such systems are still in the developing field he change in the qualitative character of such a system can be carried out by bifurcation analysis and Lyapunov exponents Bifurcation is characterized by a growth rate of a perturbation he Lyapunov Exponents of a system are a set of invariant geometric measures that describe the dynamical content of the system Lyapunov Exponents quantify the average rate of convergence or divergence of nearby trajectories in a global sense A positive exponent implies divergence and a negative one implies convergence A system with positive Lyapunov Exponent is said to be chaotic [12] he modeling of chaotic systems, based on the output time series is also a challenging problem Fortunately, artificial neural networs have the required self-learning capability to tune the networ parameters (ie weights) to identify highly nonlinear and chaotic systems [10][11] In the present wor, two important chaotic systems are modeled using dynamic neural networs he proposed neural networ system models the chaotic time series effectively he state variables continue to generate the state space evolution of the system, responsible for generating the time series Archana R is with Federal Institute of Science &echnology Angamaly, Kerala, India archanasreenivasan@rediffmailcom A Unnirishnan is with Naval Physical & Oceanographic Laboratory, Cochin, Kerala, India unnirishnan_a@livecom R Gopiaumari is with Cochin University of Science & echnology, Cochin, Kerala, India, gopia@cusatacin he bifurcation diagrams are plotted and Lyapunov exponents are calculated he parameters of the neural networ are estimated using the Extended Kalman Filter (EKF) algorithm, by choosing the weights of the neural networ as the states of the Extended Kalman Filter Further, the Expectation Maximization algorithm is used to effectively arrive at the initial states and the state covariance, required in the EKF algorithm hereby the efficacy of modeling a chaotic system using dynamic neural networs has been demonstrated For the analysis standard nonlinear system represented in the following format is used x +1 = f ( x, W) (1) y =h(x,y(-1),y(-2), ν, ν -1, ν -2,W) (2) II BIFURCAION ANALYSIS A bifurcation diagram is a plot that shows the value of the changing parameter, on one axis and the solution to the system on the other axis In other words change in the qualitative character of a solution as a control parameter is varied is nown as a bifurcation his occurs where a linear stability analysis yields an instability which is characterized by a perturbation III LYAPUNOV EXPONENS he Lyapunov Exponents of a system are a set of invariant geometric measures that describe the dynamical content of the system In particular, they serve as a measure of how easy it is to perform prediction on the system under consideration Lyapunov Exponents quantify the average rate of convergence or divergence of nearby trajectories in a global sense A positive exponent implies divergence and a negative one implies convergence Consequently, a system with positive exponents has positive entropy in that trajectories that are initially close together move apart over time he more positive the exponent, the faster they move apart Similarly, for negative exponents, the trajectories move together A system with both positive and negative Lyapunov Exponent is said to be chaotic [9] Mathematically Lyapunov Exponent can be defined by n 1 1 f ( xi ) λi = lim ln n n i= 0 xi (3) where x i, is the i th state variable of the system and f(x i ) is the output of the system 198

2 IV RECURREN NEURAL NEWORK he recurrent networs have the potential to be used in unison in systems with dynamic elements and feedbac [2] In effect recurrent neural networs used for modeling or model based predictive control are multi-layer neural networs with a delay element in their feedbac loop Recurrent neural networs could be built with multi-layer networs in their feedbac loop, creating a system where the structures compute in tandem In recurrent networ nodes are connected bac to other nodes or themselves he Information flow is multidirectional Such networs inherently possess sense of time and memory of previous state(s)biological nervous systems show high levels of recurrency Hence the networs could be used in unison creating systems with both dynamic elements and feedbac he presence of feedbac loops has a profound impact on the learning capability of the networ and its performance Moreover the feedbac loops involve unit delay elements denoted by z -1, which results in a nonlinear dynamical behavior [2] Fig 1 Structure of RNN V MODELING WIH EXENDED KALMAN FILER A System representation Consider a discrete time non linear dynamic system, described by a vector difference equation with additive white Gaussian noise that models unpredictable disturbances he Kalman filter deals with linear systems We can see that Kalman filter needs modifications for adapting the nonlinear behavior of the system he dynamic plant equation is given by the following nonlinear equations x = f( x 1, u, w 1) (4) where x is an n dimensional state vector u is an m dimensional nown input vector, and w is a sequence of independent and identically distributed zero mean white Gaussian process noise with covariance E( ww ) he measurement equation is = Q (5) z hx (, v) = (6) Where v is the measurement noise with covariance E( vv ) = R (7) he functions f and h and the matrices Q and R are assumed to be nown B Extended Kalman Filter he Extended Kalman filtering process has been designed to estimate the state vector in a non linear stochastic difference model[5]o estimate a process with non-linear difference and measurement relationships, we begin by writing new governing equations that linearize equation (3) and equation (5) Expanding the functions f and h along the aylor series, one gets the following equations for the Extended Kalman filter [5] ~ ^ x = x + A( x x ) + Ww (8) ~ ~ z = z + H ( x x ) + Vv (9) Where vectors, x and ~ x and z are the actual state and measurement ~ z are the approximate state and measurement vectors x ^ is an aposteriori estimate of the state at step, w and v are the random variables and represent the process and measurement noise A is the Jacobian matrix of partial derivatives of f with respect to x, H is the Jacobian matrix of partial derivatives of x with respect to h, W is the Jacobian matrix of partial derivatives of with respect to w, V is the Jacobian matrix of partial derivatives of with respect to v CEKF time update equations: Project the state ahead ^ ^ x = f( x 1, u,0) (10) Project the error covariance ahead 1 1 P= AP A+ W Q W (11) D EKF measurement update equations: Compute the Kalman gainz 1 K = PH ( HPH + VRV ) (12) Update estimate with measurement ^ ^ ^ x = x+ K( z hx (,0) (13) Update error covariance P= ( I KH P ) (14) One of the basic problems in the implementation of the Kalman filter is the choice of the initial values of the state x and the state co-variance P Since arbitrary choices can lead to the divergence of the filter, the present paper has used the EM 199

3 algorithm [3] to compute the initial values of state and the state co variance F RNN raining Using EKF Algorithm In the modified Kalman algorithm the state and measurement equations are modified as follows: Considering the parameter optimization as a state estimation as described above allows us to use the extended Kalman filter to update the weight estimates as well as the optimal hidden states[6], [7] he augmented state vector is thus [w 1, w 2 w n, x 1, x 2, x p ] he algorithm is explained below: 1 All the weights and states are initialized to small random values he state covariant matrix P(0/-1) is initialized to a diagonal matrix, with relatively small values Let the covariant matrix for measurement noise is R and that of process noise is Q 2 As before compute the out put at each node of the recurrent networ 3 Find the Jacobian matrix with respect to the state of the process and output at the current estimate of internal state and weights of the RNN hese matrices are given by H and A defined as follows[3] as outlined in Sec are plotted he Fig 2 depicts that the model output exactly follows the input time series he state space evolution of Lorenz system and the generated model is plotted and are seen to be matching perfectly (Fig 3) With the model as given by Eqn 4 and 6 established correctly, the Lyapunov exponents Eqn 3 using are calculate, by taing the derivative of the function f he time evolution of the Lyaponov exponent is plotted in Fig 4 It can be seen that the change over of the Lyapunov exponent from negative to positive synchronizes with the period doubling bifurcation point he concept of bifurcation matching with the change in Lyapunov exponent is also demonstrated for the Rossler system also, given by Eqn 2426 Fig 5,6 and 7 confirms this fact H () h( ) h = w w (15) I0 A = f () (16) 0 x he output of the neural networ is computed using z = g (Σ y -n w n + ν l w l + w 1 x 1 + w 2 x 2 + w 3 x 3 ), where g is any chosen non linear function he networ wors with EKF algorithm as per the equations described in section IIB with the state vector x replaced by the weights of the RNN Fig 2 ime series of Lorenz system V SYSEM SIMULAION (a) Lorenz system Edward Lorenz formally awoe the scientific world to the idea of deterministic chaos through his 1963 paper, "Deterministic Nonperiodic Flow"he system is governed by the following nonlinear equations x = σ ( y x) (17) y = xz + γ x y (18) z = xy β z (19) In order to demonstrate the efficacy of the proposed method, a Lorentz system is simulated in Matlab with the constant values σ =3, γ=28 and β=8 he output time series corresponding to x output and the model generated from Neural Networ simulating the model, Fig 3 State space evolution of Lorenz system-butterfly 200

4 Fig 7 Bifurcation diagram,lyapunov exponent and modeling error of Rossler system Fig4Bifurcation diagram, Lyapunov exponent and modeling error of Lorenz system V (b) Rossler system Otto Rössler designed the Rössler attractor in 1976 he system is described by the following nonlinear equations x = y z y = x + ay z = bx cz + xz a = 02, b = 02, c = 57 Fig5 ime series of Rossler system Fig 6 State space evolution of Rossler system (20) (21) (22) In the table I below the results are listed It can be seen that the neural networ model could accurately calculate the Lyapunov exponents and bifurcation points Also the mean square error is estimated ABLE I System Positive Lyapunov exponent VI CONCLUSION It is demonstrated that an efficient model for chaotic systems can be built using neural networs rained with EKF- EM algorithm he model successfully reproduced the time series and strange attractors of Lorenz and Rossler system he proposed method results in vey low modelling error With the model available as a differentiable function, the Lyapunov exponents of the system are directly calculated A positive Lyapunov exponent shows the hidden chaotic nature of system generating the time series he proposed method has high ability of locating the occurrence of the period doubling bifurcation, which is a path way to the onset of chaos REFERENCES Bifurcation point Mean Square Error Lorenz x10 (-5) Rossler x10 (-5) [1] NK Sinha and B Kuszta, Modeling and Identification of Dynamic systems,van Nostrand Reinhold Company, New Yor,1983 [2] Simon Hayin, Neural Networs a comprehensive Foundation, Prentice Hall International Editions, 1999 [3] Fa-Long Luo and Rolf Unbehauen, Applied Neural Networs for Signal Processing, Cambridge University Press, 1997 [4] Yaaov Bar-Shalaom and Xiao-Rong Li, Estimation and racing: Principles, echniques and Software Artech House, Boston, London, 1993 [5] Steven H Strogatz Nonlinear dynamics and Chaos Perceus Boos,Massachusetts,1994 [6] Greg Welch and Garry Bishop An introduction to KalmanFilter cccsuncedu/~tracer/s2001/alman 201

5 [7] ASPoznya,Wen Yu and EN Sanchez, Identification and control of unnown chaotic systems via dynamic neural networs, IEEE rans on Circuits and Systems I: Fundamentalheoy and Applications, Vol 46, No 12 pp ,1999 [8] GV Pusorius and LA Feldamp, Neuro Control of nonlinear dynamical systems with Kalman Filter trained recurrent networs IEEE ransactions on neural networs, Vol 5, No2, pp , 1994 [9] KS Narendra and K Parthasarathy, Identification and control of Dynamical systems using neural networs, IEEE ransactions on Neural Networs, Vol 1, No2, pp 4-27, March 1990 [10] Geist K U Parlitz and W Lauterborn Comparison of different methods for computing Lyapunov exoonents Progress of theoretical Physics 83-5 pp [11] Zhiven Zhu and Henry Leung Identification of Linear Systems Driven by Chaotic Signals IEEE transactions on circuits and systems vol49no2,february 2002 [12] Ramon A Felix,Edgar N Sanchez and Guanrong Chen Reproducing Chaos by Variable structure Recurrent Neural Networs,IEEE transactions on neural networs,vol15,no6,november 2004 [13] Xuedi Wang; Wenwen Chen Bifurcation analysis and control of the Rossler system Proceedinggs of Seventh International Conference on Natural Computation (ICNC),Volume: 3, pp: ,2011 [14] Xuedi Wang; Wenwen Chen Bifurcation analysis and control of the Rossler system Proceedinggs of Seventh International Conference on Natural Computation (ICNC),Volume: 3, pp: ,