Intenational Confeence on Infomation echnology and Management Innovation (ICIMI 05) Gadient-based Neual Netwok fo Online Solution of Lyapunov Matix Equation with Li Activation unction Shiheng Wang, Shidong Dai and Ke Wang, a Depatment of Automotive Engineeing, Nanyang Vocational College of Agicultue, Nanyang 473000, P.R. China Depatment of Mathematics, College of Sciences, Shanghai 00444, P.R. China a kwang@shu.edu.cn coesponding autho Keywods: Gadient-based neual netwok, Laypunov matix equation, Activation function Abstact. A new type of activation function, named Li activation function, is used in gadient-based neual netwok (GNN) to solve Lyapunov matix equation. With this activation function, theoetical analysis shows that GNN can convege in finite time, while it can convege only in infinite time with two conventional activation functions linea and powe-sigmoid. Compute simulation esults confim that GNN with Li activation function can not only globally convege to the solution of the Lyapunov matix equation but also convege in finite time. GNN with the conventional two activation functions ae also simulated as a contast. Intoduction he Lyapunov (o Lyapunov-like) matix equations ae widely used in many diffeent engineeing and scientific computing aeas, such as linea algeba, contol theoy, bounday value poblem, signal pocessing and optimization []. In ecent yeas, due to the in-depth eseach in ecuent neual netwoks (RNN), a vaiety of computational methods based on neual solves have been poposed to solve matix equation. A quintessential example should be cited in [], whee by taking advantage of Lyapunov functional theoy to ensue the asymptotic stability of uncetain fuzzy ecuent neual netwoks with Makovian jumping paametes a novel linea matix inequality-based stability citeion was obtained. he Neual netwoks with distibuted and/o time-vaying delays have also been studied [3]. Subsequently, Zhang neual netwok (ZNN) was poposed to solve Sylveste matix equation and matix invesion with time-vaying coefficient matix [4, 5]. o constuct a neual netwok, the fist and foemost thing is to define a scala-valued nom-based enegy function. he minimum point (it is geneally a global minimum) of the enegy function coesponds to the solution of the oiginal poblem. Next one should minimize the enegy function. he most common method is to find the negative gadient diection. heefoe, in [6], the gadient-based neual netwok (GNN) model was poposed fo solving Lyapunov matix equation. And the authos of [7] impoved the GNN model by using diffeent activation functions. Howeve, the GNN model without activation function is equivalent to the impoved GNN model with linea activation function. hey also poved that the impoved GNN model with powe-sigmoid activation function has a supeio convegence. But the impoved GNN model with the suggested activation functions neve conveges to the accuate value in finite time. In this pape, the impoved GNN model is pesented with a new activation function suggested in [8], efeed to Li activation function, fo solving the Lyapunov matix equation. he global convegence and finite-time convegence ae poved in theoy. he uppe bound of the convegent time is also given. Compute simulation esults demonstate that, by using Li activation function, the GNN model can eally convege in finite time. As a compaison, GNN models with powe-sigmoid and linea activation function ae also simulated. 05. he authos - Published by Atlantis Pess 955
Gadient-based Neual Netwok (GNN) Model Conside the Lyapunov matix equation A X + XA = C, () whee A R n n is the coefficient matix, and C R n n is positive definite. Accoding to the taditional gadient-based algoithm, the fist and foemost is to define an enegy function ε(x) based on a nonnegative scala-valued nom: AX + XA + C ε ( X ) =, whee denotes the obenius matix nom, i.e., A =, and tace(a A) is the tace of A A. hus, it follows tace(( A X + XA + C) ( A X + XA + C)) ε ( X ) =. With the basic diffeential popeties of the tace of a poduct matix PZQ: tace( PZQ) tace( PZ Q) = P Q, = QP, Z Z whee P, Z and Q ae abitay matices with appopiate ode, the following can be obtained ε ( X ) = A( A X + XA + C) + ( A X + XA + C) A. X By evolving along the negative gadient of such an enegy function ε(x), the following classical GNN model is taken X & ( t) = Γ( A( A X + XA + C) + ( A X + XA + C) A ), whee Γ is a positive definite matix, and the time vaying matix X(t), stating fom an initial condition X 0 = X(0) R n n, is the activated state matix coesponding to the theoetical solution X (t) of (). Usually Γ is simply taken as γi with constant scala γ > 0 and I is identity matix. And γ should be set as lage as the hadwae pemit and is geneally used to scale the convegence ate [9]. In 005, Zhang et al combined fou kinds of activation functions with the ZNN model [5]. In [7], to solve the Lyapunov matix equation the autho added the fou kinds of activation functions to the classical GNN model, whee the autho called the new GNN model the impoved GNN model. Hee is the impoved GNN model, X & ( t) = Γ( A ( A X + XA + C) + ( A X + XA + C) A ), () whee ( ) is a function of matix, defined as follows: n n ( A) = f ( a ), A R, i, j =,, L, n. he following is the fou kinds of activation functions: ij (i) linea activation function f (x) = x; (ii) bipola sigmoid activation function f (x) = ( - exp(- ξx))/( + exp(- ξx)) with ξ ; (iii) powe activation function f (x) = x p with odd intege p 3; (iv) powe-sigmoid activation function p x, x, f ( u) = + exp( ξ ) exp( ξx), x <, exp( ξ ) + exp( ξx) with suitable design paametes ξ and p 3. In this pape, we will combine an activation function pesented in 03 [8] and efeed to Li activation function to the GNN model (). Both theoetical analysis and numeical simulation show that when using GNN model () with this type of activation function to solve the Lyapunov matix equation, the state matix X(t) can convege to the accuate solution in finite time. Hee is the Li activation function, 956
f ( x) = +, (3) whee x R, > 0 is a paamete. he function sig (x) is defined as follows x, if x > 0, = 0, if x = 0, (4) x, if x < 0. o the Li activation function defined in (3) and (4), it's easy to see that fo a positive constant ρ, the case = ρ is always same to the case =. hen it is only needed to conside the Li activation function ρ with 0 < o. It can be seen that the Li activation function is educed to the linea activation function when =, and fo x >> with inceasing, the Li activation function appoaches sig( x ). Global Convegence and inite-time Convegence It's easy to see that both sig (x) and sig ( x) ae monotonically inceasing odd functions. hen it can be said that the Li activation function f ( x) = + is a monotonically inceasing odd function, thus the following theoem holds. heoem 3. If heoem is satisfied. X is the unique solution of Lyapunov matix equation (). By using the GNN model () with Li activation function, the state matix X(t) staing fom any initial state X 0 always conveges to X. he following theoem shows that by using the GNN model () with Li activation function the exact solution to the Lyapunov matix equation can be obtained in finite time. heoem 4. If heoem is satisfied. X is the unique solution of Lyapunov matix equation (). By using the GNN model () with Li activation function with 0<<, the state matix X(t) could convege X X to the exact solution X 0 in finite time t <, whee X α (+ ) 0 is the initial state matix, α is the minimum eigenvalue of matix M γ (+ )( ) n = A A and n is the ode of matix A. Remak If >, similaly, an uppe bound of the convegence time can also be obtained. Howeve, by the definition of Li activation function, and 0 < < is the same. So it is only need to conside 0 < < o >. And in eithe case, the convegence time t has the same supemum which is smalle than the uppe bound obtained in heoem 4. Illustative Example o illustation and compaison, conside Lyapunov matix equation () with the following coefficients (which is the same as Example 4.9 in [9]) 0 0 A = 0 0. 3 aking C as an identity matix, if the outine X = lyap(a,c) is used, the theoetical solution can be obtained:.3. 0.5 X =. 4.6.3. 0.5.3 0.6 957
By the global convegence (heoem 3), the initial matix X 0 is andomly geneated within [-,] 3 3. As X is a symmetic matix, we only need to compute x (t), x (t), x 3 (t), x (t), x 3 (t), x 33 (t). With Li activation function and taking =3, γ=0, it can be seen that the neual netwok output X(t) eaches the exact solution X in a peiod of time (appoximately 6 seconds which should be close to the supenum). And the uppe bound is about.6 seconds by heoem 4. his shows the finite-time convegence. he finite-time convegence of GNN model () with Li activation function is compaed with the GNN model () with the linea and powe-sigmoid functions, whee =3, p=3, ξ=4, and in all the thee cases γ and X 0 ae all chosen as, γ=0, 0. 0. 0. X 0 = 0. 0. 0., 0. 0. 0. and the eo is defined as X ( t) X. It can seen that the GNN model () with Li activation function can convege to the exact solution in about 6 seconds compaed to the GNN model () with the linea and powe-sigmoid activation functions which can convege to the exact solution only in infinite time. In contast, the GNN model () with powe-sigmoid and linea activation functions still have a elative lage eo at t = 6. It is also seen that at the end of the simulation the GNN model () with powe-sigmoid and linea activation functions still can't etun the tue solution X. Compaisons of the GNN model with the Li activation function and =, = 3, = 4 and = 5 show that when > a faste convegence ate can be obtained with inceasing. Conclusion In this pape, a new activation function, named Li activation function, is combined with the GNN model fo solving Lyapunov matix equation. Compaed with taditional activation functions such as the linea and powe-sigmoid activation functions, the GNN model with Li activation function can convege to the exact solution of Lyapunov matix equation in finite time. Also, the global convegence and finite time convegence ae analyzed and poved. he uppe bound of the convegence time is also given. Numeical example illustates the global convegence and finite time convegence. Acknowledgments he thid autho was suppoted by National Natual Science oundation of China (30330) and the gant he ist-class Discipline of Univesities in Shanghai. Refeences []. J. Laffey, H. S migoc, Sufficient conditions on commutatos fo a pai of stable matices to have a common solution to the Lyapunov equation, SIAM J. Matix Anal. Appl. 3 (00) 07 08. [] Y. Liu, S. Z. Li, W.Wu, R. Huang, Dynamics of a mean-shift-like algoithm and its applications on clusteing, Infom. Pocess. Lett. 3 (03) 8 6. [3] R. Yang, Z. Zhang, P. Shi, Exponential stability on stochastic neual netwoks.with discete inteval and distibuted delays, IEEE ans. Neual,Netwoks (00) 69 75. [4] Y. Zhang, D. Jiang, J. Wang, A ecuent neual netwok fo solving Sylveste equation with time-vaying coefficients, IEEE ans. Neual Netwoks 3 (00) 053 063. [5] Y. Zhang, S. S. Ge, Design and analysis of a geneal ecuent neual net wok model fo time-vaying matix invesion, IEEE ans. Neual Netwoks 6 (005) 477 490. [6] D. Guo, C. Yi, Y. Zhang, Zhang neual netwok vesus gadient-based neual netwok fo time-vaying linea matix equation solving, Neuocomputing 74 (0) 3708 37. [7] C. Yi, Y. Chen, Z. Lu, Impoved gadient-based neual netwoks fo online solution of Lyapunov matix equation, Infom. Pocess. Lett. (0) 780 786. 958
[8] S. Li, S. Chen, B. Liu, Acceleating a ecuent neual netwok to finite-time convegence fo solving time-vaying Sylveste equation by using a sign-bi-powe activation function, Neual Pocess. Lett. 37 (03) 89 05. [9] C. Mead, M. Ismail, (Eds.), Analog VLSI Implementation of Neual Systems, Kluwe Academic, Boston, 989. 959