Application Research of Fireworks Algorithm in Parameter Estimation for Chaotic System

Application Research of Fireworks Algorithm in Parameter Estimation for Chaotic System Hao Li 1,3, Ying Tan 2, Jun-Jie Xue 1 and Jie Zhu 1 1 Air Force Engineering University, Xi an, 710051, China 2 Department of Machine Intelligence, School of Electronics Engineering and Computer Science, Peking University, Beijing, 100871, China 3 Department of Intelligence, Air Force Early-Warning Academy, Wuhan, 430019, China Abstract. Chaotic system is a nonlinear deterministic system, and parameter identification for chaotic system is an important issue in nonlinear science, such as secure communication, et al. By set up appropriate objective function, the parameter identification can be converted into a multi-dimensional optimization problem which can be solved by evolutionary algorithms. Emerging as an evolutionary algorithm, Fireworks Algorithm () has shown its good computational performance and robustness. In order to expand the application of, several types of are applied to estimate the parameters for two typical chaotic system which three parameters are totally unknown, simulation results show most s can attain better estimation precision and robustness, is a new effective parameter identification method for chaotic systems. 1 Introduction Chaos embodies characteristics including complexity, internal randomness, initial value sensitivity, irregular order and so on [1]. In scientific research and engineering application, chaotic characteristics is presented in various systems and therefore chaotic system s control and synchronization have become an important research field and also widely applied in fields such as secure communication, medicine, biography and chemistry [2]. Many nonlinear system control methods like adaptive control [3], active control [4] and PI control [5] can be used in chaotic system s control and synchronization. However, if parameters of chaotic system are unknown, all of these methods above are not applicable anymore. In engineering practice, there exist some unknown parameters caused by chaotic system s complexity making some parameters hard to measure or communication security [6]. Consequently, parameter estimation of chaotic system becomes a key issue in chaotic control and synchronization. Essentially, parameter estimation of chaotic system is an optimization problem for multi-dimensional complex function. Intelligent optimization algorithms have widely applied in resolving such kind of problems. In reference [7], genetic algorithm (GA) was used for single parameter estimation of Lorenz system, achieved better results but slower convergence, Single parameter s estimation performance of ant colony algorithm for chaotic system has been studied in

2 Manuscript for The Sixth International Conference on Swarm Intelligence 2015 both noisy and non-noisy backgrounds in reference [8], but as its text spoken ant colony algorithm for chaotic parameter estimation also had slow convergence problem. Based on opposition-based learning and harmony search algorithm, a hybrid biogeography-based optimization (HBBO) is proposed in reference [9] and achieve better results of both Lorenz system and Rossler system, In addition, adaptive spatial contraction bee colony algorithm [10], chaotic invasive weed optimization [11], cuckoo adaptive search, simulated annealing hybrid algorithm [12], oppositional seeker optimization algorithm [13] and other algorithms are also used for parameter estimation of chaotic systems. However, according to the famous No Free Lunch Theorems that was proposed by Wolpert and Macready in 1997, there doesn t exist one method that can solve all kinds of problems. Therefore, different parameter estimation method should be developed aiming at specific problem [14]. The Fireworks Algorithm () is a newly developed evolutionary algorithm that was published by Tan and Zhu in 2010 (Tan, & Zhu, 2010) [15]. Like other evolution algorithms, it also aims to find the vector with the best (usually minimum) fitness in the search space. It is inspired by fireworks explosion at night and is quite effective at finding global optimal value. As a firework explodes, a shower of sparks is shown in the adjacent area. Those sparks will explode again and generate other shows of sparks in a smaller area. Gradually, the sparks will search the whole solution space in a fine structure and focus on a small place to find the optimal solution. It has favorable global search ability as well as computational robustness [16]. Therefore we use several types of for parameter estimation of two typical chaotic system. Experiments show that the method has better adaptability, reliability and high precision. It is proved to be a successful approach in parameter estimation for chaotic systems. 2 Fireworks Algorithm 2.1 Framework Assume that the number of fireworks is N and the number of dimensions is d, then the explosion amplitude A (Eq. 1) and the number of explosion sparks s (Eq. 2) for each firework X i are calculated as follows: A i = Â f(x i ) y min + ε N i=1 (f(x i) y min ) + ε. (1) s i = M e y max f(x i ) + ε N i=1 (y max f(x i )) + ε. (2) where y max = max(f(x i )), y min = min(f(x i )), Â and M e are two constants to control the explosion amplitude and the number of explosion sparks, respectively, and ε is the machine epsilon. To avoid the overwhelming effects of fireworks at

Manuscript for The Sixth International Conference on Swarm Intelligence 2015 3 good locations, the number of sparks is bounded by: round(am e ) if s i < am e, s i = round(bm e ) if s i > bm e, round(s i ) otherwise. (3) where a and b are constant parameters that confine the range of the population size. Based on A i and s i, the explosion operator is performed. For each of the s i explosion sparks of each firework X i, Algorithm 1 is performed once. In Line 7 of Algorithm 1, the operator % refers to the modulo operation (remainder of division), and X k min and Xk max refer to the lower and upper bounds of the search space in dimension k. Algorithm 1 Generating explosion sparks in 1: Initialize the location of the explosion sparks: ˆX i = X i 2: Calculate offset displacement: X = A i rand( 1, 1) 3: Set z k = round(d rand(0, 1)), k = 1, 2,..., d 4: for each dimension of ˆ Xk i {z k dimensions of ˆX i } do 5: ˆ Xk i 6: if ˆ Xk i 7: ˆ Xk i 8: end if 9: end for = X ˆ i k + X out of bounds then = Xmin k + X ˆ i k %(Xk max Xmin) k After the explosion, another type of sparks, the Gaussian sparks, are generated based on a Gaussian mutation process. This algorithm is performed M g times, each time with a randomly selected firework X i (M g is a constant to control the number of Gaussian sparks). Algorithm 2 Generating Gaussian sparks in 1: Initialize the location of the Gaussian sparks: X i = X i 2: Calculate offset displacement: e = Gaussian(1, 1) 3: Set z k = round(d rand(0, 1)), k = 1, 2,..., d 4: for each dimension of Xk i {z k dimensions of X i} do 5: Xk i 6: if Xk i 7: Xk i 8: end if 9: end for = X i k e out of bounds then = Xmin k + X i k %(Xk max Xmin) k In order to retain the information and pass it to the next generation, a new population of fireworks is selected at the end of each iteration. All original fireworks, as well as all explosion and Gaussian sparks can be selected for the next iteration (in total, N fireworks/sparks are selected). selected based on a distance based selection operator. For location X i, the selection probability p i is calculated by: R(X i ) p(x i ) = j K R(X j). (4)

4 Manuscript for The Sixth International Conference on Swarm Intelligence 2015 R(X i ) = j K d(x i, X j ) = j K X i X j. (5) where K is the set of all current locations including original fireworks and both types of sparks (without the best location). As a result, fireworks or sparks in low crowded regions will have a higher probability to be selected for the next iteration than fireworks or sparks in crowded regions. N individuals choose as the next iteration of the fireworks from the fireworks, explosion sparks and Gaussian sparks population, return optimization results. This cycle is repeated until you find the global optimal value. 2.2 Experiments To test the performance of the proposed, we conducted experiments on nine benchmark functions. The expression of the functions, initialization intervals and dimensions are listed blew: Table 1. Nine benchmark functions Function Expression Initialization D Sphere F 1 = D i=1 x2 i [30, 50] D 30 Rosenbrock F 2 = ( D 1 100(xi+1 x 2 i=1 i ) 2 + (x i 1) 2) [30, 50] D 30 Rastrigrin F 3 = ( D i=1 x 2 i 10cos(2πx i ) + 10 ) [30, 50] D 30 Griewank F 4 = 1 + D (x 2 i ) D cos( x i i=1 4000 i=1 i ) [30, 50] D 30 Ellipse F 5 = i 1 D 104 D 1 x 2 i=1 i [15, 30] D 30 Cigar F 6 = x 2 1 + D i=2 104 x 2 i [15, 30] D 30 Tablet F 7 = 10 4 x 2 1 + D i=2 x2 i [15, 30] D 30 Schwefel F 8 = ( D (x1 x 2 i=1 i ) 2 + (x i 1) 2) [15, 30] D 30 ( ) 1 D F 9 = 20 + e 20exp 0.2 D i=1 Ackley x2 i [15, 30] D 30 exp ( 1 D D i=1 cos(2πx2 i ) ) We compare the performance of the with the CPSO and the SPSO in terms of optimization accuracy. The parameters of both the CPSO and the SPSO are set as those in Ref. [17]. For the, the parameters are set as those in Ref. [15]: N = 5, M e = 50, a = 0.04, b = 0.8, Â = 40 and M g = 5, Table 2. are statistical mean and standard deviation of solutions found by the, the CPSO and the SPSO over 20 independent runs:

Manuscript for The Sixth International Conference on Swarm Intelligence 2015 5 Table 2. Statistical Results Function Function Mean CPSO Mean [15] SPSO Mean [15] evluations (StD) (StD) [15] (StD) [15] Sphere 500000 Rosenbrock 600000 Rastrigrin 500000 Griewank 200000 Ellipse 500000 Cigar 600000 Tablet 500000 Schwefel 600000 Ackley 200000 0.000000 0.000000 1.909960 (0.000000) (0.000000) (2.594634) 12.438269 33.403191 410.522552 (14.392756) (42.513450) (529.389139) 0.000000 0.053042 167.256119 (0.000000) (0.370687) (42.912873) 0.000000 0.632403 2.177754 (0.000000) (0.327648) (0.294225) 0.000000 0.000000 53.718807 (0.000000) (0.000000) (68.480173) 0.000000 0.000000 0.002492 (0.000000) (0.000000) (0.005194) 0.000000 0.000000 1.462832 (0.000000) (0.000000) (1.157021) 0.000000 0.095099 0.335996 (0.000000) (0.376619) (0.775270) 0.000000 1.683649 12.365417 (0.000000) (1.317866) (1.265322) According to Table 2, under the same standard, all the statistical mean and standard acquired by computing various complicated functions with are superior to that attained by other two algorithms. s outstanding global optimum searching ability and computation stability are both verified in experiments. 2.3 Extensions Fireworks algorithm works quite well on test functions. but there are still some places for improvement. Zheng et al. (Zheng, Janecek, & Tan, 2013) proposed an enhanced fireworks algorithm (E) in Ref [18], Zheng S proposed a dynamic search in fireworks algorithm (dyn) in Ref [19], and adaptive fireworks algorithm (A) is proposed by Li et al in 2014 [20]. In this paper, we will take advantage of these types of fireworks algorithm to estimate the parameters

6 Manuscript for The Sixth International Conference on Swarm Intelligence 2015 of chaotic systems. These algorithms are not specific elaborate in detail here, please refer to the relevant paper. 3 Estimation Framework of Chaotic System Generally speaking, an n-dimensional chaotic system could be expressed as follow formula: Ẋ = F (X, X 0, θ) (6) Where X = (x 1, x 2,..., x n ) T R n denotes n-dimensional state variable of original system, X 0 denotes system s initial state and θ = (θ 1, θ 2,..., θ m ) T R m is the unknown parameters vector, in the context that system structure is known, estimation system can be expressed as: Ẏ = F (Y, X 0, θ) (7) Where Y = (y 1, y 2,..., y n ) T R n denotes n-dimensional state variable of estimation system, and θ = ( θ 1, θ 2,..., θ m ) T R m denotes estimation values of parameters. Consequently, parameters estimation for chaotic system is transformed into an optimization problem as follow: min J( θ) = 1 M M X k Y k 2 (8) k=1 In the expression above, M denotes sequence length of state variables in parameter estimation; X k and Y k denote state variables at kth moment of the real system and estimation system respectively. Obviously, parameter estimation of chaotic system is equivalent to a multi-dimensional continuous optimization problem which needs to search for the optimum of decision variable θ to acquire minimum value of target function J( θ). Accordingly, the schematic diagram of chaotic system s parameter estimation is depicted in Fig. 1. X X F X X X X X M J best Y F Y X Y Y YM Fig. 1. The Flowchart of Chaotic Parameter Estimation

Manuscript for The Sixth International Conference on Swarm Intelligence 2015 7 4 Simulations and Analysis 4.1 Lorenz chaotic system Presented by Lorenz in 1963, can describe several different physical systems such as disk dynamos, laser devices and several problems related to convection [21]. Dynamic equation of Lorenz system is expressed as follow: ẋ = a(y x) ẏ = bx xz y ż = xy cz In which x, y and z represent state variables of system, a = 10, b = 28 and c = 8/3 are real parameter values. Using to estimate a, b and c in Lorenz. The initial range of estimated parameters is 9 a 11, 20 b 30 and 2 c 3. For the, the parameters are set as those: is 100 times, N = 5, M e = 50, a = 0.04, b = 0.8, Â = 40 and M g = 5. Table 3 are statistical optimums, mean and worst values of parameter estimation found by the, the E, the dyn, the A, the GA, the PSO and the BBO over 20 independent runs: Fig. 2 and Fig. 3 are the average fitness evolution curve of and convergence curves of parameter estimation after 20 times independent running. (9) 1.4 1.2 E dyn A 10 1 10 0 E dyn A 1 10 1 Fitness 0.8 0.6 Fitness(Log) 10 2 10 3 10 4 0.4 10 5 0.2 10 6 0 0 50 100 10 7 0 50 100 Fig. 2. Average fitness evolution curve for Lorenz system

8 Manuscript for The Sixth International Conference on Swarm Intelligence 2015 Table 3. Results of parameter estimation for Lorenz system Fitness Algorithm a b c J GA [22] 10.0671 27.9221 2.6635 4.3107 PSO [22] 9.9953 28.0071 2.6670 0.0486 Optimums Averages Worst values BBO [9] E 10.0068 27.9968 2.6667 2.36 10 5 10.0019 28.0011 2.6703 1.09 10 3 10.0000 28.0000 2.6667 9.41 10 9 dyn 9.9977 28.0030 2.6663 2.25 10 5 A 10.0000 28.0000 2.6667 4.02 10 9 GA [22] 10.1398 27.7427 2.6486 943.7629 PSO [22] 10.0184 27.9934 2.6663 4.1828 BBO [9] 10.0183 27.9913 2.6671 0.0033 9.9796 28.0001 2.6652 2.14 10 2 E 10.0000 28.0000 2.6667 2.95 10 7 dyn 9.9989 28.0017 2.6671 1.67 10 3 A 10.0002 28.0000 2.6667 1.30 10 4 GA [22] 10.9290 26.1276 2.5621 6461.4801 PSO [22] 10.6082 27.7044 2.6572 39.4060 BBO [9] 9.9440 28.0360 2.6509 0.0289 9.8497 28.0715 2.6641 5.40 10 2 E 10.0006 27.9991 2.6667 1.61 10 6 dyn 10.0730 27.9326 2.6741 1.35 10 2 A 10.0203 27.9806 2.6681 1.05 10 3

Manuscript for The Sixth International Conference on Swarm Intelligence 2015 9 10.2 10.1 10 E dyn A a 9.9 9.8 9.7 0 20 40 60 80 100 28.3 28.2 E dyn A b 28.1 28 27.9 0 20 40 60 80 100 2.7 2.68 E dyn A c 2.66 2.64 2.62 0 20 40 60 80 100 Fig. 3. Average parameter estimation curve of for Lorenz system According to Table 2, Fig. 2 and Fig. 3, Overall speaking, compared with other 3 algorithm, and its variants can get a better parameter estimation accuracy, specifically E and A, can be achieved 10 9. In these types

10 Manuscript for The Sixth International Conference on Swarm Intelligence 2015 of fireworks algorithms, A has the fastest fitness convergence rate, E has the best fitness optimums and evolution curve. 4.2 Rossler chaotic system As a simplified model of a chemical reaction system [23], Rossler system s expression showed as below is also a very famous function in nonlinear dynamics: ẋ = y z ẏ = ay + x (10) ż = b + z(x c) Rossler parameters real values are set as a = 0.2, b = 0.4, c = 5.7. Search range of estimation parameters is [0, 10]. For the, the parameters are set as those: is 100 times, N = 5, M e = 10, a = 0.04, b = 0.8, Â = 40 and M g = 5 (in total, 20 fireworks/sparks). 5 times of independent parameter estimation of Rossler system have been processed with and simulation results are compared with results of HBBO in reference [9] and DE introduced in reference [24] in Table 4. The average fitness evolution curve of and convergence curves of parameter estimation are drawn in Figure 4 and Figure 5 respectively. It is show that s have a fast convergence speed as well as outstanding performance in global optimum searching according to Figure 4 and Figure 5. Simulation results outline both solving precision and computation stability of s. 0.01 0.009 0.008 E dyn A 10 2 10 3 E dyn A 0.007 Fitness 0.006 0.005 0.004 0.003 0.002 0.001 Fitness(Log) 10 4 10 5 10 6 0 0 50 100 10 7 0 50 100 Fig. 4. Average fitness evolution curve for Rossler system

Manuscript for The Sixth International Conference on Swarm Intelligence 2015 11 Table 4. Results of parameter estimation for Rossler system Fitness Algorithm a b c J DE [24] 0.2000 0.3530 5.6710 0.0090 Optimums HBBO [9] 0.2000 0.4001 5.7000 2.02 10 10 0.1996 0.3978 5.6544 3.75 10 8 E 0.2000 0.4001 5.7023 4.09 10 11 dyn 0.1998 0.3945 5.6226 4.88 10 8 A 0.1999 0.3935 5.5969 7.19 10 8 DE [24] 0.1999 0.3728 5.7994 0.0248 Averages HBBO [9] 0.1995 0.3910 5.6639 3.37 10 5 0.1991 0.4243 6.0550 2.07 10 6 E 0.2000 0.4023 5.7373 2.37 10 7 dyn 0.2006 0.4015 5.7229 2.25 10 7 A 0.2001 0.4050 5.7793 6.85 10 7 DE [24] 0.1852 0.2000 5.5839 0.884 Worst values HBBO [9] 0.1978 0.3575 5.5296 1.60 10 4 0.1960 0.4540 6.4914 5.34 10 6 E 0.2004 0.4197 6.0053 5.80 10 7 dyn 0.2020 0.4167 5.9494 5.50 10 7 A 0.2007 0.4320 6.2059 1.52 10 6

12 Manuscript for The Sixth International Conference on Swarm Intelligence 2015 0.5 0.4 E dyn A a 0.3 0.2 0.1 0 20 40 60 80 100 b 0.7 0.65 0.6 0.55 0.5 0.45 0.4 0.35 E dyn A 0 20 40 60 80 100 7 6.5 E dyn A c 6 5.5 5 0 20 40 60 80 100 Fig. 5. Average parameter estimation curve of for Rossler system

Manuscript for The Sixth International Conference on Swarm Intelligence 2015 13 5 Conclusion In this paper, chaotic system parameter estimation problem is transformed into a class of multi-dimensional parameter optimization problem. In order to expand the application of which emerging recently, was introduced, validated and finally applied to estimate the unknown parameters of chaotic systems. With Lorenz and Rossler chaotic system, the simulation results show that the can get better effect than other algorithms, it is a new and effective chaotic system parameter estimation method. 6 Acknowledgment This work is supported by National Natural Science Foundation of China under Grant No.61472442, No.61472443 and 61203268. References 1. Wang L, Xu Y: Expert. Syst. Appl. 38 (2011) 15103 15109. 2. Liu L Z, Zhang J Q, Xu G X, Liang L S, Wang M S: Acta Phys. Sin. 63 (2014) 010501 010501 (in Chinese) 3. Hegazi A S, Agiza H N, Dessoky M M E: Internatinal Journal of Bifurcation and Chaos 12 (2002) 1579 1597. 4. Huang L L, Feng R P, Wang M: Physic Letters A 320 (2004) 271 275 5. Cheng D L, Huang C F, Cheng S Y, Yan J J: Expert Systems with Applications 36 (2009) 6854 6858 6. Liu Y, Wallace K S: Nonlinear. Dyn. 66 (2011) 213 229 7. Dai D, Ma X Q, Li F C, You Y: Acta Phys. Sin. 51 (2002) 2459 2462 8. Li L X, Peng H P, Yang Y X, Wang X D: Acta Phys. Sin. 56 (2007) 51 55 9. Lin J, Xu L: Acta Phys. Sin. 62 (2013) 030505 030505 10. Gao F, Fei F X, Xu Q, Deng Y Y, Qi Y B, Balasingham I: Applied Mathematics and Computation 219 (2012) 552 568 11. Ahmadi M, Mojallali H: Chaos, Solitons & Fractals 45 (2012) 1108 1120 12. Sheng Z, Wang J, Zhou S D, Zhou B H: CHAOS 24 (2014) 013133 013133 13. Lin J, Chen Chang: Nonlinear. Dyn. 76 (2014) 509 517 14. Wang L, He W P, Wang S Q, Liao L J, He T: Acta Phys. Sin. 63 (2014) 019203 019203 15. Tan Y, Zhu Y: Fireworks algorithm for optimization Berlin: Springer (2010) 355 364 16. Tan Y, Yu C, Zheng S Q: International Journal of Swarm Intelligence Research 4 (2013) 39 70 17. Tan Y, Xiao Z M: Proceedings of IEEE Congress on Evolutionary Computation (2007) 2303 2309 18. Zheng S, Janecik A, Tan Y: IEEE Congress on Evolutionary Computation (2013) 2069 2077 19. Zheng S, Janecik A, Li J: IEEE Congress on Evolutionary Computation (2014) 3222 3229 20. Li J, Zheng S, Tan Y: IEEE Congress on Evolutionary Computation (2014) 3214 3221

14 Manuscript for The Sixth International Conference on Swarm Intelligence 2015 21. Mahmoud G. M., Alkashif M. A., Aly S. A.: Int. J. Mod. Phys. C. 18 (2007) 253 265 22. Kusum D, Manoj T: Appl. Math. Comput 188 (2007) 895 895 23. Rossler O E: Physics Letters A 57 (1996) 397 400 24. Ho W H, Chou C Y: Nonlinear. Dyn. 61 (2010) 29 41