The Algorithms Optimization of Artificial Neural Network Based on Particle Swarm

Transcription

1 Send Orders for Reprints to The Open Cybernetics & Systeics Journal, 04, 8, Open Access The Algoriths Optiization of Artificial Neural Network Based on Particle Swar Yang Xin-quan,* Coputer and Inforation Engineering Departent, HeZe University of Heze, Shandong, 7405, China; Ebedded systes and Internet of Things institute, HeZe University of Heze, Shandong, 7405, China Abstract: As a new kind of swar intelligence algorith, particle swar optiization (PSO) algorith can be calculated conveniently to achieve fast convergence and good convergence perforance advantages. However, it shows shortcoing of falling into local extree point. In this paper, a harony search algorith was used to iprove PSO. Harony Search Algorith, as a new optiization algorith, presents a good global search perforance. By exaining four standard test functions, the accuracy of convergence speed or convergence using iproved PSO harony search algorith was validated. Keywords: Neural network, optiization, particle swar optiization algorith.. INTRODUCTION With the developent of artificial neural network research, theoretical researches on the neural network focus on the neural basis of theoretical utilization according to research results and explore ore functions. The perforance of neural network atheatical odel is ore superior [- 4]. The studies on network algoriths and perforance such as stability, convergence, fault tolerance, and robustness have been widely conducted. Optiization of neural networks has attracted increasing attentions. PSO algorith, as a typical swar intelligence algorith, is perfored by siulating behaviors of birds foraging to for swar intelligence-based iterative evolutionary coputation techniques [5-7]. This algorith is conducted to realize the particle solution space extree optiization. Each particle has an objective function deterined by fitness value. On the basis of their speeds and own optial solutions, the optial solution groups were deduced to finally converge to global optial solution. PSO algorith is ainly utilized to perfor function optiization, neural network training, engineering applications, and in bioengineering, electroagnetic, data ining and other fields. The results achieved are excellent.. BASIC PARTICLE SWARM ALGORITHM Suppose there are particles in n-diensional search space, the weight and volue of each particle are unknown, in the search space at a certain flight speed, the flight speed and direction can be dynaically adjusted according to flight experiences of individuals and groups. X i denotes the current position of particle i; V i is the flight speed for particle i; P i is the optial location of particle i 874-0X/4 experienced. It is best known as the location of the individual. i has experienced fitness value corresponding to the iniu position. For iniization proble, the saller the objective function value, the better it adapts to the corresponding value. Considering the iniization proble [8-], the subject position of the target function is the optial f(x), i is the particle deterined by using the forula (): P i (t +) = P(t), if f ( X (t +)) f (P(t)) i i i () X i (t +), if f ( X i (t +)) < f (P i (t)) Based on particle swar, all particles have experienced the optial location for P g (t), are called as the best position in the group. Then: P g (t) {P 0 (t), P (t),, P s (t)} f (P g (t)) = in{ f (P 0 (t)), f (P (t)),, f (P s (t))} The basic particle swar optiization evolutionary is (t +) = (t) + c r j (t)[p ij (t) X ij (t)]+ c r j (t)[p gj (t) X ij (t)] The process of basic particle swar optiization is presented as follows: Step : Initializing the position and velocity of the particle swar. Step : Calculating the fitness of each particle. Step 3: Illustrating the current positions and personal best positions of each particle respectively prior to coparing their results. If the fitness of the current position of the personal best position is better than before, ost individuals are updated to excellent location. Step 4: For all particles, their current positions and the best positions before drawing groups are copared. If the 04 Bentha Open () (3)

2 50 The Open Cybernetics & Systeics Journal, 04, Volue 8 Yang Xin-quan individual fitness value is better than the current position of the group before conducting optial position, the best position is updated in groups. Step 5: Particle swar particles are updated based on the speed and position in the forulae 3. and 3.3. Step 6: If the fitness value fails to eet the required standard conditions, or the nuber of iterations does not reach the set value, step two needs to be repeated. Otherwise ends. In the case of the particle swar in evolution, cognitive part is (t +) = (t) + c r j (t)[p ij (t) X ij (t)] (4) Then each particle can be updated based on their experiences. If the inforation sharing and exchange of inforation aong the particles are insufficient, it shows no significance in swar intelligence. If the particle swar evolves only in the social part, we obtain (t +) = c r j (t)[p gj (t) X ij (t)] (5) The groups only with PSO have no eory capacity when the inforation is updated on the optial position. Although the convergence rate will be accelerated, it is likely to fall into local optia. 3. APPLICATION OF PSO ALGORITHM IN THE HARMONY SEARCH PSO algorith has the global search ability, is apt to fall into local optial solution drawback. Aiing at this shortcoing, this research uses the global search perforance and better harony search algorith for iproving particle swar algorith. By initializing the group velocity and position of all the particles, PSO algorith is to update the particle velocity and position of each particle according to their own best positions and global best positions. Consequently, a global search for obtaining the optial solution can be carried out. Particle swar of each particle represents a vector solution; while harony eory is coposed by a nuber of vector solutions, then one can take the whole particle swar as a harony eory. Each vector solution is particle swar in each particle. In this case, harony search algorith and particle swar optiization can be cobined together. By integrating PSO algorith with harony search, the basic principle is to initialize the particles in particle swar initialization. The initial position of each particle is X k = ( X k, X k, X n k ) (6) The equivalent of the entire particle swar harony eory is HM = X X n X X n X X n X X X (7) According to previous forulae for updating the speed and position of particle swar particle, we obtain (t +) = (t) + c r j (t)[p ij (t) X ij (t)]+ c r j (t)[p gj (t) X ij (t)] X ij (t +) = X ij (t) + (t +) (9) This research calculates the fitness of each particle to generate a rando nuber between rand 0 and. If the rand value is less than the probability of harony eory HMCR, a new particle is generated. The value of each diension that the particles correspond to HM atrix colun vector diension randoly selected in the harony eory. Otherwise, the new particles are generated fro the solution space of any value. If the rando nuber is less than the tone adjustent rand probability PAR, the position of the swar particles are disturbed. The new particle fitness value (Xp) is calculated. If the fitness is better than that of the worst fitness value in particle swar of new particles, it will be replaced with a new generation of particles worst fitness value. To achieve a final result, we continue to deterine the optial position of the particle positions and ost groups, and update the speed and position of the particle swar particles. 4. PERFORMANCE OF PARTCLE SWARM SMULATON In order to verify the perforance of integrated PSO algorith and harony search, this paper perfored four standard test functions experient. 4.. Swar Function y = (x 0cos(x ) +0) (0) Fig. (). Restringing function. Restringing function is a ulti-peak continuous function, with a large nuber of local extree points of inflection caused by the cosine and each independent variable. Owing to Restringing function has a large nuber of extree points, optiization algoriths in search global optial so- (8)

3 The Algoriths Optiization of Artificial Neural Network The Open Cybernetics & Systeics Journal, 04, Volue 8 5 lution is likely to fall into local extree point (Fig. ). Therefore, the function optiization algorith is relatively coplex. Its extree value probles are difficult to be solved. 4.. Gırewank Function x i y = 4000 cos( x ) + () ii Fig. (). Girewank function. Girewank function refers to that the coplex ulti-peak function aong various variables is interrelated. Because Girewank function has a large nuber of local inia and tall obstructions, so the optiization algorith is also ore likely to fall into local extree point (Fig. ). The function optiization algorith is also considered to be relatively coplex. It extree value probles are ore difficult to be deals with Ackley Functıon y = 0exp(0. x ) exp( i cos(x ) ) e () Fig. (3). Ackley function. Ackley function indicates that a function of the cosine function odulation index generate a ulti-peak continuous function. Due to the superposition of adjustent cosine function, Ackley function surface has a lot of local extree points (Fig. 3). In coparison with the first two functions, global extree points of Ackley function show soe optiization extrees in optiization algorith when function is siple. 5. OPTIMIZATION BP NETWORK OF PARTICLE SWARM BP network is one of the ost coon and widely used neural networks. In the section two, the artificial neural network and BP algorith are deonstrated in detail. The BP network connection weights are ainly deterined using traditional BP algorith gradient optiization ethod. However, this ethod is not only inefficient, but also apt to fall into local optia. The particle swar optiization algorith is likely to achieve rapid calculation. Besides, it is able to act sae role as the genetic algoriths evolutionary algorith in optiization of the neural network. 5.. Design Ideas of the Algorith Step : Deterine the network structure, as well as all the connection weights in the network including the all thresholds. This figure shows that the solution vector of particle swar particle diension. The solution vector of each diension corresponds to a connection weight. In this case, each particle group of a particle represents a vector consisting of the weights of different particles. This is different with a collection of weights. Step : Initialization of particle swar. The particle swar particle size, evolutionary ties, reconciliation space particle velocity range, the initial velocity and position of each particle are deterined. Meanwhile, the individuals and groups initialized present ost optial position locations. Step 3: Training of neural network. The particle swar particle which represents each solution vector is apped to the weight in the network to construct the neural network. In order to ensure the generalization ability of neural network, the saples collected are divided into training saples and test saples. The input value is introduced into the training saple in the neural network. Afterwards, the actual output value is calculated. Then copared with the output values of the saples, the error values are obtained to calculate each ean square error of the network, so as to adapt the particle values. Step 4: Updating particle based on PSO algorith. The fitness value of each particle is used to deterine the optial location and the optial position of each particle populations. Then, according to the evolution equation for the particle position, velocity of the position and each particle are updated. The particle position is changed. The solution vectors of each particle show changes. By repeating step three, a new fitness value of each particle is calculated again to have loop eeting the condition requireent. Step 5: Test of neural network. Convergence of the optial solution space is apped to the location of the particle weights in the neural network to construct the final neural

4 5 The Open Cybernetics & Systeics Journal, 04, Volue 8 Yang Xin-quan network. Then test saples are input into the final version of the neural network to calculate the error value and the desired output of the actual output, as an evaluation based on neural networks. 5.. Evaluation And Analyses of Algoriths Evaluating capabilities of particle swar optiization algorith in neural networks can generally be ade based on the perforance of the trained neural network. BP network is used to carry out a specific object class or function fitting. There are four indicators in particle swar optiization BP networks. () Classifying error rate of training saples The total nuber of training saple is set by classifying M and BP network. A total of saples show classification errors in the training. The error rate classified of training saple is: in training. The classifying error rate of training saples: Test = n 00% (4) N (3) Mean square errors of training saples M MSE Train = M (Y Y ') (5) When the BP network is used to solve classification probles, the four perforances of tested neural network can be referred. Train = 00% (3) M () Classification for the error rate of test saples The total nuber of test saple is set based on N and BP network. A total of n saples present the classification error (a) (a) (b) (b) Fig. (4). Non-optiized BP network prediction. (c) Fig. (5). BP network prediction algorith optiization using PSO.

5 The Algoriths Optiization of Artificial Neural Network The Open Cybernetics & Systeics Journal, 04, Volue 8 53 (a) (b) as a black box syste. The first BP network training syste is endowed with input and output data, so that BP network can fit the unknown syste. Then, one can use the trained BP neural network to predict the output of the syste. In experients, BP network is not optiized. Basic PSO algorith is used to optiize BP network; while PSO-HS algorith is applied to optiize the siulation results of BP network MATLAB. 00 sets of experients are conducted. Aong which, each case is tested for 00 ties. The experiental results are shown in Figs. (4-6). The results are presented as follows: Based on the optiization process evolving fro BP network to the particle swar optiization harony search, the convergence rate of the robustness has significantly iproved in particle. However, the convergence of the particles is lower than the original PSO algorith. For non-optiized BP network, the ean squared error and variance in test saples are and.659 respectively. Data distribution is ore concentrated. After optiizing BP network using PSO algorith, the ean squared error of the test saple is The ratio of non-optiized variance (8.7608) decreases apparently. Besides, data distribute ore dispersedly. By optiizing BP network using PSO-HS algorith, the ean squared error and the sallest variance are.3556 and respectively. All the data range between [0, 3] and distribute in ore concentrated way. Optiizing BP network using PSO HS algorith is significantly superior to non-optiized BP network and the BP network optiizing by PSO algorith. CONCLUSION This research integrates particle swar optiization and harony search algorith to optiize the neural network of particle swar algorith based on neural network optiization theory, design ideas and perforance evaluation. Through conducting odeling, paraetric design, prograing, and siulation and referring non-optiized BP network, basic PSO algorith is used to optiize BP network using PSO-HS algorith. Optiization neural network are conducted 00 experients. The results verified that optiizing neural network using PSO-HS algorith has a significant effect. CONFLICT OF INTEREST The author confirs that this article content has no conflict of interest. (c) Fig. (6). BP network prediction results with PSO-HS algorith optiization. 6. EXPERIMENT AND SIMULATION In practical engineering, there are usually a nuber of coplex nonlinear systes with coplicated equations. Those systes are difficult to be odeled atheatically in accurate way. In this case, BP neural network can be utilized to present these nonlinear systes. This approach is known ACKNOWLEDGEMENTS Declared none. REFERENCES [] E. Kse, Deterination of color changes of inks on the uncoated paper with the offset printing during drying using artificial neural networks, Neural Coputing and Applications, vol. 5, no. 5, pp. 85-9, 04. [] F. Rezaei, P. Karii, and S.H. Tavassoli, Effect of self-absorption correction on LIBS easureents by calibration curve and artificial neural network, Applied Physics B: Lasers and Optics, vol. 4, no. 4, pp , 04.

6 54 The Open Cybernetics & Systeics Journal, 04, Volue 8 Yang Xin-quan [3] K. Aarasinghe, D. Wijayasekara, and M. Manic, EEG based brain activity onitoring using Artificial Neural Networks, In: Proceedings-of 7 th International Conference on Huan Syste Interactions HSI, 04, pp [4] K.P. Waidyarathne, and S. Saarasinghe, Artificial neural networks to identify naturally existing disease severity status, Neural Coputing and Applications, vol. 5, no. 5, pp , 04. [5] R. Poli, J. Kennedy, and T. Blackwell, Particle swar optiization, Swar Intelligence, vol., no., pp , 007. [6] G. Venter, and J. Sobieszczanski-Sobieski, Particle swar optiization, Aerican Institute of Aeronautics and Astronautics Journal, vol. 4, no. 8, pp , 003. [7] R.C. Eberhart, and Y. Shi, Particle swar optiization: developents, applications and resources, In: Evolutionary Coputation, Proceedings of the 00 Congress on (IEEE), vol., pp. 8-86, 00. [8] E.B. Azardoost, and A. Jashidi, Providing a genetic algorith optiized based on regression equations for solving flexible jobshop scheduling proble (With objective function iniization for akespan), In: IIE Annual Conference and Expo 00 Proceedings, 00. [9] O. Bokanowski, and H. Zidani, Minial tie probles with oving targets and obstacles, In: IFAC Proceedings Volues (IFAC- Papers Online), vol. 8, (PART ), pp , 0. [0] R.Y. Sianchev, and I.V. Urazova, An integer-valued odel for the proble of iniizing the total servicing tie of unit clais with parallel devices with precedences, Autoation and Reote Control, vol. 7, no. 0, pp. 0-08, 00. [] Y. Ohtsubo, Miniizing risk odels in stochastic shortest path probles, Matheatical Methods of Operations Research, vol. 57, no., pp , 003. [] L. Guo, and M. Garland, The use of entropy iniization for the solution of blind source separation probles in iage analysis, Pattern Recognition, vol. 39, no. 6, pp , 006. Received: Septeber, 04 Revised: Noveber 30, 04 Accepted: Deceber 0, 04 Yang Xin-quan; Licensee Bentha Open. This is an open access article licensed under the ters of the Creative Coons Attribution Non-Coercial License ( licenses/by-nc/3.0/) which perits unrestricted, non-coercial use, distribution and reproduction in any ediu, provided the work is properly cited.