Sborník vědeckých prací Vysoké školy báňské - Technické univerzity Ostrava číslo 2, rok 2007, ročník LIII, řada strojní článek č.

Transcription

1 Sborník vědeckých prací Vysoké školy báňské - echnické univerzity Ostrava číslo 2 rok 2007 ročník LIII řada strojní článek č 1570 Jolana ŠKUOVÁ * AN ORHOGONAL NEURAL NEWORK FOR NONLINEAR FUNCION ODELLING ODELOVÁNÍ NELINEÁRNÍ FUNKCE OROGONÁLNÍ NEURONOVOU SÍÍ Abstract Neural netorks provide one of the means for identification or control of a nonlinear system ultilayer feedforard netork is often used in those areas Using this kind of neural netork may result in premature termination of learning before the netork eights reach a global minimum a small training convergence an unsuitable eight parameters initialize and the stability for on-line implementation In this paper an orthogonal neural netorks (ONN) is presented he ONN s hidden layer consists of neurons ith orthogonal activation function he ONN have a much higher learning speed than the multilayered feedforard netorks Abstrakt V oblasti modelování nebo řízení neuronových sítí se často využívá neuronová síť se zpětným šířením tzv perceptronová neuronová síť která se potýká s problémy lokálního minima nízkou rychlostí konvergence volbou počátečních parametrů vah a počtem neuronů V tomto příspěvku je prezentována ortogonální neuronová síť s jednou skrytou vrstvou která využívá ortogonální aktivační funkce ato neuronová síť umožňuje odstranit výše uvedené problémy Ortogonální neuronová síť má mnohem vyšší rychlost učení než dopředné sítě 1 INRODUCION In area of a function approximation a prediction as the identification of unknon system a system control and optimization robotics are no common used the neural netorks here are multilayer feedforard neural netorks are often applied concretely perceptron and radial neural netork by type of the activation function With their application feedforard neural netork have very problematic point of learning convergence speed an acquirement of global minimum a settings of initial eight parameters here are many publications deal ith various methodologies hich remove this problems and one of the radical approach is the selection novel structure of neural netork to removing this sticking point his contribution describes novel model of neural netork ith unconventional internal ordering of neuron s connection and an orthogonal type of activation function Finally it s proposed conclusion of simulation modelling of orthogonal neural netork for chosen nonlinear function 2 ORHOGONAL NEURAL NEWORK SRUCURE ONN (orthogonal neural netork) is feedforard neural netork ith multi inputs and single output (ISO multi-input-single-output) and a hidden layer ith orthogonal activation function in hidden neurons (Fig 1) he inputs of neural netorks are distributed into orthogonal neurons blocks * Ing Department of Control Systems and Instrumentation Faculty of echanical Engineering VSB echnical University of Ostrava 17 listopadu 15/2172 Ostrava-Poruba tel (+420) jolanaskutova@vsbcz 153

2 for each input he number of neurons for each input signals is arbitrary and ith neuron correspond to ith orthogonal function order he number of orthogonal neurons is given by m the number of inputs [-] N i the number of neurons for each input [-] m N ortg = N i he other layer of ONN is arranged to nodes hich consist of products combinations of the particular outputs from orthogonal neurons and it s defined as m φn 1 K nm i= 1 ( x) = φ ( x ) m the dimension of input vector [-] ni i i= 1 x = x x ] [ 1 2 K φ i the output of orthogonal function implemented by each hidden layer neuron [] - x m (1) (2) Orthogonal function Produkt function Sum function x K Nm 2 1K Nm x 2 N1 1K N m 1 2 K Nm 2 2 K Nm Σ y 1 N 2 K N m 2 N 2 K N m x m N 1 N 2 K N m Fig 1 he structure of orthogonal neural netork 154

3 he ONN output is given by sum of the all output nodes from previous layer (2) and one is can be mathematically expressed as yˆ Φ the transformed input vector [-] N N 1 m ( x v) L φ ( x) = Φ ( x) ˆ = ŵ the transformed eight vector [-] n1 = 1 nm = 1 n1lnm n1 Lnm (3) Orthogonal Activation Function he rate of convergence due to using of orthogonal functions as activation function in neural netork is larger then the rate of convergence for perceptron or radial neural netorks hen it s sticking point he set of one-dimensional orthogonal functions φ are defined as Φ δ ij Kronecker delta function [-] hich is given by x ( N ) ( x) φ ( x) max ( N ) φi j dx = δij xmin 1 δij = 0 pokud pokud n i (4) i = j (5) i j here has been number of the orthogonal polynomials eg Hermite Legendre Laguerre Chebyshev and Fourier polynomials [5] hus far for modelling given nonlinear function applied Legendre and Fourier polynomial only he orthogonal neural netork ith Fourier activation function had a sufficient modelled outputs hence next experiments as realized by the ONN ith Legendre polynomial 3 RAINING EHOD OF ORHOGONAL NEURAL NEWORK Generally the learning process is performed by adapting the netork eight such that the expected value of the mean squared error beteen netork output and training output is minimized he gradient descent-based learning algorithms are popular training algorithm for neural netorks o train proposed netork learning rules are determined from Lyapunov-like stability analysis he cost function ONN is given by e the learning error [-] y ) the output of ONN [-] y the actual output [-] ) E = e = ( y y) 2 (6)

4 he gradient descent algorithm adjusts netork eights in such a ay that the square of the neural netork learning error changes in a negative gradient direction and then eight adaptation is given by ) the variation of netork eights [-] δ the netork learning rate [-] ) the netork learning rate [-] E the mean square error [-] ) E = δ ) (7) hus the ONN s eight update la for the instantaneous gradient descent algorithms is given as ) ) ( t) = ( t 1) + δeφ( t) (8) ) the transformed input vector consisting of orthogonal functions [-] e the learning error [-] o guarantee of ONN s training stability the learning rate δ must be bounded by 0 2 δ < (9) Φ Φ < his learning rate is one of the parameters that are the part of experiment to the acquisition of neural netork optimal model depending on chosen modelled system [1] 4 ODELLING OF NONLINEAR FUNCION his paper presents the nonlinear function modelling results that are step to development in modelling of real-time system as ell as the orthogonal neural netork apply to control system subsequently he algorithm for orthogonal neural netork implementation in atlab/simulink program have already been designed and compiled Further the necessary source code optimization has been performed regarding to decrease in the computational time he verification and experiments ith particular structure design has been implemented on an approximation of nonlinear function that is expressed as 2 y ( x) = 1 (10) 2x 1+ e he neural netork structure is given by one input and one output of nonlinear function and hidden layer includes 20 neurons ith orthogonal function of Legendre activation function kind concretely from 1st to 20th degree he orking range of input signal correspond to the values on the interval [-25 25] he ONN s training carry out in epochs (one epoch presents one training algorithm loop for hole training data) o quality results of the neural netork eight update a number of epochs have been chosen maximally 100 epochs he learning rate value has been the question of experiments and one as settings on fourth part of maximum parameter value δ in Eq (9) 156

5 he initial eight as selected in interval [-1; 1] randomly he global minimum acquisition as demonstrated by multiple running of training process ith different initial eight values he final eight values have been steadied (Fig 4) compared ith eight values in first training epoch he orthogonal neural netork as implemented in Simulink program ith minimal number of the blocks Further the individual degrees of Legendre polynomial ere realized by particular -files he very sticking point as the composition of ONN s product nodes in Simulink program Fig 2 he ONN s error for given nonlinear function Fig 3 he desired actual output and ONN s output for given nonlinear function Fig 4 he ONN s eight values for first epoch (solid line) and last epoch (dash line) of training process Fig 5 he ONN training convergence 5 CONCLUSIONS he orthogonal neural netorks pruning aay some fundamental sticking point of perceptron neural netorks applications as a rate of convergence speed of the ONN learning process an achieving of global minimum hile neural netork training In contrast ith perceptron netorks it requires different training data to acquirement of a robust neural netork It no proceed tests and experiments ith the selection of inputs number and inputs pattern the selection of training data sets and the number of orthogonal neurons (Legendre polynomials order) for real pressure-air system For a system control using neural netork exists many control strategies using neural netorks that design of a suitable neural netorks structure is examined and testing in order to a quality control 157

6 his ork as developed ith the support of the research project GACR 101/06/0491 REFERENCES [1] LEONDES C Control and Dynamic Systems Academic Press pp ISBN [2] YANG S S & SENG C S An Orthogonal Neural Netork for Function Approximation In IEEE ransaction on Systems an Cybernetics Part B 1996 Vol 26 Issue 5 pp ISSN [3] SOLOWAY D & HALEY P J Neural Generalized Predictive Control A Neton-Raphson Implementation In Proceedings of the IEEE International Symposium on Intelligent Control Dearborn: ISICs 1996 pp ISBN [4] GUO B & YU J odel Adaptive Control Based on a Compound Orthogonal Neural Netork International Journal of Information echnology 2006 Vol 12 Issue 5 pp ISSN [5] WIKIPEDIA he Free Encyclopedia [online] available from : <URL: Revieer: doc Ing Zora Jančíková CSc VŠB - echnical University of Ostrava 158