Joural of Modr Applid Statistical Mthods Volum Issu Articl 6 --03 O Som Proprtis of a Htrogous Trasfr Fuctio Ivolvig Symmtric Saturatd Liar (SATLINS) with Hyprbolic Tagt (TANH) Trasfr Fuctios Christophr Godwi Udomboso Uivrsity of Ibada, Ibada, Nigria, cg.udomboso@gmail.com Follow this ad additioal works at: http://digitalcommos.way.du/jmasm Part of th Applid Statistics Commos, Social ad Bhavioral Scics Commos, ad th Statistical Thory Commos Rcommdd Citatio Udomboso, Christophr Godwi (03) "O Som Proprtis of a Htrogous Trasfr Fuctio Ivolvig Symmtric Saturatd Liar (SATLINS) with Hyprbolic Tagt (TANH) Trasfr Fuctios," Joural of Modr Applid Statistical Mthods: Vol. : Iss., Articl 6. Availabl at: http://digitalcommos.way.du/jmasm/vol/iss/6 This Emrgig Scholar is brought to you for fr ad op accss by th Op Accss Jourals at DigitalCommos@WayStat. It has b accptd for iclusio i Joural of Modr Applid Statistical Mthods by a authorizd admiistrator of DigitalCommos@WayStat.
Joural of Modr Applid Statistical Mthods Novmbr 03, Vol., No., 47-435. Copyright 03 JMASM, Ic. ISSN 538 947 Emrgig Scholars: O Som Proprtis of a Htrogous Trasfr Fuctio Ivolvig Symmtric Saturatd Liar (SATLINS) with Hyprbolic Tagt (TANH) Trasfr Fuctios Christophr Godwi Udomboso Uivrsity of Ibada Ibada, Nigria For trasfr fuctios to map th iput layr of th statistical ural twork modl to th output layr prfctly, thy must li withi bouds that charactriz probability distributios. Th htrogous trasfr fuctio, SATLINS_TANH, is stablishd as a Probability Distributio Fuctio (p.d.f), ad its ma ad variac ar show. Kywords: variac Statistical ural twork, SATLINS, TANH, SATLINS_TANH, ma, Itroductio Adrs (996) proposd a statistical ural twork modl giv as (, ) y f X w + u () whr y is th dpdt variabl, X ( x x x ),,, I is a vctor of 0 idpdt variabls, w (α, β, γ) is th twork wight: α is th wight of th iput uit, β is th wight of th hidd uit, ad γ is th wight of th output uit, ad u i is th stochastic trm that is ormally distributd (that is, u i ~ N( 0, σ I )). Basically, f (X, w) is th artificial ural twork fuctio, xprssd as H I f ( X, w) αx + βh g γhixi h i 0 Th author is a lcturr i th Dpartmt of Statistics. Email him at: cg.udomboso@gmail.com. 47
PROPERTIES OF SATLINS WITH TANH TRANSFER FUNCTIONS whr g(.) is th trasfr fuctio. Th proposd covolutd form of th artificial ural twork fuctio usd i this study is H I I f ( X,w) αx + βh g γhixi g γhixi h i 0 i 0 ad thus, th form of th statistical ural twork modl proposd is H I I y αx + β g γ x g γ x + uu () h hi i hi i i j h i 0 i 0 whr y is th dpdt variabl, X ( x x x ) 0,,, I is a vctor of idpdt variabls, w (α, β, γ) is th twork wight: α is th wight of th iput uit, β is th wight of th hidd uit, ad γ is th wight of th output uit, u i ad u j ar th stochastic trms that ar ormally distributd (that is, u i, u j ~ N( 0, σ I )) ad g (.) ad g (.) ar th trasfr fuctios. Th distributioal proprtis of th htrogous modl arisig from th covolutio of SATLINS ad TANH is ivstigatd hr. Lt g (.) Symmtric Saturatd Liar fuctio (SATLINS), dfid as (.) ( ) satlis g f <,,, > (3) Lt g (.) Hyprbolic Tagt fuctio (TANH), dfid as tah g(.) f( ) + (4) 48
CHRISTOPHER GODWIN UDOMBOSO Symmtric Saturatig Liar ad Hyprbolic Tagt (i) Lt ( ) ( ) ( ) ( ) ( ) a b f f f f m f m dm (5) For <, f (), which implis also that f ( m). f ( m) Thrfor, m m + m m f( ) f( ) ( ) dm, r < < r + r -r - m -m + log ( + ) log - r + (6) (ii) Similarly, for, f (), which implis that f ( m) m, such that m. Thrfor, ( ) ( ) ( - ) ( ) - f f f m f m dm ( m) dm + (7) Usig itgratio by part, ad otig that uv uv u ' v Lt u m. This implis that du dm. 49
PROPERTIES OF SATLINS WITH TANH TRANSFER FUNCTIONS ad v d +. This implis that v log ( m m ) + +. Thus, ( ) ( ) ( ) log ( + ) + log ( + ) f f m dm (8) I (6), lt log ( + ) I dm Now, lt x log ( m m ) +, which implis that x m m + But x k for m. Hc I 0. Thrfor, ( ) ( ) ( ) ( ) f f + log + (9) (iii) Also, for >, f () a. This implis that f ( m) Thrfor, ( ) ( ) ( ) ( ) f f f m f m dm + dm log + + (0) Th summary of th drivd fuctio is giv as 430
CHRISTOPHER GODWIN UDOMBOSO I I g x g x f i 0 i 0 + log, for < + ( + ) log ( + ), for - + log, for > + γhi i γhi i ( ) () () is th drivd trasfr fuctio for th Symmtric Saturatd Liar trasfr fuctio ad th Hyprbolic Tagt trasfr fuctio. Distributioal Proprtis of th SATLINS_TANH SNN Modl Nxt it is show that th drivd trasfr fuctios ar probability dsity fuctios. By dfiitio, th probability dsity fuctio (p.d.f) of fuctio f(x) of a radom variabl X :Ω is said to b a propr p.d.f if for x, +, x X, thus, ( ) ( ), f x dx x X From th drivd trasfr fuctio i (), ( ) ( ) f f d + log d ( ) log ( ) d + + + + + + log d + 43
PROPERTIES OF SATLINS WITH TANH TRANSFER FUNCTIONS ( ) log ( ) log + + d ( ) ( ) + log + + d ( ) ( ) + log log + + d log ( ) d log ( ) ( ) d log ( ) d () + + + + + ( ) ( ) ( ) log log log + + + + + + log ( + ) log ( + ) Th ma ad variac of th drivd trasfr fuctios ar obtaid xt. For f ( ) f ( ) ( ) ( ) f f + log, for < + ( + ) log ( + ), for + log, for > + ( ) ( ) ( ) ( ) E f f d + + log ( ) log ( ) log + + d + + + d + d 43
CHRISTOPHER GODWIN UDOMBOSO ( ) log ( ) log + d + d ( ) log ( ) + + + d ( ) log ( ) + log + d + d ( ) ( d) ( ) ( ) d ( ) ( d) log + log + + log + 3 r r log ( + ) log ( + ) + log ( + ) 3 ( ) ( log log ) log ( + + ) + + 3 3 Hc, th ma of drivd trasfr fuctio i is ( ) log E ( + ) (3) 3 Similarly, ( ) ( ( ) ( )) E f f d + + log ( ) log ( ) log + + d + + + d + d ( ) log ( ) log + d + d 3 ( ) ( ) + log + d log d d ( ) log ( ) + + + 433
PROPERTIES OF SATLINS WITH TANH TRANSFER FUNCTIONS 3 4 3 3 r r log ( + ) + log ( + ) + log ( + ) 3 4 3 3 ( ) ( log log ) log ( + + + ) 3 3 + + 4 3 4 3 3 3 log + 3 ( ) ( ) Thrfor, variac of f ( ) f ( ) is var ( ) E( ) E( ) log + 3 3 + 4 log + + 3 9 + 4 log ( + ) log ( + ) 9 3 ( ) log ( ) ( ) ( ) log ( ) (4) Thus, I I g γhixi g γhixi f ( ) i 0 i 0 + log, for < + ( + ) log ( + ), for + log, for > + E log + 3 4 + + 9 3. with ma, ( ) ( ) ad variac, ( ) ( var log ) log ( ) 434
CHRISTOPHER GODWIN UDOMBOSO Rfrcs Akaik, H. (974). A w look at th statistical modl idtificatio. IEEE Trasactios o Automatic Cotrol 9, 76 73. Adrs, U. (996). Statistical Modl Buildig for Nural Ntworks. AFIR Colloqium. Nubrg, Grmay. Adrso, J. A. (003). A Itroductio to ural tworks. Uppr Saddl Rivr, NJ: Prtic Hall. Battiti, R. (99). First- ad scod-ordr mthods for larig: Btw stpst dsct ad Nwto s mthod. Nural Computatio 4, 4 66. Carlig, A. (99). Itroducig Nural Ntworks. Ammaford, Eglad: Sigma Prss. Fors F. D., & Haga, M. T. (997). Gauss-Nwto Approximatio to Baysia Rgularizatio. I IEEE Itratioal Cofrc o Nural Ntworks (Vol. 3), 930 935. Nw York: IEEE. Gold, R.M. (996). Mathmatical Mthods for Nural Ntwork Aalysis ad Dsig. Cambridg, MA: MIT Prss. Lawrc, J. (994). Itroductio to ural tworks: Dsig, thory, ad applicatios. Nvada City, CA: Califoria Scitific Softwar Prss. Mar, A., Harsto, C., & Pap, R., (990). Hadbook of Nural Computig Applicatios. Sa Digo, CA: Acadmic Prss. Nlso, M. M. & Illigworth, W. T. (99). A Practical Guid to Nural Nts. Addiso-Wsly Publishig Compay. Smith, M. (993). Nural tworks for statistical modlig. Nw York: Va Nostrad Rihold. Taylor, J. G. (999). Nural tworks ad thir applicatios. Nw York: Wily. Warr, B., & Misra, M. (996). Udrstadig ural tworks as statistical tools. Th Amrica Statisticia, 50(4), 84 93. 435